A naukri.com initiative
VentureBeat
1M
420
Meta unveils AI tools to give robots a human touch in physical world
- Meta has released 3 AI research artifacts including Sparsh, Digit 360 and Digit Plexus to help robots have a human touch in physical world. Sparsh provides robots with touch perception, Digit 360 is an artificial finger-shaped tactile sensor and Digit Plexus aims to facilitate the development of robotic applications. Meta is also releasing PARTNR a new benchmark evaluating planning and reasoning in human-robot collaboration. Meta's Sparsh can be applied to different types of vision-based tactile sensors and various tasks and it uses self-supervised learning, which obviates the need for labeled data. Digit 360 has on-device AI models to reduce reliance on cloud-based servers and it captures various sensing modalities to provide a richer understanding of the environment and objects interactions.
- Digit Plexus can integrate various fingertip and skin tactile sensors onto a single robot hand, encode the tactile data collected from the sensors and transmit them to a host computer through a single cable.
- PARTNR includes 100,000 natural language tasks in 60 houses and involves more than 5,800 unique objects. It is designed to evaluate the performance of LLMs and VLMs in following instructions from humans and is built on top of Habitat, Meta's simulated environment.
- Meta will be manufacturing Digit 360 in partnership with tactile sensor manufacturer GelSight Inc. They will also partner with South Korean robotics company Wonik Robotics to develop a fully integrated robotic hand with tactile sensors on the Digit Plexus platform.
- Meta's Sparsh gains an average 95.1% improvement over task- and sensor-specific end-to-end models under a limited labeled data budget as it overcomes the challenges faced by previous generations of touch perception models.
- Sparsh is created to train robots with touch perception capabilities such as determining how much pressure can be applied to a certain object to avoid damaging it.
- Meta is publicly releasing the code and designs for Digit 360 to stimulate community-driven research and innovation in touch perception.
- Digit 360 also has potential applications, such as from medicine and prosthetics to virtual reality and telepresence, are significant as it reduces the reliance on cloud-based servers.
- Meta's new benchmark, PARTNR, joins a growing number of projects exploring the use of LLMs and VLMs in robotics and embodied AI settings.
- Foundation models, such as large language models and vision-language models provide renewed hope in the industry that robots can accomplish more complex tasks that require reasoning and planning.
Read Full Article
25 Likes
For uninterrupted reading, download the app