A naukri.com initiative
Mit
1M
264
Image Credit: Mit
Can robots learn from machine dreams?
- MIT researchers have developed a new training approach to accelerate the deployment of adaptable machines in the real world. The non-invasive system called LucidSim bridges gaps between different technologies. It uses large language models to create various structured descriptions of virtual environments, which then transform into images using generative models. The images reflect real-world physics because of an underlying physics simulator, which is used to guide the generation process.
- Typically, when roboticists want to improve robots' abilities, they invent tasks that push the boundaries of their capabilities. But this process has a scaling problem: the demand for high-quality training data, which is needed to improve the robot, outpaces the humans' ability to provide it.
- LucidSim combines physics simulation with generative AI models to create diverse and realistic virtual training environments. It helps robots achieve expert-level performance in challenging tasks without the need for any real-world data. The system addresses one of the most persistent challenges in robotics, which is the ability to transfer skills learned in simulation to the real world.
- LucidSim's images reflect real-world physics due to an underlying physics simulator, which guides the generation process. LucidSim outperforms the go-to method of domain randomization. Although that technique generates diverse data, it lacks realism.
- The LucidSim could potentially apply beyond quadruped locomotion and parkour, its main test bed, to the mobile manipulation where a mobile robot is tasked to handle objects. The robots still learn from real-world demonstrations at the moment. LucidSim could make data collection easier and more scalable by moving it into a virtual environment.
- One of the challenges in sim-to-real transfer for robotics is achieving visual realism. The LucidSim framework could provide an elegant solution by using generative models to create diverse and highly realistic visual data for any simulation.
- When comparing LucidSim to 'expert training,' where an expert teacher demonstrates the skill for the robot to learn from, the results were surprising. Robots trained by the expert struggled, succeeding only 15% of the time, even when the amount of expert training data quadrupled. But when robots collected their own training data through LucidSim, success rates increased to 88%. "And giving our robot more data monotonically improves its performance — eventually, the student becomes the expert," says Yang.
- The researchers presented their work at the Conference on Robot Learning (CoRL) in early November. Their work was supported, in part, by a Packard Fellowship, a Sloan Research Fellowship, the Office of Naval Research, Singapore’s Defence Science and Technology Agency, Amazon, MIT Lincoln Laboratory, and the National Science Foundation Institute for Artificial Intelligence and Fundamental Interactions.
Read Full Article
15 Likes
For uninterrupted reading, download the app