One of the most powerful ways to train robots to navigate a home and perform useful real-world tasks is to teach them in simulation. Exploring the virtual world allows AI agents to practice a task thousands or even millions of times faster than they could in real physical space.
In 2019, Facebook rolled out the first version of the platform called “AI Habitat”, an open-source simulation platform of photorealistic 3D home environments that could be used to train robots to navigate these environments. Now the business is taking it to the next level.
This week the company announced Habitat 2.0, a next-generation simulation platform that will allow artificial intelligence (AI) researchers to train machines not only to navigate 3D virtual environments, but also to interact with objects. just as they would in a real kitchen, dining room or other commonly used space.
The new platform offers vastly improved speeds, new benchmarks, and a rebuilt data set. This will help researchers train robots to navigate these environments much faster and more efficiently than before and to perform tasks in this environment, such as filling the refrigerator, setting the table for dinner, loading the dishwasher, and even take out the trash.
The second version of Facebook’s simulation platform relies on a new dataset called ReplicaCAD. Habitat 2.0 uses a mirror image dataset compared to its predecessor, but previously static 3D scans have been converted into individual 3D models with physical parameters, collision proxy shapes. This means robots can be trained to move and manipulate them in a whole new way.
To ensure that robots are taught effectively, the new dataset includes information on material composition, geometry and texture. The interactive recreations also incorporated information about size and friction, whether an object (such as a refrigerator or a door) had compartments that could open or close, and how these mechanisms worked, among other considerations. ReplicaCAD offers 111 unique living space layouts and 92 objects that took 3D artists 900 hours to create.
The speed of Habitat 2.0 also shows a marked improvement over the previous version. The platform can simulate a Fetch robot interacting in ReplicaCAD scenes at 1200 steps per second (SPS) while existing rigs typically operate at 10-400 SPS. Such speeds dramatically reduced experimentation time, allowing researchers to perform experiments that typically lasted more than six months in as little as two days.
The researchers believe their new platform will provide a research framework for embodied AI training for years to come. “We hope that the ability to perform more complex tasks in simulation will bring us closer to AI which can help make our daily lives easier and better.“, noted Dhruv Batra, Facebook researcher.