MIT introduces new data-rich approach for training robots

MIT has unveiled a new method for training robots that scales up data in a way similar to large language models (LLMs), marking a shift from the narrow, task-focused data sets traditionally used in robotics. Imitation learning, where robots learn by observing humans, often struggles with new variables like lighting changes or unexpected obstacles. By adopting a vast data approach similar to that used in models like GPT-4, MIT’s researchers aim to help robots adapt more flexibly in varied environments.

The team developed a new architecture called Heterogeneous Pretrained Transformers (HPT), which combines information from multiple sensors and diverse settings to build robust training models. Larger transformers yielded improved outcomes, aligning with trends seen in LLMs, as HPT integrates data from multiple sources for more adaptable robotic responses.

Ultimately, researchers aspire to create a universal ‘robot brain’ that can be downloaded and used immediately without extra training. While still in early stages, the project has support from Toyota Research Institute, which recently partnered with Boston Dynamics to integrate learning research with advanced robotic hardware.