Natural language meets robotics in MIT’s on-demand object creation system

MIT researchers have developed a speech-to-reality system that allows users to create physical objects by describing them aloud, combining generative AI with robotic assembly. The system can produce simple furniture and decorative items in minutes using modular components.

The workflow translates spoken instructions into a digital design using a large language model and 3D generative AI. The design is then broken into voxel-based parts and adapted to real-world fabrication constraints before being assembled by a robotic arm.

Researchers have demonstrated the system by producing stools, shelves, chairs, tables and small sculptures. The approach aims to reduce manufacturing complexity by enabling rapid construction without specialised knowledge of 3D modelling or robotics.

Unlike traditional fabrication methods such as 3D printing, which can take hours or days, the modular assembly process operates quickly and allows objects to be disassembled and reused. The team is exploring stronger connection methods and extensions to larger-scale robotic systems.

The research was presented at the ACM Symposium on Computational Fabrication in November. The team said the work points toward more accessible, flexible and sustainable ways to produce physical objects using natural language and AI-driven design.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!