Researchers at Stanford and Caltech have connected the GPT-6 Astra model directly to a Unitree G1 robot to test autonomous cleaning in an unfamiliar kitchen. The resulting system, named HomeBody, removes the usual trained control layer between the language model and the hardware. Instead, the swappable vision-language model calls straight into an extensible skill library for grasping, navigating, or opening drawers. The robot first explores the room to build a digital twin in Nvidia’s Isaac Sim and logs objects in spatial memory. This approach allows the machine to locate items even after they move out of its field of view. For commands like cleaning the kitchen, the language model plans each step and self-corrects on errors.
The experiment highlights a shift toward direct integration of large models with physical actuators, though practical deployment faces significant hurdles. Current limitations include the latency of the Astra model, overheating finger servos, and high compute costs. Earlier benchmarks noted greatly improved spatial reasoning, while separate tests flagged safety issues when the model controls a robot. OpenAI has already announced plans to return to robotics, including applications for personal use. The code is available on GitHub for independent review.
- Unitree G1 robot
- Nvidia Isaac Sim
- HomeBody system
