From Dreams to Reality: Synthetic Data From Neural Simulation for Robot Training

Generalist robots must reason, plan, and act across many environments and tasks when given instructions. To learn new tasks, developers train robot models on real world data. But human demonstrations are costly to capture.Groot Dreams is a blueprint for synthetic data generation and neural simulation built on NVIDIA Cosmos. Using a single image and natural language, developers generate synthetic world states or dreams. These passive dreams can be generated at scale, but prompting with natural language has i ...