Over the next two decades, the market for humanoid robots is expected to reach $38 billion. To address this significant demand, particularly in industrial and manufacturing sectors, NVIDIA is releasing a collection of robot foundation models, data pipelines, and simulation frameworks to accelerate next-generation humanoid robot development efforts.
NVIDIA Isaac GR00T Blueprint for Synthetic Motion Generation
Announced by NVIDIA founder and CEO Jensen Huang today at the CES trade show, the NVIDIA Isaac GR00T Blueprint for synthetic motion generation helps developers generate exponentially large synthetic motion data to train their humanoids using imitation learning.
Imitation learning — a subset of robot learning — enables humanoids to acquire new skills by observing and mimicking expert human demonstrations. Collecting these extensive, high-quality datasets in the real world is tedious, time-consuming, and often prohibitively expensive. Implementing the Isaac GR00T blueprint for synthetic motion generation allows developers to easily generate exponentially large synthetic datasets from just a small number of human demonstrations.
Starting with the GR00T-Teleop workflow, users can tap into the Apple Vision Pro to capture human actions in a digital twin. These human actions are mimicked by a robot in simulation and recorded for use as ground truth.
The GR00T-Mimic workflow then multiplies the captured human demonstration into a larger synthetic motion dataset. Finally, the GR00T-Gen workflow, built on the NVIDIA Omniverse and NVIDIA Cosmos platforms, exponentially expands this dataset through domain randomization and 3D upscaling.
World Foundation Models Narrow the Sim-to-Real Gap
NVIDIA also announced Cosmos at CES, a platform featuring a family of open, pretrained world foundation models purpose-built for generating physics-aware videos and world states for physical AI development. It includes autoregressive and diffusion models in a variety of sizes and input data formats. The models were trained on 18 quadrillion tokens, including 2 million hours of autonomous driving, robotics, drone footage, and synthetic data.
In addition to helping generate large datasets, Cosmos can reduce the simulation-to-real gap by upscaling images from 3D to real. Combining Omniverse — a developer platform of application programming interfaces and microservices for building 3D applications and services — with Cosmos is critical, because it helps minimize potential hallucinations commonly associated with world models by providing crucial safeguards through its highly controllable, physically accurate simulations.
An Expanding Ecosystem
Collectively, NVIDIA Isaac GR00T, Omniverse, and Cosmos are helping physical AI and humanoid innovation take a giant leap forward. Major robotics companies have started adopting and demonstrated results with Isaac GR00T, including Boston Dynamics and Figure.
Humanoid software, hardware, and robot manufacturers can apply for early access to NVIDIA’s humanoid robot developer program.
Conclusion
The NVIDIA Isaac GR00T Blueprint for synthetic motion generation, along with the NVIDIA Omniverse and Cosmos platforms, is poised to revolutionize the development of next-generation humanoid robots. By generating large synthetic datasets and reducing the simulation-to-real gap, developers can accelerate their efforts to create more advanced and capable humanoids.
FAQs
Q: What is the expected market size for humanoid robots?
A: The market for humanoid robots is expected to reach $38 billion over the next two decades.
Q: What is imitation learning?
A: Imitation learning is a subset of robot learning that enables humanoids to acquire new skills by observing and mimicking expert human demonstrations.
Q: What is the GR00T-Teleop workflow?
A: The GR00T-Teleop workflow captures human actions in a digital twin using the Apple Vision Pro and records them for use as ground truth.
Q: What is Cosmos?
A: Cosmos is a platform featuring a family of open, pretrained world foundation models purpose-built for generating physics-aware videos and world states for physical AI development.
