The next frontier of AI is physical AI. Physical AI models can understand instructions and perceive, interact, and perform complex actions in the real world to power autonomous machines like robots and self-driving cars. Similar to how large language models can process and generate text, physical AI models can understand the world and generate actions.
Global leaders in software development and professional services are using NVIDIA Omniverse, powered by OpenUSD, to build new products and services that will accelerate the development of AI and controllable simulations to enable the creation of true-to-reality virtual worlds, known as digital twins, that can be used to train physical AI with unprecedented accuracy and detail.
Generate Exponentially More Synthetic Data With Omniverse and NVIDIA Cosmos
At CES, NVIDIA announced generative AI models and blueprints that expand Omniverse integration further into physical AI applications such as robotics, autonomous vehicles, and vision AI. Among these announcements was NVIDIA Cosmos, a platform of state-of-the-art generative world foundation models, advanced tokenizers, guardrails, and an accelerated video processing pipeline – all designed to accelerate physical AI development.
Developing physical AI models is a costly, resource- and time-intensive process that requires vast amounts of real-world data and testing. Cosmos’ world foundation models (WFM), which predict future world states as videos based on multimodal inputs, provide an easy way for developers to generate massive amounts of photoreal, physics-based synthetic data to train and evaluate AI for robotics, autonomous vehicles, and machines. Developers can also fine-tune Cosmos WFMs to build downstream world models or improve quality and efficiency for specific physical AI use cases.
See Cosmos in Action for Physical AI Use Cases
Cosmos WFMs are revolutionizing industries by providing a unified framework for developing, training, and deploying large-scale AI models across various applications. Enterprises in the automotive, industrial, and robotics sectors can harness the power of generative physical AI and simulation to accelerate innovation and operational efficiency.
- Humanoid robots: The NVIDIA Isaac GR00T Blueprint for synthetic motion generation helps developers generate massive synthetic motion datasets to train humanoid robots using imitation learning. With GR00T workflows, users can capture human actions and use Cosmos to exponentially increase the size and variety of the dataset, making it more robust for training physical AI systems.
- Autonomous vehicles: Autonomous vehicle (AV) simulation powered by Omniverse Sensor RTX application programming interfaces lets AV developers replay driving data, generate new ground-truth data, and perform closed-loop testing to accelerate their pipelines. With Cosmos, developers can generate synthetic driving scenarios to amplify training data by orders of magnitude, accelerating physical AI model development for autonomous vehicles.
- Industrial settings: Mega is an Omniverse Blueprint for developing, testing, and optimizing physical AI and robot fleets at scale in a USD-based digital twin before deployment in factories and warehouses. The blueprint uses Omniverse Cloud Sensor RTX APIs to simultaneously render multisensor data from any type of intelligent machine, enabling high-fidelity sensor simulation at scale. Cosmos can enhance Mega by generating synthetic edge case scenarios to amplify training data, significantly improving the robustness and efficiency of training robots in simulation.
Get Plugged Into the World of OpenUSD
For more on Cosmos, watch the replay of NVIDIA CEO Jensen Huang’s CES keynote, and get started with Cosmos WFMs available now under an open model license on Hugging Face and the NVIDIA NGC catalog. Join the upcoming livestream on Wednesday, February 5 for a deep dive into Cosmos WFMs and physical AI workflows.
Conclusion
The future of AI is physical AI, and NVIDIA Omniverse and OpenUSD are leading the charge. With the power of generative world foundation models, developers can generate massive amounts of synthetic data to train and evaluate AI for robotics, autonomous vehicles, and machines. The possibilities are endless, and the future is bright.
FAQs
Q: What is physical AI?
A: Physical AI models can understand instructions and perceive, interact, and perform complex actions in the real world to power autonomous machines like robots and self-driving cars.
Q: What is OpenUSD?
A: OpenUSD is a universal scene description language that enables the creation of true-to-reality virtual worlds, known as digital twins, that can be used to train physical AI with unprecedented accuracy and detail.
Q: What is NVIDIA Cosmos?
A: NVIDIA Cosmos is a platform of state-of-the-art generative world foundation models, advanced tokenizers, guardrails, and an accelerated video processing pipeline – all designed to accelerate physical AI development.
Q: How can I get started with Cosmos WFMs?
A: You can get started with Cosmos WFMs available now under an open model license on Hugging Face and the NVIDIA NGC catalog.
