Nvidia has released a spate of new physical AI research tools, agent workflows and open source models to train more advanced AI systems for the real world.
Unveiled this week at the Computer Vision and Pattern Recognition conference in Denver, the updates build on Nvidia’s recently launched Cosmos 3 world foundation model and are designed to help researchers automate key stages of physical AI development, including simulation, synthetic data generation, policy training and evaluation.
Physical AI refers to AI systems that interact with and operate in the physical world, including self-driving vehicles, industrial robots and embodied AI agents.
The company said the new capabilities address a major challenge facing engineers in the industry: creating scalable workflows to train and test AI virtually before real-world deployment.
“The core challenge in physical AI research isn’t simply developing stronger models. It’s building a full workflow around them,” Nvidia said in a blog post. “Today, these steps are fragmented across separate tools, slowing the pace of experimentation as researchers struggle to piece them together.”
Agent Skills
Among the announcements are new agent skills integrated across Nvidia Omniverse, Isaac Sim, Isaac Lab and Cosmos, enabling developers to automate tasks such as scene reconstruction, simulation setup, environment generation and reinforcement learning workflows.
For autonomous vehicle development, Nvidia introduced tools to help researchers address the industry’s “long-tail problem” –difficult-to-capture driving scenarios that are critical for training and validation.
To bridge this gap, Nvidia said its AI agents can now automate the reconstruction of real-world driving environments from fleet data and generate synthetic edge-case scenarios for testing.
The AI giant also introduced Alpamayo 2 Super, a 32-billion-parameter vision-language-action model for autonomous driving. The system is designed with advanced reasoning capabilities, enabling it to autonomously act across the full driving stack.
In the vision AI arena, Nvidia expanded its video analysis capabilities with updates to its Metropolis platform, including tools for video search, summarization and synthetic data generation.
The company said these capabilities will help developers build AI agents capable of understanding complex scenes, identifying events and generating alerts from video streams.
Robotics was another major focus, with new agent skills designed to automate simulation and training workflows. This reduces the manual labor typically required to create virtual environments and train robots within them.
The releases highlight Nvidia’s increased focus on physical AI as a key growth area. With the updates, the company is positioning virtual environments as a critical tool for developing AI systems that can safely operate in the physical world.
Nvidia’s new physical AI suite is now available through GitHub, while its synthetic data generation tools (Neural Reconstruction, Video Augmentation and Defect Image Generation) are available on Nvidia Brev with free trial credits for researchers.
