NVIDIA's Cosmos 3: real-time interactive world simulation for physical AI

The content received is the landing page of NVIDIA's Cosmos lab for "Interactive World Simulation with Cosmos for Physical AI." It is not a long article: it is a product page with performance figures and brief descriptions. This summary is therefore limited to what appears on it.

The central idea is that a single post-training recipe turns Cosmos 3 into interactive world models that run in streaming mode and in real time. They are "worlds that respond to control." New controls condition the next video chunk. The model generates it through diffusion (denoising) and adds the controls and the now-clean video to a cached context. It then repeats the cycle. This makes it possible to simulate camera and robot actions, generate RGB observations from spatial controls or produce synchronized driving views, always chunk by chunk.

NVIDIA presents three models, each for a type of simulation.

1) Cosmos3-Nano-Sim-Bimanual. Simulates the world from actions. It predicts visual dynamics from camera navigation and robot actions, and uses the generated history to produce the next observation as new controls arrive. According to the page, it reaches 48.1 FPS with 166 ms latency at 480p on a single GB300 GPU. The page shows four generated robot rollouts, with the synchronized 3D input actions. It also claims higher video quality and higher PSNR relative to the reference frame than DreamDojo 2B and 14B on AgiBot-World and AgiBot-GEAR.

2) Cosmos3-Nano-Sim-Transfer. Turns spatial controls into visual worlds. It takes depth, segmentation, edges or blurred video and generates detailed RGB observations. Language controls appearance, while the simulator maintains the physical state. It runs at 15.85 FPS with 240 ms latency at 640 × 640 on a single GB200 GPU. The examples on the page are 20-second rollouts at 30 FPS. With the same depth or segmentation spatial guides, text prompts change materials, colors and environment (for example, different types of kitchen cabinets, countertops and toys). NVIDIA claims stronger spatial control across the four modalities evaluated, with 59.2% higher edge F1 and 26.5% lower depth error than the Cosmos 3 GA baseline.

3) Cosmos3-Nano-Sim-Auto. Generates a world shared across multiple views, with synchronized multi-view RGB for autonomous driving. It conditions each new video chunk on language, spatial controls and available sensor context. According to the page, it delivers 16.7 FPS with 7 RGB views plus LiDAR, at 480p per view, using 16 GB200 GPUs. The site also mentions an interactive demo with five driving scenes, eleven synchronized cameras, WSM overlays and interactive LiDAR.

Each model has its own model card. The page also links to the technical report, the code on GitHub and the product site. The report is cited as NVIDIA et al., 2026.

Limitations of this summary. The page does not detail the architecture, training data, evaluation methodology or the exact conditions of the comparisons. The performance figures and improvements are the vendor's own claims. Assessing them requires consulting the technical report.