New work with @nvidia: evaluating robot policies entirely inside a world model. The policy acts, the model imagines the consequences, and the imagined evals predict real-world results. 🧵 real vs world-model rollout side by side📷

New collaboration between NVIDIA and Physical Intelligence: We propose SC3-Eval that evaluates robot policies entirely inside a world model post-trained from Cosmos3-Nano. The policy acts, the model imagines the consequences, and the imagined evals predict real-world results. 🧵

🎉 Meet vLLM-Omni v0.22.0, a major upgrade for omnimodal world models and production-grade multimodal serving. 🌍 Day-0 @NVIDIAAI Cosmos 3 world models: text, image, audio, video, and action, in and out. 🤖 Robot serving: DreamZero OpenPI realtime API. 🎙️ Production TTS: Qwen3-TTS, Qwen3-Omni, VoxCPM2 and more. 🎨 Faster image/video/diffusion: Wan 2.2, HunyuanVideo 1.5, LTX-2.3. ⚡ Broader quantization (FP8/INT8, MXFP4/MXFP8, W4A16, ModelOpt) and hardware coverage. 339 commits, 124 contributors, 52 of them new. Thank you all. 🙌 🔗 github.com/vllm-project/vllm…

Bill Freeman gives us first a list of warm-up bitter lessons. He keeps the bigger ones for later in the talk. #cvpr2026

Presenting DreamControl at ICRA 2026 today! 💫 @DvijKalaria, @pushkalkatara and Sangkyung Kwak will present our workflow for building whole-body humanoid AI skills — combining diffusion models and RL to train skills without expensive real-world data collection. 17:35 – 17:45 | Session WeBT3 | Lehar 1-4 More about DreamControl: genrobo.ai/CXtp1 #ICRA2026 #DreamControl #Humanoid #PhysicalAI

Today we're shipping Nemotron 3 Ultra. A 550B MoE frontier-intelligence open model built for long-running agents. It delivers 5x faster inference and lowers the cost of complex agentic tasks by up to 30% versus other open frontier models.

Built a traffic control system with Hermes Agent using NVIDIA Cosmos 3 and blockchain using Alpenglow Solana protocol. Hermes Agent monitors fleet state and sends control notifications. loom.com/share/21c9b69d83a14… How it works: Cars stop at an intersection, must get distributed consensus on Solana protocol, then must receive a token, ordered by consensus resolution time, not arrival order before crossing. It simulates a 3×3 city grid. Cosmos 3 video language model (VLM) generates maneuver intents per car or what to do (straight, turn, merge) and how urgently (physics deadline), from text scene descriptions like "dashcam view of a 4-way urban intersection at night, heavy rain…" and returns structured JSON. Alpenglow consensus protocol runs parameterized gossip and voting rounds among all cars approaching an intersection to agree on crossing order. Each intent becomes a transaction that must finalize before the car can proceed. Every car stops at stop sign lines, waits for distributed agreement to finalize, then gets a token in FIFO order by resolution time. The coordination pipeline: Cosmos intent, Alpenglow protocol consensus, get intersection token. A car can't cross until both the intent is generated and consensus has settled. If consensus latency exceeds the physics deadline, the intent "misses" and the car must resolve a safety override.

Playing with Nvidia Cosmos3 Super models for image and video generation. Here's obligatory Will Smith eating spaghetti. First few renders were pretty boring, so I went all out on an "energetic stuffing face with spaghetti prompt" here. That's some bottomless spaghetti.

At 4:20pm today, I will give a talk on Cosmos3 in wangywust.github.io/cvpr-tut… If you are at CVPR and interested about Cosmos3, please come.

No data, no problem introducing agentic synthetic data generation with Cosmos 3 share a few examples, generate more data, automate model training, automatically deploy the latest version with no downtime in a benchmark run with Corning Incorporated's optical fiber manufacturing engineering team, a model trained on 8 real defect images plus synthetic examples generated by Cosmos reached 0.95 mean average precision and perfect recall on the toughest defect class, beating a baseline trained on real data alone. "The Roboflow Agent powered by NVIDIA allows us to generate the training data we need, fine-tune our models, and strengthen model performance and inspection quality while increasing the speed, scalability, and adoption of next-generation technologies,” - Jeremy Knopf, chief information officer, Corning Optical Communications

.@NVIDIA’s Cosmos 3 launched today… and guess who had early access? Agile Robots SE! They’ve been running it across their full portfolio: Thor single- and dual-arm, FR3 Duo. Focus? Simulation. Using Cosmos 3 as a neural simulator; a learned world model that generates realistic environments for testing and validating robot policies before they ever touch real hardware. The practical upside is significant! Less dependency on real-world trials, faster deployment cycles, and edge cases you can stress-test at scale without stopping a production line. Agile Robots is an anchor customer of the European Industrial AI Cloud (Deutsche Telekom × NVIDIA) and has 20,000 robotic solutions deployed worldwide. A bit more than a research project. On the technical side? Cosmos 3 unifies synthetic world generation, vision-reasoning, and action simulation in one World Foundation Model. Their team is also exploring NVIDIA’s Video Augmentation Skill built on top of Cosmos (automating data curation, generation, and evaluation). The infrastructure for Physical AI in Europe is coming together… finally! ——— Weekly robotics and AI insights. Subscribe free: 22astronauts.com

Introducing NVIDIA Cosmos 3 We released NVIDIA Cosmos 3 last night. And today, seeing it take the top spots across 8 open model leaderboards feels surreal. We spent months working towards this moment. Here’s the breakdown: The Leaderboard Wins World Reasoning 🏆 #1 open model on VANTAGE-Bench for vision AI 🏆 #1 overall on Traffic Anomaly Reasoning (TAR) World Generation 🏆 #1 open model on Artificial Analysis Image-to-Video leaderboard 🏆 #1 open model on Artificial Analysis Text-to-Image leaderboard 🏆 #1 open model on PAI-Bench for physical AI synthetic data generation 🏆 #1 open model on Physics-IQ, which measures accuracy on physical laws 🏆 #1 open model on R-Bench for world generation quality World Action 🏆 #1 on RoboArena for specialized policy 🏆 #1 on RoboLab for action generation But the leaderboards are only part of the story. The real story is why we built Cosmos 3 in the first place. The Problem Training robots and autonomous systems in the real world is painfully hard. Robots need to try the same thing numerous times before they succeed reliably. Self-driving cars need rare edge cases that may never happen naturally. Smart machines need to understand physics, motion, contact, failure, and surprise. And real-world data is slow, expensive, and sometimes dangerous to collect. At some point, the answer cannot just be “collect more data.” You can’t collect your way out of an infinite physical world. You have to generate it. That… was the question behind Cosmos: Can one model understand the physical world deeply enough to reason about it, simulate it, and generate actions inside it? What We Built Cosmos 3 is the first omni-model for physical AI. It can understand and generate across: language · images · video · audio · action sequences It is not just a VLM. Not just a video generator. Not just a robot policy model. It is all of them, in one single model. That matters because physical AI has been fragmented for a long time. Cosmos 3 is our attempt to collapse that fragmentation. Depending on how you configure the inputs and outputs, the same model can act as a vision-language model, a video/world generator, a world simulator, or a world-action model. No separate architecture required. The Architecture Under the hood, Cosmos 3 uses a dual-tower Mixture-of-Transformers architecture. One tower is autoregressive for reasoning. It handles next-token prediction for language and discrete understanding. The other tower is diffusion-based- for generation. It denoises images, video, audio, and action trajectories. Two towers. Dual-stream joint attention. One shared world representation. Each modality gets its own tools: visual encoders, video VAEs, audio VAEs, and action projectors that can map different embodiments into a unified action space. Action is a first-class modality in Cosmos 3. That’s what makes it more than a video model. It doesn’t just predict and generate what the world might look like. It can connect reasoning and world modeling to physically grounded action. Why This Matters One of the most interesting findings from the ablation work is that training action domains together creates positive transfer. That means adding more embodiments does not just add more use cases. It can actually make the model better. This is the heart of why omnimodal training matters. A shared world representation is not just convenient. It can make each individual task stronger. That’s the part that feels like the beginning of something much bigger. The part I’m most excited about is that Cosmos 3 is fully open. Developers get the models, scripts, optimization, inference endpoints, post-training recipes, datasets, and benchmarks. Everything is available under the Linux Foundation’s OpenMDW 1.1 License. You can use Cosmos 3 out of the box. You can use the VLM, world model, or world-action pieces separately. You can post-train it for your own domain, embodiment, or accuracy target. That’s what makes this feel different. Cosmos 3 is not just a model release. It is the foundation for building intelligence for autonomous machines. For me, Cosmos 3 feels like a step toward a world where physical AI development becomes much more scalable and accessible - to a new age of developers and agents. That’s what we built Cosmos 3 for. I cannot wait to see what you build with it. Download Models on Hugging Face huggingface.co/collections/n… Customize Models on GitHub github.com/NVIDIA/cosmos Read the Tech Blog to Learn More developer.nvidia.com/blog/de…

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception Prediction Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

NVIDIA's Cosmos 3 is what we've never seen before ‒ an Omnimodal World Model. It's closing the loop for physical AI. All stack in one system: world and multimodal understanding, future generation, reasoning and action This is the next step in Jensen Huang’s AI progression: perception AI → generative AI → agentic AI → physical AI Here is what you need to know about it ↓

NVIDIA's Cosmos 3 lands at #1 among open weights models in both Text to Image and Image to Video on the Artificial Analysis Leaderboards! Cosmos 3 is a family of omnimodal world models for Physical AI from @nvidia, unifying language, image, video, audio and action in a single Mixture-of-Transformers architecture that pairs an autoregressive reasoner with a diffusion generator. The family comes in four variants: base Nano (16B: 8B reasoner tower 8B generator tower) and Super (64B: 32B reasoner tower 32B generator tower) models, with the Super model also having Text2Image and Image2Video fine-tuned variants, which are the versions listed in the Artificial Analysis Arena Leaderboards. Cosmos3-Super-Text2Image (agentic) runs through an agentic prompt-upsampling harness, and takes the #1 open weights spot in Text to Image, surpassing HiDream-O1-Image-Dev-2604, Alibaba's Qwen Image Max 2512 and Black Forest Labs' FLUX.2 [dev]. Cosmos3-Super-Image2Video takes #1 open weights in Image to Video (No Audio), ahead of Lightricks' LTX-2, and Alibaba's Wan 2.2 A14B. Cosmos 3 generators take structured JSON prompts rather than plain text, so prompt upsampling is needed to reproduce these results. This upsampling can be handled by an external harness or by the model's own reasoner branch, so it can also run self-contained. Cosmos 3 is fully open under the OpenMDW 1.1 license, shipping with weights, code, curated datasets and fine-tuning recipes available on @huggingface. First-party and third-party APIs are expected over the next few weeks, with pricing to follow. See the thread below for example generations and a link to try Cosmos 3 in our arena 🧵

Check out the Cosmos 3 family for yourself on the Artificial Analysis Image leaderboards: Text to Image: artificialanalysis.ai/video/… Image to Video : artificialanalysis.ai/video/…

Look what we’re cooking! Cosmos 3 is a family of unified omnimodal world model (language, image, video, audio, action), topping multiple benchmarks! Proud to have led Cosmos3-Super-Image2Video, now the #1 open I2V model on Artificial Analysis. Hope it empowers the community!

🚀 Excited to partner with @NVIDIAAI on day-0 support for Cosmos 3 on vLLM-Omni! A unified Mixture-of-Transformers fusing an AR reasoner diffusion generator across text, image, video, audio & robot action - all behind a single OpenAI-compatible API, with a ready-to-deploy Docker image! 📖Check out the detailed deployment guide👇 github.com/NVIDIA/cosmos#gen…

Cosmos 3 ties everything together. Previous releases separated world generation, physical understanding, and controlled scene generation. Cosmos 3’s MoT architecture unifies these capabilities by pairing an autoregressive reasoner tower with a diffusion-based generator tower.