Wonderful to be back from #CVPR2026, and excited to share the release of our follow-up work: VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation VoLo introduces the idea of a physical orchestrator for open-vocabulary, long-horizon manipulation. Our goal is to move toward robots that can reason, plan, act, monitor, and recover by adaptively using VLA/WAMs, vision models, and action primitives as tools. We introduce three main contributions: 🤖 VoLoAgent — a physical orchestrator that plans, monitors, and recovers by adaptively using, halting, and redirecting robot actions with tools. 📊 RoboVoLo — a high-fidelity benchmark with 126 open-vocabulary long-horizon manipulation tasks spanning common sense, memory/state tracking, complex references, and world knowledge. 📈 A large-scale empirical study comparing action models, code-as-policy systems, TAMP-style systems, and ablations of the VoLoAgent orchestrator, complemented by real-robot experiments. This work was done during my internship at @NVIDIA and would not have been possible without my brilliant collaborators: Hugo Hadfield, Alexander Zook, @mikacuy, @luke_ch_song, @erwincoumans, @xuningy, Faisal Ladhak, @qu_1006, @BirchfieldStan, Jonathan Tremblay, and @robovalts. Huge thanks to everyone! 🔗 Project: chicychen.github.io/VoLo/ 🔗 Previous work, SpaceTools: spacetools.github.io/ #Robotics #EmbodiedAI #VisionLanguageModels #VLAModels #RobotLearning #NVIDIA #CVPR2026 #LongHorizonManipulation #AI #ComputerVision

NVIDIA Research Advances Robotics From Simulation to the Real World blogs.nvidia.com/blog/icra-r…

When every generalist robot model scores 95% on a benchmark, the numbers become meaningless. What if we built a photorealistic benchmark that never saturates and can generate new scenes and tasks with AI Workflows in minutes? We introduce RoboLab! 🧵(1/6)

Excited to share that CARI4D has been accepted to #CVPR2026 @CVPR And our code is also released at: github.com/NVlabs/CARI4D Welcome to play with it and share your feedback🤗!

Introducing CARI4D: category agnostic 4D reconstruction of human object interaction. We reconstruct consistent 4D human-object from just an RGB video. Website: nvlabs.github.io/CARI4D Paper: arxiv.org/abs/2512.11988 Code: github.com/NVlabs/CARI4D

A new milestone for real-time accurate 3D spatial computing! Introducing ⚡️Fast-FoundationStereo⚡️, a real-time zero-shot stereo depth estimation model that accelerates the original FoundationStereo by >10x with comparable quality. Details in threads 🧵 (1/N)

[1/N] 🎥 We've made available a powerful spatial AI tool named ViPE: Video Pose Engine, to recover camera motion, intrinsics, and dense metric depth from casual videos! Running at 3–5 FPS, ViPE handles cinematic shots, dashcams, and even 360° panoramas. 🔗 research.nvidia.com/labs/tor…

Come and say 👋 tomorrow (06/13) for our oral (1pm, Karl Dean Ballroom) and poster sessions (4pm, ExHall D, #81)! #CVPR2025 @CVPR @CVPRConf @NVIDIAAIDev @NVIDIARobotics #NVIDIA

🚗 NVIDIA Research named Autonomous Grand Challenge winner for End-to-End Autonomous Driving at #CVPR2025 in Nashville. Theme of challenge: Towards Generalizable Embodied Systems. Learn more about NVIDIA Research at CVPR: nvda.ws/45MH4NB

Imagine if robots could fill in the blanks in cluttered scenes. ✨ Enter RaySt3R: a single masked RGB-D image in, complete 3D out. It infers depth, object masks, and confidence for novel views, and merges the predictions into a single point cloud. rayst3r.github.io

📢Time to upgrade your depth camera! Introducing **FoundationStereo**, a foundation model for stereo depth estimation in zero-shot (accepted to CVPR 2025 with full scores) [1/n] Code: github.com/NVlabs/Foundation… Website: nvlabs.github.io/FoundationS… Paper: arxiv.org/abs/2501.09898

Could #AI revolutionize stereo depth estimation? Utilizing a massive dataset of stereo pairs, FoundationStereo is designed as a foundation model with strong zero-shot generalization and automatic self-curation pipeline. Details from #NVIDIAResearch ➡️ nvlabs.github.io/FoundationS…

How can robots learn household tasks from videos using just an iPhone, no robot hardware? Introducing SPOT, an object-centric framework that learns from minimal human demos, capturing the task-related constraints. (1/n)

The new BOP challenge 2024 pushes model-free and generalizable 6D object pose estimation. 🏆 Learn about the current state of 6D object pose estimation and the next challenges by visiting our poster at the mixed reality workshop (10:30-11:15) #CV4MR at #CVPR2024 📷

📢 Benchmark for 6D Object Pose Estimation 📢 BOP challenge 24 has been opened! bop.felk.cvut.cz/challenges/… Results to be presented at the R6D workshop at #ECCV2024 Details in comments below👇

Dense tracking of deformable objects can unlock applications in robotics, gen-AI and AR. We present DeformGS (previously MD-Splatting) and release the code and data.  Join us at #WAFR where we will present new real-world results! 👇deformgs.github.io 1/9🧵

HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions paper page: huggingface.co/papers/2308.0… present the HANDAL dataset for category-level object pose estimation and affordance prediction. Unlike previous datasets, ours is focused on robotics-ready manipulable objects that are of the proper size and shape for functional grasping by robot manipulators, such as pliers, utensils, and screwdrivers. Our annotation process is streamlined, requiring only a single off-the-shelf camera and semi-automated processing, allowing us to produce high-quality 3D annotations without crowd-sourcing. The dataset consists of 308k annotated image frames from 2.2k videos of 212 real-world objects in 17 categories. We focus on hardware and kitchen tool objects to facilitate research in practical scenarios in which a robot manipulator needs to interact with the environment beyond simple pushing or indiscriminate grasping. We outline the usefulness of our dataset for 6-DoF category-level pose scale estimation and related tasks. We also provide 3D reconstructed meshes of all objects, and we outline some of the bottlenecks to be addressed for democratizing the collection of datasets like this one.

📣 Introducing FoundationPose, a unified foundation model for 6D object pose estimation and tracking for novel objects from #NVIDIAResearch. See the details below. 🧵 👇

.@nvidia has released the source code for their 2024 CVPR paper, "FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects." We're really excited to see what the #ROS community can make happen with this code. nvlabs.github.io/FoundationP…