Guanzhi Wang (@guanzhi_wang)

Pinned Post

Guanzhi Wang

@guanzhi_wang

Jun 30

Introducing ASPIRE, the first automated /𝚜𝚔𝚒𝚕𝚕 discovery system for robotics. Rather than solving tasks one by one, ASPIRE continuously discovers and accumulates reusable skills. These persistent skills become the building blocks of robot intelligence, enabling multi-task transfer, sim-to-real transfer, and cross-embodiment transfer. 🔗 research.nvidia.com/labs/gea… From @NVIDIA, @UMichCSE, @ECEILLINOIS, @Berkeley_AI, @CMU_Robotics. Check out how it works in 🧵:

0:40

6

35

190

29,940

Guanzhi Wang reposted

Nadun Ranawaka

@NadunRanawakaA

Jul 2

Real robot data is expensive. Real robot evaluations are slow. Excited to share SimFoundry - a system that turns real scenes into sim-ready worlds for training and benchmarking robots at scale - ✅Automated Scene Reconstruction with asset generation ✅Handles clutter, articulated objects, multiple robot embodiments ✅High Correlation Real-to-Sim Evals ✅Zero-shot Sim-to-Real ✅Generates diverse digital cousins Less manual environment authoring, more scalable feedback for robot learning. 🌐research.nvidia.com/labs/gea… 🧵1/9

1:30

3

11

63

8,943

Guanzhi Wang reposted

Yuke Zhu @yukez

Jul 2

Today, we're introducing SimFoundry, our real2sim2real framework at NVIDIA GEAR that automatically turns real-world scenes into simulation-ready worlds from a single image or video. Website: research.nvidia.com/labs/gea… Paper: arxiv.org/abs/2606.28276 This work marks a major step for our team toward leveraging simulations and synthetic data for foundation model training and systematic policy evaluation at scale. Code will be open-sourced soon. Stay tuned!

1:32

12

73

485

46,073

Guanzhi Wang reposted

Jim Fan

@DrJimFan

Jul 1

ENPIRE -> ASPIRE, our 2nd work in the series for Physical AutoResearch. We are building the components for robot self-improvement, one /skill at a time.

Jim Fan

@DrJimFan

Jun 30

Today, we give robots a /skills library that self-evolves and compounds indefinitely! Introducing ASPIRE: a robot solving its 100th task is no longer as clueless as solving its first. Coding agents observe multimodal sensory traces from simulation and real robots, launch an evolutionary search over control programs, and distill the best know-how into an ever-expanding library. ASPIRE is a new type of continual learning: "training" is skill refinement instead of gradient descent. "Trained model" is a repo of sensorimotor skills instead of floating weights. “Distributed training” is a panel of agents each practicing a different skill instead of sharded minibatches. Here's the beauty: ASPIRE gives the tired terms "sim2real transfer" and "cross-embodiment transfer" a whole new meaning. Bridging the sim-to-real gap is notoriously brutal. An end-to-end policy has to swallow both the visual shift (sim looks toyish next to a real camera) and the subtle contact physics it never quite gets right. ASPIRE sidesteps the mess, because it doesn't ship pixels or weights across the gap, but ships the know-how. The robot still has to practice in the real world, not zero-shot, but it gets there way faster because it isn't rediscovering the strategy from scratch. Same for going single-arm to bimanual hardware, which usually requires new data and retraining from zero. ASPIRE achieves up to ~10x cut in "transfer learning” tokens (yes, tokens are the new unit of *training* compute ;) Check out our gallery of 150 tasks and 90 skills the robots taught themselves, all on the website! Kind of wild that we can ship the "learned weights" as an HTML page rather than a GGUF. We'll open-source the full stack so your own robot library starts compounding from ours! Deep dive in thread:

0:40

34

22

275

46,014

Guanzhi Wang reposted

Ruijie Zheng @ruijie_zheng12

Jul 1

Check out the latest GEAR’s work on skills acquisition for robotics. Excited to see the synergy with foundation policies (vla,wams)! More to come

Guanzhi Wang

@guanzhi_wang

Jun 30

Introducing ASPIRE, the first automated /𝚜𝚔𝚒𝚕𝚕 discovery system for robotics. Rather than solving tasks one by one, ASPIRE continuously discovers and accumulates reusable skills. These persistent skills become the building blocks of robot intelligence, enabling multi-task transfer, sim-to-real transfer, and cross-embodiment transfer. 🔗 research.nvidia.com/labs/gea… From @NVIDIA, @UMichCSE, @ECEILLINOIS, @Berkeley_AI, @CMU_Robotics. Check out how it works in 🧵:

0:40

3

14

2,008

Guanzhi Wang reposted

Jim Fan

@DrJimFan

Jun 30

Today, we give robots a /skills library that self-evolves and compounds indefinitely! Introducing ASPIRE: a robot solving its 100th task is no longer as clueless as solving its first. Coding agents observe multimodal sensory traces from simulation and real robots, launch an evolutionary search over control programs, and distill the best know-how into an ever-expanding library. ASPIRE is a new type of continual learning: "training" is skill refinement instead of gradient descent. "Trained model" is a repo of sensorimotor skills instead of floating weights. “Distributed training” is a panel of agents each practicing a different skill instead of sharded minibatches. Here's the beauty: ASPIRE gives the tired terms "sim2real transfer" and "cross-embodiment transfer" a whole new meaning. Bridging the sim-to-real gap is notoriously brutal. An end-to-end policy has to swallow both the visual shift (sim looks toyish next to a real camera) and the subtle contact physics it never quite gets right. ASPIRE sidesteps the mess, because it doesn't ship pixels or weights across the gap, but ships the know-how. The robot still has to practice in the real world, not zero-shot, but it gets there way faster because it isn't rediscovering the strategy from scratch. Same for going single-arm to bimanual hardware, which usually requires new data and retraining from zero. ASPIRE achieves up to ~10x cut in "transfer learning” tokens (yes, tokens are the new unit of *training* compute ;) Check out our gallery of 150 tasks and 90 skills the robots taught themselves, all on the website! Kind of wild that we can ship the "learned weights" as an HTML page rather than a GGUF. We'll open-source the full stack so your own robot library starts compounding from ours! Deep dive in thread:

0:40

69

221

1,624

185,242

Guanzhi Wang

@guanzhi_wang

Jun 30

Please check out the project website for more details. 🔗 research.nvidia.com/labs/gea… The work is co-authored by @runyulu_x (our awesome intern, co-lead, and co-first), @yubowu25 (our awesome intern and co-first), Ethan Kou (our awesome intern and co-first), @letian_fu, @_wenlixiao, @AjayMandlekar, @YinzhenXu, @GuanyaShi, @Ken_Goldberg, Ang Chen, @mosharaf, @yukez (co-lead), @DrJimFan (co-lead), and @guanzhi_wang (co-lead).

1

11

452

Guanzhi Wang

@guanzhi_wang

Jun 30

/𝚎𝚟𝚊𝚕 ASPIRE improves substantially over prior coding agents and end-to-end VLAs across three benchmark families: (a) short-horizon manipulation on LIBERO-Pro, (b) contact-rich manipulation on Robosuite, and (c) long-horizon mobile manipulation on BEHAVIOR-1K.

1

7

475

Guanzhi Wang

@guanzhi_wang

Jun 30

/𝚜𝚒𝚖_𝚝𝚘_𝚛𝚎𝚊𝚕 Can skills learned in sim help real robots? ASPIRE transfers skills discovered in sim (w/ Franka Panda arms) to real robots (w/ YAM arms) as reusable knowledge for program synthesis. These skills consistently reduce real-world debugging effort by up to an order of magnitude in token usage, improve success on challenging tasks, and generalize across robot embodiments and API differences.

0:44

1

5

413

Guanzhi Wang

@guanzhi_wang

Jun 30

/𝚎𝚟𝚘𝚕𝚞𝚝𝚒𝚘𝚗𝚊𝚛𝚢_𝚜𝚎𝚊𝚛𝚌𝚑 ASPIRE uses evolutionary search to broaden exploration beyond single-trajectory repair. At each iteration, the coding agent proposes a population of candidate robot programs conditioned on the best-performing programs so far, residual failure traces, and relevant skills retrieved from the library. Each candidate is executed in the robot execution engine, producing new multimodal execution traces and task outcomes. Rather than repeatedly patching one failed trajectory, ASPIRE evolves a population of promising programs over successive search rounds.

0:20

1

6

441

Guanzhi Wang

@guanzhi_wang

Jun 30

/𝚜𝚔𝚒𝚕𝚕_𝚕𝚒𝚋𝚛𝚊𝚛𝚢 ASPIRE accumulates reusable robot skills instead of discarding experience after each task. The skill library stores diverse knowledge: localization heuristics, perception prompts, grasping constraints, navigation recovery strategies, motion primitives, scene-understanding routines, and debugging workflows. We never hand-design this taxonomy. As ASPIRE encounters more tasks, its skill library grows, enabling increasingly strong zero-shot transfer to long-horizon novel tasks. ASPIRE outperforms prior agents by up to 6.1× in policy rollout success.

0:13

1

5

468

Guanzhi Wang

@guanzhi_wang

Jun 30

/𝚍𝚎𝚋𝚞𝚐 ASPIRE turns robot programming into a debugging problem. Each ASPIRE-written program utilizes low-level primitives to interface with the environment. The robot execution engine records per-primitive multimodal traces for perception, planning, and control calls, exposes the traces to the coding agent, and executes agent-written repairs for validation. Instead of only observing task-level failures, the agent can pinpoint why a failure occurred and propose a targeted fix.

0:36

1

6

737

Guanzhi Wang

@guanzhi_wang

Jun 30

/𝚊𝚜𝚙𝚒𝚛𝚎 ASPIRE operates in an open-ended learning loop, consisting of three key components: 🤖 A robot execution engine that provides rich multimodal traces for interactive program debugging. 📚 A continually expanding skill library that distills validated patterns into transferable knowledge. 🧬 An evolutionary search procedure that explores diverse hypotheses and task-solving strategies.

1

13

1,080

Guanzhi Wang reposted

Jim Fan

@DrJimFan

Jun 16

Today, we enable AutoResearch in the physical world for the first time! Introducing ENPIRE: we give 8 Codex agents a fleet of robots, an allocation of GPUs, and generous token budget. We set them free with a simple goal: solve the task as quickly as possible, keep the robots busy but stay safe, don't waste precious compute. Make no mistake. Then humans step aside and our watch begins. The robot fleet starts to come alive: they learn to look for visual clues, reset the scene, practice novel skills, tinker with control stack, read papers online, debate, reflect, get stuck, and try again directly on the hardware. All we did is to give Codex an API to the world of atoms, and the rest is emergence. ENPIRE is able to solve high-precision tasks like tying zip-ties, organizing fine pins, and installing GPUs all by itself. We also discovered a new type of "physical scaling": 8 robots exploring in parallel improves significantly faster than fewer ones. A part of our NVIDIA GEAR lab now self-improves tirelessly over night. We just read the reports in the morning. /goal: we all take a holiday and Jensen wouldn't even notice ;) We will be open-sourcing everything, so you can host your self-running robot lab at home too! Deep dive in the thread:

1:16

187

570

3,845

653,052

Guanzhi Wang reposted

Wenli Xiao

@_wenlixiao

Jun 16

Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce 𝐄𝐍𝐏𝐈𝐑𝐄: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment → search the literature → implement ideas and build the infra → train and deploy → self-verify → analyze the logs and rewrite the code → repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting — from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. 🔗 research.nvidia.com/labs/gea… From @NVIDIA @CMU_Robotics @Berkeley_AI 🧵

1:16

18

96

470

258,911

Guanzhi Wang reposted

Siyi Chen

@ChenSiyich

Jun 10

Wonderful to be back from #CVPR2026, and excited to share the release of our follow-up work: VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation VoLo introduces the idea of a physical orchestrator for open-vocabulary, long-horizon manipulation. Our goal is to move toward robots that can reason, plan, act, monitor, and recover by adaptively using VLA/WAMs, vision models, and action primitives as tools. We introduce three main contributions: 🤖 VoLoAgent — a physical orchestrator that plans, monitors, and recovers by adaptively using, halting, and redirecting robot actions with tools. 📊 RoboVoLo — a high-fidelity benchmark with 126 open-vocabulary long-horizon manipulation tasks spanning common sense, memory/state tracking, complex references, and world knowledge. 📈 A large-scale empirical study comparing action models, code-as-policy systems, TAMP-style systems, and ablations of the VoLoAgent orchestrator, complemented by real-robot experiments. This work was done during my internship at @NVIDIA and would not have been possible without my brilliant collaborators: Hugo Hadfield, Alexander Zook, @mikacuy, @luke_ch_song, @erwincoumans, @xuningy, Faisal Ladhak, @qu_1006, @BirchfieldStan, Jonathan Tremblay, and @robovalts. Huge thanks to everyone! 🔗 Project: chicychen.github.io/VoLo/ 🔗 Previous work, SpaceTools: spacetools.github.io/ #Robotics #EmbodiedAI #VisionLanguageModels #VLAModels #RobotLearning #NVIDIA #CVPR2026 #LongHorizonManipulation #AI #ComputerVision

3

16

75

8,998

Guanzhi Wang reposted

Jan Kautz @jankautz

Jun 5

Excited that NitroGen won a Best Paper Honorable Mention at #CVPR. @NVIDIAAI

#CVPR2026 @CVPR

Jun 5

Honorable Mentions👏

2

7

45

5,837

Guanzhi Wang reposted

Jim Fan

@DrJimFan

Jun 5

NitroGen just won CVPR Best Paper Honorable Mention!! We are making strides towards general-purpose embodied agents that master not only the real world physics, but also all possible physics across a multiverse of simulations. It’s been 4 years since MineDojo, our first embodied agent in Minecraft, won NeurIPS Best Paper. Congrats to everyone on the team!!

59

48

398

42,890

Guanzhi Wang reposted

Kevin Lin @linkevin0

May 28

Humanoids need data. Lots and lots of data. Introducing HumanoidMimicGen: a method that automatically generates 1000s of humanoid loco-manipulation demonstrations from a single teleoperated demonstration.

1:07

12

45

258

28,840

Guanzhi Wang reposted

tingwu.wang

@TingwuWang

Apr 29

What is missing to bring real-time motion research into AAA games and real-world robotics? We present MotionBricks, a step toward bridging this gap with two key components: - a single generative latent motion backbone covering 350,000 motion skills, running at 15,000 FPS with 2 ms latency and substantially improved quality and reliability. - a unified smart primitive interface for locomotion, object / scene interaction, with fine-grained control over generated behaviors. Webpage: nvlabs.github.io/motionbrick… Code: github.com/NVlabs/GR00T-Whol… Paper: arxiv.org/abs/2604.24833 (ACM TOG / SIGGRAPH 2026)

6:25

26

149

1,197

152,875