@physical_int | PhD Student at @CMU_Robotics | Ex GEAR Lab @NvidiaAI | Physical AI & Robotics

Joined August 2019
39 Photos and videos
Pinned Post
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
18
96
470
258,911
Wenli Xiao reposted
ENPIRE -> ASPIRE, our 2nd work in the series for Physical AutoResearch. We are building the components for robot self-improvement, one /skill at a time.
Today, we give robots a /skills library that self-evolves and compounds indefinitely! Introducing ASPIRE: a robot solving its 100th task is no longer as clueless as solving its first. Coding agents observe multimodal sensory traces from simulation and real robots, launch an evolutionary search over control programs, and distill the best know-how into an ever-expanding library. ASPIRE is a new type of continual learning: "training" is skill refinement instead of gradient descent. "Trained model" is a repo of sensorimotor skills instead of floating weights. โ€œDistributed trainingโ€ is a panel of agents each practicing a different skill instead of sharded minibatches. Here's the beauty: ASPIRE gives the tired terms "sim2real transfer" and "cross-embodiment transfer" a whole new meaning. Bridging the sim-to-real gap is notoriously brutal. An end-to-end policy has to swallow both the visual shift (sim looks toyish next to a real camera) and the subtle contact physics it never quite gets right. ASPIRE sidesteps the mess, because it doesn't ship pixels or weights across the gap, but ships the know-how. The robot still has to practice in the real world, not zero-shot, but it gets there way faster because it isn't rediscovering the strategy from scratch. Same for going single-arm to bimanual hardware, which usually requires new data and retraining from zero. ASPIRE achieves up to ~10x cut in "transfer learningโ€ tokens (yes, tokens are the new unit of *training* compute ;) Check out our gallery of 150 tasks and 90 skills the robots taught themselves, all on the website! Kind of wild that we can ship the "learned weights" as an HTML page rather than a GGUF. We'll open-source the full stack so your own robot library starts compounding from ours! Deep dive in thread:
34
22
275
46,014
Wenli Xiao reposted
Introducing ASPIRE, the first automated /๐šœ๐š”๐š’๐š•๐š• discovery system for robotics. Rather than solving tasks one by one, ASPIRE continuously discovers and accumulates reusable skills. These persistent skills become the building blocks of robot intelligence, enabling multi-task transfer, sim-to-real transfer, and cross-embodiment transfer. ๐Ÿ”—ย research.nvidia.com/labs/geaโ€ฆ From @NVIDIA, @UMichCSE, @ECEILLINOIS, @Berkeley_AI, @CMU_Robotics. Check out how it works in ๐Ÿงต:
6
35
190
29,939
Open-sourced on Maker World makerworld.com/en/models/298โ€ฆ
Are the gripper jaws used for ENPIRE open-source?
9
32
289
33,594
The most inspiring thing I took from this paper: there's far more to squeeze from simulation than sim-to-real training of task-specific policies. RATs shows a coding agent can self-propose tasks, self-construct scenes in sim, and acquire skills that transfer to real-world deployment. It's promising to imagine handing coding agents a bunch of simulation clusters on top of ENPIRE to enable Sim-and-Real Co-research, where agents massively learn skills and try ideas in sim while continuously grounding them in the real world. Then robot skill acquisition can really scaling like everything else in the deep learning era.
Children learn from play. Can robots do the same? We propose ๐๐ฅ๐š๐ฒ๐Ÿ๐ฎ๐ฅ ๐€๐ ๐ž๐ง๐ญ๐ข๐œ ๐‘๐จ๐›๐จ๐ญ ๐‹๐ž๐š๐ซ๐ง๐ข๐ง๐ , a paradigm that gives embodied coding agents a play stage before downstream tasks arrive, and instantiate it with ๐‘๐€๐“๐ฌ (Robotics Agent Teams), where robots discover reusable skills through curious play. Co-led with @jiaxin_ge_
5
9
94
11,237
Wenli Xiao reposted
I made Physical AutoResearch sound simple (conceptually), but it took a village to pull off and lots of design thinking into the robot /loopcraft. The hardest part is everything we need to setup *before* pressing Enter. Here's a behind-the-scene tour: 1. Safety harness Letting 8 robots run unattended overnight means safety has to be more than a hint in the system prompt. ENPIRE hardwires it in 2 layers: (1) hard kinematic limit that trips an immediate task failure and auto-resets as soon as a robot leaves its safety envelope, and (2) a torque-limited compliant gripper so a bad contact or misaligned insertion ends in a safe stall, instead of crushing the robot or the object at hand. We make safety more conservative than usual so humans can sleep tight. In reality, we still need a few human operators to watch over the "robots of loving grace". 2. Definition of /done An agent that can edit its own reward will game it for sure. ENPIRE fixes the goalposts before the fleet can move them. Here's the recipe: Collect a few minutes of success & failure demos -> Ask agent to write code using computer vision tools to classify success and measure against groundtruth -> Agent hill-climbs on classifier until reliably good -> This classifier becomes the real-time reward function that directly computes on sensor streams -> *Freeze* the reward function before AutoResearch. It's sacred, enshrined in a Gym env that no one can touch. 3. System telemetry design Robot-seconds is by far the scarcest resource, followed by GPU-seconds, and finally tokens. We instrument all three and surface them to ENPIRE for live resource awareness rather than letting it hill-climb in a vacuum. We define: - Mean Robot Utilization ("MRU"): the fraction of wall-clock time when the robot is actively executing an experiment. Otherwise the hardware is sitting idle and waiting for the next code commit. - Mean Token Utilization ("MTU"): tokens consumed per minute, our proxy for how hard the agent is actually thinking. A low MTU means the agent is stalled, waiting on a robot rollout to finish instead of doing research. - GPU utilization: fraction of wall-clock time when GPU is active. ... and evaluate on two budget-to-outcome metrics: 1. Tokens-to-Success: token budget the fleet burns to complete /goal. 2. Time-to-Success: wall-clock time to /goal
Today, we enable AutoResearch in the physical world for the first time! Introducing ENPIRE: we give 8 Codex agents a fleet of robots, an allocation of GPUs, and generous token budget. We set them free with a simple goal: solve the task as quickly as possible, keep the robots busy but stay safe, don't waste precious compute. Make no mistake. Then humans step aside and our watch begins. The robot fleet starts to come alive: they learn to look for visual clues, reset the scene, practice novel skills, tinker with control stack, read papers online, debate, reflect, get stuck, and try again directly on the hardware. All we did is to give Codex an API to the world of atoms, and the rest is emergence. ENPIRE is able to solve high-precision tasks like tying zip-ties, organizing fine pins, and installing GPUs all by itself. We also discovered a new type of "physical scaling": 8 robots exploring in parallel improves significantly faster than fewer ones. A part of our NVIDIA GEAR lab now self-improves tirelessly over night. We just read the reports in the morning. /goal: we all take a holiday and Jensen wouldn't even notice ;) We will be open-sourcing everything, so you can host your self-running robot lab at home too! Deep dive in the thread:
46
87
762
101,191
Amazing effort! Open-sourcing 3,500 hours of data will surely enable a lot of meaningful research in the field. Congrats to the team!
Introducing ABC: open data, training, and infrastructure for robotics. We release the largest teleop dataset to date, and extensively investigate design decisions, pretraining, and post-training techniques. @arthurallshire @Cinnabar233 @adamrasb @redstone_hong @davidrmcall
2
5
88
14,893
โ€ผ๏ธBig misunderstanding here. The pass@8 isn't best-of-8 independent samples on the whole task โ€” it's within a single long-horizon rollout. On each RL subtask the agentic loop gets up to 8 in-context retries before moving on, and every retry is conditioned on the previous failures. So it's measuring emergent retry/recovery, not i.i.d. sampling luck โ€” which is exactly why we put the uncut 5-min video on the site instead of a cherry-picked clip. That's why the independent-sampling intuition doesn't transfer: retries only help to the extent the policy can actually recover. If your 13% policy can't recover, retrying won't save it: pass@8 stays ~pass@1, nowhere near 99%. And if it could recover well enough to hit 99% over 8 in-context retries, then that recovery IS the capability we're reporting.
99% w/ pass@8 is pretty heinous marketing. i expect my 13% success rate policy to also hit 99% pass@8 too ๐Ÿ˜‚
3
2
35
7,699
Wenli Xiao reposted
We believe AGI will play a key role in building physical AGI. Today, we introduce ENPIRE: fleet agentic autoresearch meets the physical world. Across precise manipulation tasks, teams of coding agents can autonomously hill-climb performance using heuristic learning, behavior cloning, and reinforcement learning. But ENPIRE is about more than automating physical algorithm search. The coding agents drive the entire research loop: reviewing the literature, proposing algorithms, building reset and verification mechanisms, designing rewards, improving training infrastructure, running experiments, and learning from real-world outcomes. The results suggest that strong multimodal reasoning and coding capabilities are key ingredients for closing the self-improvement loop in the real world. The age of experience is already here. By learning from online interaction, we hope multimodal models can learn to probe, experiment, and ultimately build in the physical world.
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
1
8
23
2,858
Wenli Xiao reposted
Going to try this and hoping it could find MolmoAct2 Bimanual YAM checkpoint and solve the tasks in one iteration cycle. Really cool work!
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
1
5
30
4,078
Wenli Xiao reposted
Buzzwords in 2026 AI: recursive self-improvement, autoresearch, agents that get better by iterating. Underneath them is a simple pattern: once a domain has a repeatable feedback loop, agents can propose, test, observe, and revise. This is why games, ML experiments, GPU kernels, codebases, and proof search are becoming increasingly agent-solvable. But robotics is where this story breaks. In the digital world, โ€œrun the experimentโ€ is a command. In the physical world, a rollout requires resetting the scene, executing safely, verifying the outcome, interpreting logs/videos/rewards, and refining the next policy. Our bet with ENPIRE: the missing abstraction for automated robotics research is a physical feedback loop: reset โ†’ execute โ†’ verify โ†’ refine A key observation makes this feasible: many robotics tasks have a generationโ€“verification gap. Completing the task is hard, but resetting the setup and checking success are often easier โ€” and can be built by composing existing tools like motion planners, OpenCV, segmentation models, robot APIs, and safety constraints. ENPIRE turns this into a harness with four modules: EN: Environment construction for auto-reset verification PI: Policy Improvement R: Rollout on real robot hardware E: Evolution across hypotheses / robot workers Once this loop is closed, coding agents can read failures, edit training code, try BC/RL/heuristics/code-as-policy variants, keep what works, and discard what fails โ€” directly on real robots. Maybe the next step after pretraining, RLHF, and large-scale RL/RLVR is not just a new algorithm, but a broader recipe: close the feedback loop wherever the world lets us. Digital loops came first. ENPIRE is a step toward physical loops. Check out @_wenlixiaoโ€™s thread๐Ÿ‘‡
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
4
6
109
51,067
Wenli Xiao reposted
Autoresearch via code-as-policies. LLM agent writes code and tests it on real robots. Robots are just software in the real world after all.
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
2
3
41
5,793
Wenli Xiao reposted
Auto robot policy improvement with codex!
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
5
8
97
13,684
Wenli Xiao reposted
ENPIREใ‚ทใ‚นใƒ†ใƒ ใ€ใƒญใƒœใƒƒใƒˆใŒ่‡ชๅพ‹็š„ใซใƒญใƒœใƒƒใƒˆใฎไฝœใ‚Šๆ–นใ‚’ๅญฆใ‚“ใงใƒญใƒœใƒƒใƒˆใ‚’ไฝœใ‚‹ๆ™‚ไปฃใธใ€‚ใ‚ณใƒผใƒ‡ใ‚ฃใƒณใ‚ฐAIใŒใ™ในใฆใฎAIใƒ•ใƒญใƒผใฎใ‚ณใ‚ขใซใชใ‚Šใคใคใ‚ใ‚‹ใ€‚
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
1
8
63
18,496
Wenli Xiao reposted
Robotics research just got AUTOMATED by coding agents. Introduce ENPIRE: an agent auto-research harness for real-world dexterous manipulation. Tokens in, success rate out. Coding agents construct environment, reset a scene, iterate algorithms and evaluate results ALL BY IT's OWN, directly on your hardware. Highlights: ENPIRE teaches your robot how to tie a zip-tie. No more burning PhDs: agent team takes care of your experiment. Image segmentation give ultra fast & reliable feedback. Codex/Claude ablates training configurations. Takeaways: 1. Multimodal agents (codex) are way stronger in robotics auto-research. 2. Scaling up robots and agents give you faster convergence. Website & Paper: research.nvidia.com/labs/geaโ€ฆ
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
5
18
103
52,099
Wenli Xiao reposted
Curious about a key โ€œunknown unknownโ€ in physical AGI? This work on physical autoresearch is worth your time. ENPIRE lets frontier coding agents fully evolve robotics research in the real world. They propose ideas, execute experiments on physical robots, auto-reset the environment, analyze results, and iterate. All autonomously in a continuous hill-climbing loop. Six months ago with CaP-X, we set out to understand why frontier LM capabilities werenโ€™t transferring effectively to robotics foundation models, which often lack even basic generalization. That effort gave us a clear baseline for what pure LMs can achieve without VLAs. Today we introduce ENPIRE, our latest step: addressing physical autoresearch as an ultra-long-horizon challenge. What started as a task has become autonomous evolution. Looking ahead, I believe the real leap will come from natively agentic robotics models: systems that inherently carry agency to gather context, follow checklists, generate actions, and self-verify completion, all inside one model. Excited to see where this goes ๐Ÿ‘€ Huge thanks to the whole team. Incredibly proud of what weโ€™ve built together. ๐Ÿ”— Project: research.nvidia.com/labs/geaโ€ฆ CaP-X: capgym.github.io
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
3
3
69
8,421
Wenli Xiao reposted
When VLM agents get better at 1) coding and 2) scene understanding, a trend of revisiting code-as-policy with agents appears.
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
1
1
55
7,894
Wenli Xiao reposted
robots training robots, then robots building robots
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
3
8
725
Wenli Xiao reposted
Very impressive! Coding agents might be the most promising way to solve robotic continual learning beyond pick-and-place.
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
2
4
595
Wenli Xiao reposted
Today, we enable AutoResearch in the physical world for the first time! Introducing ENPIRE: we give 8 Codex agents a fleet of robots, an allocation of GPUs, and generous token budget. We set them free with a simple goal: solve the task as quickly as possible, keep the robots busy but stay safe, don't waste precious compute. Make no mistake. Then humans step aside and our watch begins. The robot fleet starts to come alive: they learn to look for visual clues, reset the scene, practice novel skills, tinker with control stack, read papers online, debate, reflect, get stuck, and try again directly on the hardware. All we did is to give Codex an API to the world of atoms, and the rest is emergence. ENPIRE is able to solve high-precision tasks like tying zip-ties, organizing fine pins, and installing GPUs all by itself. We also discovered a new type of "physical scaling": 8 robots exploring in parallel improves significantly faster than fewer ones. A part of our NVIDIA GEAR lab now self-improves tirelessly over night. We just read the reports in the morning. /goal: we all take a holiday and Jensen wouldn't even notice ;) We will be open-sourcing everything, so you can host your self-running robot lab at home too! Deep dive in the thread:
187
570
3,845
653,052
Wenli Xiao reposted
What if coding agents can perform autoresearch on a fleet of robots to study novel robot learning algorithms and improve policy? Excited to introduce ENPIRE: a harness loop in which the coding agent first construct its own task-specific interfaces (env.reset / env.get_reward / env.get_done) via Code-as-Policy, wrapping deployment infra into a structured gym environment, then autonomously hill-climbs BC, RL, or heuristic policies on real robots. ENPIRE reframes policy improvement in the real world to a compute problem โ€” it scales with token & robot utility input, not researchersโ€™ intellectual and labor input. We also show a viable path to build a robotic data flywheel for challenging long-horizon tasks with coding agents: orchestrating available vision/control primitives to solve trivial phases (pick-n-place, free-space movement), then iterating on challenging phases (contact-rich, precision-critical) autonomously. Project website: research.nvidia.com/labs/geaโ€ฆ Work done with co-leads @_wenlixiao @jiaxie_jason @TongheZhang01 ย @NVIDIA @CMU_Robotics @Berkeley_AI
Autoresearch just left the sandbox and entered the embodied world. We are excited to introduce ๐„๐๐๐ˆ๐‘๐„: a system that drops frontier coding agents onto a fleet of real robots and hands them the entire loop: reset the environment โ†’ search the literature โ†’ implement ideas and build the infra โ†’ train and deploy โ†’ self-verify โ†’ analyze the logs and rewrite the code โ†’ repeat, until the policy is reliable in the real world. No human in the loop. Guided only by the robot's self-proposed, heuristic-based success signal, the agents hill-climb to 99% on dexterous real-world tasks: organizing pins into a box, seating GPUs, tying zip-ties. We envision the bottleneck in robotics shifting โ€” from building smarter algorithms to building the closed physical feedback loops an agent can finally turn on its own. ๐Ÿ”— research.nvidia.com/labs/geaโ€ฆ From @NVIDIA @CMU_Robotics @Berkeley_AI ๐Ÿงต
1
14
65
7,537