Learning and Control for Agile Robotics Lab at @CarnegieMellon @SCSatCMU @CMU_Robotics.

Joined August 2023
Photos and videos
LeCAR Lab at CMU reposted
@OpenAI is so lucky to have @TairanHe99! @TairanHe99 is the first PhD student to graduate from my group, and it only took him two years and eight months to get this degree (Sep 2023 → Apr 2026). When @TairanHe99 joined my group, I thought general-purpose humanoids were an interesting research problem, rather than a solution. Humanoid whole-body control barely worked. Hardware broke everyday. Physical AGI was not even a legitimate dream. I always joke that “nothing really works in robotics,” but looking back, the community has made incredible progress—and Tairan has been a big part of that journey. I am incredibly proud of everything @TairanHe99 has achieved: @CMU_Robotics Presidential Fellowship, @nvidia Graduate Fellowship, many open-source contributions to the community, and WhynotTV podcast—one of the most influential deep tech podcasts, to name just a few. @TairanHe99’s thesis: Scalable Sim-to-Real Learning for General-Purpose Humanoid Skills. Committee members: me, @ChangliuL, @kkitani, Marco Hutter, and @pabbeel.
some news: I’ve joined OpenAI. After wrapping up my PhD in Robotics, I’m excited to keep working toward AGI in the physical world. exciting journey ahead :)
3
14
461
25,819
LeCAR Lab at CMU reposted
some news: I’ve joined OpenAI. After wrapping up my PhD in Robotics, I’m excited to keep working toward AGI in the physical world. exciting journey ahead :)
236
58
2,416
228,242
LeCAR Lab at CMU reposted
Congrats to @LeCARLab's 2026 #CMUGrad!
4
10
276
15,491
LeCAR Lab at CMU reposted
My lab at CMU @LeCARLab is hiring a postdoc! (Vibe-made the poster via @ChatGPTapp)
1
24
198
24,977
LeCAR Lab at CMU reposted
I’m so tired of writing rebuttals to this kind of “lack of novelty” review: “This paper trivially combines A, B, and C, so the algorithmic novelty is limited.” Technically, most (if not all) robotics papers are convex combinations of existing ideas. I still deeply appreciate A B C papers—especially when they deliver: - New capabilities: the “trivial combination” unlocks behaviors we simply couldn’t achieve before - Sensible & organic design: A B C is clearly the right composition—not some arbitrary A′ B C′ - Nontrivial interactions: careful analysis of the dynamics, coupling, or failure modes between A, B, C - Rehabilitating old ideas: A was dismissed for years, but paired with modern B/C, it suddenly works—and teaches us why - System-level & "interface" insight: the contribution is not any single piece, but how the pieces talk to each other - Scaling laws or regimes: identifying when/why A B C works (and when it doesn’t) - Engineering clarity: making something actually work robustly in the real world is not “trivial” - New problem formulations: sometimes the real novelty is in the reformulation—only under this view does A B C make sense. Maybe worth keeping these in mind when reviewing the next A B C paper : )
30
121
985
115,758
LeCAR Lab at CMU reposted
Generative models (diffusion/flow) are taking over robotics 🤖. But do we really need to model the full action distribution to control a robot? We suspected the success of Generative Control Policies (GCPs) might be "Much Ado About Noising." We rigorously tested the myths. 🧵👇
16
90
546
111,507
LeCAR Lab at CMU reposted
We tried to rethink modern robot learning policy design in this paper. TL;DR: Generative policies perform well NOT because of their distributional-learning formulation or their ability to capture “multi-modality.” Our proposed Minimal Iterative Policy (MIP) achieves similar performance to flow with much lower inference & training cost. Some interesting takeaways: 1. “Multi-modal behavior” is a very nuanced concept in robotics. Robot learning cares about the conditional distribution P(a|o), where a is the action (chunk) and o is the observation. The objective of generative modeling in vision/language is fundamentally different from the goal in control. In vision/language, we want diverse samples from the data distribution. In control, any action that leads to better downstream performance is sufficient. Also, robotics has very little (labeled) action data, and they lie on a low-dimensional manifold. As a result, even if the marginal P(a) is multi-modal in many tasks (e.g., PushT), the conditional P(a|o) is often not multi-modal in sparse-data regions. Empirically, we observe that the common explanation—“flow/diffusion policies win because demonstrations are multi-modal”—does not hold for most studied behavior cloning benchmarks. 2. Architecture and action representation matter more than flow vs. regression. We find that the strong performance of flow/diffusion policies is largely due to their modern architectures and action representations. UNet / Transformer designs and action chunking play a vital role. In fact, L1/L2 regression policies can match flow/diffusion when paired with the right architectures and chunking, except in a few high-precision tasks. 3. What actually helps in flow/diffusion? Stochasticity injection and supervised iterative computation. To isolate these factors, we performed a “surgery” on flow to design Minimal Iterative Policy (MIP), a deterministic two-step policy that keeps only these mechanisms. Surprisingly, with far lower compute, MIP is comparable to—or even better than—flow in most studied behavior cloning benchmarks. Why? We hypothesize that these mechanisms provide good inductive biases that improve manifold adherence, especially for OOD states. Flow and MIP exhibit significantly better manifold adherence than regression. I don’t think these conclusions are complete—there are still many mysteries. But I strongly feel we need a new subfield: “the physics/science of robot learning,” aimed at understanding fundamental learning mechanisms and principles for data-driven robotics. As @ilyasut said for LLMs: “We’re moving from the age of scaling to the age of research.” For robot learning, I believe we must embrace the age of research to unlock better scaling to ignite the age of scaling.
Generative models (diffusion/flow) are taking over robotics 🤖. But do we really need to model the full action distribution to control a robot? We suspected the success of Generative Control Policies (GCPs) might be "Much Ado About Noising." We rigorously tested the myths. 🧵👇
3
30
178
14,445
LeCAR Lab at CMU reposted
🕸️ Introducing SPIDER — Scalable Physics-Informed Dexterous Retargeting! A dynamically feasible, cross-embodiment retargeting framework for BOTH humanoids 🤖 and dexterous hands ✋. From human motion → sim → real robots, at scale. 🔗 Website: jc-bao.github.io/spider-proj… 🧵 1/n
12
59
233
65,129
LeCAR Lab at CMU reposted
Introduce SPIDER - a unified scalable dynamics-level retargeting method for both dex hand and humanoid! What does "dynamics-level retargeting" mean? Human motions in, physically feasible high-quality robot motions out (BOTH state and action). These robot motions are so good that you can directly open-loop execute the action sequence in the real world! Also, dynamics-level retargeting makes the downstream RL policy learning super easy since it only needs to figure out a "small residual" on top of the already physically feasible refs. Even better, we can easily do physics-level data augmentation & robustification, such as adding external forces and changing frictions. SPIDER can be used as a data engine for real2sim2real and learning from human data. How it works? It is based on the diffusion-inspired sampling-based optimal control framework we developed in the past (MBD, DIAL-MPC) and virtual contact guidance. Led by @ChaoyiPan and collaboration with @Meta FAIR.
🕸️ Introducing SPIDER — Scalable Physics-Informed Dexterous Retargeting! A dynamically feasible, cross-embodiment retargeting framework for BOTH humanoids 🤖 and dexterous hands ✋. From human motion → sim → real robots, at scale. 🔗 Website: jc-bao.github.io/spider-proj… 🧵 1/n
1
9
69
8,185
LeCAR Lab at CMU reposted
Meet BFM-Zero: A Promptable Humanoid Behavioral Foundation Model w/ Unsupervised RL👉 lecar-lab.github.io/BFM-Zero… 🧩ONE latent space for ALL tasks ⚡Zero-shot goal reaching, tracking, and reward optimization (any reward at test time), from ONE policy 🤖Natural recovery & transition
7
75
269
81,556
LeCAR Lab at CMU reposted
Excited to release BFM-Zero, an unsupervised RL approach to learn humanoid Behavior Foundation Model. Existing humanoid general whole-body controllers rely on explicit motion tracking rewards, on-policy PG methods like PPO, and distillation to one policy. In contrast, BFM-Zero directly learns an effective shared latent representation that embeds motions, goals, and rewards into a common space, which enables a single policy zero-shot perform multiple tasks: (1) natural transition from any pose to any goal pose, (2) real-time motion following, (3) optimize any user-specified reward function at test time, etc. How it works? We don't give the model any specific reward in training. It builds upon recent advances in Forward-Backward (FB) models, where a latent-conditioned policy, a deep "forward dynamics model" and a deep "inverse dynamics model"are jointly learned. In such a way, the learned representation space understands humanoid dynamics and unifies different tasks. More details: lecar-lab.github.io/BFM-Zero…
Meet BFM-Zero: A Promptable Humanoid Behavioral Foundation Model w/ Unsupervised RL👉 lecar-lab.github.io/BFM-Zero… 🧩ONE latent space for ALL tasks ⚡Zero-shot goal reaching, tracking, and reward optimization (any reward at test time), from ONE policy 🤖Natural recovery & transition
2
12
97
18,627
LeCAR Lab at CMU reposted
✈️🤖 What if an embodiment-agnostic visuomotor policy could adapt to diverse robot embodiments at inference with no fine-tuning? Introducing UMI-on-Air, a framework that brings embodiment-aware guidance to diffusion policies for precise, contact-rich aerial manipulation.
9
33
213
63,908
LeCAR Lab at CMU reposted
100% human data -> long-horizon💡insertion on an aerial manipulator Key: Embodiment-aware diffusion policy (EADP) steers UMI's embodiment-agnostic DP using the gradient of the low-level controller's tracking error. My favorite: We quantify how "UMI-able" different robots are.
✈️🤖 What if an embodiment-agnostic visuomotor policy could adapt to diverse robot embodiments at inference with no fine-tuning? Introducing UMI-on-Air, a framework that brings embodiment-aware guidance to diffusion policies for precise, contact-rich aerial manipulation.
4
13
1,330
LeCAR Lab at CMU reposted
Didn't get chance to attend #CoRL2025 in person, but students from @LeCARLab will present three papers: - Sampling-based system ID for legged sim2real lecar-lab.github.io/spi-acti… at Oral5 by @NikhilSoban353 - HoldMyBeer: learning humanoid end-effector stabilization control lecar-lab.github.io/SoFTA/ at Spotlight3 by @li_yitang - Humanoid Policy ~ Human Policy human-as-robot.github.io/ at Spotlight1 by @chaitanya_cha Check them out!
1
11
60
5,361
LeCAR Lab at CMU reposted
We present HDMI, a simple and general framework for learning whole-body interaction skills directly from human videos — no manual reward engineering, no task-specific pipelines. 🤖 67 door traversals, 6 real-world tasks, 14 in simulation. 🔗 hdmi-humanoid.github.io/
24
141
733
116,427
LeCAR Lab at CMU reposted
On my way ✈️ to ATL for @ieee_ras_icra! @LeCARLab will present 8 conference papers (including DIAL-MPC as the Best Paper Finalist) and one RA-L paper. Details: lecar-lab.github.io/ Hope to meet old & new friends and chat about building generalist 🤖 with agility 🚀
18
107
9,080
LeCAR Lab at CMU reposted
🦾How can humanoids unlock real strength for heavy-duty loco-manipulation? Meet FALCON🦅: Learning Force-Adaptive Humanoid Loco-Manipulation. 🌐: lecar-lab.github.io/falcon-h… See the details below👇:
4
51
181
57,614
LeCAR Lab at CMU reposted
I've been working on dynamics model learning in robotics for >6 years (from the Neural-Lander paper). Training a small DNN some regularization some online weight adaptation have proven to be effective for specific robots in specific tasks. LLMs have shown the power of transformer, large-scale pertaining, fine-tuning, and in-context adaptation. Can we do the same thing for dynamics model learning? Excited to introduce AnyCar🚗, a transformer-based universal vehicle dynamics model that adapts to various cars & envs & tasks & state estimators, via in-context adaptation. It has some interesting designs akin to LLMs: 1. A seq2seq model aware of history. Input: K-step noisy state/action history, H-step future action. Output: H-step future state prediction; 2. Large-scale pertaining using 100M sim data collected in diverse sims, including Isaac Sim, MuJoCo, and F1-level Assetto Corsa; 3. Real-world "alignment" via fine-tuning on 20K real data; 4. Online "reasoning" by sampling-based MPC that rolls out action sequences using the learned model. AnyCar shows strong adaptability in many scenarios, such as drifting in the wild with 3D-printed plastic tires. Fully open-sourced (including the 1/16 racing car design): lecar-lab.github.io/anycar/
🚨🚨Can generalist robotics models perform agile tasks? Introducing 𝗔𝗻𝘆𝗖𝗮𝗿🏎️ 🚚 🚗, a transformer-based 𝗴𝗲𝗻𝗲𝗿𝗮𝗹𝗶𝘀𝘁 vehicle dynamics model that can adapt to various cars, tasks, and envs via in-context adaptation, 𝗼𝘂𝘁𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗶𝗻𝗴 𝘄𝗲𝗹𝗹-𝘁𝘂𝗻𝗲𝗱 𝘀𝗽𝗲𝗰𝗶𝗮𝗹𝗶𝘀𝘁𝘀 by up to 𝟱𝟰%! Key recipe: large-scale pretrain in diverse sim (100M data) fine-tuning in real (20K) Sampling-based MPC Website and code: lecar-lab.github.io/anycar/ 🧵👇 1/6
2
26
125
12,921
LeCAR Lab at CMU reposted
🎙️I gave a talk "Building Generalist Robots with Agility via Learning and Control: Humanoids and Beyond" at the CMU RI Seminar and Michigan AI Symposium. It covers @LeCARLab's research and my thoughts on building agile generalist robots. Recording: youtu.be/Uym3Tr6t5TM
1
24
101
6,825
LeCAR Lab at CMU reposted
H2O (👉human2humanoid.com) and OmniH2O (👉omni.human2humanoid.com) are open-sourced! Check out our fully open-source code: github.com/LeCAR-Lab/human2h…, featuring simulation training, motion data retargeting, and real-world deployment. Have fun with your humanoids!
🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗: human2humanoid.com
1
42
154
23,867