Foundation Models for Generalizable Autonomy in Robotics. Reinforcement Learning. Faculty in AI Robotics @GeorgiaTech. Prev @nvidia @UCberkeley @stanford

Joined January 2010
552 Photos and videos
Pinned Post
Physical AI is making a classic Moneyball error by mis-pricing data. Optimizing for "cumulative operational hours" or relying on low-variance production telemetry is the robotics equivalent of chasing batting averages, when we should be isolating on-base percentage. Unlike text, every useful hour of robot data is paid for. If you aren't calculating marginal utility per dollar based on data novelty and intrinsic dimensionality, you're allocating capital based on vibes. In physical AI, useful data is strictly resource-constrained, shifting the goal from maximizing compute efficiency to maximizing marginal loss reduction per dollar. A framework breaking down: - Interventional vs. Observational channels - Why deployment telemetry behaves like a decaying oil well - The "Convergence Trap" of narrow commercial niches - Metrics to replace raw operational hours check out the full post here: open.substack.com/pub/praxis…
11
21
229
21,802
"load-bearing" and "gate" are the new —
5
1
23
5,558
Animesh Garg reposted
x.com/i/article/207275948551…
7
36
354
67,639
Animesh Garg reposted
Today, we're introducing SimFoundry, our real2sim2real framework at NVIDIA GEAR that automatically turns real-world scenes into simulation-ready worlds from a single image or video. Website: research.nvidia.com/labs/gea… Paper: arxiv.org/abs/2606.28276 This work marks a major step for our team toward leveraging simulations and synthetic data for foundation model training and systematic policy evaluation at scale. Code will be open-sourced soon. Stay tuned!
12
73
485
46,075
Animesh Garg reposted
Gears are the "muscles" of robotics; but with so many AI/ML folks getting into robotics now, you may not know what people mean when they say cycloid/strain wave/planetary gear, and, more importantly why they matter What are the advantages and disadvantages? What actually ARE these things? I made this over the weekend to try to help provide an intuition: cpaxton.github.io/gears/inde… Probably has some mistakes as I am no mechanical engineer but I think it's useful
14
18
230
16,463
It is great to see Robot Park in action. to see so many Apollos is really exciting, and that hip/waist flex on the walk, great touch. lfg 🚀@jeffcardn and team.
Welcome to Robot Park, where we’re building the future with Apollo 2. Robot Park is where Apollo learns today, getting the experience needed to make a difference tomorrow. #Apptronik #Robotics #AI #Humanoids
1
12
2,864
Animesh Garg reposted
Great piece by @animesh_garg on the taxonomy of robot data and the importance of data novelty praxiscurrents.substack.com/…
1
12
2,386
Animesh Garg reposted
Can’t wait to see all the awesome science automation people will build on NEO
10
6
124
31,970
Animesh Garg reposted
Robots (esp personal and home) will feel more like ambient computing than pure labor automation. It needs to balance presence/companionship and utility/productivity. It will look and more and more like a computing platform that can proactively & collaboratively take action.
The computing container has evolved from -> the beige box (desktop) -> laptops (backpack) -> smartphones (pocket) Arguably, they are now growing arms and legs! personal computing will take the form of agentic interfaces in "embodied" agents (aka robots) as well as smaller "carry-with-you" form factors. In both forms, computing becomes ambient.
1
2
1,826
Animesh Garg reposted
Europe cannot rent its way to AI sovereignty. TLDR, here's my take I shared with frontier AI lab leadership this week. When Washington can disable a model overnight, the question is not whether AI is safe but who controls it: A week ago the United States government ordered Anthropic, the world's most valuable AI startup, to shut off its most capable model, Fable, for every foreign national on earth - whether they worked for Anthropic or not. This was not an export ban on a weapon sold to an adversary. It was an instruction to disable a commercial product, four days after its release, after officials acted on a claim - which Anthropic disputed as narrow and unproven - that its safeguards could be jailbroken to expose cyber-offense capabilities. I have spent my career around this technology, first as a graduate student and for the past decade as an investor @airstreet. In that time I have watched AI move from recommending movies to driving cars, speaking with a human voice, and editing the genome. I have also watched the debate about its risks settle on only half the question. That debate is mostly about capability: how powerful these systems are becoming, and whether one might escape human control. Those are real questions. But they are not the only ones, and the Anthropic episode exposed the half we have neglected: access and control. The most advanced AI is built by a handful of American companies, on American soil, under American law, and what the rest of us are allowed to do with it can change on a Friday afternoon. The risk that matters today is not only that AI goes rogue, but that we do not control access to it at all. Consider what "renting intelligence" now means in practice. A European hospital triaging scans, a bank screening fraud, a defense ministry planning for a conflict: increasingly each runs on an American AI system that's governed by its export regime. A single directive in Washington cascades, instantly, through every institution wired to that model. We have built core economic and public infrastructure on a supply that a foreign government can shut off. And while there are open-source alternatives, they're either Chinese or not at the frontier, and building European infrastructure on Chinese open weights trades one dependency for a thornier one. And these systems are starting to improve themselves. As they do, AI stops being one industry among many and becomes the input to all the others - writing the code, running the research, designing the products, and, increasingly, generating the growth itself. Once intelligence is the engine of an economy, a country without a frontier model of its own does not lose a sector; it loses control of the inputs to everything else, and the independence that depends on them. Worse, the gap compounds: capability that improves itself gets harder to chase with every month it runs ahead. This is not a race Europe can plan to enter in a decade. The window to be a builder rather than a buyer is measured in the time it takes to stand up a cluster, not a career. This should sting, because Europeans invented much of modern AI. DeepMind was founded in London and sold to Google in 2014, and a great deal of the talent that followed now lives in California. Today Europe faces a company worth almost $1 trillion and American tech giants spending an estimated $450 billion a year on AI infrastructure. Its answer has been the EU AI Act and a capital commitment that is a rounding error by comparison. A single American site, xAI's Colossus in Memphis, runs more than half a million GPUs. Europe has nothing remotely at that scale. The instinct to govern this technology is right, but we're off on the ambition by orders of magnitude. It is fair to object that regulation is itself a form of power. But a rulebook is not a substitute for the thing it governs. You cannot regulate, or be cut off from, an industry you do not have. Europe's instinct, when it is cut off, is not to build but to ask. We saw it within the week. The G7 convened in Evian and floated a "trusted partners" scheme to win back the access it had just lost, while Emmanuel Macron feted Donald Trump beneath the gilt of Versailles, the palace where France once helped midwife American independence. Two and a half centuries on, the dependency has reversed, and the posture is courtship. None of this means Europe can match the American frontier dollar for dollar. With today's capital, it cannot, and pretending otherwise only wastes the little it has. But the goal is not parity, it is leverage. A country does not need the best model in the world to be sovereign; it needs a credible one of its own, on its own soil, good enough that being cut off is survivable rather than catastrophic. That is the difference between negotiating your access from dependence and negotiating it with an alternative in hand. The point is not to win the race. It is to make sure no one else can end it for you. Sovereignty of that kind is something you build, and Europe has done it before. The Financial Conduct Authority's regulatory sandbox, launched in 2016, let startups test products with real customers under supervision instead of waiting years for authorization. The pro-innovation culture it signaled helped make London the fintech capital of Europe, home to Revolut, Wise, and Monzo. Government should be AI's most demanding early customer rather than writing rules for systems it has only ever imported. Industry has to stop behaving like a tenant. Too many European companies rent the entire stack from American providers and build a thin product on top. That earns a margin and owns nothing: when the lab that supplies you decides to compete with you, or its government decides to cut you off, you have no ground to stand on. Where it counts, build and hold your own models and compute. And our universities, which should be the source of all this, still work against it. I have argued here before that Europe's spinout system is broken, and it remains so. Too many institutions treat the companies their research creates as something to extract value from, rather than as the vehicle through which a discovery reaches the world. The best research should leave the building as a company, in addition to a paper. We keep framing AI safety and AI ambition as a tradeoff, as though a country must choose between governing this technology and building it. It is not a choice. The safest position is not the most heavily regulated one. It is the one where the model runs on your terms, in your jurisdiction, and no one on the far side of an ocean can reach over and turn it off. Right now that finger is not ours. Until it is, every other conversation about AI risk is one we are having with someone else's permission. --- END
32
61
283
86,327
The computing container has evolved from -> the beige box (desktop) -> laptops (backpack) -> smartphones (pocket) Arguably, they are now growing arms and legs! personal computing will take the form of agentic interfaces in "embodied" agents (aka robots) as well as smaller "carry-with-you" form factors. In both forms, computing becomes ambient.
No one knows yet what the next form factor for computing will be, and yet there will be a next form factor, and it will seem obvious in retrospect.
3
6
22
5,974
They say "you can just do things" This is what they meant @DavidSHolz kudos on taking on such a hard problem and with no obvious connection to Image2Text Hopefully many of us will not heed too much to the conventional advice of "stay in your lane" lfg 🚀
Announcing a new division of Midjourney called "Midjourney Medical"
4
1
18
3,328
Earlier today @notmahi and team's paper debuted on a similar idea to learn from videos x.com/animesh_garg/status/20…
Enabling learning motion directly from videos rather than using them for action supervision is a superior method and likely more scalable. While it is early this line of work suggests replicating the playbook that made robots walk. --> Real videos provide state supervision (not action) --> retargeting provides reference trajectories. --> RL tracks these trajecotries. This is a very good example of the separation of the "What" and the "How"
2
2
1
2,933
Multiple folks buying into the WHAT and HOW framework The slide is from the talk last year at ICRA keynote debate. (youtube.com/watch?t=196&v=Pf…) Another great example of using human data primarily. from reference generation not action supervision.
We trained a dexterous hand 🤖 from internet-scale human videos 📺 with ZERO real-world robot or teleoperation data We pair a world model 🌎 that predicts task intent with a generalist sensorimotor policy ✋ trained sim2real to zero-shot fulfill ANY intent Introducing LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition 🧵
5
4
25
5,614
Enabling learning motion directly from videos rather than using them for action supervision is a superior method and likely more scalable. While it is early this line of work suggests replicating the playbook that made robots walk. --> Real videos provide state supervision (not action) --> retargeting provides reference trajectories. --> RL tracks these trajecotries. This is a very good example of the separation of the "What" and the "How"
Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵
7
10
52
10,531
Remember the clone wars! The home robot arena is increasingly reminiscent. Even though there isn't an IBM standard yet, a lot of them are starting to look alike: colorful, soft-touch, bimanual, and wheeled! PCs of the 1980s were increasingly similar in ability and form factor, see advert from 40 years ago (May 6, 1986) in @nytimes. Fierce competition drives both uniformity and capability, and good times for the consumer! I, for one, can't wait to actually have one.
4
4
23
2,990
I am willing and able to marry any Anthropic MTS who needs their Mythos access back
2
16
3,476
"why humanoid" gets another case study? Expansion of capability envelope of humanoids and importantly in a very data and compute efficient manner. the recipe that worked for locomotion in increasingly working on loco-manipulation and perceptive locomotion. - data to generate reference ("what") - RL in sim to create experts ("how") - Distillation to add visual observations - Deployment on real env.
🪜 What if humanoids could climb ladders and work on them straight out of simulation? Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation. Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
13
6
41
11,119
Animesh Garg reposted
really interesting economic case for how robotics could create abundance, while turbocharging concentrated wealth accumulation. It really all comes down to natural resources (power and materials) capital ownership
Grandma’s handmade recipe vs. mass-produced bottled good For a century, industrialization forced a trade-off: cheaper standardization or expensive customization. Physical AGI stands to change this trade-off entirely. What happens when When Labor Becomes a Utility? New post on Praxis Currents: praxiscurrents.substack.com/…
1
3
2,376