When people share a space, their movements become intertwined. Embodied agents need to understand these social dynamics to interact effectively. Introducing MAGNet 🧲, a unified autoregressive diffusion forcing model for multi-agent motion generation that captures these interactions. MAGNet is flexible: predict the future, fill in missing motion, or have people react to each other, all while naturally scaling to N>2 people and generating ultra-long motion sequences.

New paper: Back into Plato’s Cave Are vision and language models converging to the same representation of reality? The Platonic Representation Hypothesis says yes. BUT we find the evidence for this is more fragile than it looks. Project page: akoepke.github.io/cave_umwel… 1/9

Grateful for my amazing collaborators: @kiehoriuchi, @LeaMue27, Evonne Ng, @JitendraMalikCV, and @akanazawa Check out our paper, project page and code for more details: 📎Paper: arxiv.org/abs/2512.17900 🌐Page: von31.github.io/MAGNet 🖥️ Code: github.com/Von31/MAGNet-code

Thanks @lisabdunlap for organizing a long overdue talk. -----Biltong Vs Beef Jerky Poll Results------ They came. They chewed. They voted. Biltong got 75% — Swi nandzika ngopfu!!

Happy to share our new work on Navigation World Models! 🔥🔥 Navigation is a fundamental skill of agents with visual-motor capabilities. We train a single World Model across multiple environments and diverse agent data. w/ @GaoyueZhou, Danny Tran, @trevordarrell and @ylecun.

We wrote a paper: estimating body and hand motion from a pair of glasses! 👓 website: egoallo.github.io code: github.com/brentyi/egoallo

Please see the website for more details. synNsync🪩is joint work with my awesome ✨co-authors✨: @LeaMue27 @jathushan @geopavlakos @shiryginosar @akanazawa @JitendraMalikCV Website🖥️: von31.github.io/synNsync/ Data💾: github.com/Von31/swing_dance… Arxiv📜: arxiv.org/abs/2409.04440 🧵6/6

Sampling multiple predictions👯👯👯 from our model shows that knowing about the "other" helps generate diverse motion that is closer to the ground truth. 🧵5/6

Conditioned on past dyadic motions of Alice and Bob 🪩🕺🏾💃🏾, our method can predict plausible and realistic motions better than the case where you only have Alice's past motion, i.e. unary case🪩🕺🏾 🧵4/6

We used a combination of VQ-VAE and autoregressive architecture to predict 🔮the next set of tokens in the motion sequence representing the 4D humans. ▶️🪩🕺🏾💃🏾⏭️💃🏾⏭️💃🏾⏭️💃🏾⏭️💃🏾 🧵3/6

We created an in-the-wild Swing Dance dataset 🪩 to study this. We lift the video into 4D using SLAHMR to extract 3D human motion in the world from 680 unique videos. The dataset features about 30 hours of videos of Alice (Person A 🕺🏾) and Bob (Person B💃🏾). 🧵2/6

To what extent does social interaction affect behavior in couples swing dancing? Our work looks at how we can predict a dancer's next moves based on their past motion alone or by considering both the dancer's and their partner's motions. 🪩🕺🏾 🆚 🪩🕺🏾💃🏾 🧵1/6

I really had no idea who else was coming...lol

Super excited to share that I will be starting as an Assistant Professor at UT Austin @UTCompSci in January 2024! 🥳🥳 I'm extremely grateful to my amazing mentors and colleagues for their unwavering support every step of the way! Looking forward to this exciting new chapter!

While most existing methods that estimate 3D human pose and shape (HPS) do so in camera coordinates, many applications require 3D humans in global coordinates. At #CVPR2023, we introduce TRACE, which addresses this using a novel 5D representation. 1/6

I'll be talking about nerfstudio at #GTC2023 today at 2pm PT! Come learn about #NeRFs, the latest updates in @nerfstudioteam, and see cool results from the community 🥳 bit.ly/3lzhtTU

I delivered the 110th Annual Martin Meyerson UC Berkeley Faculty Research Lecture on March 20, 2023. youtube.com/watch?v=f6fDpKDx…

Psyched to finally release our new work that decouples human🧑🏽‍🤝‍🧑🏻 & camera motion🎥 from an unconstrained video✨ We place people in the world, enabling high quality recovery of motion and spatial relationships between people 🤼 CVPR'23: Code & more video: vye16.github.io/slahmr

By the way - if you are a journalist looking to make sense of the bias issues with generative AI, I highly recommend speaking to those that have been thinking about this much longer than I have: @vonekels for example has an excellent paper on bias in face generation models.

Super proud of @vonekels and @neerjathakkar. Come check out some thought-provoking work at poster 43.