Inferring 3D human poses from video is highly ill-posed because of depth ambiguity. Our work accepted to #NeurIPS2024, ManiPose, gets one step closer to solving this, by leveraging prior knowledge about poses topology and cool multiple-choice learning techniques.

We’re sharing the next major milestone in our non-invasive brain-to-text decoder research: Brain2Qwerty v2. Building on v1, which was published today in @Nature, Brain2Qwerty v2 is the highest-performing end-to-end pipeline capable of real-time sentence decoding from raw brain signals. It advances beyond character-level performance to decoding words and semantics, enabling accuracy for overall communication. We believe this research has the potential to make a real difference for the millions of people who suffer from brain lesions or disorders that prevent them from communicating. 🧵👇

Decoding typing from brain activity — no surgery needed. Brain2QwertyV2, led by @JarodLevy and @LucyZ47712090, narrows the gap with invasive BCIs using LLMs on MEG and paves the way toward real-time language decoding! Proud to have collaborated with this talented team!

Human-Computer Input via a Wrist-Based sEMG Wearable meta.com/blog/quest/surface-…

Thanks for the nice workshop ! It was great to be able to share our work, listen to great talks and have many interesting conversations !

Yet another great work on multi-hypothesis learning by @VLetzelter accepted to #NeurIPS2024 ! In this paper they show that simulated annealing can help to make the winner-takes-all loss more stable and robust, demonstrating its useful in many ill-posed real-world applications!

Our team at @RealityLabs is also open-sourcing two EMG datasets at #NeurIPS2024 : emg2pose and emg2qwerty Check it out if you want to try predicting hand poses and typed text from a new challenging data modality!

✨Joint work with the amazing @VLetzelter , @nemka_ , @RenaudMarlet, @quobbe , @ptrkprz and Eduardo Valle✨

If you’re at #Neurips2024 next week, come meet us in poster session 3 on Thu 12 Dec 11 a.m.! Or at our oral presentation during the @neur_reps workshop on Saturday 14th! Paper: arxiv.org/abs/2312.06386 Github: github.com/cedricrommel/mani…

We train it with the resilient winner-takes-all loss, which allows the model to optimally quantize the space without requiring many heads. In the end, our model works as a conditional density estimator, taking the shape of a mixture of Dirac deltas.

Limbs length and directions are disentangled to constrain predicted poses to an estimated manifold. A multi-head subnetwork is used to predict different possible rotations for each joint, together with their corresponding likelihoods. Both are then merged into predicted poses.

We prove the *only* way of conciliating consistency with accurate predictions is to output multiple 3D poses for each 2D input. We hence propose ManiPose, a manifold-constrained multi-hypothesis deep network capable of better dealing with depth ambiguity.

Previous approaches constrain poses to an estimated manifold by disentangling limbs lengths and directions. But they lag behind unconstrained models in terms of joint position error (MPJPE). In our work, we prove this is unavoidable because of points 1 and 2.

There are 3 main reasons to this: 1. Existing training&evaluation metrics (MPJPE) are blind to such inconsistencies ; 2. Many possible 3D poses can map to the same 2D input ; 3. Pose sequences cannot occupy the whole space: they lie on a smooth manifold because of limbs rigidity.

While standard approaches directly map 2D coordinates to 3D, prior works noticed that predicted poses’ limbs could shrink and stretch along a movement. In our work, we prove these are not isolated cases and that these methods always predict *inconsistent* 3D pose sequences.

Many intelligent systems, like autonomous cars and smart/VR glasses, need to understand human’s movements and poses. This can be achieved with a single camera by detecting human keypoints on a video, then lifting them into a 3D pose.

Oh boy, something definitely happened in 2022 ! 😅

If you are at #ICML2024 , come chat with @VLetzelter , David and I about conditional density estimation and ill-posed ML tasks this afternoon ! Poster session 4 - 1:30pm - poster # 1506

I’ll be attending #ICML2024 next week to present this excellent work led by @VLetzelter ! Looking forward to chat about ill-posed machine learning tasks, data augmentation, pose estimation, eeg decoding or anything else ML at the poster session or around some coffee !

Interested in ill-posed learning tasks, uncertainty prediction, conditional density estimation or multi-head deep neural networks ? In our new paper, accepted at #ICML24, we tackle these challenges by exploring the Winner-Takes-All (WTA) training scheme. [1/n]