@Tobealegend24 profile picture
Yihao Sun ✈️ ICML 26 @Tobealegend24

PhD student at @Mila_Quebec & @UMontrealDIRO. Focusing on model-based decision making.

Joined February 2023
  • Posts 15
  • Following 48
  • Followers 55
5 Photos and videos
A really nice coffee chat with the OpenAI Robotics team! We’re truly honored that OpenAI has taken an interest in our work, “Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models”. We’ll be presenting our ICML poster at 5pm on July 8th.
3
18
698
Yihao Sun ✈️ ICML 26 reposted
Lu Li | ICML 2026
 @luli_airl
12h
🎉 Thrilled to share that our paper, "The Three Regimes of Offline-to-Online Reinforcement Learning," received the Best Paper Award 🏆 at the ICML 2026 Workshop on Decision-Making from Offline Datasets to Online Adaptation! I'll be giving an oral presentation at the workshop on July 11 at 1:30 PM KST in ASEM Ballroom 203. If you're interested in our work, feel free to reach out. I'd be happy to discuss it further in Seoul! 📄 Paper: arxiv.org/abs/2510.01460 📝 Blog: twni2016.github.io/blogs/pol… 📍 Workshop: decision-making-offline2onli… For a deeper dive into the paper, check out @twni2016's thread. #ICML2026 #ReinforcementLearning

The Three Regimes of Offline-to-Online Reinforcement Learning

Offline-to-online reinforcement learning (RL) has emerged as a practical paradigm that leverages offline datasets for pretraining and online interactions for fine-tuning. However, its empirical...

arxiv.org
Tianwei Ni @twni2016
Feb 9
Offline-to-online RL fine-tuning feels unpredictable: methods that work in one task can collapse in another. In work led by @luli_airl, we argue this isn’t noise — it’s a stability–plasticity mismatch driven by where prior knowledge lives. Paper: arxiv.org/abs/2510.01460 🧵
4
11
3,813
Our paper VLA-MBPO got into ICML! 🎉 Model-based RL has always been the “high potential, painful to tune” corner of RL. But our work pushes a classic MBRL algorithm (MBPO) to a new level: one shared set of hyperparameters sweeps every sim & real-robot env — no per-task tuning.
1
9
45
2,449
You can take a sneak peek: arxiv.org/abs/2603.20607

Towards Practical World Model-based Reinforcement Learning for...

Vision-Language-Action (VLA) models show strong generalization for robotic control, but finetuning them with reinforcement learning (RL) is constrained by the high cost and safety risks of...

arxiv.org
1
1
1
169
Three key design choices to make it happen: (i) adapting unified multimodal models (UMMs) for data-efficient world modeling; (ii) an interleaved view decoding mechanism to enforce multi-view consistency; and (iii) chunk-level branched rollout to mitigate error compounding.
1
1
141
Most VLA-RL frameworks inherit the complexity of LLM-RL infra but we found that none of it is necessary. We therefore introduce VLARLKit: A simple yet fast VLA RL framework. Code link: github.com/VLARLKit/VLARLKit
4
20
113
9,083
2. No long-context problem: Each step’s input is the current observation (plus a few recent frames) — not a growing conversation or tool-use history. Nothing to shard, nothing to KV-cache across steps.
1
1
283
3. Relatively small models: Inference-latency requirements keep modern VLAs around 3–4B params. Therefore simple distributed training tool is enough like FSDP/DDP.
1
256
Some personal perspective on World Models: peaceful-waste-8f5.notion.si…

World Models | Notion

World models are having a moment. Here are some of my personal thoughts 🧵

peaceful-waste-8f5.notion.site
1
1
7
131
LLMs are already a form of WM, but one that operates purely in the language modality. We shouldn’t draw a hard line between LLMs and WMs — what we actually need is a multimodal WM capable of flexible reasoning across both language and vision.
1
69
This is a new startup founded by my friend! Cool!
2
118
📢 My First tweet! 🚀 We explored how to bring the human brain’s imagination and reasoning into multimodal language models. 🧠 Uni-Plan uses a single Unified Multimodal Model (UMM) as policy, dynamics, self-discriminator, and value for decision-making. 🔗 uni-plan.github.io/
5
11
454
Load more