Wrote a deep dive on implementing a language model from scratch in JAX and scaling it with distributed training! If you’re coming from PyTorch and want to see how the same ideas look in JAX, or just want a hands-on intro to distributed training, check out this blog post: chuyishang.com/blog/2026/jax… Comes with code an assignment and test cases so you can follow along!

Everyone is talking about memory lately: Micron, SanDisk, etc. Here, we zoom out from FlashAttention/device memory to the next bottleneck: data-center communication. That is where photonics matters. winterrykim.github.io/blog/2… w/@punhojark If interested, come see our work at ICML.

Visual deep dive on FlashAttention by hand ✍️ (drawn with Excalidraw) winterrykim.github.io/blog/2…

Presenting this work today at @CVPR from 11:45 AM – 1:45 PM, Poster 451! Swing by if interested, would love to chat! #CVPR2026

Visual reasoning does not need to be hand-designed! By changing the training dynamics, we can let VLMs discover their own visual abstractions for solving multimodal tasks. 📝Paper: arxiv.org/pdf/2512.21218 🌐Website: chuyishang.com/livr/ 🧑‍💻Code/Data: Releasing later this week! Work with amazing collaborators @kelvinli01 @roeiherzig @leokarlin @RogerioFeris and @trevordarrell

💪 This simple trick consistently outperforms SFT and remains competitive with approaches that rely on expensive intermediate supervision, such as long CoT, image annotations, or costly RL-style training. In contrast, our approach uses substantially less data and requires no intermediate supervision: while many baselines train on full reasoning traces, we train only on the original question/answer pairs!

⏳ Stage 1: Visual bottleneck. We prevent the answer tokens from directly attending to image tokens. The only path from the image to the answer is through the latent tokens, forcing them to encode task-useful visual information. Stage 2: Restore standard attention. Now the model can attend both to the original image tokens and to the learned latent tokens. The latents act as an internal visual workspace: compact, task-adaptive representations learned without explicit intermediate supervision.

🔍We ask a simple question: Can a VLM learn useful visual abstractions on its own? Instead of supervising intermediate visual states, we add latent tokens and modify the attention mask so the model is forced to use them as a visual reasoning workspace. These latent tokens are not generated autoregressively, unlocking the power of expressive latents without the inference overhead.

Prior methods for “visual” or latent reasoning supervise intermediate representations: boxes, crops, masks, or other human-designed visual steps. This is often costly! • needs extra task-specific data • expensive annotation/generation pipelines • imposes human priors on what the model’s reasoning should look like

🧠 Humans often solve visual problems by forming internal visual abstractions. For many visual tasks, we don't need to verbalize every intermediate step, or render a new image, to reason about what we see. So why should VLMs?

What if the best visual reasoning steps are ones humans can’t specify? 🤔 Existing VLM reasoning is often constrained by language, pixels, and human-designed intermediates. We introduce Latent Implicit Visual Reasoning, where we show that VLMs can discover the best visual reasoning steps by themselves — no bboxes, no intermediate images, no extra supervision. Presenting this week at CVPR! (1/n)🧵

Latent Implicit Visual Reasoning Current LMMs are text-centric and struggle with visual reasoning tasks. LIVR trains models to discover visual reasoning tokens implicitly—no supervision needed—enabling task-adaptive visual abstraction that outperforms explicit methods.

The Berkeley Bowl Berkeley Bowl at California Memorial Stadium

i am once again asking @wandb to make a phone app so that i can monitor the situation when i'm outside

I wrote a new blog on TPUs -- it's been fun seeing how different they are from GPUs and also drawing things on excalidraw again✏️ henryhmko.github.io/posts/tp…

people could upload their code and findings for experiments from old projects, failed experiments, ablations, etc this may have a few benefits: 1. reduces duplicate efforts 2. incentivizes reporting negative results 3. allows others to find mistakes in experiments and fix them

something i wish exists is an large, open database for experiments. for example, if i wanted to see how model X does on Y dataset when I make some minor modification, i can search this database to see if someone else has done this experiment already, before i run it myself.

shanghai doing its best amsterdam impression:

Happy to share that our 𝐌𝐮𝐥𝐭𝐢𝐦𝐨𝐝𝐚𝐥 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 work🤖 has been accepted to #EMNLP2024 Main conference! Hope to see everyone in Miami!🥳🌞🌅 Kudos to all authors and collaborators: @chuyishang, @amooseyou, @sanjayssub, and @trevordarrell.