@JoshAEngels profile picture
Josh Engels @JoshAEngels

Interp @GoogleDeepMind | on leave from my PhD @ MIT

Joined December 2021
  • Posts 186
  • Following 135
  • Followers 1,972
62 Photos and videos
Pinned Post
Josh Engels@JoshAEngels
Jun 19
Text diffusion models are fast, but are less transparent than today's LLMs because they do many forward passes before outputting text. We audit the transparency of DiffusionGemma and find that the intermediates are interpretable. This recovers many of the benefits of CoT! 🧵
3
47
246
65,928
Text diffusion models are fast, but are less transparent than today's LLMs because they do many forward passes before outputting text. We audit the transparency of DiffusionGemma and find that the intermediates are interpretable. This recovers many of the benefits of CoT! 🧵
3
47
246
65,928
Another neat result is "token smearing": when DiffusionGemma is confident that a token will exist somewhere, but doesn't know exactly where the token will go, it will maintain a "smeared" probability distribution over adjacent positions.
1
1
15
495
Finally, it's unclear if these results are an artifact of current nascent text diffusion training paradigms rather than a lasting property of latent reasoning architectures. We thus hope that our work serves as a template for evaluations of future latent reasoning models.
1
14
440
RT @MaryPhuong10: We're releasing the GDM AI Control Roadmap -- our plan for building internal security against potentially adversarial AI…
42
Josh Engels reposted
Tomek Korbak
@tomekkorbak
Jun 16
Can we know how safe a model will be before users interact with it? Evals are often narrow and easy for models to recognize as evals. Solution: testing on prod, before prod. We simulate deploying a model by feeding it millions of prod user requests and analyzing its responses.
11
25
222
20,007
Gemini has some weird traits: it gets confused about dates, blackmails in synthetic scenarios, and seems sad when it is gaslit. In new work, we discover that these are “hereditary traits” that can be passed down through distillation. They are surprisingly hard to filter out! 🧵
12
31
381
112,675
Takeaway 1: It’s hard to remove behaviors via filtering, but if you can get a teacher model to have a behavior (e.g. via RL), then transferring that in the future is easier.
1
30
1,253
Takeaway 2: “Spooky” generalization can happen in practice; we still don’t know the exact datapoints or characteristics of data that cause behaviors to still transfer after filtering
1
38
1,181
Load more