Engram cofounder @jxmnop just raised $98M to build a new type of AI. He says models don't need to get smarter over time. Instead, they just need to know you better and better over time. Jack describes what he's building: "Our product is a new type of AI. We have a pretty different vision from a lot of the frontier labs, which are working on one model per lab, and trying to make that model smarter every month." "There's another way to think about it, which is that the model doesn't need to get smarter every month. It needs to know you better." "So we're working on a whole different stack, which is a way to train models that train themselves to know your world better and adjust to the things that you say." "So: new ways of training, new ways of running the models."

Today's AI models train once. We don't work that way. We learn continuously, forget what doesn't matter, and retain what does. That gap is what @dan_biderman and @realJessyLin are closing at @EngramLab. AI that never stops learning, with memory that lives inside the model instead of bolted on as an afterthought. In our latest Training Data episode we get into why memory is the next frontier: why the brain forgets on purpose, why RAG is a band-aid, and what becomes possible when a model is always training. 00:00 Introduction 00:59 Always Training Explained 01:51 Beyond Context Windows 03:29 Ngram Product Overview 04:34 Adapters And Training Signals 05:32 Internalize Vs Externalize 06:49 Compute And Token Savings 08:19 Teams First Then Individuals 08:51 Memorization Vs Understanding 12:47 Dreams And Offline Digestion 14:08 Training Beats Curation 15:19 Why Everyone Needs A Model 21:44 Bitter Lesson And Architecture 24:44 RAG Killer And KV Cache 31:38 Future Of Memory And Models

Today's AI models train once. We don't work that way. We learn continuously, forget what doesn't matter, and retain what does. That gap is what @dan_biderman and @realJessyLin are closing at @EngramLab. AI that never stops learning, with memory that lives inside the model instead of bolted on as an afterthought. In our latest Training Data episode we get into why memory is the next frontier: why the brain forgets on purpose, why RAG is a band-aid, and what becomes possible when a model is always training. 00:00 Introduction 00:59 Always Training Explained 01:51 Beyond Context Windows 03:29 Ngram Product Overview 04:34 Adapters And Training Signals 05:32 Internalize Vs Externalize 06:49 Compute And Token Savings 08:19 Teams First Then Individuals 08:51 Memorization Vs Understanding 12:47 Dreams And Offline Digestion 14:08 Training Beats Curation 15:19 Why Everyone Needs A Model 21:44 Bitter Lesson And Architecture 24:44 RAG Killer And KV Cache 31:38 Future Of Memory And Models

Today, I’m excited to formally announce @mirendil with my amazing co-founders Harsh Mehta, Shayan Salehian, and Tara Rezaei! We’re fortunate to work with @a16z and @kleinerperkins, who led our seed round of $200M, followed by a major investment from NVIDIA, among others. Mirendil exists to accelerate science and technology, and through them, to help solve humanity's most pressing problems. Self-accelerating AI R&D is the most direct path to delivering on AI's broader promise, which is why we believe the most important application of AI is AI itself. Get this loop right, and it compounds. It fundamentally changes the rate of progress itself across all domains. We believe this capability should be democratized. It should be used to power all scientific efforts trying to innovate at the frontier. There are far more important problems—and broader ones—than any single lab can take on, so more groups should be able to pursue them. This pulls concentration of power away from a few labs: businesses and science labs can own their AI and infrastructure, keep their margins, and control their own destiny instead of ceding it all to a single AI lab. We’re a small team with a singular focus. Our founding team consists of 20 researchers and engineers from frontier institutions including Anthropic, xAI, Google DeepMind, and OpenAI, united by a passion for science and a drive to build the technologies that move it faster. If you want to build the system that builds systems, join us! @HarshMeh1a, @shayan_, @tararezaeikh

All reliable clusters are alike; each unreliable cluster is unreliable in its own way.

Thank you to @Nasdaq for supporting Engram on our launch day yesterday! Some have commented that this photo looks AI-generated. It's not. This really happened. Feel free to send this picture to your moms. We're certainly going to.

The models we use every day are brilliant strangers. They forget your organization the moment a chat ends, then relearn it on the next query. @EngramLab fixes that. It learns your world once and reuses that memory, matching frontier systems on 1-10% of the tokens. @Microsoft, @NotionHQ, and @Harvey are already testing it within their organizations. Congratulations to the team, and hear directly from @dan_biderman (CEO and co-founder) and Sabri Eyuboglu (CTO and co-founder) with @LM_Braswell ⬇️

I've spent years studying how human memory works: how we learn and forget, and how our memories shape what we do. I'm thrilled to share that I've joined the founding team of Engram, which is coming out of stealth today. Now is the time to understand how to build machines with neural memories. Excited to do it with the best team for the task. If you're curious, reach out.

I'm excited to share what we're building at Engram! This team is incredible, and we're working on one of the most interesting problems in AI right now: how to build models that are tailored to each person and continually learn from experience. Come join us!

we started a company!! so, we’re tackling continual learning: what’s the learning algorithm to take arbitrary data — documents, conversations, the models’ own experience — and make better models? how do we scale compute in the same way we’ve already seen with pre-training and inference time, but scaling on the same data we see as humans, day after day with no labels, no rewards? A lot of the ingredients are out there already (rl, distillation, long-context, sparse / param-efficient architectures, etc.). our team is at the frontier of these topics, and we’re singularly focused on this. we want to understand this problem better than anyone else in the world. nobody’s solved this problem yet, but even today it’s extremely greenfield opportunity to co-develop research & useful products. in our space, how people interact with the models defines what the data distribution is - and working on this problem end-to-end, from core science to end user, gives us incredible freedom to define the problem and imagine new kinds of experiences. i expect we’ll use models that continually learn much differently than we’re using them today. it’ll feel different when the models _just know_, and build on our thinking and direction in ways we can’t even imagine. we don’t even know the queries we’re not asking, the things we would do but aren’t able to today. i’m so excited to share what we’re doing with the world in the coming months!! and the team is extremely cracked :) tackling this grand challenge and working alongside @jxmnop @EyubogluSabri @dan_biderman @MayeeChen @__howardchen @shizhehe and many others has made every day so fun. come work with us!

I'm really excited to be working with Engram to bring their models into Custom Agents in Notion. There's something magical that happens when the agent stops needing to search your workspace, and just knows it.