Today's AI models train once. We don't work that way. We learn continuously, forget what doesn't matter, and retain what does.
That gap is what
@dan_biderman and
@realJessyLin are closing at
@EngramLab. AI that never stops learning, with memory that lives inside the model instead of bolted on as an afterthought.
In our latest Training Data episode we get into why memory is the next frontier: why the brain forgets on purpose, why RAG is a band-aid, and what becomes possible when a model is always training.
00:00 Introduction
00:59 Always Training Explained
01:51 Beyond Context Windows
03:29 Ngram Product Overview
04:34 Adapters And Training Signals
05:32 Internalize Vs Externalize
06:49 Compute And Token Savings
08:19 Teams First Then Individuals
08:51 Memorization Vs Understanding
12:47 Dreams And Offline Digestion
14:08 Training Beats Curation
15:19 Why Everyone Needs A Model
21:44 Bitter Lesson And Architecture
24:44 RAG Killer And KV Cache
31:38 Future Of Memory And Models