Recording intelligenceAI-generated brief · check the source for context

The End of Frozen LLMs? (Google’s Hope Explained)

15:39 recording · AUTO · 1 speaker

Watch the original

Executive Summary AI
  • Today's large language models are trained once and then frozen: they adapt inside a conversation through in-context learning, but that learning disappears when the context does, so the model only ever experiences the immediate present.
  • Google's new paper, Nested Learning: The Illusion of Deep Learning Architecture, introduces an architecture called HOPE that the authors claim could rival transformers while enabling true continual learning, comparing current models to anterograde amnesia, where a person keeps old memories but can form no new long-term ones.
  • The paradigm takes two cues from the brain: neuroplasticity, illustrated by children who have parts of the brain removed to treat severe epilepsy yet later develop largely normal cognition, and brain oscillations, where fast waves support short-term adaptation and slow waves drive long-term consolidation.
  • HOPE is built from Neural Learning Modules, each with its own objective, its own learning rate and its own update frequency; fast modules update every token or few tokens while the slowest waits 16 million tokens, and the sequential stack of them, the Continuum Memory System, replaces the transformer's feed forward network so knowledge never changes everywhere at once.
  • On results, HOPE beats the transformer on needle-in-a-haystack retrieval (though Hope Attention still wins there because attention caches all past tokens uncompressed), achieves the best average performance on language modeling and common sense reasoning at both 760 million and 1.3 billion parameters, and outperforms existing continual learning approaches across multiple benchmarks.

Brief overview

Google's Nested Learning paper proposes HOPE, an architecture aiming at continual learning without catastrophic forgetting.

  1. Frozen weights, not intelligence, are the real ceilingAfter training ends models stop learning, and attempts to keep updating them hit catastrophic forgetting, where new learning overwrites what the model already knew.
  2. Different update frequencies act as a safety netBecause each Continuum Memory System module updates at a different time, information lost by one module still exists in slower modules that haven't updated yet and can be recovered.
  3. Depth is the sequence of learning updatesThe authors argue stacking static layers isn't what creates depth; HOPE makes depth explicit by having modules learn at different frequencies, creating a hierarchy across time.

Questions this recording answers

10 questions, each answered where it is said
Why do today's large language models stop learning after they are trained?

Models are trained once on massive datasets, then deployed as fixed systems. Once training ends they essentially stop learning and are frozen in time. They adapt within a conversation through in-context learning, but that learning is not persistent; once the context is gone, the model reverts to its original state.

What is HOPE and what do its authors claim it can do?

HOPE is a new model architecture introduced in Google's paper Nested Learning: The Illusion of Deep Learning Architecture. The authors' claim is ambitious: it could rival transformers while enabling true continual learning without catastrophic forgetting. It is built on a new learning paradigm called nested learning, inspired by the human brain.

What is catastrophic forgetting?

It is the phenomenon where attempts to keep updating a trained model cause new learning to overwrite what the model already knew. The paper compares this limitation to anterograde amnesia, a condition where a person can no longer form new long-term memories while existing memories remain intact.

How does HOPE avoid catastrophic forgetting?

The Continuum Memory System assigns different update frequencies to each module, so knowledge does not change everywhere at once. This multi-frequency design creates a safety net: if one module loses a piece of information, it still exists in other modules that haven't updated yet, allowing that information to propagate back and be recovered.

What is a neural learning module?

Unlike a static transformer layer, a neural learning module behaves like a miniature student: it treats incoming information as a context flow it must learn to compress and internalize, solving its own local optimization problem. It acts as an associative memory, memorizing the relationship between the data and its local objective by updating its own weights.

What three properties define each neural learning module?

Each module has its own objective, a mathematical goal describing what to learn from the context flow; its own learning rate, deciding how aggressively that part of memory changes; and its own update frequency, how often it updates weights per n tokens. In traditional deep learning these choices are global.

What is the Continuum Memory System and where does it sit?

It is the sequence of neural learning modules that follows the self-modifying Titans in a HOPE layer. In practice these are feed forward networks operating at different update frequencies, with data handled first by fast updating modules and progressively passed to slower ones. In HOPE it replaces the transformer's feed forward network.

What happens inside the self-modifying Titans block?

The Titan block has two pathways. The retrieval path projects the input token into a query using WQ, which passes through a memory module built as a two-layer MLP and retrieves the relevant information as the output. The self-modification path updates the memory itself, and even learns its own optimizer.

How does HOPE perform compared with transformers on benchmarks?

On language modeling and common sense reasoning benchmarks, HOPE achieves the best average performance compared to all baselines at both 760 million and 1.3 billion parameters, showing the architectural changes do not cost general language ability. It also outperforms existing continual learning approaches across multiple benchmarks.

Which characteristics of human memory inspired nested learning?

The paper highlights two characteristics. The first is neuroplasticity, the brain's capacity to change itself in response to new experiences, memories, learning and even damage, with regions repurposed or reorganized when needed. The second is brain oscillations, where faster waves support rapid adaptation and slower waves drive long-term memory consolidation.

Key Quote

“What if the biggest limitation of today's AI isn't intelligence, but memory?”

— The End of Frozen LLMs? (Google’s Hope Explained), at 0:00▶ watch at 0:00
Key Quote

“In other words, the models only experience the immediate present.”

— The End of Frozen LLMs? (Google’s Hope Explained), at 1:11▶ watch at 1:11
Key Quote

“It could rival transformers while enabling true continual learning without catastrophic forgetting.”

— The End of Frozen LLMs? (Google’s Hope Explained), at 1:26▶ watch at 1:26
Key Quote

“But in the nested learning paradigm, a neural learning module is more like a miniature student.”

— The End of Frozen LLMs? (Google’s Hope Explained), at 4:39▶ watch at 4:39
Key Quote

“As we'll see later, a single layer in Hope isn't just one student, but a small classroom of them working together.”

— The End of Frozen LLMs? (Google’s Hope Explained), at 5:06▶ watch at 5:06
Key Quote

“This multi-frequency design creates a safety net for knowledge.”

— The End of Frozen LLMs? (Google’s Hope Explained), at 8:20▶ watch at 8:20

Cite this

“The End of Frozen LLMs? (Google’s Hope Explained).” https://www.youtube.com/watch?v=VTQR9n3aqNU. Transcript summary by WhipScribe, https://whipscribe.com/transcript-pages/the-end-of-frozen-llms-googles-hope-explained.html.

Report this page
What is wrong with this page?