The Muscle Memory benchmark: does an AI that learns beat one that takes notes?

Abstract

Frontier language models are frozen at the end of training and adapt at deployment through context: notes, retrieved records and tools. Whether learning in the weights offers anything beyond a well-kept notebook is an open empirical question, and the usual continual-learning tests cannot answer it because they remove the context between tasks. We propose the Muscle Memory benchmark, a controlled comparison in which two copies of one agent play the same sequence of game levels with memory left on. The notebook agent appends everything it observes to a record it rereads before every decision; the learning agent updates its weights after every level and carries no record. The levels are built so that their lessons are of two kinds a record carries badly: regularities spread over hundreds of trials with no single statement to write down, and timing skills that must be executed rather than recalled. We add levels until one agent falls behind, and we report win rate and cost per decision as functions of levels played. The design is preregistered here; no results exist yet.

1. Background

Two positions are defensible today. The first holds that a frozen model plus memory is enough: long contexts and retrieval hold the full record of a project, the model reasons over that record at every step, and nothing is ever forgotten because nothing in the model changes. The second holds that rereading a record is not learning: the model's judgment at the last step is the same as at the first, the record grows and slows every decision, and the skills that matter most, the feel for which action is worth taking, are not the kind of thing a record can contain. Each side has evidence in its favour: writing facts into weights is less reliable than placing them in context, while long-horizon tasks remain the clearest failure mode of frozen models.

The gated level test on this site asks whether a lesson reached the weights at all, by wiping context between levels. That isolates weight learning but says nothing about its value relative to context. This proposal is the missing comparison.

2. Hypothesis

H1. As the number of levels grows, the learning agent's win rate stays level or rises while the notebook agent's win rate flattens and then falls, on levels whose lessons are distributed or executional. H0. The two win-rate curves stay within a small margin of each other at every point, at a cost per decision that remains within a fixed budget. H0 is the position that notes are enough; the design must be able to produce it.

3. Method

3.1 Environment

The Skirmish worlds used by the gated level test: small real-time battles with hidden rules, a fixed observation as numbers, up to eight orders per half second of game time. Levels are generated from the same world maker with held-out rule combinations, so neither agent has seen them.

3.2 Players

Both agents start from one checkpoint, one policy trained on the ungated worlds. The notebook agent keeps its weights fixed. After every decision it appends the observation, the orders and the outcome to a record; before every decision it is given the full record, or, when the record exceeds the context, a retrieval over it. The learning agent keeps no record. After every level it runs a fixed update on that level's own play, using the forgetting recipe described on this site: learn from own play, fence the update, replay earlier levels. Both agents see identical levels in identical order and are scored identically.

3.3 Lessons

Each level carries one lesson of two kinds. Distributed: a regularity visible only across many trials, such as which of several apparently equal mines runs dry first, drawn from a distribution per world rather than fixed; no single observation reveals it. Executional: a timing skill, such as the moment to commit a wave so it arrives as a gate opens, where the lesson is a reaction, not a fact. Control levels carry a third kind, declarative: a rule that one line could state, such as which gate a switch is wired to, where a notebook is expected to do well. Lessons return in later levels in new worlds.

3.4 Protocol

Levels are played in blocks of fifty. After each block both agents are frozen and evaluated on a fixed test set of twenty held-out levels covering all three lesson kinds; the evaluation uses the notebook agent's record as of that block and the learning agent's weights as of that block. Blocks continue until one agent's test win rate is below the other's by more than ten points on two consecutive blocks, or until a thousand levels. Three seeds per agent.

4. Measures

Primary: test win rate against levels played, per agent, per lesson kind. Secondary: cost per decision in tokens processed and wall-clock time; for the notebook agent this grows with the record. Tertiary: the learning agent's win rate on the earliest test levels at the end, as a forgetting check.

5. Predictions

If H1 holds, the gap opens first on executional lessons, then on distributed ones, and last or never on declarative ones, where the notebook should keep up. If H0 holds, the curves coincide on all three kinds. A gap that opens only once the record exceeds the context would show that the notebook agent's failure is a capacity limit rather than a learning limit, and the result would be reported as such.

6. Limitations

One environment, one base checkpoint and one update recipe. A better notebook strategy, such as the agent summarising its own record, might narrow the gap; a stronger learning recipe might widen it. Cost is compared on one hardware setup. Nothing here speaks to safety or to tasks outside games.

Status

Preregistered design, draft v0, November 2026. Not run. The plain-language version is the research page.