Continual learning research · proposed direction · November 2026
Teaching an AI new things without losing the old
A recipe for keeping every earlier level passable while new gates are learned, and what it should be measured against.
Proposed research direction · November 2026
A bet on how to forget less
Three separate findings from the past year each cut how much a network forgets when it learns something new, and none of them alone looks like the whole answer. The bet I want to test is their combination: learn from the agent’s own play, fence each weight update into the directions the network barely uses, and replay a little of the past for whatever the fence cannot protect.
Proposal only. Nothing here has been run on my agents. The numbers quoted are the papers’ own, measured on language models, not on games.
On-policy reinforcement learning forgot less than supervised fine-tuning at the same new-task accuracy, apparently because it drifts less from the starting model (RL’s Razor, ICLR 2026).
Subspace Tuning keeps full-weight training but confines each update to the bottom 1% of each attention and MLP weight matrix’s singular directions. On an 8B model, the drop across five retention benchmarks shrank from 25–57 points to under 4, over two- and three-task chains (Chavan et al., October 2026).
Rehearsing a small sample of earlier experience is still the most widely used defence in continual pre-training (2026 survey). Whatever the fence lets through, replay of earlier levels might catch.
Train through a chain of levels; after each new one, play every earlier level again with the context wiped, so only the weights can carry the lesson. Retention is the earlier-level score. Learning is whether the next gate can be passed at all.
A fence that keeps old levels but cannot learn the gate has failed; so has a learner that passes the gate and loses level A.
Why combine them, and what could go wrongThe fence adds no room, own play needs the right reward, replay needs memory
The fence protects; it does not make room
Confining updates to 1% of a weight matrix protects the other 99% exactly, but every new skill must fit in that 1%. The Subspace Tuning paper already met the limit once: on a 7B model one task did not fit, and the free share had to grow to 10%. A refresh step that re-measures the directions after learning makes more room, at the cost of weaker protection of the original model. The paper tested chains of two and three tasks; whether the room runs out over a long chain, and how fast, is untested. A level chain with dozens of gates is a direct way to find out.
Why own play fits games
Reinforcement learning from the agent’s own play is already how the arena models are trained, so the first ingredient costs nothing extra here. The RL’s Razor result also suggests a measurement: forgetting tracked how far the new policy drifted from the old one on the new task, so drift can be logged during training and compared with the retention measured afterwards.
What the fence should be measured against
The protected directions are chosen from the weights themselves, not from what earlier levels actually use, so protection is a bet, not a guarantee. The test records, after each level, how much of the free subspace the accumulated updates have filled and how much the earlier-level scores have moved. If the two curves track each other, the fence is the bottleneck; if forgetting appears while room remains, the fence is protecting the wrong directions.
First experiment
Four learners on the same gated level chain with the same budget: plain updates; fenced updates; fenced updates plus replay of earlier levels; and plain updates plus replay. Each is retested on every earlier level after each new one, with the context wiped. Repeat across seeds and level orders. Report retention and gate passes separately; a single combined score would hide the trade-off this experiment exists to expose.
Related: the Long Game toolkit holds the levels this test would run on, with a held-out set of rules kept out of all training. Gated levels, which cannot be won without a lesson from an earlier level, are in development.
More research