Continual learning research · the benchmark in one page · November 2026
What Muscle Memory tests that ARC-AGI does not
ARC-AGI measures how well you solve a task you have never seen. It never measures whether solving it changed you. Muscle Memory is built so that it has to.
The benchmark, in one page · November 2026
What Muscle Memory tests that ARC-AGI does not
Muscle Memory, the main benchmark · play it · the two players · no AI run on it yet
ARC-AGI asks: can you solve a new puzzle?
Muscle Memory asks: do you still have yesterday’s trick while you learn today’s?
What we test
- Learn by playing
Nothing is explained. A worker finds the switch.
- Remember it
Next level, the switch is the only way in.
- Build on it
Level 8: two rings, two older moves.
- Memory stays on
Context, notes or weights: your choice.
- Weights vs context
The same AI, played both ways, same board.
- Real time
The enemy knows the move and comes for you.
- It keeps growing
More levels until one player falls behind.
What ARC-AGI tests
- ARC-AGI-1 2019
Grid puzzles. See a few examples, draw the answer.
- ARC-AGI-2 2025
The same, harder. One puzzle at a time.
- ARC-AGI-3 2026
Little games, no instructions. Each game scored alone.
Side by side
| ARC-1 | ARC-2 | ARC-3 | Muscle Memory | |
|---|---|---|---|---|
| Rules never explained | ||||
| Learn by acting | ||||
| Later task needs an earlier one | ||||
| Memory across tasks is scored | ||||
| Weights vs context, same model | ||||
| A live opponent, time counts |
yes partly no
ARC-AGI scores the solving. We score what it did to you.
Why the ARC-AGI-3 harnesses should struggle here
They are a frozen model, a long context and a notebook it rereads (agno, executable world models). Our guess, not yet measured:
- No line to write down
Two plates, three seconds. A timing, not a sentence.
- The clock runs while it thinks
Reread the notebook and the enemy is already in.
- The context overflows
A whole board twice a second. Summaries drop the exact bit.
- It restarts every level
From notes, not from skill. The bell lifts the gates for eight seconds.
Build it yourself
Click a rule to read it in full.
If they keep up anyway, our bet is lost. Either way it is a number ARC-AGI never takes. Run one.
More research