Past the window: what the agent still remembers

The question

An agent that works on one task for a long time builds up far more history than its live window can hold. Something has to leave the window. What can the agent still answer about things that happened long ago?

The same Qwen3.8-27B model read a 240K-token stream of short stories in 147 batches, with about three counter changes per batch. Its live window was capped at 16K or 32K tokens, so the history was 7 to 15 times larger than what it could see at once. Every 25 batches it was asked three questions about a batch a known distance back:

• Current value: what a counter is now.
• Old value: what a counter was before it was overwritten.
• Who did it: a small detail that only appeared once, in the story text.

At the end it answered 14 final questions about the current state. Distance is counted in batches back from the question.

Four ways to manage the window

Window-drop

When the window is full, the oldest turns are dropped. Nothing is kept. This is the simplest baseline, and it is lossy.

Compaction

What coding agents commonly do: old tool output is replaced with a short stub and old thinking is removed, then the oldest turns go. Also lossy. Called D or P in the evidence files.

Self-editing, read mode

The model reads each batch and keeps name value lines in a pinned state at the top; processed batches leave the window. How self-editing with a protected state works.

Self-editing + archive and search

The same pinned state, and every batch that leaves the window goes word for word to a read-only archive. A recall tool searches it. How the archive works.

Results

Each score is right/asked. The distance columns split the same probe questions by how far back the answer was stated; the type columns split them by question type. At 16K the scores are pooled over two seeds, except where noted. The 32K rows are one seed each.

Second round: two B70 GPUs, server window 65,536 tokens, exact prefix cache, one run at a time. Seed 1 is void: it hit the step cap and answered keys that were not asked. Only seed 0 is scored. Probe analysis for this table.

What it shows

• Archive and search at 32K is the first method above the lossy baselines on every axis at once. It had the best probe score (13/15), all 14 final answers, the best total (27/29), and at 15 minutes it beat window-drop and read mode. Only compaction finished sooner (13 minutes), with far worse final answers. The verbatim text is still there, so “who did it” questions can be answered.
• Compaction that keeps old state visible reports stale values as current. At 32K it answered the probes well (13/15), then got only 4 of 14 final questions right: six answers were values that used to be true. It also loses “who did it” details, because those live in the tool output it stubs out (1/10 at 16K).
• Window-drop is honest but forgets. It answered nothing correctly beyond about 25 batches back in any run: 0/5 at 16K, 0/2 at 32K, and 1/6 then 0/4 even with a 48K window.
• 16K starves the methods that work inside the window. With the batch, the state, the instructions and room to think all inside 16K, the self-editing read mode fell to 9/30. The archive method had no room either: the harness refused to fetch the next batch 121 times and refused to fold old batches 35 times, so old values stuck (8/14 final). That was lack of room, not a fault in how recalled text was written back. At 32K neither failure appeared.
• Speed did not degrade as history grew. Every method took about 4–10 minutes per 100K tokens of history, in both the first and second half of the stream.
• End-of-run questions alone do not separate methods. At 16K three methods answered every final question while scoring 4/15 to 10/15 on the probes. They mostly ask about state that is still in view.

Limits

The 32K results are one seed; the 16K results are two seeds, and the archive method has only one valid 16K seed. Each distance bin holds few questions (as few as 2 at 25–99 batches back), so a difference of one or two answers is not decisive. One synthetic task family, one model and one machine. The stream reached 240K tokens, not the millions this program is aimed at.

Lab note with both rounds and the post-mortem · Raw evidence: checks, logs, probe analyses and server settings · Website measurement record

What we are trying to solve now

The target is an agent that works on one task for weeks or months, with 1 to 10 million tokens of history, while its live window stays capped: 200K tokens on a dedicated GPU, or 32K–48K when the GPU is shared. The model must stay fast, accurate and on goal. The model’s own arithmetic stays exact; how the history is managed may lose information, but then the loss is measured.

Why the model limits the options

Qwen3.8-27B is a hybrid. Most of its layers carry a running summary state forward instead of keeping one entry per past token. That state cannot have a piece cut out of the middle. The exact operations are: add new text at the end, or roll back to a saved checkpoint and read everything after it again. Cutting a block out of the middle of an ordinary attention cache is only approximate too: the later entries were computed while looking at the removed text. So an exact edit in the middle means rolling back and re-reading the tail, which the second, idle GPU could do between turns.

How the next tests will measure it

• Probe families at 1–4, 5–24, 25–99, 100–399 and 400+ batches back: current values, overwritten values, small details, updated facts and questions the history cannot answer.
• A rule stated at the start that must still hold at the end, and a procedure learned early that must be reused late.
• A compaction-count axis: force a set number of compactions and plot accuracy against it.
• Costs for every method: time per 100K tokens of history, tokens spent on management, peak window and cache reuse.

Next steps

1. A second seed at 32K for all four methods.
2. Harden the archive method for small windows: limit how much one search returns, tell the model how to free room instead of refusing again, and answer with what it has when steps run out.
3. Add the rule-adherence and procedure-reuse questions.
4. Measure saving and restoring the attention cache from fast disk for recalled batches, so recalled text need not be read again from scratch.

The full program charter: the problem, every idea so far and the benchmark design · Recommendations so far