Working past the context limit
Compare ways to preserve facts and keep long tasks moving.
Keep a small working context, and keep the originals searchable. That combination gave the strongest results on our long-history tests. A keeper handles the notes and archive outside the working agent, so the agent can concentrate on the task. Run as a small proxy in front of the model server, for an agent that manages nothing, it answered 226 of 228 questions over a million tokens in 50 minutes.
Download and use the proxy · Visual guide to the approaches · Search the research
Long-history comparison
The same model read a 240K-token history through a 32K working window, answering questions about facts, earlier values and rules stated at the start. Here is the first stream, with the same 39 questions for every approach.
| Approach | What stays available | Correct |
|---|---|---|
| Keeper | Notes, protected rules, searchable originals | 39/39 |
| Keeper proxy | The same policy in a small program between client and server | 39/39 |
| Retrieval only (RAG) | Every sentence in a store; the best matches shown with each question; no notes | 39/39 |
| Agent-managed archive | A state table, pinned rules, searchable originals | 35/39 |
| Window-drop | Recent turns only | 23/39 |
| Tool-output compaction | Short stubs in place of old tool output | 20/39 |
The keeper also answered every question on a second stream. So does the proxy, now that it keeps its notes the fast way; its earlier form missed two. Retrieval alone matched the keeper here because the rule names what it covers. It never found a rule that names nothing (0 of 16, against 16 of 16 for the keeper), and it fell behind late in a million-token history: retrieval alone, measured. These are synthetic tasks on Qwen3.8-27B, not a general ranking of agents. Timing, both streams and test details.
A small desk, with the full history on file
Before old turns leave, the keeper saves them word for word and asks the model to refresh a short note: rules, current state and open work. It restores omitted rules and repeats them beside new questions. The model can search the originals whenever the note is not enough.
Follow the keeper step by step · How this differs from self-editing
Same stream and seed: 203 questions during the task, plus 25 at the end. One run per bar; all four finished and were graded by the task’s own checker. The keeper runs inside the test harness. The proxy is a separate small program between an ordinary agent and the model server. In-place notes keep the same notes at lower cost: the model writes down only what changed, and nothing is read twice. On a second million-token stream the keeper with in-place notes answered 210 of 222 and the proxy 214 of 222. Qwen3.8-27B on two B70 GPUs; total elapsed time.
Each bar is one run. The proxy also carried a real code-repair task for a separate agent program: 40 of 40 bugs fixed, every rule kept. Twelve well-known open-source coding agents have since run the same task through it, and eleven of the twelve fixed 39 or 40 of 40: which agents. How the proxy caught up · Why in-place notes are faster · What the earlier runs missed and the original measurements.
Choose what the task needs to remember
- For a long, changing conversation: keep searchable originals, short working notes and protected rules. Search finds things; notes keep the rules and the current state in view. A rule that names nothing it covers stayed in the notes (16 of 16) and was never found by search alone (0 of 16). On a harder test the two together beat either alone. Keeper and archive results · A rule that names nothing · Retrieval alone · The two together.
- For a ledger or repeated calculation: let code maintain the totals. Files and code finished the 121K ledger in 1.9 minutes with all answers correct. For prose, the model can submit quoted changes for code to apply. See both workflows.
- When the source fits comfortably: try supplying it together. Whole-history reading led on the 56-question chat-memory test. Compare with archive and summaries.
Summaries can work: they matched the archive on the smaller retention test, but took longer. A summary alone cannot recover a detail it omitted. Accuracy and time, side by side.
Compare the other tasks and earlier approaches
| What was tested | Best so far | Compared with |
|---|---|---|
| A 121K ledger of counter updates | Files and code: 24/24 in 1.9 min | Summaries: 24/24 in 41 min |
| The same ledger, no files allowed | Self-editing, thinking only when needed: 24/24 and 24/24 in 6.6 min and 5.7 min | Original CLM agent: 19/24 in 64 min |
| A million-token story read through a 32K budget | Quoted events: 24/24 in 47.9 min | Read mode: 23/24 in 62.2 min |
| Questions about text that was dropped (119K) | Archive and search: 36/36 in 6.1 min | Summaries: 36/36 in 26 min |
| A public chat-memory benchmark (LongMemEval, 56 questions) | Whole history in one call: 46/56; archive and search at 32K: 42/56 | Summaries: stopped at 19/23 |
| A 240K history behind a 32K window, questions planted at known distances (two seeds) | Archive and search: 63/65 right, 15–16 min per run | Window-drop: 56/65, and 0/5 beyond 25 batches back; compaction: 48/65 |
| Summarising at a threshold, on the first of those two histories (one seed) | Archive and search: 27/29 in 15 min | Summarising at 75% of the window: 23/29 in 29 min; window-drop: 25/29 in 20 min |
| A rule and a method stated once at the start of a 240K history (two streams) | A keeper that asks nothing of the agent: 39/39 and 46/46; run as a proxy with in-place notes: 39/39 and 46/46 (39/39 and 44/46 in the proxy’s earlier form) | On the first stream, archive and search with pinned rule lines: 35/39; window-drop: 23/39. Four methods with no pinned place for a rule: 0/20 rule questions |
| A 1,000,000-token history through a 32K window | The keeper as a proxy, with in-place notes: 226/228 right in 50 min on one stream, and 214/222 in 48 min on a second. The keeper inside the test harness, with in-place notes: 228/228 in 50 min on the first stream and 210/222 in 50 min on the second. All graded by the task’s own checker | The keeper with a full notes rewrite: 226/228 in 83 min; the proxy’s earlier form: 222/228 in 95 min. Archive and search with pinned rule lines: 187/228 in a stopped run that was not verified |
| Retrieval alone (“RAG”) against the keeper, on the same three histories | The keeper with in-place notes: 39/39 and 46/46 at 240K; 203/203 of the questions asked along the 1,000,000-token history, and 25/25 final questions | Retrieval only: 39/39 and 45/46 at 240K; 189/203 along the 1,000,000-token history, where it did not answer the final questions (a stopped run) |
| A rule that names nothing it covers, on two 240K histories | The keeper with in-place notes applied it every time: 16/16 | Retrieval only: 0/16. It gave the true number every time; the rule had left the window and no question leads a search back to it |
| A harder 240K history that says “it” and makes some values depend on others (the comparison is one seed, exploratory) | The keeper with in-place notes and retrieval together: 47/50 and 46/50, and 46/49 on a second seed | The keeper with notes only: 42/50; retrieval only: stopped with 10/15 right on the first three question sets |
| A real-code repair marathon: 40 planted defects and three standing instructions that arrive as notices (three seeds) | The proxy between agent and server: 40/40 and 40/40 clean; the audit of the oldest fixes 7/7, and 29.5 min at medium thinking effort | Summaries: 40/40 then 33/40 (one summary came back as work, not a summary; audit 0/7); at the same medium effort summaries matched the proxy, 40/40 clean, audit 7/7, 20.9 min; window-drop: 30/40 and 22/40 clean, and 35/40 without the task’s trap tickets |
Compare within each row. The ledger, narrative-reading, chat-memory and keeper tests ask different questions.
For speed rather than memory, prefix caching reuses the work of reading unchanged text. Saving text and moving GPU memory are different operations.