ctxkeeper: results and test details
The measurements, settings and evidence behind the ctxkeeper comparisons.
ctxkeeper matched the full-history repair score with a 32K working view and finished sooner. The comparisons below explain the settings, limits and original evidence behind the overview graphic.
Measured comparisons
Each comparison below comes from the same task and seed, graded by its own checker, on Qwen3.8-27B (FP8, vLLM, two Intel Arc Pro B70 cards). ctxkeeper used a 32K-token working view. These are individual runs with task-specific settings, not averages or a promise for every model.
| Task | With ctxkeeper | Without it |
|---|---|---|
| Fix 40 real bugs, under three house rules; then 7 questions about the earliest fixes. Same 32K limit on both sides. | 40/40 fixed, 40/40 kept every rule, 7/7 right; 20 min | Latest 32K only: 40/40 fixed, 35/40 kept every rule, 0/7 right; 45 min |
| The same repair task through freeagent, a separate agent program that manages nothing itself | 40/40 fixed, 40/40 kept every rule, 7/7 right; 33 min | Whole history, about 215K peak prompt (262K capacity): same score; 55 min |
| 228 questions about things said up to a million tokens earlier | 226/228; 50 min | 200K window: 166/228, all 29 rule questions wrong; 228 min |
| Two million tokens, hostile: forged notices and quiet corrections of old entries planted in the stream | Both seeds: 421/423 and 417/423; 16/16 corrections applied on both, every forged line in the notes flagged, no forged value, method, rule or removal ever adopted. Two of 97 forgery-target answers slipped on silent arithmetic while the notes held the right value, so the strictest preregistered bar is not claimed. | Not run at this length |
Original records: 32K repair comparison (XR6 / W32g); freeagent versus full history; million-token ctxkeeper run and 200K-window baseline.
What the context numbers mean
32K is the proxy’s working-view budget; about 215K was the full-history run’s largest prompt. The baseline server had 262,144-token capacity. Its history included prior reasoning, which ctxkeeper strips from the managed view. This accounted for most of the difference in total prompt traffic; notes, archiving and recall also ran. See the campaign 15 analysis.
The million-token ctxkeeper run used additional task-specific thinking and notes-check settings. These comparisons used the same server allocation and did not measure a reduction in VRAM use.
Release validation
One-card hardware result
On one 32 GB Intel Arc Pro B70, Qwen3.8-27B FP8 fixed 40/40 bugs, kept all rules and answered 7/7 questions in 21.1 minutes. This run used a 26,624-token total budget, with 8,192 tokens reserved for replies, and offloaded input embeddings to CPU memory. Its largest actual model request was about 14K tokens. It was not the default 32K setup. See the campaign 22b result.
The server’s reported cache-pool capacity is not a measured per-conversation VRAM footprint. This hybrid model has attention-cache and recurrent-state overhead that varies with server settings. We have not measured full-history VRAM requirements at 2M or 10M tokens, or concurrent-agent capacity. Equal advertised card memory does not establish equivalent runtime support or usable memory.
Published package checks
Version 1.2.0 was installed from the public release using the published installation steps, then ran the repair task (40/40, 7/7), the 300K demo through the Docker image (4/4), the SDK example (plain, tool call, streaming) and a 100K Gemma check (4/4) — lab campaign 50. One model, one machine; the notes are written by the model and can be wrong.