ctxkeeper

How it works, in plain words

ctxkeeper is a small program that sits between your AI app and the model. Your app sends the whole conversation, as it always does. ctxkeeper passes the model a short, current version instead:

1. Your instructions, kept word for word, always.
2. A notebook the model keeps for itself: the rules it must follow, what it has done, what is still open. When the view fills, the model updates its notes before older turns leave the view.
3. The newest turns of the conversation.

Turns moved out of the view are filed away on disk, word for word. The model gets a recall tool to search that archive whenever it needs an exact detail from earlier, the way you would look something up in a filing cabinet.

Does the history have to stay in VRAM?

No. The archive and notes live on disk. The model server processes a small working view. If its cache has expired or the server restarts, it reads that view again; ctxkeeper does not save and reload precomputed GPU state.

You can return to a saved conversation later. Keep your app’s complete conversation and ctxkeeper’s state folder. They have no built-in expiry in ctxkeeper. How to save and resume →

Measured, not promised

Each pair below is the same task on the same model and server (Qwen3.8-27B FP8, vLLM, two Intel Arc Pro B70 cards), graded by its own checker. Shorter bars are faster.

On the hardest test, two million tokens with forged instructions and quiet corrections planted in the stream, ctxkeeper answered 421 of 423 and 417 of 423 on two runs, applied all 16 corrections, and never adopted a forged rule. Full results, settings and original records →

Why you can trust it

• Free and public domain. No account, no telemetry, no cloud. It runs on your machine and talks only to the model server you point it at.
• Small and readable. Source on GitHub: two Python files using only the standard library. You can read every line before you run it.
• Published the normal way. On PyPI, with a release page listing a SHA-256 checksum for every file, and 100+ automated tests in the source.
• Tested in public, failures included. Every test run that shaped it, including the ones that went wrong, is in the research trail, with the raw records on GitHub.

Install and run

You need Python 3.10 or newer and a model server that speaks the OpenAI API (vLLM, llama.cpp’s server and most local servers do). Start ctxkeeper, then point your AI app at http://127.0.0.1:18200/v1 instead of the model server.

python3 -m venv .venv-ctxkeeper
. .venv-ctxkeeper/bin/activate
python -m pip install ctxkeeper

ctxkeeper --recipe qwen \
  --upstream http://127.0.0.1:8080

docker build -t ctxkeeper:1.2.0 \
  "https://github.com/steveseguin/b70-optimization-lab/releases/download/ctxkeeper-v1.2.0/ctxkeeper-1.2.0.tar.gz"

docker run --rm --network host \
  -v ctxkeeper-state:/state \
  ctxkeeper:1.2.0 --host 127.0.0.1 --recipe qwen \
  --upstream http://127.0.0.1:8080

Replace http://127.0.0.1:8080 with your model server’s address. Your app should keep sending its full conversation history; ctxkeeper does the trimming. The model server must accept requests without an API key (ctxkeeper does not forward your app’s key), so keep it on your own machine or a trusted network. The full guide covers every option, the status page and the tests.

Which model do I need?

• Tested most on Qwen3.8-27B (FP8). The one-card test used a smaller 26,624-token budget and CPU-offloaded embeddings; see the hardware settings.
• A quick check on Gemma 4 26B (Q8, llama.cpp) passed the demo 4 of 4. Other models should work if they call tools reliably and keep their notes in the expected format; try your own task first.
• Let the main model keep its own notes. A separate, smaller note-taking model was worse and slower (19 of 39 against 39 of 39 on a 240K-token test).

Connect your AI app

Choose your app for setup instructions.

Tested means local models over the Chat Completions API. Adapter and gateway routes are untested. The coding agents in the next table also ran through it.

Tested with 12 coding agents

Twelve open-source coding agents ran the bug-fixing task through ctxkeeper, each as it ships (its own history trimming off). One run each, an hour at most, sorted by time.

Out of 40 bugs and 7 questions. The lab’s own simple agent, for comparison: 40, 39, 7 in 21 min. pi wrote its 7 answers and then deleted them, because a wrong line in its notes said the questions had never been asked (see Limits). ForgeCode was still fixing bugs correctly when its hour ran out. Running the agents found four ways their requests differed from what ctxkeeper expected, such as sending two opening instructions instead of one; version 1.0.1 and later handle all four.

Settings

A recipe is a set of options that were measured together; any option you give still wins. Both recipes use a 32K working view, about the fastest size found:

Limits, stated

• The notes are written by the model, so they can be wrong. The originals stay archived and searchable, but nothing forces the model to look. In one agent test a wrong notes line said seven questions had never been asked, and the agent deleted answers it had already written.
• Silent arithmetic slips exist. In the two-million-token runs, two answers were mis-added in the model’s head while its notes held the right value. The built-in arithmetic check only catches wrong working it can see; a check for answers given with no working is in development.
• No login; state is plain text on the machine running it. Bind to localhost (the commands above do). The Docker default binds to all interfaces, so keep the explicit --host 127.0.0.1 if you change the run command.
• Your opening instructions are always kept, so they must fit well inside the working view (default 32,768 tokens).
• Tested on one model family most. Results on other models depend on how reliably they use tools and keep notes.

Evidence and source

• Source on GitHubCode, issues and the full guide.
• ctxkeeper on PyPIpip install ctxkeeper
• Release 1.2.0Package, source archive, program files, demo, SHA-256 sums. Public domain.
• Full guideEvery option, the status page, where state is kept, how to run the tests.
• Results and test detailsEvery comparison on this page, with settings and links to the original records.
• 2M hostile auditsHash-bound receipts behind the two-million-token result.
• The research trailEvery campaign that shaped the defaults, including what failed.