Use it: the ctxkeeper proxy

What it does

Think of a desk and a filing cabinet. The model works at a small desk: the opening instructions, a page of notes and the newest messages. Everything else goes into the cabinet, unchanged. Three things keep the desk useful:

1. Notes, kept up to date. Every so often the model writes down only what changed: a new rule, a changed figure (with the working), a new open question. ctxkeeper checks each line before accepting it; a sum is recomputed, a rule can never be dropped.
2. Search. The model gets one extra tool, recall, that searches the cabinet. ctxkeeper answers it itself; your agent never sees that step.
3. Rules stay in force. Standing rules are repeated next to every new instruction, so a rule given hours earlier still applies.

Nothing about the model or the server changes. Anything that can be pointed at an OpenAI-compatible address can use it. Follow the keeper step by step.

Does it help?

Measured with Qwen3.8-27B on two Intel Arc Pro B70 graphics cards, with a working view of 32K tokens. Each row is a real run, graded by the task’s own checker.

One model on one machine. The repair task in detail · The million-token runs · The two-million-token stream.

Try it

You need Python 3.10 or newer (or Docker) and a model server that speaks the OpenAI API: vLLM, llama.cpp’s server, SGLang, Ollama’s OpenAI endpoint and others. ctxkeeper uses Python’s standard library only.

R=https://github.com/steveseguin/b70-optimization-lab/releases/download/ctxkeeper-v1.0.0
pip install $R/ctxkeeper-1.0.0-py3-none-any.whl
# or with Docker: docker build -t ctxkeeper $R/ctxkeeper-1.0.0.tar.gz

ctxkeeper --upstream http://127.0.0.1:8000 --recipe agent

Give --upstream your server’s address without /v1. Then point your agent or app at http://127.0.0.1:18200/v1 instead of the server. To see it work, run the demo: it holds a conversation several working views long, says a codename, a budget and a standing rule at the start, and asks about all three at the end.

curl -LO $R/long_chat.py
python3 long_chat.py --base-url http://127.0.0.1:18200/v1

Use it with your agent or app

It works with anything that takes an OpenAI-compatible base URL, if the program:

• uses Chat Completions (/v1/chat/completions); other requests pass straight through, unmanaged;
• sends its whole history each time, with its own trimming or summarizing turned off;
• asks for one reply at a time. Streaming works, but the reply arrives in one piece at the end.

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:18200/v1", api_key="unused")
reply = client.chat.completions.create(model="your-model", messages=history)

If new instructions arrive inside tool results, such as tickets, emails or files, tell ctxkeeper what they look like so it repeats the rules after each one: --recite-when '(?m)^TICKET \d+'. The bug-fixing task did not need it: its tickets arrive that way, and the rules held from the notes alone.

Settings

A recipe is a set of options that were measured together; any option you give still wins.

The Qwen recipe writes its notes without thinking and keeps the prompt identical, so the server’s cache does most of the note-taking work. Both use a 32K working view. On the repair task 24K did as well (13 minutes); at 16K one rule slipped once. A bigger view is slower, because the model reads more on every turn.

Tested before release

Version 1.0.0 was installed the way you would install it and run as its instructions say:

Limits

• Measured on one model family and one machine. Other models should work if they use tools reliably and can keep the notes format; that has not been measured.
• The notes are written by the model, so they can be wrong. On the hardest test about 1 notes line in 130 was silently wrong; on the million-token test, 1 in 1,435. The originals stay searchable, but nothing forces the model to look.
• No login, and everything is stored as plain text on the machine running it. It listens only on the local machine unless you tell it otherwise.
• The opening instructions are always kept, so they must fit well inside the working view.