Research FAQ

The demo learns patterns in your ratings and uses them to score or adjust drawings. Its calculations and simple made-up preferences have been tested; we have not yet shown that people prefer its suggestions.

How learning works · Preference tests

Not consistently in our tests. The demo’s mix of likely likes, uncertain examples and random examples performed worse with noisy ratings than random sampling; selecting uncertain examples did better on the simple test rules.

Actual demo sampler: comparison and learning curves

A single linear model struggles with separated tastes. A nonlinear feature model helped in a synthetic test; keeping separate models for explicitly named contexts also helped in a small alternating-context test. Neither demonstrates learning a person’s changing moods.

More than one taste · Context and retention tests

No. A model can confidently prefer the wrong thing. We tested selection from the same candidate pool and retained cases where maximizing the score disagrees with the known preference; a person’s new rating is still the real check.

Candidate selection and failure cases

No. A small classifier can read its internal numbers and return a label directly. Returning a number instead of one trained token did not, by itself, show a reliable speed advantage.

Decisions without text · Matched output test · Try real moderation messages in your browser · Try the small trained classifier · Search documents locally · Compare a real shared model on your device

A shared four-layer BERT model can answer after layer 2. On 500 unused messages, its frozen rule preserved every full-depth decision while proposing 400 early exits, or 40% fewer transformer blocks. Actual browser execution also works. Only 18 new test messages were toxic, so rare-error reliability remains uncertain.

500-message fresh validation · Run the shared model yourself · Compare early, middle and late layers · Earlier Qwen stopping results

On 100 tested messages, reusing the fixed instructions cut time by 37.9% with identical decisions. Processing messages in groups of four cut queue time by 9.7%. Both still use all 24 layers; these checks do not establish identical behavior on every possible input.

Reuse the instructions · Process a queue together

The original pilot roughly halved time, but introduced a toxic-message miss. On 100 new messages, a stricter fallback matched Qwen’s decisions while avoiding Qwen only twice; its measured time saving was less than 1%.

How the cascade works · Its exact test results · New messages and stricter fallback

Not reliably in the tests so far: opposite rules exposed failures even at full depth. We have not tested these decision classifiers on live conversations with audio or video context.

Rule changes and unfamiliar requests

Yes, a pointer model can return a location without writing coordinates as text. It found 25/30 targets in the first study, but only 8/25 on a harder new benchmark. A confidence cutoff withheld all 25 infeasible requests there, while also withholding 21 valid ones and still allowing two wrong clicks.

Harder screenshots and infeasible requests · Click-target results · Inspect real screenshot results · Run a small image-to-action model

A separate small grid model completed 7 of 10 mazes after more training and a rule against wall moves. That is not a completed GUI task: the screenshot tests return points and do not execute clicks.

Maze outcomes · Recorded click locations

Saving text to files can let a long task use a short working context. It does not give a GPU unlimited active memory. This model needs about 12.2 GiB of attention cache for 200,000 tokens, plus other working state. Parking an inactive numerical cache and streaming it during generation are different designs; neither has a completed benchmark in this research snapshot.

Disk, GPU memory and the calculations · The measured file-using task

The revised self-editing agent processed a 121K-token ledger within a 32K working budget. It got 24/24 in 15 minutes on one seed and 21/24 in 21 minutes on another. The three misses came from a bug in its update script, with every batch retained. The 480K-stream result is still pending. Keeping the data available does not by itself ensure correct computation.

Self-editing and summaries · Task comparison

Prefix caching cuts repeated reading: cached 200K questions started in about 2 seconds, versus 114 seconds in a separate cold probe. Standard-gate decode stayed near 89 tokens/s with or without caching. Longer active context still slowed generation; the recall probes ranged from about 127 tokens/s at 8K to 27 at 250K. Recall was 708/720 across the test, including 120/120 at 60K. See the tables for sample sizes and differences between the runs.

Caching versus decoding · Speed and recall tables

A notes file is stored outside the active conversation and must be loaded when needed. A CLM mirror edit changes the message history sent to the model on the next call. On the 121K-token ledger, both file-using agents got 24/24 in about 1.9 minutes while holding under 9K active tokens. That is one task result, not a guarantee that every detail was archived or that every long document will be equally fast.

Files, state and retrieval · How the editable mirror works

The measured cleanup rules did not: conservative deduplication and formatting cleanup removed 1.83% of roughly five million tokens. Moving large outputs to files was a separate 7.50% reduction, with a proxy indicating some text would need retrieval later. A trained CPU pruning classifier was not benchmarked. Removing earlier reasoning blindly left a separate model run stuck without an answer.

What the census measured · More context questions

Explicit bounds can prevent a known forbidden edit; new conflict cases check that mechanism. A high preference score is not a safety guarantee, and personalized instruction-following has not been validated.

Constraint-conflict checks

The browser demo keeps its ratings on your device. That does not prove an encoded image hides private information; we have not run a reconstruction or privacy-attack study.

Local preference demo

Not in the current experiments. Generator transfer, music, feeds, mutual preferences and engineering designs are proposed applications; validating them needs task data, appropriate models and, where relevant, participating users.

Original project and current scope

A separate implementation replay reproduced saved decisions and layer stopping, including a published runtime version. We have CPU timings and a full-depth browser demo, but no matched GPU/NPU comparison or evidence that the trained classifiers transfer to another model.

Reproduction checks · Try Qwen on real messages