Early tests
Decisions without a written reply
Sometimes an app only needs a label, such as “support” or “spam”.
A trained classifier reads the model's internal representation and returns one of its learned categories. Our strongest test uses fixed banking topics. A separate changing-rule test failed; following arbitrary instructions remains unproven.
A confidence score can decide whether to keep processing. Confidence can be wrong; no stopping rule has passed our reliability checks yet.
On a public banking-query test, a trained Qwen classifier got 2,551 of 3,080 requests right across 77 topics. A simple word-based classifier got 2,464 right.
Technical details and sources
Latest classifier and specialist comparisons · Maze follow-up: training and legal moves.
600 real chat decisions: results and measured runtime · Independent review · Watch recorded maze decisions.
Second-dataset validation · Testing an unknown-request output · Evidence requirements.
Separate implementation replay and portable classifiers · Changing-rule test: the current classifier failed · Adversarial reassessment.
Data, classifier comparisons and per-query results. This test uses fixed banking intents; arbitrary changing rules remain unvalidated.
Classifier implementation and original evidence · Run a local Qwen output-format comparison
Keep the context. Change the output.
A readout position after the input can attend to the instructions and relevant context. A learned classifier maps its hidden state to the task’s labels. The application receives numerical scores or an enum directly.
An API can serialize that result as JSON without making the model generate JSON tokens. The useful comparison is with minimal constrained label output, not a deliberately verbose baseline.
Answer and control flow are separate
| Control state | Meaning |
|---|---|
| RESOLVED | A reliable task result is ready. |
| IGNORE | Confidently no action is needed. |
| CONTINUE | Run more of the same model. |
| ESCALATE | Use another model, tool, or reviewer. |
An early head’s uncertainty does not mean the full model cannot solve the input. A confident head may also be wrong. Exit policies need calibration and independent tests.
The first pilot
The local pilot uses a pinned Qwen2.5-0.5B-Instruct model and synthetic topic policies. It compares constrained one-token decisions, a final-layer head, intermediate heads, and a runtime that can actually stop executing layers.
This small controlled test does not establish moderation quality or arbitrary instruction following. All measured outcomes, including weak results, belong on the results page.