Exploratory results
One model, two chances to answer
Compute the beginning once; continue only when needed.
Compare accuracy and time on real messages. This demo uses a four-layer BERT model that can stop after layer 2; the Qwen tests below are separate.
Back to decisions without text
The first two layers do the shared work. A small classifier checks the result. Easy cases can finish there; other cases continue through layers three and four, using the same internal state.
| Method | Correct / 100 | Average layers |
|---|---|---|
| Four layers, final answer only | 80 | 4 |
| Train answers at layers 2 and 4; use all four | 83 | 4 |
| Same jointly trained model; allow early answers | 83 | 2.56 |
72 of 100 messages stopped after layer 2. The adaptive route preserved all 100 decisions of its own four-layer model. That skips 36% of its transformer-block executions on this sample. It does not establish equal quality to Qwen, which scored 86/100 on these messages.
Actual execution on the first 50 messages took 491 ms at full depth and 344 ms with the check, averaged over three passes. That was 30.1% less time.
Training and limits
Google BERT small has four layers and about 11.1 million parameters here. Both classifier heads and the shared encoder were trained on 1,398 labeled messages. Separate development data selected the epoch, thresholds and stopping rule. The 100 evaluation messages had already been inspected in earlier experiments.
This is one jointly trained backbone with multiple exits, not two unrelated pretrained models fused together. The comparison changes the training objective as well as the exit rule. One seed and one small consumed test cannot establish a general quality improvement.