Use case · Related prototype
Route between small and large models
Use the larger model when it adds value.
- Request or small-model result
- Estimate difficulty or risk
- Accept or use a larger model
An app could send a routine request to a small model and use a larger one for a difficult request. It could also try the small model first, then escalate when a learned check rejects its result.
Where the small model fits
A router predicts which model to use, or a confidence checker decides whether to accept a candidate answer. These are different policies. Separate models normally process the input separately; a shared encoder or intermediate handoff needs compatible, trained components.
What would need to work
Measure useful answers, routing mistakes and total time or cost including fallback. The lab's existing cascade returns moderation labels, not general chat replies, and does not demonstrate interchangeable hidden states between unrelated models.
Try the related work
The browser demo uses BERT classifiers. The recorded Qwen cascade is a separate experiment.