Use case · Related prototype
Live chat moderation
Return an allow, block or review decision.
- Chat message
- Small decision head
- Allow, block or review
A chat application needs a decision for each incoming message, usually without a written explanation. A review route can handle messages that the model cannot classify reliably.
Where the small model fits
A trained head reads a text model's internal representation. A separate optimization could try a smaller model first or check an earlier layer, continuing when the stopping rule does not accept the result.
What would need to work
The lab has ToxicChat experiments and browser demos. They do not establish reliable context-aware live moderation, including abuse split across users or messages. Count missed harmful messages and false blocks separately.
Try the related work
The browser demo compares direct vocabulary scores with generated output; the trained-head and stopping studies use different models or weights.