SkinDeepRESEARCHSteve Seguin

Browser experiment

Try 100 real messages

Run Qwen locally and compare accuracy and time.

Results are measured when you run this demo. It does not reproduce the trained banking classifier or the early-stop studies.

Back to decisions without text

Compare direct SAFE/BLOCK scores, a one-token reply, and JSON. All three use Qwen 0.5B and all 24 layers.

Run downloads about 800 MB and processes messages locally. A 100-message run can take tens of minutes; choose 10 for a quick check.

Ready. No model or dataset has been downloaded.

0

Your results

The table fills as each message finishes. Incorrect and invalid replies count as errors. Toxic recall is the share of toxic messages blocked.

OutputCorrectTotal timeToxic recallSafe blockedInvalid

Which messages, and what does this show?

The ToxicChat0124 dataset contains real interactions with an AI chatbot. We use the same previously inspected, human-annotated 100-message subset as the local experiments: 50 toxic and 50 benign. Smaller runs use its fixed first 10 or 50 messages and can have a different balance.

These are exploratory checks, not a representative live chat queue. Qwen has not been fine-tuned for moderation here. The two output instructions can change the decisions, so quality parity is not assumed. Messages are bounded to the same first 256 Qwen tokens in all three paths; truncation is reported.

The dataset is downloaded from a pinned revision and checked with SHA-256. Its messages stay in memory on this device; the saved report contains row IDs, labels and outputs, not source message text. ToxicChat is CC BY-NC 4.0.

What work is actually skipped?

Direct mode calls the model once and compares two scores from its final vocabulary output. It does not generate or decode answer tokens. The standard vocabulary projection still runs: this is not a new trained classifier or a replacement numerical head.

JSON mode performs a greedy text generation with a 16-token limit. Invalid JSON is retained and counted wrong. The one-token control uses the exact direct-mode prompt and restricts generation to SAFE or BLOCK. It checks whether direct reading improves on an already minimal reply.

All modes process the full 24-layer network; none uses the early-exit classifiers from our other tests. Per-format time includes input tokenization, truncation, prompt preparation, inference and output reading. Dataset download, model loading and one warm-up per format are shown separately. Execution order rotates between messages; one timed pass is exploratory and can be affected by browser load.

Every output and timing
Message IDFormatExpectedOutputCheckGenerated tokensTime

Method and limits · Worker source · Dataset IDs and scoring · Original subset protocol