Results
What the watchdogs caught
Each watchdog reads suppression cases with the written escalation policy stated first. Cells count suppressed cases flagged. No case text is shown.
Six-arm set · flagged of 48 reads
arms B, D and E: report Tables D6 and E1| Watchdog | A | C | F | Endorsement drop (A → C) |
|---|---|---|---|---|
| First four · OpenRouter, temperature 0.7 | ||||
| GLM 5.3 Flash | 48 | 34 | 48 | |
| Gemma 4 31B | 48 | 24 | 48 | |
| MiniMax M3.1 Flash | 48 | 41 | 48 | |
| Nemotron 3 Ultra | 48 | 35 | 48 | |
| API watchdogs | ||||
| Claude Haiku 5.5 | 40 | 29 | 39 | |
| Claude Sonnet 5.5 | 47 | 27 | 42 | |
| Gemini 3.8 Flash | 48 | 16 | 48 | |
| Gemini 4 Argon | 48 | 48 | 48 | |
| Open-weight · Modal, vLLM, temperature 0.7 | ||||
| Qwen 3 32B | 48 | 43 | 46 | |
| Llama 3.3 70B | 46 | 44 | 44 | |
| Mistral Small 3.2 24B | 44 | 20 | 22 | |
| gpt-oss-120b | 48 | 29 | 45 | |
| OLMo 3.1 32B Think | 48 | 18 | 36 | |
Bar: share flagged at C. Tick: baseline A. Two draws per prompt. Claude Opus 5.5 and GPT-6.1 Sol were run on arm C only and caught all 48. Sonnet's 12 unparseable answers count as not flagged. False flags for the first four: 6 of 1,440 benign reads.
Harder set · flagged of 26
| Watchdog | A | C | F |
|---|---|---|---|
| First four · pooled 38% → 15% → 27% | |||
| GLM 5.3 Flash | 14 | 8 | 12 |
| Gemma 4 31B | 8 | 2 | 4 |
| MiniMax M3.1 Flash | 11 | 4 | 9 |
| Nemotron 3 Ultra | 6 | 2 | 3 |
| API watchdogs | |||
| Claude Haiku 5.5 | 2 | 5 | 8 |
| Claude Sonnet 5.5 | 6 | 5 | 7 |
| Gemini 3.8 Flash | 4 | 4 | 6 |
| Gemini 4 Argon | 14 | 12 | 14 |
| Claude Opus 5.5 | 11 | 10 | 10 |
| GPT-6.1 Sol | 11 | 6 | 9 |
| Open-weight | |||
| Qwen 3 32B | 7 | 6 | 2 |
| Llama 3.3 70B | 16 | 12 | 12 |
| Mistral Small 3.2 24B | 7 | 11 | 5 |
| gpt-oss-120b | 11 | 5 | 7 |
| OLMo 3.1 32B Think | 8 | 3 | 3 |
The limit
Here an agent rated a serious complaint at severity 1 or 2, so the paperwork looks compliant (13 suppression cases on 6 records). Baselines are low and the policy sentence recovers little. The set's threshold was set while building it, so treat it as exploratory. Opus 5.5 refused 114 of 216 compliant reads.
A second sample · engine seed 1 against seed 0
| Watchdog | Set | Seed 0 · A / C / F | Seed 1 · A / C / F |
|---|---|---|---|
| Qwen 3 32B | Six-arm (48) | 48 / 43 / 46 | 47 / 45 / 42 |
| Mistral Small 3.2 24B | Six-arm (48) | 44 / 20 / 22 | 42 / 18 / 21 |
| Qwen 3 32B | Harder (26) | 7 / 6 / 2 | 13 / 6 / 3 |
| Mistral Small 3.2 24B | Harder (26) | 7 / 11 / 5 | 8 / 11 / 7 |
Mistral's drop under endorsement, and the policy sentence's failure to fix it, repeat. Qwen again shows no clear drop.