AI Jailbreak Benchmark Reveals 100x Safety Gap Between Models
New benchmark shows up to 100x safety gaps among frontier AI models against jailbreak attacks; some models yield zero jailbreaks.
Researchers introduced the Minimal Standard for Safeguards v1.0, a new benchmark measuring how well frontier AI models resist jailbreak attacks. Building a taxonomy of 67 publicly known jailbreak techniques and combining them into a large attack space, they tested Claude Fable 5, GPT-5.6 Sol, Gemini 3.1 Pro, and Grok 4.5 against 360 attacker goals spanning CBRNE (chemical, biological, radiological/nuclear, explosive) threats and offensive cyber use cases.
Results show stark inconsistency across developers. Random search uncovered 63 universal jailbreaks (single prompts working on over 75% of a domain's goals) against Grok 4.5 and 18 against Gemini 3.1 Pro, at average costs of roughly $58 and $278 respectively; expert-guided attacks pushed these figures to 385 and 231. By contrast, neither Claude Fable 5 nor GPT-5.6 Sol produced a single universal jailbreak under either strategy.
The team also introduced a cost-to-jailbreak metric modeling attacker spend directly, with lower-bound estimates when no working jailbreak was found. Since successful attacks rely only on techniques already publicly documented and deployed elsewhere, the authors argue these safety gaps are closable with existing methods, and recommend defense-in-depth combining reasoning-layer, activation-layer, and input/output monitoring safeguards.