ExploitGym Benchmark Tests If AI Agents Can Build Real Exploits

Researchers from UC Berkeley, the Max Planck Institute, Anthropic, OpenAI, and Google have released ExploitGym, a benchmark measuring whether AI agents can turn known software bugs into working exploits. Across 869 real-world vulnerabilities in userspace programs, Chrome's V8 engine, and the Linux kernel, agents are given a crash-triggering input and tasked with achieving unauthorized code execution.

Standard mitigations like ASLR, stack canaries, and V8's heap sandbox reduce success rates but don't eliminate them; agents found bypasses including partial-pointer overwrites, sandbox escapes, and kernel tricks like overwriting modprobe_path. Notably, agents frequently abandoned the intended bug entirely, auditing source code or fuzzing to discover alternative exploitable flaws — a sign of genuine autonomous reasoning rather than scripted replay.

Different frontier models solved largely non-overlapping sets of targets, and extended time budgets kept improving results for the strongest agents with no clear plateau. In one striking case, GPT-5.4 turned a five-line crash into a full V8 exploit chain in 71 minutes — though it failed once ASLR and the sandbox were re-enabled. The authors argue defenders must now model AI agents as active adversaries rather than relying solely on traditional mitigations.