Building a Model-Agnostic Vulnerability Discovery Harness

In the ongoing Project Glasswing effort, the team shares architectural lessons from using frontier AI models to hunt vulnerabilities across an enterprise codebase. Their core argument: systems built around a single model or single-agent sessions fail because context windows fill up and erase earlier findings, crashes force full restarts, and isolated repo sessions can't see cross-repo dependencies. Their fix is a harness that treats models as interchangeable parts, deliberately swapping models between discovery and validation stages.

The team started with a roughly 450-line security-audit skill run as a single seven-phase session—recon, per-class attack hunting, adversarial validation, reporting, schema checking, and independent re-verification. That skill alone caught only about half the bugs a multi-run process would find; the real gain came from externalizing state so the LLM acts as a stateless compute engine, solving context exhaustion, crash recovery, and cross-repo reasoning. Over six weeks, this skill evolved into a fleet scanner covering 128 repositories with no per-language tuning, automatically tracing dependencies between them.

The resulting architecture rests on a two-stage framework: the Vulnerability Discovery Harness (VDH) and the Vulnerability Validation System (VVS). The key takeaway for engineers is designing the harness to be model-agnostic from day one—building persistent, resumable, deduplicated infrastructure that reduces thousands of raw findings into a trusted, triaged queue, regardless of which frontier model currently leads.