CAI Dataset: The Largest Corpus of LLM-Driven Hacker Trajectories

CAI Dataset is presented as a fourteen-month corpus of cybersecurity LLM interactions gathered through the open-source CAI agent framework. The work responds directly to PentestGPT's finding that the real bottleneck for cybersecurity LLM performance isn't base-model capability but the scarcity of expert operator trajectories. The result is a massive aggregation: 230,935 session logs and over 26 million user prompts from 16,768 source IPs across 123 countries, spanning 4,187 unique LLM identifiers and 23,147 target domains, stored across more than 18 terabytes.

The content skews heavily hands-on, with roughly a third offensive, a fifth attacker-intent, about a quarter business/integration-focused, and a small defensive slice. It's released in tiered sizes (10, 1k, 200k samples) to partner organizations and select customers, and the authors describe it as the largest known corpus of LLM-driven hacker trajectories to date.

The more consequential finding is systemic: operators routinely paste live credentials, production hostnames, and bearer tokens into prompts, fully aware their inputs are being logged, accepting that exposure to stay competitive. Aggregated industry-wide, this concentrates a huge share of the world's offensive and defensive operator knowledge inside a handful of frontier-model API providers, creating a single failure surface whose breach or politically motivated misuse could cascade into nation- and enterprise-scale disruption. The authors argue the only setup that preserves both productivity gains and confidentiality is an on-premise, privately hosted, cybersecurity-specialized LLM running within the operator's own trust boundary — something CAI Dataset is explicitly shaped to make feasible.