CAI: An Open, Bug Bounty-Ready Cybersecurity AI
About
By 2028 most cybersecurity actions will be autonomous, with humans teleoperating. We present the first classification of autonomy levels in cybersecurity and introduce Cybersecurity AI (CAI), an open-source framework that democratizes advanced security testing through specialized AI agents. Through rigorous empirical evaluation, we demonstrate that CAI consistently outperforms state-of-the-art results in CTF benchmarks, solving challenges across diverse categories with significantly greater efficiency -up to 3,600x faster than humans in specific tasks and averaging 11x faster overall. CAI achieved first place among AI teams and secured a top-20 position worldwide in the "AI vs Human" CTF live Challenge, earning a monetary reward of $750. Based on our results, we argue against LLM-vendor claims about limited security capabilities. Beyond cybersecurity competitions, CAI demonstrates real-world effectiveness, reaching top-30 in Spain and top-500 worldwide on Hack The Box within a week, while dramatically reducing security testing costs by an average of 156x. Our framework transcends theoretical benchmarks by enabling non-professionals to discover significant security bugs (CVSS 4.3-7.5) at rates comparable to experts during bug bounty exercises. By combining modular agent design with seamless tool integration and human oversight (HITL), CAI addresses critical market gaps, offering organizations of all sizes access to AI-powered bug bounty security testing previously available only to well-resourced firms -thereby challenging the oligopolistic ecosystem currently dominated by major bug bounty platforms.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Capture The Flag | CTF challenges rev category | -- | 1 | |
| Capture The Flag | CTF challenges misc category | -- | 1 | |
| Capture The Flag | CTF challenges pwn category | -- | 1 | |
| Capture The Flag | CTF challenges web category | -- | 1 | |
| Capture The Flag | CTF challenges crypto category | -- | 1 | |
| Capture The Flag | CTF challenges forensics category | -- | 1 | |
| Capture The Flag | CTF challenges robotics category | -- | 1 | |
| Capture The Flag | CTF challenges all categories | -- | 1 | |
| Cybersecurity Challenge Solving | CTF Challenges Very Easy 2025 | -- | 1 | |
| Cybersecurity Challenge Solving | CTF Challenges Easy 2025 | -- | 1 |