Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

CAI: An Open, Bug Bounty-Ready Cybersecurity AI

About

By 2028 most cybersecurity actions will be autonomous, with humans teleoperating. We present the first classification of autonomy levels in cybersecurity and introduce Cybersecurity AI (CAI), an open-source framework that democratizes advanced security testing through specialized AI agents. Through rigorous empirical evaluation, we demonstrate that CAI consistently outperforms state-of-the-art results in CTF benchmarks, solving challenges across diverse categories with significantly greater efficiency -up to 3,600x faster than humans in specific tasks and averaging 11x faster overall. CAI achieved first place among AI teams and secured a top-20 position worldwide in the "AI vs Human" CTF live Challenge, earning a monetary reward of $750. Based on our results, we argue against LLM-vendor claims about limited security capabilities. Beyond cybersecurity competitions, CAI demonstrates real-world effectiveness, reaching top-30 in Spain and top-500 worldwide on Hack The Box within a week, while dramatically reducing security testing costs by an average of 156x. Our framework transcends theoretical benchmarks by enabling non-professionals to discover significant security bugs (CVSS 4.3-7.5) at rates comparable to experts during bug bounty exercises. By combining modular agent design with seamless tool integration and human oversight (HITL), CAI addresses critical market gaps, offering organizations of all sizes access to AI-powered bug bounty security testing previously available only to well-resourced firms -thereby challenging the oligopolistic ecosystem currently dominated by major bug bounty platforms.

V\'ictor Mayoral-Vilches, Luis Javier Navarrete-Lozano, Mar\'ia Sanz-G\'omez, Lidia Salas Espejo, Marti\~no Crespo-\'Alvarez, Francisco Oca-Gonzalez, Francesco Balassone, Alfonso Glera-Pic\'on, Unai Ayucar-Carbajo, Jon Ander Ruiz-Alcalde, Stefan Rass, Martin Pinzger, Endika Gil-Uriarte• 2025

Related benchmarks

TaskDatasetResultRank
Capture The FlagCTF challenges rev category--
1
Capture The FlagCTF challenges misc category--
1
Capture The FlagCTF challenges pwn category--
1
Capture The FlagCTF challenges web category--
1
Capture The FlagCTF challenges crypto category--
1
Capture The FlagCTF challenges forensics category--
1
Capture The FlagCTF challenges robotics category--
1
Capture The FlagCTF challenges all categories--
1
Cybersecurity Challenge SolvingCTF Challenges Very Easy 2025--
1
Cybersecurity Challenge SolvingCTF Challenges Easy 2025--
1
Showing 10 of 13 rows

Other info

Follow for update