Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Automated Vulnerability Repair on AIxCC CP Dataset 1.0 (final round)
Loading...
91.3
Overall Success Rate
PatchIsland
14.444
34.397
54.35
74.303
Jan 24, 2026
Overall Success Rate
Overall Fix Ratio (Count)
Success Rate (commons-compress, java)
Success Rate (poi, java)
Success Rate (curl, c)
Success Rate (dav1d, c)
Success Rate (freerdp, c)
Success Rate (lcms, c)
Success Rate (libavif, c)
Success Rate (libexif, c)
Success Rate (libxml2, c)
Success Rate (log4j2, java)
Success Rate (mongoose, c)
Success Rate (pdfbox, java)
Success Rate (shadowsocks, c)
Success Rate (systemd, c)
Success Rate (tika, java)
Success Rate (wireshark, c)
Success Rate (xz, c)
Overall Success Rate (Total Summary)
Updated 5mo ago
Evaluation Results
Method
Method
Links
Overall Success Rate
Overall Fix Ratio (Count)
Success Rate (commons-compress, java)
Success Rate (poi, java)
Success Rate (curl, c)
Success Rate (dav1d, c)
Success Rate (freerdp, c)
Success Rate (lcms, c)
Success Rate (libavif, c)
Success Rate (libexif, c)
Success Rate (libxml2, c)
Success Rate (log4j2, java)
Success Rate (mongoose, c)
Success Rate (pdfbox, java)
Success Rate (shadowsocks, c)
Success Rate (systemd, c)
Success Rate (tika, java)
Success Rate (wireshark, c)
Success Rate (xz, c)
Overall Success Rate (Total Summary)
PatchIsland
LLMs=GPT o4-mini, Clau...
2026.01
91.3
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Buttercup
configuration=Default...
2026.01
62
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
RoboDuck
configuration=Default...
2026.01
17.4
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Feedback
Search any
task
Search any
task