Share your thoughts, 1 month free Claude Pro on usSee more

Vulnerability Detection on Vulnerability Detection Benchmark

60.27Pass@1

DeepSeek-V3.1

Updated 3mo ago

Evaluation Results

Method	Links
DeepSeek-V3.1 2026.02		60.27	71.27	26.09	82.47	39.64	22.5	71.95	3.59	1.95
GPT-OSS-120B 2026.02		59.77	70.51	26.36	79.46	39.58	22.16	71.03	4.2	2.61
gemini-2.5-flash 2026.02		58.9	70.1	22.97	81.64	35.85	19.66	75.18	3.31	1.85
Qwen3-4B-GRPO 2026.02		58.1	72.17	24.1	75.3	36.51	19.28	71.4	4.71	4.62
DeepSeek-R1-0528 2026.02		57.97	71.42	25.03	73.36	37.32	21.55	69.37	3.48	5.61
o4-mini 2026.02		57.45	66.09	18.19	84.69	29.94	15.98	80.74	2.21	1.08
Qwen3-235B-A22B 2026.02		56.64	68.17	18.46	78.06	29.86	16.26	78.55	2.21	2.98
Qwen3-4B-DPO 2026.02		56.41	70.41	19.84	73.9	31.28	16.94	76.05	2.89	4.11
Qwen3-4B-SFT 2026.02		55.99	67.77	17.34	76.4	28.26	14.95	79.7	2.39	2.97
Qwen3-4B-ORPO 2026.02		55.31	68.73	18.07	70.81	28.79	15.13	77.42	2.94	4.51
GPT-OSS-20B 2026.02		55.24	63.35	15.88	74.64	26.18	12.98	81.62	2.89	2.5
DeepSeek-R1-0528-Qwen3-8B 2026.02		53.97	66.55	13.35	71.18	22.48	11.78	82.82	1.57	3.83
Qwen3-4B-MARCO 2026.02		53.27	67.21	13.78	65.54	22.77	12.04	80.71	1.74	5.51
Qwen3-4B 2026.02		51.85	61.68	8.63	63.61	15.2	7.45	87.61	1.18	3.76
VulnLLM-R-7B 2026.02		50.76	61.42	6.83	56.22	12.18	5.08	89.61	1.75	3.57
Qwen3-4B-R2VUL 2026.02		48.7	64.31	7.17	42.32	12.26	5.36	84.87	1.81	7.96
R2VUL-7B 2026.02		47.11	59.7	9.1	37.94	14.68	3.78	81.33	5.32	9.57
Qwen3-4B-ReVD 2026.02		44.91	64.16	7.01	28.96	11.28	4.63	78.19	2.37	14.81