Share your thoughts, 1 month free Claude Pro on usSee more

Software Vulnerability Detection on OSV-Bench 1.0 (test)

97.18Pointer Errors (Pass@1)

Claude Opus 4.6

Updated 8d ago

Evaluation Results

Method	Links
Claude Opus 4.6 2026.04		97.18	95.54	94.59	98.15	96.3	97.96	96.73
Claude Sonnet 4 2026.04		77.46	66.96	75.68	79.63	70.37	85.71	75.1
Claude Haiku 4.5 2026.04		76.06	66.07	70.27	75.93	72.22	83.67	76.33
Devstral 2 2026.04		74.65	72.32	75.68	85.19	77.78	85.71	80
Claude Opus 4.6 2026.04		70.42	72.32	60.81	62.96	69.44	79.59	74.69
Nova Premier 2026.04		56.34	53.57	52.7	37.04	57.41	63.27	58.78
Claude Sonnet 4 2026.04		54.93	51.79	52.7	46.3	55.56	75.51	61.22
Qwen3 2026.04		54.93	37.5	44.59	38.89	47.22	55.1	49.8
Llama 4 Maverick 2026.04		52.11	48.21	39.19	46.3	57.41	59.18	55.1
Doubao-1.5-pro 2026.04		50.7	48.21	45.95	40.74	52.78	63.27	55.1
DeepSeek-V3.2 2026.04		47.89	53.57	51.35	59.26	52.78	63.27	56.73
Devstral 2 2026.04		46.48	41.96	47.3	42.59	48.15	51.02	48.16
Qwen3 2026.04		43.66	29.46	32.43	20.37	33.33	44.9	38.78
Claude Sonnet 3.5 2026.04		39.44	41.96	39.19	48.15	39.81	46.94	44.9
Claude Haiku 4.5 2026.04		39.44	41.96	41.89	42.59	43.52	59.18	50.2
Nova Premier 2026.04		39.44	35.71	37.84	27.78	40.74	48.98	43.67
DeepSeek-Chat 2026.04		38.02	39.29	36.49	44.44	43.52	51.02	46.53
DeepSeek-R1† 2026.04		35.21	33.93	27.03	44.44	37.04	53.06	40.41
Llama 4 Maverick 2026.04		33.8	33.04	25.68	35.19	40.74	42.86	38.78
DeepSeek-V3.2 2026.04		30.99	36.61	31.08	29.63	34.26	38.78	38.78
DeepSeek-R1† 2026.04		21.13	22.32	18.92	29.63	28.7	24.49	26.53