SOTA Parallel Code Generation on ParEval 1.0 (full) and PapersWithCode

98Build@1

Gemini-2.5-Pro

Updated 1mo ago

Evaluation Results

Method
Gemini-2.5-Pro 2026.03	98	77	10.4
Qwen2.5-Coder-32B 2026.03	98	35	15.61
Qwen2.5-Coder-32B 2026.03	97	39	12.86
StarCoder2-15B 2026.03	97	26	12.32
Gemini-3-Pro 2026.03	97	82	7.42
DeepSeek-Coder-V2-Lite-Base 2026.03	96	39	16.06
StarCoder2-15B 2026.03	95	30	11.7
Gemini-2.5-Parlay 2026.03	94	71	23.84
Qwen2.5-Coder-32B 2026.03	93	11	9.98
DeepSeek-Coder-V2-Lite-Base 2026.03	92	14	6.79
Qwen2.5-Coder-32B-Instruct 2026.03	92	65	14.43
Qwen2.5-Coder-32B-Instruct 2026.03	91	65	13.36
Qwen3-Coder-30B-Instruct 2026.03	91	56	15.17
DeepSeek-6.7B-Base 2026.03	89	11	3.65
Qwen2.5-Coder-32B 2026.03	86	17	15.85
Qwen3-Coder-30B-Instruct 2026.03	86	55	16.53
Qwen2.5-Coder-32B 2026.03	86	38	4.48
DeepSeek-Syntax 2026.03	85	12	6.6
DS-Coder-V2-Lite-Base 2026.03	85	25	5.21
Gemini-2.5-Parlay (ParEVO) 2026.03	84	33	106.87
DeepSeek-Coder-V2-Lite-Base 2026.03	82	24	8.86
Qwen2.5-Coder-32B 2026.03	82	45	5.64
StarCoder2-15B 2026.03	82	25	5.66
StarCoder2-15B 2026.03	81	15	37.75
DeepSeek-Coder-V2-Lite-Base 2026.03	80	9	2.57
StarCoder2-15B 2026.03	80	27	20.2
DeepSeek-Parlay (ParEVO) 2026.03	79	35	16.4
Gemini-3-Pro 2026.03	78	72	23.13
StarCoder2-15B 2026.03	77	27	3.58
GPT-5 Thinking 2026.03	73	63	14.03
DS-Coder-V2-Lite-Base 2026.03	73	29	6.26
Qwen2.5-Coder-32B-Instruct 2026.03	70	48	6.55
Qwen3-Coder-30B-Instruct 2026.03	66	49	5.64
Qwen3-Rust (ParEVO) 2026.03	64	46	6.1
Qwen2.5-Coder-32B-Instruct 2026.03	63	49	5.97
Qwen2.5-Coder-32B-Instruct 2026.03	61	41	12.91
Qwen3-Coder-30B-Instruct 2026.03	61	50	5.7
Gemini-2.5-Flash 2026.03	58	29	13.42
Qwen3-Coder-30B-Instruct 2026.03	51	28	8.61
Qwen3-Parlay (ParEVO) 2026.03	50	33	8.63
DS-Coder-V2-Lite-Instruct 2026.03	48	2	0.76
DS-Coder-V2-Lite-Instruct 2026.03	40	2	0.77
Claude Opus 4.5 2026.03	28	27	0.65
Gemini-3-Pro 2026.03	25	23	12.29