Share your thoughts, 1 month free Claude Pro on usSee more

Code Generation on MultiPL-E (Language Scores)

76.5Average Score

Qwen2.5-Coder-7B-Instruct

Updated 1mo ago

Evaluation Results

Method
Qwen2.5-Coder-7B-Instruct 2024.11	76.5	87.8	76.5	75.6	80.3	81.8	83.2	78.3	48.7
DS-Coder-V2-Lite-Instruct 2024.11	73.2	81.1	76.6	75.8	76.6	80.5	77.6	74.5	43
Yi-Coder-9B-Chat 2024.11	71.8	85.4	76	67.7	76.6	72.3	78.9	72.1	45.6
CodeQwen1.5-7B-Chat 2024.11	71.6	83.5	70.9	72	75.9	76.7	77.6	73.9	41.8
Stable-DiffCoder-8B-Base 2026.01	71.2	80.5	74.1	69.4	70.3	74.8	73.1	74.4	53.2
OpenCoder-8B-Instruct 2024.11	71	83.5	72.2	61.5	75.9	78	79.5	73.3	44.3
Seed-Coder-8B-Base 2026.01	67.6	77.4	72.8	69.6	53.8	77.4	77.6	63.9	48.1
DS-Coder-6.7B-Instruct 2024.11	66.1	78.6	68.4	63.4	72.8	67.2	72.7	68.9	36.7
Qwen2.5-Coder-14B 2026.01	62.3	83.5	46.8	69.6	63.3	69.2	61.5	64.6	39.9
OpenCoder-8B-Base 2026.01	61	66.5	63.9	63.4	54.3	68.6	65.8	61.5	44.3
Qwen2.5-Coder-7B 2026.01	58.8	72	53.2	62.1	60.8	64.2	60.3	59	38.6
OpenCoder-1.5B-Instruct 2024.11	57.5	72.5	64.6	50.9	61.4	63.5	62.1	55.3	29.7
Qwen2.5-Coder-1.5B-Instruct 2024.11	56.7	71.2	55.7	50.9	64.6	61	62.1	59	29.1
Yi-Coder-1.5B-Chat 2024.11	51.9	67.7	51.9	49.1	57.6	57.9	59.6	52.2	19
DeepSeek-Coder-33B-Base 2026.01	50.3	56.1	51.9	58.4	51.3	52.8	55.3	44.1	32.3
DS-Coder-1.3B-Instruct 2024.11	48.4	65.2	51.9	45.3	55.1	59.7	52.2	45.3	12.7
CodeQwen1.5-7B 2026.01	48.4	51.8	42.4	52.2	55.7	52.2	49.7	46.6	36.7
CodeLlama-70B-Base 2026.01	48.2	52.4	44.7	49.7	46.7	52.2	56.5	46.6	31.6
CodeGemma-7B-It 2024.11	46.5	59.8	48.1	46.6	51.9	54.7	54	46.6	10.1
Llama-3.1-70B-Base 2026.01	46.5	54.9	41.1	41	44.2	57.9	55.3	48.4	29.1
DeepSeek-Coder-6.7B-Base 2026.01	44.7	49.4	43	50.3	50	49.7	48.4	38.5	28.5
StarCoder2-15B 2026.01	41.7	46.3	46.2	47.2	53.2	42.1	43.5	39.1	15.8
DeepSeek-Coder-V2-Lite-Base 2026.01	40.4	40.9	34.8	45.9	41.7	48.4	44.7	47.2	19.6
StarCoder2-7B 2026.01	34.3	35.4	38	40.4	46.2	34	36	30.4	13.9
CodeLlama-7B-Instruct 2024.11	33	45.7	32.2	28.6	32.9	39	43.5	31.7	10.1
GRIP 16B 2026.02	25.5	-	-	-	-	-	-	-	-
GRIP 8B 2026.02	23.7	-	-	-	-	-	-	-	-
6. GRIP (Full) 2026.02	19.2	-	-	-	-	-	-	-	-
4. + Loss Replay 2026.02	16.5	-	-	-	-	-	-	-	-
5. + Diversity 2026.02	16	-	-	-	-	-	-	-	-
Random 16B 2026.02	14.8	-	-	-	-	-	-	-	-
3. + Static Replay 2026.02	14.2	-	-	-	-	-	-	-	-
Random 8B 2026.02	13.5	-	-	-	-	-	-	-	-
2. + Static Budget 2026.02	13.1	-	-	-	-	-	-	-	-
1. Random 2026.02	11.3	-	-	-	-	-	-	-	-