Code Generation on HumanEval and MBPP (Aggregated)

85.6Overall Average Score

GPT-4-1106-preview

Updated 4d ago

Evaluation Results

Method
GPT-4-1106-preview 2024.05	85.6	-	-	-	-	-	77.5
GPT-3.5-Turbo-1106 2024.05	79.7	-	-	-	-	-	70.2
WizardLM(filtered) 2025.12	78.9	77.4	-	80.4	-	-	-
DataFlow-SFT-15K(filtered) 2025.12	78.9	82.9	-	74.9	-	-	-
WizardLM(random) 2025.12	78.8	75.6	-	82	-	-	-
DataFlow-SFT-15K(random) 2025.12	77.9	79.9	-	75.9	-	-	-
Alpaca(filtered) 2025.12	74.8	73.8	-	75.7	-	-	-
Alpaca(random) 2025.12	73.6	71.3	-	75.9	-	-	-
MAmmoTH2-7B-Plus 2024.05	66.1	-	-	-	-	-	58.2
Llama-3-8B-Instruct 2024.05	65.8	-	-	-	-	-	58
MAmmoTH2-8x7B-Plus 2024.05	63.3	-	-	-	-	-	55.3
MAmmoTH2-8B-Plus 2024.05	61.9	-	-	-	-	-	53.3
DeRS-LM⁺ Coder-MoE 2025.03	60.9	65.9	62.2	63.4	51.9	-	-
DeRS-SM⁺ Coder-MoE 2025.03	60.3	64.6	60.4	63.7	52.4	-	-
Vanilla Coder-MoE 2025.03	60.2	64.6	61	63.9	51.4	-	-
C3A 2024.07	58.7	59.5	53.8	65.4	56.1	-	-
DoRA 2024.07	58.4	59.1	52.9	65.7	55.8	-	-
LoRA 2024.07	57.6	57.9	52.4	64.8	55.3	-	-
VeRA 2024.07	56.2	56.3	50.8	64	53.6	-	-
Mixtral-8x7B-Instruct-v0.1 2024.05	52.3	-	-	-	-	-	44.7
Tulu-2-DPO-70B 2024.05	51.2	-	-	-	-	-	43
Zero-Shot 2024.07	44	33.5	29.3	61.4	51.6	-	-
Mistral-7B-Instruct-v0.2 2024.05	43.4	-	-	-	-	-	36.5
Yi-34B-Chat 2024.05	38.7	-	-	-	-	-	32.6
C3A 2024.07	32.8	33.8	29	36.5	31.9	-	-
DoRA 2024.07	32.7	33.2	28.8	36.8	31.8	-	-
LoRA 2024.07	31.4	31.7	28	35.4	30.4	-	-
Llama-2-70b-chat 2024.05	31.4	-	-	-	-	-	26.5
VeRA 2024.07	29.5	28.9	24.6	35.1	29.6	-	-
Zero-Shot 2024.07	18.8	11	9.8	30.2	24.1	-	-
Chat 2025.02	-	7.9	-	16.5	-	-	-
Math 2025.02	-	15.9	-	28.8	-	-	-
Code 2025.02	-	10.4	-	21.3	-	-	-
Task Arithmetic 2025.02	-	9.1	-	24.1	-	-	-
Sens-Merging (Task Arithmetic) 2025.02	-	20.1	-	42.6	-	-	-
Ties-Merging 2025.02	-	21.3	-	40.4	-	-	-
Sens-Merging (Ties-Merging) 2025.02	-	20.6	-	41.3	-	-	-
DARE 2025.02	-	20.1	-	39.1	-	-	-
Sens-Merging (DARE) 2025.02	-	20.7	-	39.3	-	-	-