Our new X account is live! Follow @wizwand_team for updates

Code Generation on HumanEval+ (test)

81.7Pass@1

GPT4-Turbo

Updated 4d ago

Evaluation Results

Method
GPT4-Turbo 2023.12	81.7	-	-	-	-	-	-
MPSC-Lexical 2023.09	74.39	-	-	-	-	75	77.24
MPSC-Semantic 2023.09	73.54	-	-	-	-	74.46	75.26
MPSC-Label 2023.09	73.47	-	-	-	-	76.66	77.25
IF-CL 2023.12	71.3	-	-	-	-	-	-
GPT-4 2023.09	70.52	-	-	-	-	75.48	79.54
IF-CLP 2023.12	69.5	-	-	-	-	-	-
IF-CLP 2023.12	69.5	-	-	-	-	-	-
WeDLM-8B 2025.12	68.9	-	-	-	-	-	-
IF-CL 2023.12	68.3	-	-	-	-	-	-
CodeT 2023.09	67.87	-	-	-	-	68.75	69.65
MagicoderS-CLP 2023.12	66.5	-	-	-	-	-	-
GPT3.5-Turbo 2023.12	65.9	-	-	-	-	-	-
MPSC-Uniform 2023.09	65.05	-	-	-	-	69.76	71.72
WizardCoder-CLP 2023.12	64.6	-	-	-	-	-	-
WeDLM-7B 2025.12	64	-	-	-	-	-	-
Self-consistency 2023.09	63.5	-	-	-	-	64.7	65.67
Qwen3-8B 2025.12	63.4	-	-	-	-	-	-
MBR-EXEC 2023.09	62.12	-	-	-	-	67.08	71.38
GPT-3.5-Turbo 2023.09	58.75	-	-	-	-	66.58	73.96
Qwen2.5-7B 2025.12	53.05	-	-	-	-	-	-
LLaDA-1.5 2025.12	45.1	-	-	-	-	-	-
CODELLAMA-PYTHON 2023.12	42.7	-	-	-	-	-	-
ESPO 2025.12	42.7	-	-	-	-	-	-
LLaDA 2025.12	41.5	-	-	-	-	-	-
WizardCoder-CLP 2023.12	40.9	-	-	-	-	-	-
diffu-GRPO(d1) 2025.12	37.2	-	-	-	-	-	-
CODELLAMA-PYTHON 2023.12	36.6	-	-	-	-	-	-
ESPO 2025.12	36.6	-	-	-	-	-	-
CODELLAMA-PYTHON 2023.12	34.1	-	-	-	-	-	-
wd1 2025.12	32.9	-	-	-	-	-	-
LLaDA-1.5 2025.12	32.3	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	30.5	-	-	-	-	-	-
LLaDA 2025.12	30.5	-	-	-	-	-	-
Proxy (Neural) 2026.02	29.9	-	-	-	-	-	-
diffu-GRPO(d1) 2025.12	29.9	-	-	-	-	-	-
wd1 2025.12	29.9	-	-	-	-	-	-
wd1 2025.12	29.9	-	-	-	-	-	-
StarCoder 2023.12	29.3	-	-	-	-	-	-
Tokenizer-based 2026.02	29.3	-	-	-	-	-	-
Tokenizer-based 2026.02	28.7	-	-	-	-	-	-
LLaDA-8B 2025.12	28.05	-	-	-	-	-	-
Tokenizer-based 2026.02	28	-	-	-	-	-	-
CodeGen-Mono 2023.12	27.4	-	-	-	-	-	-
Proxy (Neural) 2026.02	26.8	-	-	-	-	-	-
CodeT5+ 2023.12	26.2	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	26.2	-	-	-	-	-	-
Byte-level 2026.02	24.4	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	24.4	-	-	-	-	-	-
ESPO 2025.12	24.4	-	-	-	-	-	-
Byte-level 2026.02	23.8	-	-	-	-	-	-
Mistral 2023.12	23.2	-	-	-	-	-	-
LLaDA 2025.12	23.2	-	-	-	-	-	-
LLaDA-1.5 2025.12	23.2	-	-	-	-	-	-
Proxy (Neural) 2026.02	22.6	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	22	-	-	-	-	-	-
Byte-level 2026.02	22	-	-	-	-	-	-
diffu-GRPO(d1) 2025.12	22	-	-	-	-	-	-
Tokenizer-based 2026.02	21.3	-	-	-	-	-	-
Proxy (Neural) 2026.02	21.3	-	-	-	-	-	-
CDLM 2025.12	21	-	-	-	39	-	-
StarCoder 2023.12	20.7	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	20.7	-	-	-	-	-	-
CDLM 2025.12	20	-	-	-	38	-	-
Dream-7B 2025.12	19.51	-	-	-	-	-	-
Tokenizer-based 2026.02	18.3	-	-	-	-	-	-
Byte-level 2026.02	18.3	-	-	-	-	-	-
Proxy (Neural) 2026.02	18.3	-	-	-	-	-	-
Tokenizer-based 2026.02	17.7	-	-	-	-	-	-
Tokenizer-based 2026.02	17.1	-	-	-	-	-	-
MDLM 2025.12	17	-	-	-	34	-	-
Base 2025.12	17	-	-	-	32	-	-
MDLM 2025.12	17	-	-	-	34	-	-
Base 2025.12	16	-	-	-	29	-	-
Byte-level 2026.02	15.9	-	-	-	-	-	-
Byte-level 2026.02	14.6	-	-	-	-	-	-
Proxy (Neural) 2026.02	14	-	-	-	-	-	-
Proxy (Neural) 2026.02	13.4	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	12.8	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	12.2	-	-	-	-	-	-
Byte-level 2026.02	9.1	-	-	-	-	-	-
Baseline 2024.11	-	66.5	-	-	-	-	-
SP 2024.11	-	69.5	3	-	-	-	-
MapCoder 2024.11	-	66.5	0	-	-	-	-
SD 2024.11	-	72	5.5	-	-	-	-
LDB 2024.11	-	72.6	6.1	-	-	-	-
LPW 2024.11	-	77.4	10.9	-	-	-	-
Baseline 2024.11	-	61	-	-	-	-	-
SP 2024.11	-	65.2	4.2	-	-	-	-
MapCoder 2024.11	-	73.8	12.8	-	-	-	-
SD 2024.11	-	68.3	7.3	-	-	-	-
LDB 2024.11	-	72	11	-	-	-	-
LPW 2024.11	-	76.2	15.2	-	-	-	-
Baseline 2024.11	-	32.3	-	-	-	-	-
SP 2024.11	-	34.8	2.5	-	-	-	-
SD 2024.11	-	45.7	13.4	-	-	-	-
LDB 2024.11	-	54.9	22.6	-	-	-	-
LPW 2024.11	-	62.8	30.5	-	-	-	-
LLaDA 2025.12	-	-	-	31.7	-	-	-
diffu-GRPO(d1) 2025.12	-	-	-	29.7	-	-	-

Showing 100 of 112 rows