Share your thoughts, 1 month free Claude Pro on usSee more

Code Generation on HumanEval+ (test)

84.76Pass@1

ACES-O

Updated 11d ago

Evaluation Results

Method
ACES-O 2026.04	84.76	-	-	-	-	84.76	86.59	-	-
Majority Voting 2026.04	84.15	-	-	-	-	84.76	87.2	-	-
ACES-C 2026.04	84.15	-	-	-	-	84.15	86.59	-	-
GPT4-Turbo 2023.12	81.7	-	-	-	-	-	-	-	-
ACES-O 2026.04	81.1	-	-	-	-	82.93	83.54	-	-
ACES-C 2026.04	79.88	-	-	-	-	82.32	82.93	-	-
Zero-shot 2026.04	79.4	-	-	-	-	82.97	85.96	-	-
Majority Voting 2026.04	78.66	-	-	-	-	82.32	82.93	-	-
ACES-C 2026.04	78.66	-	-	-	-	78.66	79.27	-	-
ACES-O 2026.04	76.22	-	-	-	-	76.22	78.66	-	-
Zero-shot 2026.04	75.63	-	-	-	-	81.38	85.62	-	-
MPSC-Lexical 2023.09	74.39	-	-	-	-	75	77.24	-	-
Majority Voting 2026.04	74.39	-	-	-	-	75.61	78.05	-	-
MPSC-Semantic 2023.09	73.54	-	-	-	-	74.46	75.26	-	-
MPSC-Label 2023.09	73.47	-	-	-	-	76.66	77.25	-	-
IF-CL 2023.12	71.3	-	-	-	-	-	-	-	-
Zero-shot 2026.04	71.05	-	-	-	-	75.3	79.03	-	-
GPT-4 2023.09	70.52	-	-	-	-	75.48	79.54	-	-
IF-CLP 2023.12	69.5	-	-	-	-	-	-	-	-
IF-CLP 2023.12	69.5	-	-	-	-	-	-	-	-
WeDLM-8B 2025.12	68.9	-	-	-	-	-	-	-	-
IF-CL 2023.12	68.3	-	-	-	-	-	-	-	-
CodeT 2023.09	67.87	-	-	-	-	68.75	69.65	-	-
MagicoderS-CLP 2023.12	66.5	-	-	-	-	-	-	-	-
GPT3.5-Turbo 2023.12	65.9	-	-	-	-	-	-	-	-
MPSC-Uniform 2023.09	65.05	-	-	-	-	69.76	71.72	-	-
WizardCoder-CLP 2023.12	64.6	-	-	-	-	-	-	-	-
WeDLM-7B 2025.12	64	-	-	-	-	-	-	-	-
Self-consistency 2023.09	63.5	-	-	-	-	64.7	65.67	-	-
Qwen3-8B 2025.12	63.4	-	-	-	-	-	-	-	-
MBR-EXEC 2023.09	62.12	-	-	-	-	67.08	71.38	-	-
GPT-3.5-Turbo 2023.09	58.75	-	-	-	-	66.58	73.96	-	-
Qwen2.5-7B 2025.12	53.05	-	-	-	-	-	-	-	-
LLaDA-1.5 2025.12	45.1	-	-	-	-	-	-	-	-
CODELLAMA-PYTHON 2023.12	42.7	-	-	-	-	-	-	-	-
ESPO 2025.12	42.7	-	-	-	-	-	-	-	-
LLaDA 2025.12	41.5	-	-	-	-	-	-	-	-
WizardCoder-CLP 2023.12	40.9	-	-	-	-	-	-	-	-
diffu-GRPO(d1) 2025.12	37.2	-	-	-	-	-	-	-	-
CODELLAMA-PYTHON 2023.12	36.6	-	-	-	-	-	-	-	-
ESPO 2025.12	36.6	-	-	-	-	-	-	-	-
CODELLAMA-PYTHON 2023.12	34.1	-	-	-	-	-	-	-	-
wd1 2025.12	32.9	-	-	-	-	-	-	-	-
LLaDA-1.5 2025.12	32.3	-	-	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	30.5	-	-	-	-	-	-	-	-
LLaDA 2025.12	30.5	-	-	-	-	-	-	-	-
Proxy (Neural) 2026.02	29.9	-	-	-	-	-	-	-	-
diffu-GRPO(d1) 2025.12	29.9	-	-	-	-	-	-	-	-
wd1 2025.12	29.9	-	-	-	-	-	-	-	-
wd1 2025.12	29.9	-	-	-	-	-	-	-	-
StarCoder 2023.12	29.3	-	-	-	-	-	-	-	-
Tokenizer-based 2026.02	29.3	-	-	-	-	-	-	-	-
Tokenizer-based 2026.02	28.7	-	-	-	-	-	-	-	-
LLaDA-8B 2025.12	28.05	-	-	-	-	-	-	-	-
Tokenizer-based 2026.02	28	-	-	-	-	-	-	-	-
CodeGen-Mono 2023.12	27.4	-	-	-	-	-	-	-	-
Proxy (Neural) 2026.02	26.8	-	-	-	-	-	-	-	-
CodeT5+ 2023.12	26.2	-	-	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	26.2	-	-	-	-	-	-	-	-
Byte-level 2026.02	24.4	-	-	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	24.4	-	-	-	-	-	-	-	-
ESPO 2025.12	24.4	-	-	-	-	-	-	-	-
Byte-level 2026.02	23.8	-	-	-	-	-	-	-	-
Mistral 2023.12	23.2	-	-	-	-	-	-	-	-
LLaDA 2025.12	23.2	-	-	-	-	-	-	-	-
LLaDA-1.5 2025.12	23.2	-	-	-	-	-	-	-	-
Proxy (Neural) 2026.02	22.6	-	-	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	22	-	-	-	-	-	-	-	-
Byte-level 2026.02	22	-	-	-	-	-	-	-	-
diffu-GRPO(d1) 2025.12	22	-	-	-	-	-	-	-	-
Tokenizer-based 2026.02	21.3	-	-	-	-	-	-	-	-
Proxy (Neural) 2026.02	21.3	-	-	-	-	-	-	-	-
CDLM 2025.12	21	-	-	-	39	-	-	-	-
StarCoder 2023.12	20.7	-	-	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	20.7	-	-	-	-	-	-	-	-
CDLM 2025.12	20	-	-	-	38	-	-	-	-
Dream-7B 2025.12	19.51	-	-	-	-	-	-	-	-
Tokenizer-based 2026.02	18.3	-	-	-	-	-	-	-	-
Byte-level 2026.02	18.3	-	-	-	-	-	-	-	-
Proxy (Neural) 2026.02	18.3	-	-	-	-	-	-	-	-
Tokenizer-based 2026.02	17.7	-	-	-	-	-	-	-	-
Tokenizer-based 2026.02	17.1	-	-	-	-	-	-	-	-
MDLM 2025.12	17	-	-	-	34	-	-	-	-
Base 2025.12	17	-	-	-	32	-	-	-	-
MDLM 2025.12	17	-	-	-	34	-	-	-	-
Base 2025.12	16	-	-	-	29	-	-	-	-
Byte-level 2026.02	15.9	-	-	-	-	-	-	-	-
Byte-level 2026.02	14.6	-	-	-	-	-	-	-	-
Proxy (Neural) 2026.02	14	-	-	-	-	-	-	-	-
Proxy (Neural) 2026.02	13.4	-	-	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	12.8	-	-	-	-	-	-	-	-
Proxy (Tokenizer) 2026.02	12.2	-	-	-	-	-	-	-	-
Byte-level 2026.02	9.1	-	-	-	-	-	-	-	-
Baseline 2024.11	-	66.5	-	-	-	-	-	-	-
SP 2024.11	-	69.5	3	-	-	-	-	-	-
MapCoder 2024.11	-	66.5	0	-	-	-	-	-	-
SD 2024.11	-	72	5.5	-	-	-	-	-	-
LDB 2024.11	-	72.6	6.1	-	-	-	-	-	-
LPW 2024.11	-	77.4	10.9	-	-	-	-	-	-
Baseline 2024.11	-	61	-	-	-	-	-	-	-

Showing 100 of 134 rows