Share your thoughts, 1 month free Claude Pro on usSee more

Code Repair on HumanEvalFix (test)

59.1Success Rate (Python)

WaveCoder-Pro-6.7B

Updated 1mo ago

Evaluation Results

Method	Links
WaveCoder-Pro-6.7B 2023.12		59.1	56.7	54.2	45.1	45.7	34.1	49.2	-	-
WaveCoder-Ultra-6.7B 2023.12		58.5	57.3	61	53	50	37.2	52.8	-	-
WaveCoder-DS-6.7B 2023.12		57.9	52.4	57.3	47.5	45.1	36	49.4	-	-
Deepseek-instruct-6.7B 2023.12		56.1	58.5	57.3	49.4	45.1	36.6	50.5	-	-
Magicoder-S-DS 2023.12		56.1	55.4	58.5	51.2	45.7	35.3	50.3	-	-
DeepseekCoder-CodeAlpaca-6.7B 2023.12		49.4	51.8	45.1	48.8	44.5	31.7	45.2	-	-
WaveCoder-CL-13B 2023.12		48.8	48.2	50.6	51.8	45.1	40.2	47.4	-	-
GPT-4 2023.12		47	48.2	50	50.6	47.6	43.3	47.8	-	-
CodeLLaMa-CodeAlpaca-13B 2023.12		42.7	43.9	50	45.7	39.6	37.2	43.2	-	-
Magicoder-DS 2023.12		42	43.3	50.6	41.4	38.4	29.2	40.8	-	-
WaveCoder-CL-7B 2023.12		41.4	41.4	42	47.1	42.7	34.7	41.5	-	-
WaveCoder-SC-15B 2023.12		39.3	35.1	34.8	36.2	30.2	22.5	33	-	-
CodeLLaMa-CodeAlpaca-7B 2023.12		37.8	39	42	37.8	37.2	29.2	37.1	-	-
WizardCoder 2023.12		31.8	29.5	30.7	30.4	18.7	13	25.7	-	-
OctoCoder 2023.12		30.4	28.4	30.6	30.2	26.1	16.5	27	-	-
DeepseekCoder-6.7B 2023.12		29.9	29.2	39	29.2	25	21.9	29	-	-
CodeLLaMa-instruct-13B 2023.12		29.2	19.5	32.3	24.4	12.8	0.1	19.7	-	-
CodeLLaMa-instruct-7B 2023.12		28	23.2	23.2	18.3	0.1	0.1	15.5	-	-
StarCoder 2023.12		8.7	15.7	13.3	20.1	15.6	6.7	13.4	-	-
Qwen2.5-72B + CoT 2026.06		-	-	-	-	-	-	-	61.4	1,842
Qwen2.5-72B + CoT-SC 2026.06		-	-	-	-	-	-	-	66.8	29,472
Llama-3.1-70B + CoT 2026.06		-	-	-	-	-	-	-	57.3	1,913
Llama-3.1-70B + ToT 2026.06		-	-	-	-	-	-	-	60.1	9,561
InternLM-StepProver + CoT 2026.06		-	-	-	-	-	-	-	55.8	1,987
InternLM-StepProver + CoT-SC 2026.06		-	-	-	-	-	-	-	60.3	15,896
TRI (SFT only) 2026.06		-	-	-	-	-	-	-	68.3	1,534
TRI (SFT + DPO) 2026.06		-	-	-	-	-	-	-	73.1	1,267
TRI (Full) 2026.06		-	-	-	-	-	-	-	74.9	1,268