Share your thoughts, 1 month free Claude Pro on usSee more

Code Repair on HumanEvalFix (test)

59.1Success Rate (Python)

WaveCoder-Pro-6.7B

Updated 1mo ago

Evaluation Results

Method	Links
WaveCoder-Pro-6.7B 2023.12		59.1	56.7	54.2	45.1	45.7	34.1	49.2
WaveCoder-Ultra-6.7B 2023.12		58.5	57.3	61	53	50	37.2	52.8
WaveCoder-DS-6.7B 2023.12		57.9	52.4	57.3	47.5	45.1	36	49.4
Deepseek-instruct-6.7B 2023.12		56.1	58.5	57.3	49.4	45.1	36.6	50.5
Magicoder-S-DS 2023.12		56.1	55.4	58.5	51.2	45.7	35.3	50.3
DeepseekCoder-CodeAlpaca-6.7B 2023.12		49.4	51.8	45.1	48.8	44.5	31.7	45.2
WaveCoder-CL-13B 2023.12		48.8	48.2	50.6	51.8	45.1	40.2	47.4
GPT-4 2023.12		47	48.2	50	50.6	47.6	43.3	47.8
CodeLLaMa-CodeAlpaca-13B 2023.12		42.7	43.9	50	45.7	39.6	37.2	43.2
Magicoder-DS 2023.12		42	43.3	50.6	41.4	38.4	29.2	40.8
WaveCoder-CL-7B 2023.12		41.4	41.4	42	47.1	42.7	34.7	41.5
WaveCoder-SC-15B 2023.12		39.3	35.1	34.8	36.2	30.2	22.5	33
CodeLLaMa-CodeAlpaca-7B 2023.12		37.8	39	42	37.8	37.2	29.2	37.1
WizardCoder 2023.12		31.8	29.5	30.7	30.4	18.7	13	25.7
OctoCoder 2023.12		30.4	28.4	30.6	30.2	26.1	16.5	27
DeepseekCoder-6.7B 2023.12		29.9	29.2	39	29.2	25	21.9	29
CodeLLaMa-instruct-13B 2023.12		29.2	19.5	32.3	24.4	12.8	0.1	19.7
CodeLLaMa-instruct-7B 2023.12		28	23.2	23.2	18.3	0.1	0.1	15.5
StarCoder 2023.12		8.7	15.7	13.3	20.1	15.6	6.7	13.4