Share your thoughts, 1 month free Claude Pro on usSee more

Code-to-text on HumanEvalExplain (test)

64.6Python Acc

GPT-4

Updated 1mo ago

Evaluation Results

Method	Links
GPT-4 2023.12		64.6	57.3	51.2	58.5	38.4	42.7	52.1
Deepseek-instruct-6.7B 2023.12		62.2	54.3	61	39.6	55.5	33.5	51
Magicoder-S-DS 2023.12		60.3	46.3	54.3	38.4	48.1	29.2	46.1
WaveCoder-Ultra-6.7B 2023.12		56.7	50	54.3	34.8	51.2	36.6	47.3
Magicoder-DS 2023.12		55.5	36.6	49.4	36	39.6	27.4	40.7
WaveCoder-Pro-6.7B 2023.12		53	43.3	54.9	34.1	42.7	20	41.3
WaveCoder-DS-6.7B 2023.12		48.2	47.5	49.4	32.3	48.2	22	41.3
WaveCoder-CL-13B 2023.12		45.7	42	48.2	32.3	38.4	20.7	37.9
DeepseekCoder-6.7B 2023.12		43.9	40.2	37.8	29.2	34.1	22.5	34.6
WaveCoder-CL-7B 2023.12		41.4	31.7	39	25	34.1	23.2	32.4
Deepseek-CodeAlpaca-6.7B 2023.12		40.8	37.2	42.1	29.9	31.7	22.5	34
CodeLLaMa-instruct-13B 2023.12		40.2	26.8	37.2	22.5	28	14.6	28.2
WaveCoder-SC-15B 2023.12		37.1	33.3	40.5	23.3	31.8	19.3	30.8
OctoCoder 2023.12		35.1	24.5	27.3	21.1	24.1	14.8	24.5
CodeLLaMa-CodeAlpaca-7B 2023.12		34.7	24.4	37.8	23.2	28.6	19.5	28
CodeLLaMa-instruct-7B 2023.12		33.5	36	31.7	21.3	25	16.4	27.3
WizardCoder 2023.12		32.5	33	27.4	26.7	28.2	16.9	27.5
CodeLLaMa-CodeAlpaca-13B 2023.12		32.3	28	34.1	18.9	29.9	20.7	27.3
StarCoder 2023.12		0	0	0	0	0	0	0