Share your thoughts, 1 month free Claude Pro on usSee more

Code Infilling on HumanEval single-line infilling (test)

87.4Pass Rate

code-davinci-002

Updated 1mo ago

Evaluation Results

Method	Links
code-davinci-002 2022.04		87.4	69.6
code-davinci-002 2022.04		71.8	52
INCODER-6.7B 2022.04		69	56.3
code-davinci-002 2022.04		63.7	48.4
INCODER-6.7B 2022.04		54.9	44.1
INCODER-6.7B 2022.04		48.2	38.7
CM infilling 2022.04		0.69	0.563
code-davinci-001 2022.04		0.63	0.56
L-R reranking 2022.04		0.549	0.441
code-cushman-001 2022.04		0.531	0.42
L-R single 2022.04		0.482	0.387
PLBART 2022.04		0.416	-