Share your thoughts, 1 month free Claude Pro on usSee more

Code Generation on ClassEval

27Pass@1 (Class-level)

DeepSeek-Coder-6.7B-Instruct

Updated 1mo ago

Evaluation Results

Method	Links
DeepSeek-Coder-6.7B-Instruct 2024.10		27	57.2
CodeQwen1.5-7B-Chat 2024.10		27	54.6
SelfCodeAlign-CQ-7B 2024.10		27	52.6
Llama-3-8B-Instruct 2024.10		23	52.4
CodeQwen1.5-7B-Base 2024.10		23	52.8
CodeGemma-7B-IT 2024.10		21	44.8
CodeQwen1.5-7B-OctoPack 2024.10		21	45.2
OctoCoder-16B 2024.10		19	38
StarCoder2-15B 2024.10		9	24.9