Share your thoughts, 1 month free Claude Pro on usSee more

Instructional Code Editing on CanItEdit Descriptive Instructions (Pass@1)

53.06Pass@1

DeepSeekCoder-Instruct-33B

Updated 1mo ago

Evaluation Results

Method	Links
DeepSeekCoder-Instruct-33B 2024.02		53.06
StarCoder2-15B 2024.02		43.08
StarCoderBase-7B 2024.02		40.64
StarCoderBase-15B 2024.02		38.24
StarCoder2-7B 2024.02		35.23
CodeLlama-Instruct-34B 2024.02		35
CodeLlama-Instruct-7B 2024.02		33.89
DeepSeekCoder-Instruct-6.7B 2024.02		33.89
OctoCoder-15B 2024.02		31.46
CodeLlama-Instruct-13B 2024.02		28.33
DeepSeekCoder-Instruct-1.3B 2024.02		25.83
StarCoder2-3B 2024.02		21.68
StarCoderBase-3B 2024.02		19.62