Share your thoughts, 1 month free Claude Pro on usSee more

Code Generation on NaturalCodeBench

55.7NCB (zh) Python Score

Seed-Coder-8B-Instruct

Updated 1mo ago

Evaluation Results

Method	Links
Seed-Coder-8B-Instruct 2026.01		55.7	45.7	50.7	50	47.1	48.6	49.6
Seed-Diffusion-Preview(0705) 2026.01		52.9	38.6	45.8	45.7	38.6	42.2	43.9
Stable-DiffCoder-8B-Instruct 2026.01		51.4	45.7	48.6	50	47.1	48.6	48.6
Qwen2.5-Coder-14B-Instruct 2026.01		48.6	48.6	48.6	42.9	45.7	44.3	46.4
StarCoder2-15B-Instruct 2026.01		44.3	30	37.2	38.6	42.9	40.8	39
DeepSeek-Coder-33B-Instruct 2026.01		44.3	38.9	41.6	44.3	44.3	44.3	43
Yi-Coder-9B-Chat 2026.01		41.4	45.7	43.6	38.6	44.3	41.5	42.5
DeepSeek-Coder-V2-Lite-Instruct 2026.01		41.4	47.1	44.3	41.4	37.1	39.3	41.8
OpenCoder-8B-Instruct 2026.01		40	30	35	35.7	24.3	30	32.5
Codestral-22B 2026.01		40	44.3	42.2	41.4	45.7	43.6	42.9
DeepSeek-Coder-6.7B-Instruct 2026.01		38.6	31.4	35	32.9	32.9	32.9	33.9
Qwen3-8B 2026.01		37.1	32.9	35	34.3	38.6	36.5	35.7
CodeLlama-70B-Instruct 2026.01		35.1	32.1	33.6	32.8	30.5	31.7	32.6
Qwen2.5-Coder-7B-Instruct 2026.01		34.3	37.1	35.7	34.3	35.7	35	35.4
CodeQwen1.5-7B-Chat 2026.01		30	28.6	29.3	30	27.1	28.6	25.7
Llama-3.1-8B-Instruct 2026.01		27.1	24.3	25.7	22.9	22.9	22.9	24.3
CodeLlama-7B-Instruct 2026.01		18.6	8.6	13.6	17.1	14.3	15.7	14.6