Share your thoughts, 1 month free Claude Pro on usSee more

Code Completion on APC Hard Completion 1.0 (test)

100HCR

Qwen2.5-Coder-1.5B-BASE

Updated 2mo ago

Evaluation Results

Method	Links
Qwen2.5-Coder-1.5B-BASE 2026.04		100	41.88	77.36
Qwen2.5-Coder-1.5B-SFT 2026.04		100	44.55	74.96
Qwen2.5-Coder-1.5B-GRPO 2026.04		100	41.02	70.9
Qwen2.5-Coder-3B-BASE 2026.04		100	46.74	83.19
Qwen2.5-Coder-3B-SFT 2026.04		100	48.57	82.73
Qwen2.5-Coder-3B-GRPO 2026.04		100	46.49	77.97
Qwen2.5-Coder-7B-BASE 2026.04		100	24.29	54.68
Qwen2.5-Coder-7B-SFT 2026.04		100	49.24	83.27
Qwen2.5-Coder-7B-GRPO 2026.04		100	51.08	86.07
Qwen2.5-Coder-14B-BASE 2026.04		100	30.35	57.77
Qwen2.5-Coder-14B-SFT 2026.04		100	51.32	84.21
Qwen2.5-Coder-14B-GRPO 2026.04		100	47.04	79.75
CodeGemma-2B-BASE 2026.04		100	43.24	80.56
CodeGemma-2B-SFT 2026.04		100	32.67	40.31
CodeGemma-2B-GRPO 2026.04		100	43.1	74.81
CodeGemma-7B-BASE 2026.04		100	40.51	77.7
CodeGemma-7B-SFT 2026.04		100	34.21	66.94
CodeGemma-7B-GRPO 2026.04		100	40.1	74.98
CodeLlama-7B-BASE 2026.04		100	11.54	42.78
CodeLlama-7B-SFT 2026.04		100	37.1	70.14
CodeLlama-7B-GRPO 2026.04		100	36.83	70.48
CodeLlama-13B-BASE 2026.04		100	17.07	48.99
CodeLlama-13B-SFT 2026.04		100	31.18	63.76
CodeLlama-13B-GRPO 2026.04		100	35.72	68.49
StarCoder2-3B-BASE 2026.04		100	38.46	75.47
StarCoder2-3B-SFT 2026.04		100	42.2	77.15
StarCoder2-3B-GRPO 2026.04		100	43.4	79.23
StarCoder2-7B-BASE 2026.04		100	42.9	79.78
StarCoder2-7B-SFT 2026.04		100	45.67	81.65
StarCoder2-7B-GRPO 2026.04		100	38.98	74.16
StarCoder2-15B-BASE 2026.04		100	40.53	78.43
StarCoder2-15B-SFT 2026.04		100	44.04	79.51
StarCoder2-15B-GRPO 2026.04		100	44.05	78.69