Code Completion

Benchmarks

Dataset Name	SOTA Method	Metric
APC Placeholder Completion 1.0 (test)	Qwen2.5-Coder-7B-GRPO	PCR68.26	33	15d ago
APC Hard Completion 1.0 (test)	Qwen2.5-Coder-1.5B-BASE	HCR100	33	15d ago
MBPP+	PrivCode	Pass@165.6	33	1mo ago
HumanEval+	NonDPFT	Pass@156.7	33	1mo ago
Python Code Completion downstream		Exact Similarity (ES)57.7	21	1mo ago
BigCodeBench Hard	NonDPFT	Pass@116.2	20	1mo ago
BigCodeBench Full	NonDPFT	Pass@146.1	20	1mo ago
HumanEval	NonDPFT	Pass@10.652	20	1mo ago
BigCodeBench	Qwen2.5-Coder-7B	Full Score45.8	17	1mo ago
MBPP Sanitized	OpenCoder-8B-Base	MBPP Pass Rate79.9	17	1mo ago
HumanEval	OpenCoder-8B-Base	HE Score66.5	17	1mo ago
Java code dataset	CoProtector	False Positive Rate (FPR)7	16	4d ago
C code dataset	CoProtector	FPR7	16	4d ago
MBPP (test)		3-Shot Pass Rate69.4	16	1mo ago
VerilogEval CC v2 (test)	DeepSeek-R1-671B	Pass@179.1	11	1mo ago
RepoBench-P	CoA	Similarity0.7305	10	1mo ago
DS-1000 (test)	DeepSeek-Coder-Base	Matplotlib Success Rate56.1	8	1mo ago
py150 OOD WILDS (test)	CORAL + SDG	MethodClass Accuracy68.3	7	1mo ago
HumanEval FIM	LLaDA 8B	Accuracy (HumanEval FIM)74	5	1mo ago
DS-1000 1.0 (test)	WizardCoder	Success Rate (Matplotlib)55.2	5	1mo ago
CrossCodeEval Project-level (test)	CoCoGen	C-EM9.08	4	1mo ago
RepoBench	LAVA	Pass@k Score48.92	1	17d ago

Showing 22 of 22 rows