Code Reasoning

Benchmarks

Dataset Name	SOTA Method	Metric
LiveCodeBench		Accuracy87.4	62	25d ago
CRUXEval		Input-CoT Accuracy98.8	56	1mo ago
HumanEval	DeepSeek-R1-Distill-Qwen-14B (Reasoning)	HumanEval Score95.73	40	19d ago
HumanE	Denser	Accuracy84.9	35	1mo ago
MBPP		MBPP Execution Accuracy84.7	33	1mo ago
CRUXEval-O	Kimi-K2 Base	Accuracy83.5	28	1mo ago
MBPP	Denser	Accuracy67.3	23	1mo ago
CRUX	RMoA	Accuracy87.37	23	1mo ago
CRUXEval	Qwen2.5-Math-72B-Instruct	Accuracy68.6	21	1mo ago
LiveCodeBench 1.0 (test)	A3PO	Accuracy47.2	18	1mo ago
CruxEval Output	DataFlow-Code-10K	Score51	12	1mo ago
LiveCodeBench	REBALANCE	Pass@1 Accuracy88.3	11	8d ago
LiveCodeBench (LCB) v6	INTUITOR	Accuracy15.3	8	1mo ago
LCB	SCF-RKL	pass@162.46	8	1mo ago
CodeForces	LAD	Rating1,533.64	6	1mo ago
HumanEval+	LAD	Average Score @1682.29	6	1mo ago
LiveCodeBench	LAD	Avg@1633.51	6	1mo ago
CRUX-O	HSA-UL	Accuracy40.75	6	1mo ago
LiveCodeBenchPro Med 25Q2	Nemotron-Cascade-2 30B-A3B	Pass@136.8	5	29d ago
LiveCodeBenchPro Easy 25Q2	Nemotron-Cascade-2 30B-A3B	Pass@189.3	5	29d ago
LiveCodeBench 2408-2505 v6	Nemotron-Cascade-2 30B-A3B	Pass@188.4	5	29d ago
MBPP base and extended (out-of-distribution)	InftyThink+	Accuracy55.83	5	1mo ago
HumanEval base and extended (out-of-distribution)	InftyThink+	Accuracy0.677	5	1mo ago
SciCode	Nemotron-3-Super 120B-A12B	Pass@142.1	4	29d ago
CRUXEval I	Kimi-K2 Base	Accuracy74	4	1mo ago

Showing 25 of 28 rows