Software Engineering

Benchmarks

Dataset Name	SOTA Method	Metric
SWE-bench Verified		Success Rate80.9	53	1mo ago
SWE-bench Verified	JoyAI-LLM Flash	Accuracy62.6	43	16d ago
SWE-bench Lite	Draft-OPD	Speedup4.66	36	1mo ago
SWE-bench verified (All)	ACE	Success Rate93.8	32	2mo ago
SWE-bench Verified	MemCoder	Resolution Rate83.8	32	1mo ago
SWE Lite	Draft-OPD	Throughput (tok/s)10,538	30	1mo ago
Commit0-Lite	STORM-Combined	Score88.2	24	2mo ago
SWE-Bench Lite	CoT-SC	Accuracy57.09	23	1mo ago
SWE-bench Verified		Pass@172	23	18d ago
SWE-Bench Verified	HyperAgent + Librarian	Pass Rate72	20	1mo ago
SWE-Bench Multilingual 1.0 (test)		Resolution Rate75.2	20	4mo ago
SWE-Bench Pro (public)		Resolve Rate (Pass@1)64.3	19	1mo ago
SWE-Bench Verified	Mini-SWE	Pass@184	18	3mo ago
SWE Verified		Resolution Rate77.2	17	4mo ago
SWE-bench		Resolve Rate82.4	16	2mo ago
PaperBench Code (dev)	STORM-Combined	Score78.2	15	2mo ago
SWE-bench Multilingual	SE-agent-Reflect	Pass@121	14	2mo ago
SWE-Bench Pro 1.0 (test)		Resolved Rate51.6	14	4mo ago
SWE-Bench-Verified (50 cases)		Accuracy72	12	2mo ago
SWE-Bench Verified		Resolution Rate (%)86.2	10	2mo ago
SWE-rebench 9B	EvoTrainer	Avg@8 BC%38.16	9	1mo ago
SWE-rebench 4B	EvoTrainer	Avg@8 BC%31.49	9	1mo ago
PaperBench	Claude Sonnet 4.5	Score66.8	9	4mo ago
SWE-bench Lite (300 instances)		Misalignment Rate0	8	1mo ago
SWE-rebench January 2026 (test)		Resolved Rate52.9	8	4mo ago

Showing 25 of 64 rows