Share your thoughts, 1 month free Claude Pro on usSee more

Automated Theorem Proving

Benchmarks

Dataset Name	SOTA Method	Metric
MiniF2F (test)	Seed-Prover	Success Rate99.6	93	3mo ago
MUSTARDSAUCE	KG-Prover	Accuracy34	18	8d ago
miniF2F	KG-Prover	Accuracy31.97	18	8d ago
UniGeo Plane Geometry 1.0 (test)	DreamProver	Output Tokens (Thousands)0.12	12	1mo ago
CoqGym (test)	ASTactic + hammer	Success Rate30	9	3mo ago
FormalML-Hard (Machine Learning Theory) 1.0 (test)	DreamProver	Output Tokens (k)0.4	6	1mo ago
Olympiad Plane Geometry 1.0 (test)		Output Tokens (Thousands)0.66	6	1mo ago
HS Competition Plane Geometry 1.0 (test)	DreamProver	Thousands of Output Tokens0.16	6	1mo ago
Library Plane Geometry 1.0 (test)		Output Tokens (Thousands)0.08	6	1mo ago
seL4	Stepwise (Mistral)	Proof Success Rate77.6	6	2mo ago
seL4 hard (test)	Stepwise (Mistral)	Proof Success Rate69.8	6	2mo ago
seL4 (test)	Stepwise (Mistral)	Proof Success Rate89	6	2mo ago
seL4 (val)	Stepwise (Mistral)	Proof Success Rate79.8	6	2mo ago
Metamath (val)	700m policy+value a = 32	Performance56.5	6	3mo ago
FIMO Easy Mode	Goedel-Prover-V2	Solved Problems (Pass@32)4	5	1mo ago
miniF2F Easy Mode (test)	Goedel-Prover-V2	Solved Problems (Pass@32)215	5	1mo ago
seL4 proof corpus (full library)	Stepwise	Proof Lines Count6,235	5	2mo ago
FATE-X	Seed-Prover 1.5	Pass Rate33	5	2mo ago
FATE-M	AxProverBase	Pass Rate98	5	2mo ago
CombiBench Easy Mode	Goedel-Prover-V2	Solved Problems (Pass@32)10	4	1mo ago
FVELER hard (test)	FVEL-Llama-3-8B	Solved Proofs64	4	3mo ago
FVELER (test)	FVEL-Llama-3-8B	Solved Proofs88	4	3mo ago
CombiBench Hard Mode	DAP	Total Solved (Pass@32)10	3	1mo ago
PutnamBench Easy Mode	Goedel-Prover-V2	Solved Problems (Pass@32)43	3	1mo ago
HOList complex analysis corpus (val)	Subexpression sharing 12-hop GNN	Proofs Closed Rate49.95	3	3mo ago

Showing 25 of 31 rows