Symbolic Reasoning

Benchmarks

Dataset Name	SOTA Method	Metric
Letter	Meta-Reasoning Paradigm	Accuracy92.4	67	25d ago
Last Letter Concatenation	Zero-Shot CoT	Accuracy90.4	58	25d ago
Coin	Meta-Reasoning Paradigm	Accuracy100	45	25d ago
Coin Flip	Auto-CoT	Accuracy99.9	27	25d ago
AQUA	COT	Accuracy80.3	26	1mo ago
OlyBench	QuaSAR	Accuracy36.5	25	1mo ago
MMLU Redux	QuaSAR	Accuracy80.3	25	1mo ago
Countdown	FDM-A	Accuracy49.61	24	1mo ago
Last Letter	In-Writing-BF	Accuracy0.819	21	1mo ago
Date	Automatic Model Selection with LLMs	Solve Rate90.5	14	1mo ago
Lies	Meta-Reasoning Paradigm	Accuracy100	12	1mo ago
LastLetter (test)	SGE	Accuracy86.96	11	1mo ago
Coin Flip	RM-Primed	Accuracy80.75	10	25d ago
Coinflip 4	Self-consistency	Accuracy99.5	10	1mo ago
Date Understanding (DU)	SoftCoT	Accuracy87.2	10	1mo ago
CoinFlip	SC+IC (tune)	Calibrated Accuracy100	8	1mo ago
Last Letter Concatenation (test)		Accuracy81.9	8	1mo ago
Last Letter Concatenation (LLC) out-of-distribution (test)	StrategyLLM	LLC-4 Accuracy98	7	1mo ago
COLOR (Colored Object)	SATLM	Accuracy99.4	7	1mo ago
ARC-AGI 1 (test)	RIMA	Pass@247.5	6	1mo ago
Coin Flip OOD (test)	AMPLIFY	Accuracy65.7	6	1mo ago
Coin Flip (test)	Few-shot Auto-CoT	Accuracy98.6	6	1mo ago
Symbolic Longer	SCO	Accuracy (Clean, Avg)0.187	5	1mo ago
Symbolic Equal	CD-CoT	Acc (Clean, Avg)42.7	5	1mo ago
Date Understanding (DU) (test)	SoftCoT	Accuracy67.52	4	1mo ago

Showing 25 of 42 rows