ZebraLogic
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Logical Reasoning | ZebraLogic v1.0 (test) | Cell Accuracy97.7 | 90 | |
| Logical Reasoning | ZebraLogic (test) | Grid Accuracy92.2 | 90 | |
| Logical Reasoning | ZebraLogic | Accuracy98.8 | 86 | |
| Logic Reasoning | ZebraLogic | Accuracy96 | 49 | |
| Logic Reasoning | Zebralogic | Score96.1 | 42 | |
| Reasoning | ZebraLogic | Score90.97 | 31 | |
| General Reasoning | ZebraLogic | Accuracy (%)96.3 | 29 | |
| Logic Puzzle Solving | ZebraLogic | Accuracy60.8 | 25 | |
| Local Causal Attribution | ZebraLogic | AUPC348.71 | 20 | |
| Logical Reasoning | ZebraLogic (held-out) | Accuracy71.7 | 18 | |
| Reasoning | ZebraLogic | Avg Accuracy@144.5 | 12 | |
| Logic Reasoning | ZebraLogic | Avg Accuracy @10.817 | 11 | |
| Constraint Satisfaction Reasoning | ZebraLogic | Easy Score96.8 | 9 | |
| Data-to-Text Generation | ZebraLogic | Schema Validity100 | 8 | |
| Long-horizon planning | ZebraLogic | Accuracy93.5 | 6 | |
| Logical Reasoning | ZebraLogic | Accuracy (Avg@8)96.3 | 6 | |
| Logic Reasoning | Zebralogic | Pass@877 | 6 | |
| Reasoning | ZebraLogic | Error Rate (%)4.01 | 6 | |
| Logical Reasoning | ZebraLogic 140 puzzles | Puzzle Accuracy130 | 5 | |
| General Reasoning | ZebraLogic | Mean@196.1 | 4 | |
| Reasoning | ZebraLogic | TPS2,245.5 | 3 | |
| Reasoning | Zebralogic | Accuracy (Zebralogic)78.7 | 3 | |
| Combinatorial Reasoning | ZebraLogic mc_mode, Rows/Cols <= 4 | Average Score @6473.73 | 1 |
Showing 23 of 23 rows