Sudoku
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Logical Reasoning | Sudoku | Accuracy94.3 | 119 | |
| Planning | Sudoku | Accuracy92.7 | 76 | |
| Reasoning | Sudoku | Pass@196 | 26 | |
| Sudoku | Sudoku 4x4 (test) | Accuracy (Seq Len 64)17.5 | 18 | |
| Agent Task | Sudoku | Success Rate (SR)99 | 17 | |
| Agent Behavior Adaptation | Sudoku (Su) (test) | Loop Ratio34.3 | 17 | |
| Sudoku Solving | Sudoku 512 tokens | Pass@180.3 | 15 | |
| Sudoku Solving | Sudoku 256 tokens | Pass@192.9 | 15 | |
| Sudoku Solving | Sudoku 2x2 | Final Reward1.3 | 14 | |
| Puzzle Solving | Sudoku Out Of Distribution | Average @12877.8 | 12 | |
| Puzzle Solving | Sudoku In Distribution | Average Score @12897.1 | 12 | |
| Sudoku Solving | Sudoku 10k 9x9 boards (val) | Board Accuracy93.4 | 12 | |
| Sudoku Solving | Sudoku (test) | Accuracy76.4 | 12 | |
| Constraint Satisfaction | Sudoku | CSP Result Index 3557 | 12 | |
| Visual Reasoning | Sudoku | Accuracy (Scale 40)100 | 10 | |
| Sudoku Solving | Sudoku | Success Rate (pass@1)100 | 10 | |
| Logic Reasoning | Sudoku 8B Instruct (test) | Accuracy71.7 | 9 | |
| Combinatorial Reasoning | Sudoku 4x4 | Accuracy100 | 9 | |
| Combinatorial Reasoning | Sudoku 3x3 | Accuracy100 | 9 | |
| Sequential puzzle-solving | Sudoku | Accuracy44.2 | 9 | |
| Sudoku Solving | Sudoku | Average per-epoch runtime (ms)12.15 | 9 | |
| Reasoning | Sudoku (test) | Accuracy0.161 | 9 | |
| Reasoning | Sudoku | Accuracy (Sudoku Reasoning)17.2 | 8 | |
| Sudoku Solving | 9x9 Sudoku (test) | Cell Accuracy52 | 7 | |
| Sudoku Solving | Sudoku 5x5 | Final Reward2.7 | 7 |
Showing 25 of 38 rows