Sudoku
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Logical Reasoning | Sudoku | Accuracy94.3 | 152 | |
| Planning | Sudoku | Accuracy92.7 | 141 | |
| Reasoning | Sudoku | Pass@196 | 60 | |
| Puzzle Solving | Sudoku | Test Accuracy27.2 | 42 | |
| Sudoku Solving | Sudoku (test) | Accuracy100 | 33 | |
| Reasoning | Sudoku | Accuracy (Sudoku Reasoning)91.9 | 25 | |
| Sudoku | Sudoku 4x4 (test) | Accuracy (Avg)14.7 | 25 | |
| Logical planning | Sudoku (test) | Accuracy91.7 | 24 | |
| Reasoning | Sudoku Extreme | Pass@1 Accuracy99.8 | 21 | |
| Sudoku | Sudoku | Accuracy91.82 | 19 | |
| Reasoning | Sudoku (test) | Accuracy100 | 19 | |
| Sudoku Solving | Sudoku 25x25 (90% masking) | VPR (%)52.2 | 18 | |
| Sudoku Solving | Sudoku 25x25 80% masking | VPR (%)100 | 18 | |
| Sudoku Solving | Sudoku 25x25 70% masking | VPR (%)100 | 18 | |
| Agent Task | Sudoku | Success Rate (SR)99 | 17 | |
| Agent Behavior Adaptation | Sudoku (Su) (test) | Loop Ratio34.3 | 17 | |
| Sudoku Solving | Sudoku 512 tokens | Pass@180.3 | 15 | |
| Sudoku Solving | Sudoku 256 tokens | Pass@192.9 | 15 | |
| Planning and Reasoning | Sudoku | Accuracy99.4 | 14 | |
| Sudoku Solving | Sudoku 2x2 | Final Reward1.3 | 14 | |
| Puzzle Solving | Sudoku Out Of Distribution | Average @12877.8 | 12 | |
| Puzzle Solving | Sudoku In Distribution | Average Score @12897.1 | 12 | |
| Sudoku Solving | Sudoku 10k 9x9 boards (val) | Board Accuracy93.4 | 12 | |
| Constraint Satisfaction | Sudoku | CSP Result Index 3557 | 12 | |
| Sudoku Solving | Sudoku | Success Rate (pass@1)100 | 12 |
Showing 25 of 75 rows