ARC-AGI
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Abstract Visual Reasoning | ARC-AGI 1 | Accuracy (Pass@2)98 | 15 | |
| Abstract Visual Reasoning | ARC-AGI 2 | Accuracy (Pass@2)100 | 14 | |
| Compositional Reasoning | ARC-AGI 2 | Accuracy33.6 | 11 | |
| Abstraction and Reasoning | ARC-AGI Public Training Set (Easy) (60 tasks) | Total Cost0.41 | 10 | |
| Reasoning | ARC-AGI 2 (test) | Accuracy43.3 | 10 | |
| Abstraction and Reasoning | ARC-AGI | ARC-1 Score58.2 | 6 | |
| Reasoning | ARC-AGI 2 | Accuracy50 | 4 | |
| Abstract Reasoning | ARC-AGI (concept evaluation) | Accuracy86.8 | 2 |
Showing 8 of 8 rows