AI2D
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Visual Question Answering | AI2D | Accuracy88.4 | 249 | |
| Diagram Understanding | AI2D | Accuracy94.2 | 247 | |
| Diagram Question Answering | AI2D | AI2D Accuracy96.02 | 232 | |
| Diagram Question Answering | AI2D (test) | Accuracy94.7 | 142 | |
| Diagram Understanding | AI2D (test) | Accuracy94.7 | 131 | |
| Visual Question Answering | AI2D (test) | Accuracy96.4 | 73 | |
| Diagram Understanding | AI2D 1.0 (test) | Accuracy96.3 | 58 | |
| Visual Question Answering | AI2D | EM82.48 | 42 | |
| Diagram Understanding | AI2D | AI2D Score87.14 | 33 | |
| Multimodal Understanding | AI2D | Score85.56 | 24 | |
| Visual Question Answering | AI2D 65 (test) | Score98.7 | 23 | |
| Diagram Understanding | AI2D F | Accuracy59.7 | 23 | |
| OCR-based Visual Question Answering | AI2D 2016 (test) | Accuracy84.6 | 21 | |
| Visual Perception | AI2D | Accuracy83 | 20 | |
| Diagram Understanding | AI2D | Exact Match79.11 | 19 | |
| Chart Understanding | AI2D | AI2D Score0.947 | 18 | |
| Document Understanding | AI2D (test) | Accuracy88.9 | 17 | |
| Diagram Understanding | AI2D | Pass@1 Accuracy86.5 | 16 | |
| Diagram Understanding | AI2D | Accuracy84.5 | 16 | |
| Diagram Reasoning | AI2D | Score83.44 | 16 | |
| Multimodal Understanding | AI2D | Accuracy80.8 | 16 | |
| OCR, Chat/Doc QA | AI2D (val) | AI2D Accuracy84 | 13 | |
| Multimodal Reasoning | AI2D | Score0.857 | 13 | |
| Diagram Understanding | AI2D | Score80.7 | 10 | |
| OCR-related understanding | AI2D (test) | Accuracy85 | 10 |
Showing 25 of 44 rows