MMSI-Bench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Spatial Reasoning | MMSI-Bench | Average Accuracy49.5 | 67 | |
| Spatial Reasoning | MMSI-Bench | Accuracy97.2 | 65 | |
| Spatial Reasoning (Multi-Image) | MMSI-Bench | Accuracy97.2 | 44 | |
| Spatial Reasoning | MMSI-Bench (test) | PR Score52.8 | 29 | |
| Spatial and Temporal Reasoning | MMSI-Bench (test) | Cam-Cam Accuracy43 | 25 | |
| Spatial Understanding | MMSI-Bench | Accuracy45.9 | 21 | |
| Spatial Reasoning | MMSI-Bench MindJourney Subset (162 questions) (test) | Accuracy0.358 | 19 | |
| Multi-view spatial reasoning | MMSI-Bench | Overall Accuracy42.7 | 17 | |
| Spatial Understanding | MMSI-Bench | Score48 | 16 | |
| Video Spatial Understanding | MMSI-Bench 500 QA 1.0 | Accuracy41 | 15 | |
| Image Understanding | MMSI-Bench 68 (test) | Average Score69.4 | 12 | |
| 3D/4D Video Question Answering | MMSI-Bench | Accuracy40.7 | 12 | |
| Spatial Reasoning | MMSI-Bench 79 (MV) | Accuracy43.3 | 11 | |
| Compositional Reasoning | MMSI-Bench | MSR Score54.3 | 6 | |
| Multi-image Understanding | MMSI-Bench | Accuracy36.9 | 6 | |
| Video Spatial Understanding | MMSI-Bench 500 QA pairs | Accuracy41 | 5 | |
| Video Spatial Understanding | MMSI-Bench 500 QA subset | Accuracy35.3 | 5 | |
| Embodied Cognition | MMSI-Bench | Score40.8 | 4 | |
| 3D/4D Visual Question Answering | MMSI Bench 1.0 (test) | Avg Multiple Choice Accuracy33.3 | 4 | |
| Visual Spatial Reasoning | MMSI-Bench | Cam-Cam Accuracy36.6 | 4 | |
| Multimodal Understanding | MMSI-Bench | Accuracy31.5 | 3 |
Showing 21 of 21 rows