MME-RealWorld
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| General Visual Reasoning | MME-RealWorld-Lite | Accuracy73.06 | 47 | |
| Perception | MME-RealWorld Lite | Overall Score67 | 46 | |
| Multimodal Understanding | MME-RealWorld Lite | Overall Score67.3 | 38 | |
| Reasoning | MME-RealWorld Lite | OCR Score84 | 37 | |
| Real-world Visual Question Answering | MME-RealWorld-Lite (MMERW) | Accuracy57 | 31 | |
| Autonomous Driving (Perception, Prediction & Planning) | MME-RealWorld | Overall Score (P+P+P)71.2 | 25 | |
| Real-world Multimodal Understanding | MME-RealWorld Lite | Lite Score54.9 | 25 | |
| Multimodal Understanding | MME-RealWorld Chinese | Accuracy64.3 | 25 | |
| Multimodal Understanding | MME-RealWorld English | Accuracy59.9 | 25 | |
| Multimodal Evaluation | MME-RealWorld | Accuracy71.2 | 24 | |
| Multimodal Perception and Reasoning | MME-RealWorld Lite | Overall Score59.87 | 21 | |
| General Perception and Reasoning | MME-RealWorld Lite | Overall Accuracy54.3 | 21 | |
| Real-world Understanding | MME-RealWorld EN | Score64 | 20 | |
| Real-world visual perception | MME-RealWorld CN | Accuracy73.31 | 19 | |
| Real-world visual perception | MME-RealWorld Lite | Accuracy56.38 | 19 | |
| Multimodal Question Answering | MME-RealWorld-Lite 1.0 (test) | Perception (AD) Acc57.7 | 19 | |
| Real-World Visual Reasoning | MME-RealWorld Lite | Overall Score61.2 | 18 | |
| Perception-intensive Reasoning | MME-RealWorld-Lite (MRWL) | Score55.13 | 18 | |
| Fine-grained visual reasoning | MME Realworld Lite | Avg@155.8 | 12 | |
| Reasoning | MME-RealWorld Lite (test) | OCR76 | 12 | |
| Remote Sensing Visual Question Answering | MME-RealWorld-RS | Position Score58.15 | 11 | |
| Multimodal Evaluation | MME-RealWorld Lite | Score57.8 | 10 | |
| Multimodal Perception and Reasoning | MME-RealWorld Lite | Overall Score57.4 | 9 | |
| Fine-Grained Perception & Understanding | MME-RealWorld lite | Accuracy49.87 | 9 | |
| Hallucination | MME-RealWorld EN | Hallucination Score74.9 | 8 |
Showing 25 of 34 rows