MMQA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multimodal Question Answering | MMQA | Accuracy70.5 | 36 | |
| Multi-modal Retrieval (Image Query) | MMQA | Recall@2076.52 | 21 | |
| Multi-modal Retrieval (Text Query) | MMQA | Recall@2087.17 | 21 | |
| Multimodal Question Answering | MMQA | BLEU-149.31 | 20 | |
| Multimodal Question Answering | MMQA k=1 | ROrig@k57.4 | 20 | |
| Watermark Detection | MMQA | Rank1.03 | 16 | |
| SQL execution performance | MMQA n=1105 | EM (1T)50 | 15 | |
| Table Reasoning | MMQA (test) | Accuracy50 | 13 | |
| Multimodal Retrieval-Augmented Generation | MMQA | ROrig.53.6 | 12 | |
| Table Question Answering | MMQA | Accuracy86.08 | 10 | |
| Multi-table Reasoning | MMQA 3-table | Accuracy40.88 | 9 | |
| Table Selection | MMQA 2025 (test) | Avg Tables6.1 | 8 | |
| Question Answering | MMQA QE-PE 5.2 (test) | EM55.64 | 8 | |
| Multi-table Question Answering | MMQA | List EM31.1 | 7 | |
| Membership Inference Attack | MMQA 1,000 images (non-member set) | AUC0.85 | 4 | |
| Question Answering | MMQA QH-PH 5.2 (test) | EM46.41 | 4 | |
| Question Answering | MMQA QE-PH 5.2 (test) | Exact Match (EM)45.12 | 4 | |
| Machine Comprehension | MMQA 5.2 v1.0 (test) | F1 (QE-PE)0.7443 | 4 | |
| Multimodal Question Answering | MMQA SemBench 1.0 (test) | Latency (s)23.49 | 2 |
Showing 19 of 19 rows