OKVQA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Visual Question Answering | OKVQA | Top-1 Accuracy75.29 | 283 | |
| Visual Question Answering | OKVQA (val) | VQA Score66.1 | 101 | |
| Knowledge-based Visual Question Answering | OKVQA | Accuracy0.885 | 92 | |
| Visual Question Answering | OKVQA | ASR100 | 42 | |
| Vision Question Answering | OKVQA | ASR (Success Rate)73.07 | 30 | |
| Knowledge-based Visual Question Answering | OKVQA (val) | Accuracy66.7 | 27 | |
| Visual Question Answering | OKVQA | Accuracy64.56 | 26 | |
| Knowledge-based Visual Retrieval | OKVQA Google Search (test) | PR@584.66 | 16 | |
| Cognition and Reasoning | OKVQA | Score61.92 | 16 | |
| Visual Question Answering | OKVQA | VQA Accuracy (Clean)60.3 | 14 | |
| Visual Question Answering | OKVQA | Accuracy60.13 | 14 | |
| Visual Question Answering | OKVQA (test) | Accuracy65.7 | 11 | |
| Visual Question Answering | OKVQA (I) (test) | VQA Accuracy57.8 | 11 | |
| Visual Question Answering | OKVQA N=200 | Score D61.7 | 11 | |
| Visual Question Answering | OKVQA (N=100) | CD Score56.6 | 11 | |
| Visual Question Answering | OKVQA N=40 | CD48.8 | 11 | |
| Object Hallucination Probing | OKVQA POPE Popular | Accuracy85 | 11 | |
| Uncertainty Quantification | OKVQA | AUROC69.1 | 10 | |
| Visual Question Answering | OKVQA | Score57.79 | 10 | |
| Visual Question Answering | OKVQA | AUROC0.788 | 9 | |
| Knowledge-based Question Answering | OKVQA | Score64.56 | 9 | |
| Open-ended Question Answering | OKVQA | LVM Evaluation Score71.6 | 6 | |
| End-to-end Retrieval | OKVQA | R@10041 | 6 | |
| Visual Question Answering | OKVQA (val-lite) | Accuracy48.68 | 6 | |
| Knowledge-based Visual Retrieval | OKVQA WK11M (test) | MRR@551.15 | 6 |
Showing 25 of 31 rows