PathMMU
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multiple Choice Question Answering | PathMMU (val) | Overall Accuracy80.1 | 45 | |
| Visual Question Answering | PathMMU Tiny 1.0 (test) | Overall Accuracy72.4 | 42 | |
| Visual Question Answering | PathMMU 1.0 (ALL test) | Overall Score72.2 | 22 | |
| Pathology Multimodal Understanding | PathMMU | Accuracy71.45 | 19 | |
| Pathology Visual Question Answering | PathMMU (test-all) | Overall Score77.1 | 19 | |
| Pathology Visual Question Answering | PathMMU tiny (test) | Overall Score79.6 | 19 | |
| Multi-modal Multiple Choice Question Answering | PathMMU (test) | Accuracy (Atlas)75.34 | 18 | |
| Pathological Multimodal Understanding | PathMMU ALL (test) | PubMed Accuracy72.9 | 16 | |
| Pathology Visual Question Answering | PathMMU (test) | PubMed (Tiny) Score72.9 | 15 | |
| Pathological Multimodal Understanding | PathMMU Tiny (test) | PubMed Score74.5 | 15 | |
| Pathology Visual Question Answering | PathMMU (val) | Overall Accuracy62.4 | 14 | |
| Multiple-choice visual question answering | PathMMU PubMed (test-tiny) | Accuracy77.6 | 13 | |
| Multiple-choice Visual Question Answering | PathMMU SocialPath tiny n=229 (test) | Accuracy75.1 | 13 | |
| Multiple-choice question answering | PathMMU EduContent n=255 (test-tiny) | Accuracy76.9 | 13 | |
| Visual Question Answering | PathMMU All-tiny (test) | Accuracy77.1 | 13 | |
| Multiple-choice questions | PathMMU PathCLS n = 177 (test-tiny) | Accuracy78.9 | 13 | |
| Pathological Multimodal Understanding | PathMMU (test) | ACS70.9 | 13 | |
| Multiple-choice visual question answering | PathMMU PubMed (test-all) | Accuracy75.3 | 12 | |
| Multiple-choice Visual Question Answering | PathMMU SocialPath n=1,796 (test) | Accuracy71.7 | 12 | |
| Multiple-choice question answering | PathMMU EduContent n=1,938 (test-all) | Accuracy73.5 | 12 | |
| Multiple Choice Question Answering | PathMMU Atlas (test) | Accuracy84 | 12 | |
| Visual Question Answering | PathMMU All (test) | Accuracy75.1 | 12 | |
| Visual Question Answering | PathMMU All (val) | Accuracy72 | 12 | |
| Multiple-choice questions | PathMMU PathCLS (test-all) | Accuracy74.8 | 12 | |
| Pathological Visual Question Answering | PathMMU tiny (test) | Atlas Score81.73 | 9 |
Showing 25 of 27 rows