MMAR
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Audio Reasoning | MMAR (test) | Average Score74.4 | 57 | |
| Audio Question Answering | MMAR | Average Score80.4 | 47 | |
| Audio Reasoning | MMAR | Average Accuracy83.7 | 38 | |
| Audio-Language Reasoning | MMAR 1.0 (test) | Accuracy70.3 | 27 | |
| Audio Understanding | MMAR (comprehensive evaluation) | Sound Score62.4 | 25 | |
| Multimodal Audio Reasoning | MMAR | Mean Score63.5 | 22 | |
| Audio Understanding | MMAR (test) | Performance67.1 | 20 | |
| Audio Understanding | MMAR | Average Score83.7 | 15 | |
| Audio Perception and Reasoning | MMAR within CAFE framework (overall) | Perception Accuracy63.51 | 13 | |
| Audio Understanding / Audio Reasoning | MMAR | Accuracy61.4 | 13 | |
| Massive Multimodal Audio Reasoning | MMAR | Normalized Judge Score56.4 | 9 | |
| Audio Reasoning | MMAR Agent Track | Accuracy77.4 | 8 | |
| Audio Understanding | MMAR | Speech Score45.24 | 5 | |
| Audio Reasoning | MMAR N=1,000 | Accuracy53.6 | 5 | |
| Audio Understanding & Reasoning | MMAR | Score71.9 | 3 | |
| Dense Audio Captioning | MMAR | MMAR Score46.4 | 3 | |
| Audio QA | MMAR (test) | Accuracy68.1 | 2 |
Showing 17 of 17 rows