MMAU
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Audio Understanding | MMAU v05.15.25 (test-mini) | Sound Score84.04 | 54 | |
| Audio Understanding | MMAU | Accuracy81.1 | 54 | |
| Audio Understanding | MMAU v05.15.25 (test) | Sound Score84.04 | 53 | |
| Audio Understanding | MMAU Pro | Average Score100 | 42 | |
| Audio Understanding Robustness | MMAU Sound (SNR=-10dB) | Accuracy66.83 | 36 | |
| Multiple-choice audio understanding | MMAU mini (test) | Average Accuracy73.2 | 33 | |
| Audio Question Answering | MMAU | Score77.5 | 28 | |
| Audio-Language Understanding (MCQ) | MMAU-Pro 1.0 (test) | Accuracy59.5 | 27 | |
| Audio-Language Understanding | MMAU 1.0 (test) | Accuracy77.4 | 27 | |
| Multimodal Audio Understanding | MMAU mini v05.15.25 (test) | Sound Accuracy79.6 | 25 | |
| Audio Understanding | MMAU (test) | Speech Score76.58 | 25 | |
| Audio Understanding | MMAU mini original (test) | Accuracy (Sound Domain)80.78 | 21 | |
| Audio Understanding | MMAU mini (test) | Accuracy78.5 | 20 | |
| Audio Understanding Robustness | MMAU Music SNR=-10dB | Accuracy64.82 | 18 | |
| Audio Reasoning | MMAU-Pro | Average Score61.9 | 18 | |
| Audio Reasoning | MMAU mini (test) | Average Score77.7 | 17 | |
| Massive Multi-discipline Audio Understanding | MMAU | Speech Score69.23 | 17 | |
| Omni-modal reasoning | MMAU mini v05.15.25 (test) | Overall Score75.5 | 15 | |
| Audio Reasoning | MMAU mini 1.0 (test) | Sound Score83.48 | 15 | |
| Audio Generation | MMAU Speech (Noise) | GSR80 | 15 | |
| Audio Generation | MMAU Speech (Clean) | GSR99 | 15 | |
| Audio Generation | MMAU Sound (Noise) | GSR78.33 | 15 | |
| Audio Generation | MMAU Sound (Clean) | GSR99.67 | 15 | |
| Audio Generation | MMAU Music (Noise) | GSR85 | 15 | |
| Audio Generation | MMAU Music (Clean) | GSR99 | 15 |
Showing 25 of 56 rows