Share your thoughts, 1 month free Claude Pro on usSee more

Audio Question Answering on MMAR

80.4Average Score

AUDIO-MIND

Updated 6d ago

Evaluation Results

Method
AUDIO-MIND 2026.05	80.4	78.18	63.11	87.41	100	87.61	79.27	87.5
Qwen3.5-Omni 2026.05	80	-	-	-	-	-	-	-
Qwen3.5-Omni 2026.05	78.9	75.76	61.17	85.37	90.91	87.61	79.27	87.5
AudioToolAgent 2026.05	77	73.33	63.59	82.99	63.64	83.03	81.71	79.17
Gemini 3 Pro 2026.05	76.4	72.73	61.65	84.35	72.73	81.19	78.05	83.33
Gemini 2.5 Pro 2026.05	74.7	67.3	56.8	82	100	84.9	80.5	66.7
AudioGenie-Reasoner 2026.05	70.5	70.91	54.85	79.59	54.55	72.48	76.83	58.33
Gemini-2.5-Pro 2026.02	70.03	66.06	57.77	73.13	81.82	71.1	69.51	70.83
Echo 2026.02	69.99	67.27	60.68	69.39	81.82	69.72	74.39	66.67
Gemini-2.0-Flash 2026.02	67.9	61.21	50.97	72.11	81.82	72.48	65.85	70.83
Audio-Thinker 2026.02	67.25	68.48	53.88	64.29	72.73	71.56	73.17	66.67
Gemini 2.0 Flash 2026.05	65.6	61.21	50.97	72.11	81.82	72.48	65.85	70.83
AudioRouter 2026.05	64.4	61.4	66.1	77.2	57.2	65.1	68.3	55.5
Gemini 2.5 Pro 2025.10	64.1	-	-	-	-	-	-	-
Gemini 2.5 Pro 2025.10	64.1	-	-	-	-	-	-	-
GPT-4o-Audio 2026.02	64.09	53.94	50.97	70.41	63.64	72.48	62.2	75
GPT-4o Audio 2026.05	63.5	53.94	50.97	70.41	63.64	72.48	62.2	75
Omni-R1 2026.02	63.46	67.3	51.5	64.3	55.5	70.2	64.6	70.8
AuTAgent 2026.05	63	51.52	45.15	73.81	-	-	-	-
Audio-Captioner-7B 2025.10	59.8	-	-	-	-	-	-	-
Audio-Captioner-7B 2025.10	59.8	-	-	-	-	-	-	-
GPT-4o Audio 2025.10	59.3	-	-	-	-	-	-	-
GPT-4o Audio 2025.10	59.3	-	-	-	-	-	-	-
AudioGenie-Reasoner 2026.05	58.85	49.68	43.26	69.23	45.45	64.53	65.33	59.09
Gemini 2.5 Flash 2025.10	58.2	-	-	-	-	-	-	-
Gemini 2.5 Flash 2025.10	58.2	-	-	-	-	-	-	-
Qwen2.5-Omni 2026.02	57.33	58.79	40.78	59.86	54.55	61.93	67.07	58.33
Qwen2.5-Omni-7B 2025.10	51.8	-	-	-	-	-	-	-
Qwen2.5-Omni-7B 2025.10	51.8	-	-	-	-	-	-	-
MiDashengLM-7B 2025.10	50.7	-	-	-	-	-	-	-
MiDashengLM-7B 2025.10	50.7	-	-	-	-	-	-	-
Gemini 2.0 Flash 2025.10	50.6	-	-	-	-	-	-	-
Gemini 2.0 Flash 2025.10	50.6	-	-	-	-	-	-	-
GPT-4o-mini-Audio 2026.02	49.49	38.79	35.92	58.84	45.45	60.09	57.32	50
Qwen2-Audio-7B 2025.10	44.2	-	-	-	-	-	-	-
Qwen2-Audio-7B 2025.10	44.2	-	-	-	-	-	-	-
SALMONN-13B 2025.10	42.5	-	-	-	-	-	-	-
SALMONN-13B 2025.10	42.5	-	-	-	-	-	-	-
Baichuan-Omni-1.5 2026.02	40.05	41.21	33.01	40.48	36.36	48.62	39.02	41.67
Audio-Reasoner 2026.02	36.42	43.64	33.5	32.99	45.45	42.66	31.71	25
SALMONN 2026.02	31.01	30.3	31.07	34.69	9.09	34.86	35.37	41.67
Audio-CoT 2026.02	28.97	35.76	25.24	34.01	9.09	30.73	30.49	37.5
Random Guess 2026.02	28.61	29.39	25.88	31.48	25	29.3	31.1	28.13
Qwen2-Audio 2026.02	26.53	33.33	24.27	32.31	9.09	31.19	30.49	25
GAMA 2026.02	24.83	29.09	24.27	27.89	27.27	24.77	28.05	12.5
Qwen-Audio 2026.02	21.59	27.88	20.39	22.11	9.09	25.23	25.61	20.83
Audio-Flamingo-2 2026.02	19.91	24.85	17.48	20.75	18.18	26.61	23.17	8.33