Share your thoughts, 1 month free Claude Pro on usSee more

Omni-modal agentic tasks on OmniGAIA

82Pass@1 (Easy)

GPT-5.4 xHigh

Updated 1d ago

Evaluation Results

Method
GPT-5.4 xHigh 2026.05	82	75	-	-	-	-	-	-	-	-	-	75	64.1
Gemini-3-Pro 2026.02	78.7	62.5	65.2	59.2	62.1	72	78.4	52.8	48.5	42.3	88.9	61.9	38.5
OmniAgent 2026.05	77.9	66.4	-	-	-	-	-	-	-	-	-	63.1	55.1
Gemini 3.1 Pro 2026.05	77.8	66.1	-	-	-	-	-	-	-	-	-	63.8	52.6
Claude Opus 4.6 2026.05	74.6	68.6	-	-	-	-	-	-	-	-	-	69.4	57.7
Claude Sonnet 4.6 2026.05	73.8	63.3	-	-	-	-	-	-	-	-	-	62.5	48.8
Kimi K-2.5 2026.05	72.1	55.6	-	-	-	-	-	-	-	-	-	50.6	39.7
GPT-5.4 High§ 2026.05	70.5	64.4	-	-	-	-	-	-	-	-	-	63.8	56.4
GPT-5.4 High 2026.05	70.4	61.4	-	-	-	-	-	-	-	-	-	60	50
Gemini-3-Flash 2026.02	67.2	51.7	50.7	57.1	44.8	48	59.5	55.6	54.6	38.5	61.1	46.9	37.2
Gemini 3 Flash 2026.05	67.2	51.7	-	-	-	-	-	-	-	-	-	46.9	37.2
GPT-5.4 Medium 2026.05	66.4	56.4	-	-	-	-	-	-	-	-	-	55	43.6
GPT-5.4 Low 2026.05	63.1	55	-	-	-	-	-	-	-	-	-	53.8	44.9
Codex-GPT-5.4 Low 2026.05	63.1	55	-	-	-	-	-	-	-	-	-	53.8	44.9
Agent-Omni 2026.05	54.1	41.1	-	-	-	-	-	-	-	-	-	41.9	19.2
Code-X (SFT) 2026.05	50	43.3	-	-	-	-	-	-	-	-	-	45	28.2
Direct MLLM 2026.05	44.3	38.2	-	-	-	-	-	-	-	-	-	38.1	28.6
Qwen 3.5 Omni Plus 2026.05	43.5	38.7	-	-	-	-	-	-	-	-	-	39.4	29
Gemini-2.5-Pro 2026.02	41.8	30.8	23.2	28.6	32.8	20	32.4	41.7	42.4	26.9	33.3	26.9	21.8
MiniMax-M2.7‡ 2026.05	38.5	33.3	-	-	-	-	-	-	-	-	-	34.4	23.1
Mimo V2 Omni 2026.05	33.9	25.8	-	-	-	-	-	-	-	-	-	22.5	19.1
Code-X (RL) 2026.05	32.8	28.1	-	-	-	-	-	-	-	-	-	26.3	24.4
Baseline (direct image) 2026.05	32.8	24.4	-	-	-	-	-	-	-	-	-	18.8	23.1
OmniAtlas-Qwen-3 2026.02	31.1	20.8	10.1	30.6	29.9	32	18.9	16.7	12.1	11.5	27.8	18.8	9
OmniAtlas-Omni-30B-A3B 2026.05	31.1	20.8	-	-	-	-	-	-	-	-	-	18.8	9
Code-X (SFT) 2026.05	30	23.1	-	-	-	-	-	-	-	-	-	21	17
Baseline (direct image) 2026.05	23.8	18.3	-	-	-	-	-	-	-	-	-	14.4	17.9
Direct MLLM 2026.05	23	20	-	-	-	-	-	-	-	-	-	16.2	23.1
OmniAtlas-Qwen-2.5 2026.02	22.1	13.3	8.7	18.4	16.4	4	16.2	22.2	3	7.7	22.2	11.3	3.9
Qwen-3-Omni 2026.02	19.7	13.3	8.7	14.3	11.9	28	10.8	13.9	9.1	15.4	22.2	10.6	9
Qwen 3 Omni 2026.05	19.7	13.3	-	-	-	-	-	-	-	-	-	10.6	9
LongCat-Flash-Omni 2026.02	16.4	11.1	8.7	10.2	16.4	12	10.8	8.3	6.1	11.5	16.7	9.4	6.4
OmniAtlas-Qwen-2.5 2026.02	13.9	10.3	4.4	12.2	16.7	4	16.2	11.1	3	11.5	11.1	10	5.1
Ming-Flash-Omni 2026.02	12.3	8.3	5.8	8.2	10.4	12	8.1	5.6	6.1	11.5	11.1	7.5	3.8
Gemini-2.5-Flash-Lite 2026.02	9.8	8.6	5.8	8.2	14.9	4	10.8	8.3	6.1	3.9	11.1	8.1	7.7
Qwen-2.5-Omni 2026.02	8.2	3.6	1.5	4.1	7.5	4	0	2.8	0	7.7	5.6	1.3	1.3
Baichuan-Omni-1.5 2026.02	4.9	2.8	2.9	4.1	3	4	2.7	0	3	3.8	0	2.5	0
Ming-Lite-Omni-1.5 2026.02	4.9	3.9	2.9	6.1	1.5	4	5.4	2.8	6.1	7.7	5.6	3.8	2.6
MiniCPM-O-2.6 2026.02	3.3	3.1	2.9	2	1.5	0	2.7	8.3	3	3.8	5.6	2.5	3.8
Qwen-2.5-Omni 2026.02	1.6	1.4	0	2	4.5	0	0	0	0	3.9	0	1.9	0