Abstract Reasoning on ARC-AGI 2 (pass@k)

77.1Pass@2

Gemini 3.1-Pro

Updated 1mo ago

Evaluation Results

Method
Gemini 3.1-Pro 2026.06	77.1	-	-
Claude Opus-4.7 2026.06	75.8	-	-
OpenAI GPT-5.4 2026.06	74	-	-
Claude Opus-4.6 2026.06	68.8	-	-
Ring-2.6-1T 2026.06	66.18	-	-
DS-V4-Pro 2026.06	62.43	-	-
Bespoke (Grok-4) 2025.11	29.4	-	-
Kimi-K2.6 2026.06	29.38	-	-
GLM-5.1 2026.06	21.74	-	-
Grok-4-thinking 2025.11	16	-	-
Standard TRM, our reproduction 2026.05	9.72	7.36	14.31
PTRM 2026.05	9.72	8.47	15.97
TRM 2026.05	7.8	-	-
DIS 2025.11	6	-	-
HRM 2026.05	5	-	-
Gemini 2.5 Pro 32K 2025.11	4.9	-	-
TRM 2025.11	3.3	-	-
o3-mini-high 2025.11	3	-	-
DIS-medium 2025.11	2.5	-	-
Deepseek R1 2025.11	1.3	-	-
Claude 3.7 16K 2025.11	0.7	-	-
TRM-compact 2025.11	0	-	-
DIS-compact 2025.11	0	-	-
TRM-medium 2025.11	0	-	-