Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

ARC-AGI

Benchmarks

Task NameDataset NameSOTA ResultTrend
Abstract Visual ReasoningARC-AGI 2
Accuracy (Pass@2)100
33
ReasoningARC-AGI 2 (test)
Pass@2 Exact Match Accuracy24.9
28
Abstract Visual ReasoningARC-AGI 1
Accuracy (Pass@2)98
27
Abstract reasoningARC-AGI 2
Pass@277.1
24
Abstract ReasoningARC-AGI 2
Accuracy60
18
Abstract ReasoningARC-AGI 1
Accuracy97
18
Abstract ReasoningARC-AGI v1 (test)
Accuracy98
17
Logical ReasoningARC-AGI
Pass@14.5
16
Abstraction and ReasoningARC-AGI 2 (public evaluation)
Pass@2100
13
Abstract ReasoningARC-AGI v2 (test)
Accuracy100
11
Compositional ReasoningARC-AGI 2
Accuracy33.6
11
Abstract ReasoningARC-AGI 2 (Semi-private)
Accuracy100
10
Abstraction and ReasoningARC-AGI Public Training Set (Easy) (60 tasks)
Total Cost0.41
10
General ReasoningARC-AGI 2
General Reasoning Score57.5
9
General ReasoningARC-AGI 1
Score89.9
9
ReasoningARC-AGI 2
Accuracy50
9
Abstraction and ReasoningARC-AGI
ARC-1 Score58.2
9
Fingerprint MatchingARC-AGI 1
FMR51
7
ReasoningARC-AGI public evaluation set V2
Accuracy97.9
6
Symbolic ReasoningARC-AGI 1 (test)
Pass@247.5
6
Action-efficiency Game SolvingARC-AGI 3 (public evaluation)
Score (%)78.4
5
ARC-AGIARC-AGI (test)
Accuracy (ARC-AGI Test)67
5
Abstract ReasoningARC-AGI-3 25 Public Games v2
RHAE0
4
Abstract ReasoningARC-AGI
Frugality Index (Fp)3.54
4
Abstract and compositional reasoningARC-AGI 2 (test)
Accuracy (ARC-AGI 2 Test)51
4
Showing 25 of 30 rows