Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

COUNTDOWN

Benchmarks

Task NameDataset NameSOTA ResultTrend
Mathematical ReasoningCountdown
Accuracy85
252
PlanningCountdown
Accuracy87.5
101
Mathematical ReasoningCountdown (test)
Accuracy85
84
ReasoningCountdown
Accuracy83.2
49
Constraint Satisfaction ReasoningCountdown
Pass@172.7
34
Logical ReasoningCountdown
Accuracy63.2
31
Arithmetic PlanningCountdown
Accuracy65.6
28
PlanningCountdown
Accuracy85.6
27
Arithmetic ReasoningCountdown
Accuracy63.1
25
CountdownCountdown 5
RTR (%)56.4
24
Logical planningCountdown (test)
Accuracy84.8
24
Symbolic ReasoningCountdown
Accuracy49.61
24
CountdownCountdown 4
RTR (%)86.2
20
CountdownCountdown 3
RTR (%)98.4
15
Arithmetic ReasoningCountdown 512 tokens
Pass@162.1
15
Arithmetic ReasoningCountdown 256 tokens
Pass@171.1
15
Planning and ReasoningCountdown
Accuracy81.6
14
PlanningCountdown (held-out)
Pass@187.96
14
Logical ReasoningCountdown CD34
Avg@1678.2
14
Logical ReasoningCountdown CD4
Avg@1659.4
14
Arithmetic ReasoningCountdown (val)
Validation Reward (mean@4)76.27
13
Numerical ReasoningCountdown-4
CD498.9
13
Mathematical ReasoningCountdown
Accuracy (L=128)54.3
13
ReasoningCOUNTDOWN (test)
Accuracy66.02
13
CountdownCountdown 3 (test)
Reward95
12
Showing 25 of 46 rows