Share your thoughts, 1 month free Claude Pro on usSee more

Mathematical Reasoning on MATH 500 (Accuracy, Avg. Tokens, E3)

93.16Accuracy

Vanilla

Updated 1mo ago

Evaluation Results

Method
Vanilla 2025.05	93.16	711.2	12.2
PLAN-AND-BUDGET 2025.05	93.04	1,469.29	5.89
PLAN-AND-BUDGET 2025.05	92.88	1,487.83	5.8
PLAN-AND-BUDGET 2025.05	92.64	1,557.64	5.51
Global Budget 2025.05	92.56	1,661.24	5.16
PLAN-AND-BUDGET 2025.05	92.32	1,529.98	5.57
PLAN-AND-BUDGET 2025.05	92.28	1,575.04	5.41
Vanilla 2025.05	92.12	2,022.38	4.2
PLAN-AND-BUDGET 2025.05	91.92	1,514.43	5.58
Vanilla 2025.05	91.88	539.36	15.65
Global Budget 2025.05	91.84	636.41	13.25
Global Budget 2025.05	91.84	586.18	14.39
PLAN-AND-BUDGET 2025.05	91.64	538.22	15.6
PLAN-AND-BUDGET 2025.05	91.36	525.53	15.88
PLAN-AND-BUDGET 2025.05	91.32	522.89	15.95
Global Budget 2025.05	91.24	1,552.62	5.36
Vanilla 2025.05	91.04	1,883.73	4.4
PLAN-AND-BUDGET 2025.05	90.88	525.51	15.72
PLAN-AND-BUDGET 2025.05	90.8	1,389.75	5.93
Global Budget 2025.05	90.8	1,810.83	4.55
Global Budget 2025.05	90.56	2,565.18	3.2
PLAN-AND-BUDGET 2025.05	90.56	534.45	15.34
PLAN-AND-BUDGET 2025.05	90.48	1,485.99	5.51
Vanilla 2025.05	90.44	2,286.63	3.58
PLAN-AND-BUDGET 2025.05	90.36	525	15.55
PLAN-AND-BUDGET 2025.05	90.16	1,440.7	5.64
PLAN-AND-BUDGET 2025.05	90.04	1,336.27	6.07
PLAN-AND-BUDGET 2025.05	90.04	1,371.59	5.91
PLAN-AND-BUDGET 2025.05	89.88	1,365.51	5.92
Vanilla 2025.05	89.76	2,105.12	3.83
Global Budget 2025.05	89.6	1,526.15	5.26
PLAN-AND-BUDGET 2025.05	88.68	2,397.16	3.28
PLAN-AND-BUDGET 2025.05	88.6	2,306.83	3.4
PLAN-AND-BUDGET 2025.05	88.27	2,346.94	3.32
Global Budget 2025.05	88.2	2,671.6	2.91
PLAN-AND-BUDGET 2025.05	88.13	2,346.35	3.31
PLAN-AND-BUDGET 2025.05	87.9	2,320.04	3.33
PLAN-AND-BUDGET 2025.05	87.45	2,479.46	3.08
Vanilla 2025.05	85.3	3,309.69	2.2
Vanilla 2025.05	84.88	3,523.72	2.04