Share your thoughts, 1 month free Claude Pro on usSee more

Reasoning on MMLU-Pro (Subject Averages)

57.5History Score

RAT

Updated 1mo ago

Evaluation Results

Method	Links
RAT 2025.08		57.5	55.3	73	34.2	55
Standard RAG 2025.08		57	52.3	68.6	36.2	53.5
CoT 2025.08		56.7	53.1	70.4	38.2	54.5
Self-Ask 2025.08		56.3	48.5	67.8	31.2	51
Search-o1 2025.08		53.5	55.3	69.8	35.4	53.5
UR2 (GPT Summary) 2025.08		53.2	53	72.2	35	53.3
Vanilla RL 2025.08		52.2	43.5	64	33.8	48.4
UR2 (qw Summary) 2025.08		51.2	49.7	67.8	32.6	50.3
Open-Reasoner-Zero 2025.08		50	46.6	67.5	34.2	49.6
UR2 (GPT summary) 2025.08		48.3	38.6	58	28.8	43.4
UR2 2025.08		48.3	38.6	58	28.8	43.4
General Reasoner 2025.08		47.9	44.2	65.9	30.4	47.1
UR2 (GPT summary) 2025.08		47.8	49.3	63.9	30	47.8
UR2 2025.08		47.8	49.3	63.9	30	47.8
RAT 2025.08		47.2	44.7	64.4	30	46.6
Standard RAG 2025.08		44.6	41.1	57.8	26	42.4
Vanilla RL 2025.08		44.6	36.9	53	26.4	40.2
Vanilla RL 2025.08		44.6	36.9	53	26.4	40.2
Standard RAG 2025.08		43.6	33.9	51	26.6	38.8
UR2 (qw summary) 2025.08		43.6	41.2	56.8	28	42.4
Standard RAG 2025.08		43.6	33.9	51	26.6	38.8
UR2 (qw summary) 2025.08		43	42.6	60.6	26.6	43.2
Search-o1 2025.08		42.8	45.9	63.2	29.6	45.4
CoT 2025.08		42.3	45.7	63.4	26.6	44.5
RAT 2025.08		42.3	37.7	52.6	28.6	40.3
Vanilla RL 2025.08		40.7	34.7	55	24.6	38.7
Self-Ask 2025.08		40.7	42.1	60	26.2	42.3
Vanilla RL 2025.08		40.7	34.7	55	24.6	38.7
Self-Ask 2025.08		39.8	32.1	47	23.4	35.6
Standard RAG 2025.08		37.8	36.5	51.4	23.2	37.1
CoT 2025.08		37.8	40.9	53.4	29	40.3
Standard RAG 2025.08		37.8	36.5	51.4	23.2	37.1
RAT 2025.08		37.8	33.5	50.6	18.4	35.1
CoT 2025.08		37.8	40.9	53.4	29	40.3
SimpleRL-Zoo 2025.08		35.7	36.9	55.2	25.4	38.3
Search-o1 2025.08		33.9	34.5	50.6	20.6	34.9
CoT 2025.08		33.6	32.3	48.8	20.6	33.8
CoT 2025.08		33.6	32.3	48.8	20.6	33.8
Self-Ask 2025.08		33.1	30.5	44.2	20.2	32
Search-o1 2025.08		32.6	32.5	46	28	34.8