SOTA Question Answering on QA OOD StrQA SciQA and PapersWithCode

98.3StrQA Accuracy

Qwen3-8B pass@N (Upper Bound)

Updated 1mo ago

Evaluation Results

Method
Qwen3-8B pass@N (Upper Bound) 2025.11	98.3	99.3	-	-
Qwen3-14B pass@1 2025.11	91.9	95.6	-	-
ReProbe, Attn+Logit, Self-anno 2025.11	88.6	97.1	-	-
ReProbe, Attn+Logit, DeepSeek-anno 2025.11	88.6	96.9	-	-
ReProbe, Hidden States, DeepSeek-anno 2025.11	88.6	96.9	-	-
Qwen2.5-Math-PRM-7B 2025.11	88.1	96.9	-	-
ReProbe, Hidden States, Self-anno 2025.11	88.1	95.8	-	-
RLHFlow-PRM-Mistral-Data 2025.11	87.8	94.1	-	-
Universal-PRM-Qwen2.5-Math-7B 2025.11	87.8	97.1	-	-
RLHFlow-PRM-Deepseek-Data 2025.11	87.6	94.1	-	-
Math-Shepherd-PRM-7B 2025.11	87.3	95.8	-	-
Qwen2.5-Math-7B-PRM800k 2025.11	87.1	96.9	-	-
Qwen3-8B pass@1 (Lower Bound) 2025.11	86.8	92.7	-	-
Majority Voting 2025.11	86.6	92.5	-	-
Skywork-PRM-1.5B 2025.11	86.6	96.3	-	-
H4-Qwen2.5-PRM-1.5B-0.2 2025.11	84.6	94.7	-	-
Qwen3-1.7B pass@N (Upper Bound) 2025.11	74	89.2	81.6	83.4
Math-Shepherd-PRM-7B 2025.11	54.6	47.2	50.9	62
ReProbe, Hidden States, GPT-OSS-anno 2025.11	51.8	45.9	48.9	60.6
RLHFlow-PRM-Mistral-Data 2025.11	51.2	49.8	50.5	59
MaxProb 2025.11	50.9	59.7	55.3	60.6
Perplexity 2025.11	50.9	59.7	55.3	60.6
RLHFlow-PRM-Deepseek-Data 2025.11	50	45.6	47.8	58.6
MaxEntropy 2025.11	49.4	60	54.7	60.5
Universal-PRM-Qwen2.5-Math-7B 2025.11	48.5	57.4	53	63
H4-Qwen2.5-PRM-1.5B-0.2 2025.11	45.4	47.2	46.3	57.5
Qwen2.5-Math-PRM-7B 2025.11	44.8	51.5	48.2	59.4
Qwen3-1.7B pass@1 (Lower Bound) 2025.11	43.2	48.3	45.8	52.8