Share your thoughts, 1 month free Claude Pro on usSee more

LLM Judging Agreement on Reasoning Agreement Benchmark 500-sample human-annotated

0.91Cohen's Kappa

Multi-sub RM

Updated 1mo ago

Evaluation Results

Method	Links
Multi-sub RM 2025.07		0.91
GPT-4o 2025.07		0.9
Master-RM-7B 2025.07		0.9
Qwen2.5-72B-Instruct 2025.07		0.88
Qwen2.5-32B-Instruct 2025.07		0.88
Qwen2.5-14B-Instruct 2025.07		0.88
Master-RM-32B 2025.07		0.87
Qwen2.5-1.5B-Instruct 2025.07		0.83
Qwen2.5-3B-Instruct 2025.07		0.82
Omni-Judge 2025.07		0.81
LLaMA3-70B-Instruct 2025.07		0.81
Qwen2.5-7B-Instruct 2025.07		0.8
LLaMA3-8B-Instruct 2025.07		0.73
General-Verifier 2025.07		0.7
Qwen2.5-0.5B-Instruct 2025.07		0.1