LLM-as-a-Judge

Benchmarks

Dataset Name	SOTA Method	Metric
PreferenceBench	CalibraEval	Accuracy90.71	59	3mo ago
MTbench (test)		StdDev2.24	45	4mo ago
MT-Bench	PA-GRPO	Accuracy81.4	44	3mo ago
RewardBench 1.0 (test)	CC	Rstd0.54	36	4mo ago
RewardBench	Qwen3-Next-80B-A3B-Thinking	Accuracy92.9	31	3mo ago
JudgeBench		Accuracy84.19	29	4mo ago
Peer-Support Evaluation Set	MindTailor	Empathy4.89	23	1mo ago
PreferenceBench	PA-GRPO	Accuracy90.2	21	4mo ago
High-contrast response pairs	LongCat-Flash-Chat	Discriminability (πi)0.87	20	2mo ago
ARENA	EpiPersona-A	Accuracy66.07	20	3mo ago
PRISM	EpiPersona-A	Accuracy59.38	20	3mo ago
SenseBench	SenseJudge	Math Score86.53	17	1mo ago
PRISM (test)	SynthesizeMe	Accuracy58.9	14	4mo ago
Chatbot Arena (test)	Gemini-2.5-Pro	Accuracy68.13	14	4mo ago
FairJudge Benchmark 1K (test)	FairJudge-8B	Agreement71.5	13	4mo ago
JudgeLM (test)		Agreement79.59	13	4mo ago
PandaLM Human Annotations (test)	FairJudge-8B	Agreement0.7683	13	4mo ago
TL;DR		Coverage82.6	12	2mo ago
Chatbot Arena		Coverage94.3	12	2mo ago
HH-RLHF		Coverage81.3	12	2mo ago
AlpacaEval		Coverage78.3	12	2mo ago
Preference Bench (test)	CalibraEval	Std Dev2.82	9	4mo ago
RewardBench (test)	CalibraEval	Std Dev (Reward)2.72	9	4mo ago
LLM-as-a-Judge (10-fold cross-validation)	Qwen3-14B	CG Accuracy88	8	1mo ago
JudgeBench (Merged GPT Claude)		Direct Baseline Score87.38	8	3mo ago

Showing 25 of 30 rows