JudgeBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Reward Modeling | JudgeBench | Accuracy93.3 | 117 | |
| Uncertainty Estimation | JudgeBench (test) | AUROC71.53 | 77 | |
| Reward Modeling | JudgeBench (test) | Overall82 | 40 | |
| Preference Prediction | JudgeBench | Positional Consistent Accuracy74.8 | 30 | |
| LLM-as-a-Judge | JudgeBench | Accuracy84.19 | 29 | |
| Uncertainty Calibration | JudgeBench | Kuiper0.037 | 24 | |
| LLM-as-a-Judge Evaluation | JudgeBench (test) | Score83.4 | 22 | |
| Reward Modeling | JudgeBench | Knowledge74.6 | 22 | |
| Reward Modeling | JudgeBench | Reward Modeling Score85.2 | 16 | |
| LLM Evaluation | JudgeBench (test) | Knowledge79.9 | 16 | |
| Pair-wise comparison | JudgeBench | Accuracy75.7 | 16 | |
| Reward Modeling | JudgeBench Knowledge | Accuracy74.4 | 16 | |
| LLM Judging | JudgeBench response pairs generated by GPT-4o 1.0 | Knowledge68.18 | 11 | |
| Discriminative Accuracy | JudgeBench | Knowledge Accuracy77.3 | 8 | |
| Reward Modeling | JudgeBench | Positional Consistency Score56.3 | 8 | |
| LLM-as-a-Judge | JudgeBench (Merged GPT Claude) | Direct Baseline Score87.38 | 8 | |
| Model Evaluation | JudgeBench (test) | Kuiper5.63 | 8 |
Showing 17 of 17 rows