Our new X account is live! Follow @wizwand_team for updates

LLM Judge Consistency Evaluation on RewardBench

83.25Kappa

CalibraEval

Updated 4d ago

Evaluation Results

Method	Links
CalibraEval 2024.10		83.25	96.25	96.27
CalibraEval 2024.10		82.8	95.47	95.75
GPT4o (Default) 2024.10		82.57	94.83	94.89
Pride 2024.10		82.53	94.94	94.98
CC 2024.10		81.38	94.47	94.48
Pride 2024.10		78.64	92.99	93.3
DI 2024.10		78.57	93.72	94.01
Qwen-72B (Default) 2024.10		78.28	92.77	93.13
DI 2024.10		77.33	92.27	92.63
DC 2024.10		76.89	92.28	92.35
CC 2024.10		69.23	88.71	89.92
DC 2024.10		66.61	87.03	87.9
CalibraEval 2024.10		32.02	77.25	77.6
CalibraEval 2024.10		30.32	86.51	86.66
Pride 2024.10		28.25	70.38	73.16
DC 2024.10		27.94	66.09	70.54
Pride 2024.10		27.65	72.72	73.77
CalibraEval 2024.10		26.4	75.75	76.11
Pride 2024.10		24.73	67.46	68.3
DI 2024.10		24.52	70.23	71.58
CC 2024.10		24.28	57.25	58.23
DC 2024.10		23.57	69.04	72.79
DC 2024.10		23.17	72.89	76.22
DI 2024.10		21.59	74.04	80
Llama-3-8B (Default) 2024.10		20.81	66.24	71.79
CalibraEval 2024.10		20.67	83.23	86.68
ChatGPT (Default) 2024.10		20.08	62.67	70.75
Qwen-14B (Default) 2024.10		19.69	63.41	66.86
DI 2024.10		19.63	64.98	70.87
CC 2024.10		16.89	59.24	60.09
DC 2024.10		16.04	45.1	45.26
CC 2024.10		15.49	58.77	63.7
Llama-3.1-8B (Default) 2024.10		15.02	68.82	76.65
Pride 2024.10		14.63	66.62	76.02
DI 2024.10		11.9	50.46	53.2
CC 2024.10		-1.62	-4.71	-4.93