Share your thoughts, 1 month free Claude Pro on usSee more

LLM-as-a-judge evaluation on FB Bench (Feedback Bench)

0.949Pearson's r

Qwen3-8B TRACT

Updated 1mo ago

Evaluation Results

Method
Qwen3-8B TRACT 2026.03	0.949	0.947	0.843
Qwen3-8B Standard RL 2026.03	0.94	0.942	0.835
Mistral2-7B TRACT 2026.03	0.939	0.937	0.829
Mistral2-7B Standard RL 2026.03	0.937	0.937	0.828
Mistral-7B-Instruct (RAFT on GPT-4) 2025.03	0.932	0.93	-
Mistral2-7B REAL (ours) 2026.03	0.932	0.934	0.825
TRACT 2025.03	0.931	0.93	-
Baseline B.4 (RAFT) 2025.03	0.92	0.918	-
TRACT 2025.03	0.92	0.917	-
Qwen3-8B REAL (ours) 2026.03	0.92	0.921	0.857
TRACT (Ablation: Objective CE) 2025.03	0.919	0.917	-
Qwen3-32B REAL (ours) 2026.03	0.911	0.917	0.859
Mistral-7B-Instruct (CE on GPT-4 scores) 2025.03	0.89	0.891	-
TRACT (Ablation: Data GPT-4) 2025.03	0.879	0.88	-
Mistral2-7B RAFT 2026.03	0.879	0.88	0.763
Mistral-7B-Instruct (CE on Self-gen CoT) 2025.03	0.873	0.873	-
Mistral-7B-Instruct (CE on GPT-4 CoT) 2025.03	0.872	0.872	-
Prometheus-1-13B 2026.03	0.86	0.858	0.771
Baseline B.1 (CE) 2025.03	0.857	0.857	-
Qwen3-32B RAFT 2026.03	0.854	0.865	0.729
Prometheus-2-7B 2026.03	0.853	0.853	0.729
Prometheus-1-7B 2026.03	0.847	0.849	0.767
Prometheus-2-7B 2025.03	0.845	0.847	-
Prometheus-2-7B 2026.03	0.845	0.847	0.765
Qwen3-8B RAFT 2026.03	0.843	0.855	0.73
Mistral2-7B Base (w/ warmup) 2026.03	0.837	0.843	0.702
Baseline B.2 (CE) 2025.03	0.835	0.834	-
Mistral2-7B Base (w/ warmup) 2026.03	0.831	0.833	0.748
Baseline B.3 (RAIL) 2025.03	0.683	0.689	-
TRACT (Ablation: Stage 1 Init) 2025.03	0.674	0.684	-
Qwen3-32B Base 2026.03	0.634	0.708	0.567
Qwen3-8B Base 2026.03	0.567	0.654	0.541
Qwen3-8B Base 2026.03	0.566	0.627	0.539
GPT-3.5-Turbo-0613 2026.03	0.563	0.521	0.453
CLoud (Reward model) 2025.03	0.381	0.376	-
Mistral-7B-Instruct (Zero-shot RAIL) 2025.03	0.197	0.175	-