SOTA Pairwise Preference Prediction benchmarks and papers with code

Benchmarks

Dataset Name	SOTA Method	Metric
DyCoBench-1K Overall Preference	DYCORM(OURS)	Preference Rate (A > B)78.3	17	2mo ago
DyCoBench-1K Multiple Criteria	DYCORM(OURS)	Preference Rate (A > B)65.6	17	2mo ago
DyCoBench-1K Single Criterion	DYCORM(OURS)	P(A > B)70.2	17	2mo ago
HPD v3	DiT-Reward	Accuracy77.6	9	1mo ago
HPD v2	DiT-Reward	Accuracy85.6	9	1mo ago
PickScore	PickScore	Accuracy70.5	9	1mo ago
HPD v3++ v1 (test)	HPSv3++	Aesthetic Score79.1	7	1mo ago
MJ-Bench v1 (test)	HPSv3++	Alignment Score69.8	7	1mo ago
PickScore v1 (test)	HPSv3	Accuracy72.2	7	1mo ago
ImageReward v1 (test)	HPSv3	Accuracy66.8	7	1mo ago
IUB-COSINE-S unseen (test)	PrefSQA-F	Accuracy91.61	4	1mo ago
IUB-COSINE-C unseen (test)	PrefSQA	Accuracy83.5	4	1mo ago
SpeechJudge human preference (test)		Accuracy70.4	4	1mo ago
SpeechEval human preference (test)	PrefSQA-F	Accuracy86.85	4	1mo ago
CHiLi NM simulated (test)	PrefSQA	Accuracy90.37	4	1mo ago
CHiLi M simulated (test)	PrefSQA	Accuracy96.29	4	1mo ago
SOMOS NM MOS-derived (test)	PrefSQA	Accuracy74.72	4	1mo ago
SOMOS M MOS-derived (test)	PrefSQA-F	Accuracy73.27	4	1mo ago
NISQA MOS-derived (test)	PrefSQA	Accuracy83.84	4	1mo ago
RBM-EVAL-OOD Different Quality Trajectory Pairs	ROBOMETER	Preference Accuracy (USC Franka)100	4	4mo ago
SWT-bench	PACE	Accuracy90.11	1	22d ago
SWE-bench Multimodal	PACE	Accuracy85.39	1	22d ago
SWE-bench Verified	PACE	Accuracy78.54	1	22d ago
GAIA	PACE	Accuracy83.33	1	22d ago

Showing 24 of 24 rows