Preference Prediction

Benchmarks

Dataset Name	SOTA Method	Metric
PRISM (test)	EXACT	Accuracy66.62	51	3mo ago
Arena-Expert-5K, HelpSteer3, HH-RLHF, and UltraFeedback (held-out)	PREMISE (+ pref repair)	Accuracy70.5	42	2d ago
PPE Preference (test)	Skywork-Reward-V2-Llama-3.1-8B-40M	Preference Score79.8	24	3mo ago
MMRB2 out-of-domain	AutoRubric-T2I (on PickScore)	EvalMuse Score70.7	22	15d ago
MovieLens 10k		Accuracy (Q=0.25)61	20	6d ago
PickScore (test)	DRM	Accuracy73.4	19	7d ago
RewardBench	C2	Accuracy91.8	19	21d ago
HPD v3 (test)	HPSv3 - 7B	Accuracy76.9	14	7d ago
Sushi Preference Category B		Accuracy (0.25 Quantile)67	10	6d ago
Netflix Prize 150k ratings		Accuracy (0.25 Quantile)61	10	6d ago
Netflix Prize 100k ratings		Accuracy (0.25 Quantile)62	10	6d ago
MovieLens 50k ratings		Accuracy (0.25 Quantile)59	10	6d ago
MovieLens 1k ratings		Accuracy (0.25 Quantile)62	10	6d ago
JudgeBench		Positional Consistent Accuracy63.9	10	1mo ago
RewardBench 2		Accuracy73.9	10	1mo ago
RM-Bench	C2	Accuracy87.8	10	1mo ago
Pets	SPL	Accuracy100	8	2mo ago
UltraFeedback 500 held-out users (test)	RFM(32)	Test Accuracy70.53	7	3mo ago
Sushi Preference Category A		Accuracy (Q0.25)68	5	6d ago
Eigen-taste Jokes onehot variant		Accuracy (0.25 Q)61	5	6d ago
HPSv3 (test)	HPSv3	Accuracy74	5	15d ago
Meta-World Pick-Place (Novel Task)	ReCouPLe-EC	Reward Accuracy66.3	4	2mo ago
Meta-World Pick-Place-Wall (train)	ReCouPLe-IC	Reward Accuracy65.7	4	2mo ago
Meta-World Push-Wall (train)	RFP	Reward Accuracy90	4	2mo ago
Meta-World Push (train)	ReCouPLe-IC	Reward Accuracy89.3	4	2mo ago

Showing 25 of 31 rows