Reward Prediction

Benchmarks

Dataset Name	SOTA Method	Metric
RewardPrediction	VLWM-critic	AlfWorld Reward Score82.3	13	4mo ago
RoboRewardBench 1.0	RoboReward 8B	Overall MAE0.665	12	5mo ago
D18	Bid2X	MAE4.01	10	4mo ago
D12	Bid2X	MAE5.45	10	4mo ago
D6	Bid2X	MAE7.82	10	4mo ago
BL	Bid2X	MAE170.03	10	4mo ago
BM	Bid2X	MAE31.91	10	4mo ago
BS	Bid2X	Mean Absolute Error (MAE)7.76	10	4mo ago
TR	Bid2X	MAE8.9	10	4mo ago
BCB	Bid2X	MAE5.94	10	4mo ago
UI-TARS-7B trajectories	PRORE	Accuracy96.5	9	3mo ago
M3A trajectories	PRORE	Accuracy91.4	9	3mo ago
V-Droid trajectories	PRORE	Accuracy93.1	9	3mo ago
10-task benchmark S1 classic	SARM2	Demo L (MSE)0.006	8	1mo ago
RoboRewardBench	RoboReward-8B	MAE0.67	8	4mo ago
Human preference annotation (test)		Effort30	8	5mo ago
10-task benchmark Overall	SARM2	Demo L (MSE)0.02	7	1mo ago
S2 10-task benchmark unconventional	SARM2	Demo L MSE0.031	7	1mo ago
Out-of-Domain Task Suite NuminaMath, SIQA, PIQA	MemReward	Average Score66.96	6	4mo ago
PIQA (out-of-domain)		Accuracy82.22	6	4mo ago
SIQA (out-of-domain)	MemReward	Accuracy76.89	6	4mo ago
NuminaMath (out-of-domain)	MemReward	Accuracy42.22	6	4mo ago
OSWorld Chrome	PRORE	Reward Accuracy93.5	5	3mo ago
OSWorld	PRORE	Reward Accuracy92	5	3mo ago
ManiSkill (image-based) 4-task Color Swapped (OOD)	ReCouPLe-EC	Pick Score96	5	4mo ago

Showing 25 of 31 rows