Share your thoughts, 1 month free Claude Pro on usSee more

Preference Reasoning (Recommendation) on SIMMC 2.1 (test)

38.75R@1

SiPeR

Updated 1mo ago

Evaluation Results

Method	Links
SiPeR 2026.04		38.75	54.09	58.61	45.8	46.83
SiPeR w/o STE 2026.04		33.69	47.85	52.32	40.29	41.66
SiPeR w/o BI-INF 2026.04		31.88	44.26	47.51	38.55	39.13
Qwen2.5-VL 2026.04		29.47	31.69	37.16	29.2	30.42
GPT-4o 2026.04		29.15	47.94	55.45	38.45	39.95
GPT-4o 2026.04		28.12	45.42	53.18	36.21	38.05
ReGeS 2026.04		27.68	45.45	54.12	35.49	37.46
LLaVA-NeXT 2026.04		23.67	26.84	30.18	24.89	27.77
Qwen2.5-VL 2026.04		17.12	19.66	20.02	19.14	19.45
Qwen2.5-VL 2026.04		16.72	18.35	18.61	17.65	17.92
LLaVA-NeXT 2026.04		14.36	15.26	15.48	14.76	15.1
LLaVA-NeXT 2026.04		13.01	13.92	14.12	13.45	13.52
ALBEF 2026.04		6.06	7.45	8.19	7.28	7.45