Share your thoughts, 1 month free Claude Pro on usSee more

Conversational Recommendation on REDDIT V2 (test)

11.46Recall@5

CRAG

Updated 1mo ago

Evaluation Results

Method
CRAG 2025.10	11.46	17.15	20.3	22.12	8.85	10.65	11.64	12.27
GPT-4o 2025.10	11.06	16.25	19.92	21.47	8.61	10.28	11.36	11.97
Llama-3.2-3B + SFT + Rank-GRPO (exp_inf) 2025.10	10.92	16.53	20.1	22.72	8.45	10.28	11.32	12.03
Llama-3.2-1B + SFT + Rank-GRPO (exp_inf) 2025.10	10.37	15.55	19.27	21.66	8.06	9.73	10.81	11.46
Llama-3.2-3B + SFT + Rank-GRPO (log) 2025.10	10.37	15.72	19.58	22.2	8.16	9.89	11.01	11.69
Llama-3.2-3B + SFT + Vanilla GRPO 2025.10	9.52	14.25	17.26	19.52	7.66	9.16	10.04	10.65
GPT-4o-mini 2025.10	9.49	13.48	16	16.87	7.47	8.77	9.5	9.73
Llama-3.2-3B + SFT + GSPO 2025.10	9.42	14.36	17.7	19.43	7.58	9.14	10.11	10.58
Llama-3.2-1B + SFT + Vanilla GRPO 2025.10	9.37	14.15	17.26	19.74	7.28	8.82	9.75	10.42
Llama-3.2-1B + SFT + Rank-GRPO (log) 2025.10	9.37	14.64	18.32	20.66	7.31	9.02	10.11	10.74
Qwen2.5-0.5B + SFT + Rank-GRPO (exp_inf) 2025.10	9.35	14.12	17.38	18.99	7.35	8.87	9.82	10.25
Llama-3.2-1B + SFT + GSPO 2025.10	9.01	13.89	16.93	18.99	7	8.57	9.47	10.02
Qwen2.5-0.5B + SFT + Rank-GRPO (log) 2025.10	8.18	12.56	16.04	17.64	6.44	7.84	8.86	9.3
Qwen2.5-0.5B + SFT + GSPO 2025.10	8.16	12.43	15.11	16.41	6.47	7.84	8.62	8.96
NBCRS 2025.10	8.01	12.9	16.55	20.19	6.61	8.33	9.54	10.48
Llama-3.2-3B + SFT 2025.10	7.88	11.91	14.1	15.41	6.15	7.44	8.07	8.42
Qwen2.5-0.5B + SFT + Vanilla GRPO 2025.10	7.81	11.76	14.29	15.38	6.23	7.48	8.23	8.52
Llama-3.2-1B + SFT 2025.10	7.54	11.48	13.54	14.98	5.95	7.23	7.82	8.19
UniCRS 2025.10	7.22	10.53	13.44	14.94	5.48	6.4	7.26	7.78
Qwen2.5-0.5B + SFT 2025.10	6.42	10.27	12.12	13.08	5.02	6.25	6.78	7.04
KGSF 2025.10	5.79	9.21	12.46	14.33	4.05	5.03	5.99	6.62
Llama-3.2-3B 2025.10	5.74	8.05	9.12	9.61	4.63	5.35	5.66	5.8
KBRD 2025.10	4.44	8.13	10.58	12.23	3.05	4.18	4.9	5.45
Llama-3.2-1B 2025.10	1.87	2.4	2.57	2.63	1.57	1.7	1.75	1.76
Redial 2025.10	1.03	2.28	2.74	3.22	0.73	1.13	1.28	1.43
Qwen2.5-0.5B 2025.10	0.21	0.21	0.21	0.21	0.23	0.22	0.21	0.21