Search Relevance

Benchmarks

Dataset Name	SOTA Method	Metric
Minor Language family Qwen2.5 series benchmark (test)	SERM	NDCG@184.99	18	3mo ago
Romance language family Qwen2.5 series benchmark (test)	SERM	NDCG@188.14	18	3mo ago
Germanic language family Qwen2.5 series benchmark (test)	SERM	NDCG@187.56	18	3mo ago
ESCI	DeBERTa-v3-large	Macro F161.03	14	2mo ago
WANDs	DeBERTa-v3-large	Macro F191.39	12	2mo ago
JD.com Search Traffic Online Evaluation (A/B test)	K-CARE	Bad Case Rate11.39	3	1mo ago
Taobao Visual Search (Offline Evaluation Set)	REVISION	Top-1 Relevance66.56	2	3mo ago
Manual Annotation Queries Knowledge 2,000 queries	TaoSR1	GSB18.45	1	2mo ago
Manual Annotation Queries Negative 2,000 queries	TaoSR1	GSB Score10.92	1	2mo ago
Manual Annotation Queries Alternative 2,000 queries	TaoSR1	GSB Score34.43	1	2mo ago
Manual Annotation Queries Q&A 2,000 queries	TaoSR1	GSB16.62	1	2mo ago
Online Search Platform Longtail Traffic Current	SERM (Distilled Qwen2.5-7B)	Change Query Ratio-0.1312	1	3mo ago
Online Search Platform Overall Current (Live Traffic)	SERM (Distilled Qwen2.5-7B)	User Negative Feedback-1.2081	1	3mo ago

Showing 13 of 13 rows