SPLADE-v3: New baselines for SPLADE
About
A companion to the release of the latest version of the SPLADE library. We describe changes to the training structure and present our latest series of models -- SPLADE-v3. We compare this new version to BM25, SPLADE++, as well as re-rankers, and showcase its effectiveness via a meta-analysis over more than 40 query sets. SPLADE-v3 further pushes the limit of SPLADE models: it is statistically significantly more effective than both BM25 and SPLADE++, while comparing well to cross-encoder re-rankers. Specifically, it gets more than 40 MRR@10 on the MS MARCO dev set, and improves by 2% the out-of-domain results on the BEIR benchmark.
Carlos Lassance, Herv\'e D\'ejean, Thibault Formal, St\'ephane Clinchant• 2024
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Text-to-motion generation | HumanML3D (test) | FID0.033 | 481 | |
| Text-to-motion generation | KIT-ML (test) | FID0.143 | 189 | |
| Retrieval | MS MARCO (dev) | MRR@100.387 | 84 | |
| Retrieval | TREC DL 2019 | NDCG@1075.2 | 83 | |
| Information Retrieval | NFCorpus (test) | NDCG@100.348 | 65 | |
| Information Retrieval | SciFact (test) | NDCG@100.688 | 65 | |
| Information Retrieval | BEIR | Average NDCG@100.517 | 62 | |
| Information Retrieval | MS Marco | -- | 56 | |
| Information Retrieval | TREC DL 2020 | nDCG@1075.4 | 33 | |
| Information Retrieval | FIQA BEIR (test) | nDCG@1037.4 | 32 |
Showing 10 of 43 rows