MTEB
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Text Embedding | MTEB English v2 | Mean Score74.6 | 68 | |
| Sentence Embedding Evaluation | MTEB (test) | Classification Score90.37 | 55 | |
| Text Embedding | MTEB | MTEB Score66.43 | 45 | |
| Multilingual Text Embedding | MTEB Multilingual | Mean Score (Task)72.3 | 29 | |
| Multilingual Retrieval | MTEB Multilingual v2 | nDCG@1070.9 | 28 | |
| Information Retrieval | MTEB v2 | NDCG@1045.9 | 28 | |
| Code Retrieval | MTEB Code Retrieval Average (val) | nDCG@574 | 24 | |
| Text Embedding | MTEB Turkish (test) | Overall MTEB Score65.42 | 23 | |
| Clustering | MTEB Clustering | Bior Score33.13 | 23 | |
| Retrieval | MTEB eng v2 | nDCG@1069.4 | 19 | |
| Semantic Textual Similarity (STS) | MTEB English 2023 (test) | BIO89.37 | 19 | |
| Text Embedding Evaluation | MTEB eng v2 (test) | Retrieval Score42.7 | 18 | |
| Clustering | MTEB Clustering v1 (test) | TNG58.14 | 18 | |
| Text Embedding | MTEB v2 | Clustering Score42.5 | 17 | |
| Text Embedding Evaluation | MTEB [hye] | Score78.04 | 13 | |
| Text Embedding | MTEB Multilingual V2 (test) | Bitext Mining Score83.8 | 12 | |
| Code Retrieval | MTEB Code (test) | Apps Score98.1 | 12 | |
| Sentence Embedding | MTEB Clustering standard (test) | AskU. Score63.48 | 12 | |
| Multilingual Information Retrieval | MTEB-ZH 8 subsets (test) | nDCG@100.72 | 11 | |
| Multilingual Information Retrieval | MTEB-FR 5 subsets (test) | nDCG@1057.6 | 11 | |
| Reranking | MTEB Reranking (test) | MAP (AU)54.56 | 11 | |
| Code Retrieval | MTEB Code | APPS93.75 | 10 | |
| Text Embedding | MTEB (test) | Bioranking Score44.57 | 10 | |
| Retrieval | MTEB English BEIR Retrieval | Average Score38.46 | 10 | |
| Semantic Textual Similarity | MTEB English STS subsets | Average STS Score75.88 | 10 |
Showing 25 of 51 rows