MTEB
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Text Embedding | MTEB English v2 | Mean Score74.6 | 113 | |
| Text Embedding | MTEB | Classification Score90.4 | 82 | |
| Sentence Embedding Evaluation | MTEB (test) | Classification Score90.37 | 55 | |
| Multilingual Retrieval | MTEB Multilingual v2 | nDCG@1070.9 | 40 | |
| Text Embedding | MTEB (test) | Average Score72.97 | 38 | |
| Multilingual Text Embedding | MTEB Multilingual | Mean Score (Task)72.3 | 33 | |
| Retrieval | MTEB eng v2 | nDCG@1069.4 | 31 | |
| Text Embedding | MTEB Code v1 | Average Performance70 | 30 | |
| Information Retrieval | MTEB v2 | NDCG@1045.9 | 28 | |
| Text Embedding Evaluation | MTEB eng v2 (test) | Average Score67.95 | 26 | |
| Code Retrieval | MTEB Code Retrieval Average (val) | nDCG@574 | 24 | |
| Text Embedding | MTEB Turkish (test) | Overall MTEB Score65.42 | 23 | |
| Clustering | MTEB Clustering | Bior Score33.13 | 23 | |
| Code Retrieval | MTEB Code | nDCG@1080.07 | 21 | |
| Reranking | MTEB 8-task suite | Average Reranking Score65.61 | 20 | |
| Reranking | MTEB eng v2 | Ask Score70.2 | 20 | |
| Reranking | MTEB multilingual v2 | Allo Score85.13 | 20 | |
| Semantic Textual Similarity (STS) | MTEB English 2023 (test) | BIO89.37 | 19 | |
| Clustering | MTEB Clustering v1 (test) | TNG58.14 | 18 | |
| Text Embedding | MTEB v2 | Clustering Score42.5 | 17 | |
| Text Embedding | MTEB Multilingual V2 (test) | Mean Score (TaskType)62.5 | 16 | |
| Embedding Evaluation | MTEB Corrupt | Classification Score40 | 15 | |
| Embedding Evaluation | MTEB Clean | Classification Score51.6 | 15 | |
| Text Embedding Evaluation | MTEB [hye] | Score78.04 | 13 | |
| Text Classification | MTEB classification (test) | Emotion Score67 | 12 |
Showing 25 of 61 rows