Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

MTEB

Benchmarks

Task NameDataset NameSOTA ResultTrend
Text EmbeddingMTEB English v2
Mean Score74.6
50
Sentence Embedding EvaluationMTEB (test)
Re-Rank Score61.42
48
Text EmbeddingMTEB
MTEB Score66.43
45
Multilingual Text EmbeddingMTEB Multilingual
Mean Score (Task)72.3
29
Information RetrievalMTEB v2
NDCG@1045.9
28
Text EmbeddingMTEB Turkish (test)
Overall MTEB Score65.42
23
ClusteringMTEB Clustering
Bior Score33.13
23
Semantic Textual Similarity (STS)MTEB English 2023 (test)
BIO89.37
19
ClusteringMTEB Clustering v1 (test)
TNG58.14
18
Sentence EmbeddingMTEB Clustering standard (test)
AskU. Score63.48
12
Multilingual RetrievalMTEB Multilingual v2
MTEB-M Score69.6
11
Multilingual Information RetrievalMTEB-ZH 8 subsets (test)
nDCG@100.72
11
Multilingual Information RetrievalMTEB-FR 5 subsets (test)
nDCG@1057.6
11
RerankingMTEB Reranking (test)
MAP (AU)54.56
11
Code RetrievalMTEB Code
APPS93.75
10
Text EmbeddingMTEB (test)
Bioranking Score44.57
10
RetrievalMTEB English BEIR Retrieval
Average Score38.46
10
Semantic Textual SimilarityMTEB English STS subsets
Average STS Score75.88
10
SummarizationMTEB English Summarization
Main Score32.17
10
ClusteringMTEB English Clustering
Average Score45.8
10
ClassificationMTEB English Classification subsets
Average Classification Score65.12
10
Information RetrievalMTEB (test)
MRR (Avg Med)0.677
9
Information RetrievalMTEB Retrieval (BEIR) 1.0 (test)
MSMARCO46.5
8
Dense Text EmbeddingMTEB v2 (test)
Classification Score70.1
7
Document RetrievalMTEB v2
NDCG@100.39
7
Showing 25 of 35 rows