Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

STARK

Benchmarks

Task NameDataset NameSOTA ResultTrend
Knowledge Graph RetrievalSTaRK-Amazon 1.0 (Human)
Hits@161.73
32
Knowledge Graph RetrievalSTaRK-Prime Synthetic 1.0
Hits@10.464
20
Knowledge Graph RetrievalSTaRK-MAG Synthetic 1.0
Hits@174.1
20
Knowledge Graph RetrievalSTaRK-Amazon Synthetic 1.0
Hits@164
20
RetrievalSTaRK MAG Synthetic
Recall@2078.8
20
Knowledge Graph RetrievalSTaRK-Prime 1.0 (Human)
Hits@10.4495
16
RetrievalSTaRK PRIME Human
Recall@2066.37
16
RetrievalSTaRK MAG Human
Recall@2056.4
16
RetrievalSTaRK AMAZON (Human)
Recall@2042.43
16
Knowledge Graph RetrievalSTaRK PRIME synthetic (test)
Hit@151.39
13
Knowledge Graph RetrievalSTaRK MAG synthetic (test)
Hit@173.4
13
Knowledge RetrievalSTARK-PRIME official (test)
Hit@118.44
8
Knowledge RetrievalSTARK MAG official (test)
Hit@144.36
8
Knowledge RetrievalSTARK Amazon official (test)
Hit@149.87
8
Complex RetrievalSTARK-PRIME (test)
MRR42.72
6
Semi-structured Retrieval and Query OptimizationSTARK-PRIME (test)
MRR41.75
6
RetrievalSTARK-PRIME 1.0 (leave-out set of human-generated queries)
Hit@133.03
6
RetrievalSTARK-MAG 1.0 (leave-out set of human-generated queries)
Hit@133.33
6
RetrievalSTARK-AMAZON 1.0 (leave-out set of human-generated queries)
Hit@158.32
6
Showing 19 of 19 rows