STARK
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Knowledge Graph Retrieval | STaRK-Amazon 1.0 (Human) | Hits@161.73 | 32 | |
| Knowledge Graph Retrieval | STARK Amazon | H@170.7 | 25 | |
| Knowledge Graph Retrieval | STARK PRIME | H@168.9 | 25 | |
| Reasoning | STARK | EM79.3 | 24 | |
| Knowledge Graph Retrieval | STaRK-Prime Synthetic 1.0 | Hits@10.464 | 20 | |
| Knowledge Graph Retrieval | STaRK-MAG Synthetic 1.0 | Hits@174.1 | 20 | |
| Knowledge Graph Retrieval | STaRK-Amazon Synthetic 1.0 | Hits@164 | 20 | |
| Retrieval | STaRK MAG Synthetic | Recall@2078.8 | 20 | |
| Knowledge Graph Retrieval | STaRK-Prime 1.0 (Human) | Hits@10.4495 | 16 | |
| Retrieval | STaRK PRIME Human | Recall@2066.37 | 16 | |
| Retrieval | STaRK MAG Human | Recall@2056.4 | 16 | |
| Retrieval | STaRK AMAZON (Human) | Recall@2042.43 | 16 | |
| Knowledge Graph Retrieval | STaRK PRIME synthetic (test) | Hit@151.39 | 13 | |
| Knowledge Graph Retrieval | STaRK MAG synthetic (test) | Hit@173.4 | 13 | |
| Question Answering Retrieval | STaRK synthetically generated | Hit@162 | 12 | |
| Knowledge Graph Retrieval | STARK MAG | Hits@139.7 | 11 | |
| Information Retrieval | STaRK AMAZON human-generated (test) | Hit@161.7 | 10 | |
| Information Retrieval | STaRK MAG human-generated (test) | Hit@152.4 | 10 | |
| Information Retrieval | STaRK PRIME human-generated (test) | Hit@157.1 | 10 | |
| Retrieval | STARK AMAZON | Hit@10.319 | 9 | |
| Retrieval | STARK MAG | Hit Rate @ 124.4 | 9 | |
| Retrieval | STARK-PRIME | Hit@119.9 | 9 | |
| Question Answering Retrieval | STaRK human-generated | Hit@155.8 | 9 | |
| Knowledge Retrieval | STARK-PRIME official (test) | Hit@118.44 | 8 | |
| Knowledge Retrieval | STARK MAG official (test) | Hit@144.36 | 8 |
Showing 25 of 33 rows