Snips
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Intent Classification | SNIPS (test) | Accuracy99.7 | 40 | |
| Natural Language Understanding | Snips (test) | Intent Acc99.29 | 27 | |
| Slot Filling | SNIPS (test) | F1 Score0.983 | 25 | |
| Intent Detection | SNIPS Type 3 (test) | Accuracy96.2 | 15 | |
| Intent Detection | SNIPS Type 2 (test) | Accuracy98.9 | 15 | |
| Intent Detection | SNIPS Type 1 (test) | Accuracy98.6 | 15 | |
| Spoken Language Understanding | SNIPS | Slot F197 | 15 | |
| Unknown Intent Detection | SNIPS (test) | Macro F184.1 | 15 | |
| Slot filling | SNIPS all target domains | F1 Score0.578 | 12 | |
| Slot Filling | SNIPS SearchCreativeWork zero-shot (test) | F1-score72.88 | 11 | |
| Slot Filling | SNIPS RateBook zero-shot (test) | F1 Score47.53 | 11 | |
| Slot Filling | SNIPS PlayMusic zero-shot (test) | F1 Score66.42 | 11 | |
| Slot Filling | SNIPS zero-shot Average | F1 Score61.07 | 11 | |
| Slot Filling | SNIPS zero-shot (SearchScreeningEvent) | F1 Score51.42 | 11 | |
| Slot Filling | SNIPS GetWeather zero-shot | F1 Score65.36 | 11 | |
| Slot Filling | SNIPS BookRestaurant zero-shot | F1 Score63.77 | 11 | |
| Slot Filling | SNIPS zero-shot AddToPlaylist | F1 Score68.7 | 11 | |
| Clustering | SNIPS (test) | NMI89.3 | 11 | |
| OOD Intent Detection | SNIPS standard (test) | Macro F192.32 | 10 | |
| Intent Classification | SNIPS (unsupervised) | Accuracy97 | 9 | |
| Intent Detection | SNIPS | F1 Score98.7 | 8 | |
| Slot Filling | SNIPS 5-shot | We Score89.39 | 8 | |
| Slot Filling | SNIPS | CT10 | 7 | |
| Open intent recognition | SNIPS | Accuracy96.1 | 6 | |
| Slot Filling | SNIPS 1-shot | Average Score70.44 | 6 |
Showing 25 of 38 rows