SLURP
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Slot Filling | SLURP (test) | F1 Score80.61 | 26 | |
| Intent Classification | SLURP (test) | Accuracy (IC)90.07 | 26 | |
| Intent Detection | SLURP | Accuracy89.16 | 16 | |
| ASR Rescoring | SLURP (test) | WER23.77 | 11 | |
| Spoken Language Understanding | SLURP 6 tasks | Avg Accuracy84.41 | 9 | |
| Spoken Language Understanding | SLURP 3 tasks | Average Accuracy84.41 | 9 | |
| Slot Filling | SLURP Reference transcriptions (test) | SLU F1 Score86.3 | 6 | |
| Slot Filling | SLURP Whisper Medium transcriptions (test) | SLU F1 Score66.5 | 6 | |
| Slot Filling | SLURP Whisper Small transcriptions (test) | SLU-F164.6 | 6 | |
| Slot Filling | SLURP Whisper Base transcriptions WER 26.4% (test) | SLU F1 Score59.1 | 6 | |
| Slot Filling | SLURP Whisper Tiny transcriptions (test) | SLU F1 Score56.4 | 6 | |
| Spoken Language Understanding | SLURP (test) | Scenario Accuracy78.76 | 5 | |
| Speech Intent Classification and Slot Filling | SLURP (test) | Intent Accuracy90.68 | 4 |
Showing 13 of 13 rows