MSC
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Dialogue Response Generation | MSC | B-4 Score35.8 | 38 | |
| Language Modeling | MSC Session Openings 1.0 (val) | Perplexity7.78 | 10 | |
| Language Modeling | MSC Session 5 1.0 (val) | Perplexity8.99 | 10 | |
| Language Modeling | MSC Session 4 1.0 (val) | Perplexity9.07 | 10 | |
| Language Modeling | MSC Session 3 1.0 (val) | Perplexity8.96 | 10 | |
| Language Modeling | MSC Session 2 1.0 (val) | Perplexity9.08 | 10 | |
| Language Modeling | MSC Session 1 1.0 (val) | Perplexity8.14 | 10 | |
| Text Generation | MSC | SacreBLEU1.23 | 5 | |
| Conversational Memory | MSC | RP@1077.2 | 5 | |
| Sparse Matrix-Vector multiplication | msc10848 | Memory (MB)1,014.04 | 4 | |
| Speech Mask Detection | MSC (test) | UAR72.5 | 3 | |
| Head-to-Head Comparative Evaluation | MSC (test) | Wins289 | 2 | |
| Conversational Quality Evaluation (Conversational Turns) | MSC 10% human-annotated sample | Topic Consistency64.12 | 1 | |
| Pulmonary nodule diagnosis | MSC | AUC0.927 | 1 |
Showing 14 of 14 rows