MSC
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Dialogue Response Generation | MSC | B-4 Score35.8 | 38 | |
| Membership Inference Attack | MSC | ROC-AUC1 | 24 | |
| Multi-Turn Dialogue Generation | MSC (test) | PPL8.431 | 14 | |
| Dialogue Response Generation | MSC Session 3 | BLEU-1 Score24.18 | 10 | |
| Language Modeling | MSC Session Openings 1.0 (val) | Perplexity7.78 | 10 | |
| Language Modeling | MSC Session 5 1.0 (val) | Perplexity8.99 | 10 | |
| Language Modeling | MSC Session 4 1.0 (val) | Perplexity9.07 | 10 | |
| Language Modeling | MSC Session 3 1.0 (val) | Perplexity8.96 | 10 | |
| Language Modeling | MSC Session 2 1.0 (val) | Perplexity9.08 | 10 | |
| Language Modeling | MSC Session 1 1.0 (val) | Perplexity8.14 | 10 | |
| Dialogue Response Generation | MSC Average | BLEU-120.01 | 5 | |
| Dialogue Response Generation | MSC Session 5 | BLEU-120.57 | 5 | |
| Dialogue Response Generation | MSC Session 4 | BLEU-120.19 | 5 | |
| Dialogue Response Generation | MSC Session 2 | BLEU-119.54 | 5 | |
| Text Generation | MSC | SacreBLEU1.23 | 5 | |
| Conversational Memory | MSC | RP@1077.2 | 5 | |
| Dialogue Generation | MSC N=100 (test) | Coherence4.85 | 4 | |
| Multi-Turn Dialogue Generation | MSC | Perplexity (PPL)8.385 | 4 | |
| Retrieval | MSC persona | R@1099.2 | 4 | |
| Sparse Matrix-Vector multiplication | msc10848 | Memory (MB)1,014.04 | 4 | |
| Speech Mask Detection | MSC (test) | UAR72.5 | 3 | |
| Head-to-Head Comparative Evaluation | MSC (test) | Wins289 | 2 | |
| Conversational Quality Evaluation (Conversational Turns) | MSC 10% human-annotated sample | Topic Consistency64.12 | 1 | |
| Pulmonary nodule diagnosis | MSC | AUC0.927 | 1 |
Showing 24 of 24 rows