QMSum
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Query-based meeting summarization | QMSum (test) | ROUGE-219.63 | 34 | |
| Next Token Prediction | QMSum | Next Token Accuracy47 | 32 | |
| Long-context language generation | QMSum | Average Acceptance Length (τ)3.15 | 25 | |
| Summarization | QMSum (val) | ROUGE-L0.2378 | 17 | |
| Traceback (Prompt Injection Attacks) | QMSum | Precision99 | 13 | |
| Abstractive Summarization | QMSum | BLEU6.75 | 11 | |
| Context Traceback | QMSum LongBench | Precision99 | 10 | |
| Synthetic Text Generation | QMSum | Mean Embedding Similarity52 | 10 | |
| Document Summarization | QMSum (test) | ROUGE-138.9 | 10 | |
| Document Summarization | QMSum | G-mean15.47 | 9 | |
| Summarization | QMSum | Std Dev ROUGE-10.3 | 8 | |
| Query-focused Summarization | QMSum (test) | ROUGE-138.06 | 7 | |
| Payload-splitting attack detection | QMSum | Precision (QMSum)81 | 6 | |
| Query-based Meeting Summarization | QMSum | ROUGE-L10 | 6 | |
| Query-focused Meeting Summarization | QMSum 50 samples | Fluency4.88 | 6 | |
| Summarization | QMSum (test) | Fluency4.93 | 5 | |
| Next Token Prediction | QMSum | Acc (BERT-Small, Epsilon=Inf)32.82 | 4 | |
| Abstractive Meeting Summarization | QMSum | Coreference1.67 | 4 | |
| Meeting Summarization | QMSum (all turns) | ROUGE-134.03 | 4 | |
| Long-Context Summarization | QMSum | ROUGE-L15.32 | 3 | |
| Meeting Summarization | QMSum Gold turns only | ROUGE-140.2 | 3 | |
| Query-based Summarization | QMSum SCROLLS (val) | ROUGE-130.9 | 2 | |
| Transcript Challenge Assessment | QMSum (test) | Spoken Language Score3 | 1 | |
| Meeting Transcript Evaluation | QMSum | Coherence4.5 | 1 |
Showing 24 of 24 rows