SamSum
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Dialogue Summarization | SAMSum (test) | ROUGE-233 | 80 | |
| Abstractive Summarization | SAMSum | ROUGE-228.97 | 73 | |
| Summarization | SamSum | PRR-0.113 | 66 | |
| Selective Generation | SamSum | ROC-AUC82.1 | 66 | |
| Abstractive dialogue summarization | SAMSum (test) | ROUGE-L52.7 | 53 | |
| Few-shot Learning | SAMSum | Score41.62 | 40 | |
| Summarization | SAMSum LongBench | ROUGE-L43.57 | 30 | |
| Summarization | SAMSum | ROUGE Score27.2 | 30 | |
| Summarization | SAMSum Full 2019 | F1 Score37 | 30 | |
| Summarization | SAMSum | BERTScore F191.3 | 30 | |
| Factual Consistency Evaluation | SAMSum | Spearman Correlation46.7 | 30 | |
| Answer Accuracy | Samsum | BRT Accuracy39.7 | 26 | |
| Factual Consistency Evaluation | SamSum (test) | Pearson Correlation Coefficient44.6 | 22 | |
| Meeting Summarization | SamSum | HPI6.4347 | 22 | |
| Selective Prediction | SAMSum | PRR32.9 | 20 | |
| Summarization | SAMSum | AlignScore89.5 | 19 | |
| Summarization | SamSum (test) | ROUGE-153.4 | 18 | |
| Selective Generation | SamSum | PRR (ROUGE-L)48.6 | 14 | |
| Language Modeling | SAMSum | Perplexity31.18 | 13 | |
| Summarization Faithfulness | SAMSum | SummaC41.08 | 12 | |
| Abstractive Summarization | SAMSum sampled (test) | ROUGE Score26.88 | 12 | |
| Faithfulness Evaluation | SAMSum (test) | SummaC29.58 | 12 | |
| Summarization | SAMSum | Completeness4.98 | 12 | |
| Summarization | SAMSum | ROUGE-L31.46 | 12 | |
| Dialogue Summarization | SAMSum 1.0 (test) | R151 | 11 |
Showing 25 of 57 rows