TLDR: Extreme Summarization of Scientific Documents
About
We introduce TLDR generation, a new form of extreme summarization, for scientific papers. TLDR generation involves high source compression and requires expert background knowledge and understanding of complex domain-specific language. To facilitate study on this task, we introduce SciTLDR, a new multi-target dataset of 5.4K TLDRs over 3.2K papers. SciTLDR contains both author-written and expert-derived TLDRs, where the latter are collected using a novel annotation protocol that produces high-quality summaries while minimizing annotation burden. We propose CATTS, a simple yet effective learning strategy for generating TLDRs that exploits titles as an auxiliary training signal. CATTS improves upon strong baselines under both automated metrics and human evaluations. Data and code are publicly available at https://github.com/allenai/scitldr.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Corpus Composition Inference | LLMScan Coarse-Grained 1.0 (test) | Overlap Accuracy41.52 | 48 | |
| Corpus Composition Inference | LLMScan Mid-Grained 1.0 (test) | Overlap Accuracy52.33 | 36 | |
| Extreme Summarization | SciTLDR 1.0 (test) | ROUGE-144.9 | 20 | |
| Scientific Extreme Summarization | SciTLDR (test) | ROUGE-144.3 | 14 | |
| Corpus Composition Inference | LLMScan Fine-Grained 1.0 (test) | Overlap Accuracy25.46 | 12 | |
| Abstractive Summarization | SciTLDR Abstracts | ROUGE-143.8 | 8 | |
| Abstractive Summarization | SciTLDR AIC | ROUGE-144.9 | 8 | |
| Abstractive Summarization | SciTLDR AIC (test) | Mean ROUGE-132.1 | 4 | |
| Text Compression | SciTLDR (test) | ROUGE-1 F144.9 | 4 | |
| Text Summarization | SciTLDR 76 gold papers | MRR0.54 | 4 |