Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model
About
Automatic generation of summaries from multiple news articles is a valuable tool as the number of online publications grows rapidly. Single document summarization (SDS) systems have benefited from advances in neural encoder-decoder model thanks to the availability of large datasets. However, multi-document summarization (MDS) of news articles has been limited to datasets of a couple of hundred examples. In this paper, we introduce Multi-News, the first large-scale MDS news dataset. Additionally, we propose an end-to-end model which incorporates a traditional extractive summarization model with a standard SDS model and achieves competitive results on MDS datasets. We benchmark several methods on Multi-News and release our data and code in hope that this work will promote advances in summarization in the multi-document setting.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Long-form Question Answering | ELI5 (test) | ROUGE-L22.9 | 54 | |
| Multi-document summarization | Multi-News (test) | ROUGE-216.05 | 45 | |
| Multi-document summarization | DUC 2004 (test) | ROUGE-1 Score35.78 | 9 | |
| Multi-document summarization | Multi-News (test) | Informativeness85 | 4 |