Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies

About

We present NEWSROOM, a summarization dataset of 1.3 million articles and summaries written by authors and editors in newsrooms of 38 major news publications. Extracted from search and social media metadata between 1998 and 2017, these high-quality summaries demonstrate high diversity of summarization styles. In particular, the summaries combine abstractive and extractive strategies, borrowing words and phrases from articles at varying rates. We analyze the extraction strategies used in NEWSROOM summaries against other datasets to quantify the diversity and difficulty of our new data, and train existing methods on the data to evaluate its utility and challenges.

Max Grusky, Mor Naaman, Yoav Artzi• 2018

Related benchmarks

TaskDatasetResultRank
SummarizationarXiv (test)
ROUGE-132.06
161
SummarizationPubmed
ROUGE-135.86
70
SummarizationbigPatent
ROUGE-135.99
61
SummarizationNewsroom (test)
ROUGE-249
40
Abstractive Snippet GenerationWeb Document Snippets 100 sample human study WWW '20 (test)
Fluency Agreement (Majority)0.9
12
SummarizationNewsroom Mixed
ROUGE-125.5
11
Snippet UsefulnessSnippet Usefulness Study (n=100 snippets)
F-score61.85
7
Showing 7 of 7 rows

Other info

Follow for update