Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Teaching Machines to Read and Comprehend

About

Teaching machines to read natural language documents remains an elusive challenge. Machine reading systems can be tested on their ability to answer questions posed on the contents of documents that they have seen, but until now large scale training and test datasets have been missing for this type of evaluation. In this work we define a new methodology that resolves this bottleneck and provides large scale supervised reading comprehension data. This allows us to develop a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure.

Karl Moritz Hermann, Tom\'a\v{s} Ko\v{c}isk\'y, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, Phil Blunsom• 2015

Related benchmarks

TaskDatasetResultRank
Machine ComprehensionCNN (val)
Accuracy0.618
80
Machine ComprehensionCNN (test)
Accuracy66.8
77
Machine ComprehensionCBT NE (test)
Accuracy41.8
56
Machine ComprehensionCBT-CN (test)
Accuracy56
56
Machine Reading ComprehensionDaily Mail (test)
Accuracy69
46
Machine ComprehensionCBT-NE (val)
Accuracy51.2
37
Machine ComprehensionCBT-CN (val)
Accuracy62.6
37
Machine Reading ComprehensionDaily Mail (val)
Accuracy70.5
36
Question AnsweringCNN (test)
Accuracy63
24
SummarizationEMAILSUM long 1.0 (test)
ROUGE-1 (R1)44.15
19
Showing 10 of 22 rows

Other info

Code

Follow for update