DART: Open-Domain Structured Data Record to Text Generation
About
We present DART, an open domain structured DAta Record to Text generation dataset with over 82k instances (DARTs). Data-to-Text annotations can be a costly process, especially when dealing with tables which are the major source of structured data and contain nontrivial structures. To this end, we propose a procedure of extracting semantic triples from tables that encodes their structures by exploiting the semantic dependencies among table headers and the table title. Our dataset construction framework effectively merged heterogeneous sources from open domain semantic parsing and dialogue-act-based meaning representation tasks by utilizing techniques such as: tree ontology annotation, question-answer pair to declarative sentence conversion, and predicate unification, all with minimum post-editing. We present systematic evaluation on DART as well as new state-of-the-art results on WebNLG 2017 to show that DART (1) poses new challenges to existing data-to-text datasets and (2) facilitates out-of-domain generalization. Our data and code can be found at https://github.com/Yale-LILY/dart.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Natural language generation | E2E (test) | ROUGE-L68.97 | 79 | |
| Data-to-text generation | DART (test) | BLEU49.21 | 42 | |
| Data-to-text generation | WebNLG (test) | BLEU59.95 | 39 | |
| Data-to-text generation | DART | BLEU46.89 | 16 | |
| Graph-to-text generation | WebNLG seen v1.0 (test) | BLEU52.86 | 12 | |
| Graph-to-text generation | WebNLG all v1.0 (test) | BLEU45.89 | 11 | |
| Graph-to-text generation | WebNLG unseen v1.0 (test) | BLEU37.85 | 10 | |
| Data-to-text generation | Cleaned E2E (test) | BLEU41.83 | 9 | |
| Graph-to-text generation | WebNLG Unseen 2017 (test) | BLEU37.85 | 7 | |
| Graph-to-text generation | WebNLG All 2017 (test) | BLEU45.89 | 7 |