Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context
About
Conversations have an intrinsic one-to-many property, which means that multiple responses can be appropriate for the same dialog context. In task-oriented dialogs, this property leads to different valid dialog policies towards task completion. However, none of the existing task-oriented dialog generation approaches takes this property into account. We propose a Multi-Action Data Augmentation (MADA) framework to utilize the one-to-many property to generate diverse appropriate dialog responses. Specifically, we first use dialog states to summarize the dialog history, and then discover all possible mappings from every dialog state to its different valid system actions. During dialog system training, we enable the current dialog state to map to all valid system actions discovered in the previous process to create additional state-action pairs. By incorporating these additional pairs, the dialog policy learns a balanced action distribution, which further guides the dialog model to generate diverse responses. Experimental results show that the proposed framework consistently improves dialog policy diversity, and results in improved response diversity and appropriateness. Our model obtains state-of-the-art results on MultiWOZ.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| End-to-end task-oriented dialogue | MultiWOZ (test) | Task Success Rate77.9 | 68 | |
| End-to-end task-oriented dialogue | MultiWOZ 2.1 (test) | BLEU Score18 | 49 | |
| Response Generation | MultiWOZ (test) | BLEU Score27.3 | 27 | |
| End-to-end Dialogue Modelling | MultiWOZ 2.0 (test) | Inform Rate76.33 | 22 | |
| End-to-end task-oriented dialogue | MultiWOZ 2.0 (test) | Inform Accuracy76.33 | 22 | |
| Task-oriented Dialogue | MultiWOZ 2.0 | Inform Rate86.6 | 13 | |
| Task-oriented Dialogue | MultiWOZ 5% 2.0 (train) | Inform52.5 | 10 | |
| Task-oriented Dialogue | MultiWOZ 2.0 (10% train) | Inform Rate55.3 | 10 | |
| Task-oriented Dialogue | MultiWOZ 20% 2.0 (train) | Inform62.6 | 10 | |
| Task-oriented Dialogue | MultiWOZ Attraction domain 2.1 | Inform Rate70 | 4 |