SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking
About
In goal-oriented dialog systems, belief trackers estimate the probability distribution of slot-values at every dialog turn. Previous neural approaches have modeled domain- and slot-dependent belief trackers, and have difficulty in adding new slot-values, resulting in lack of flexibility of domain ontology configurations. In this paper, we propose a new approach to universal and scalable belief tracker, called slot-utterance matching belief tracker (SUMBT). The model learns the relations between domain-slot-types and slot-values appearing in utterances through attention mechanisms based on contextual semantic vectors. Furthermore, the model predicts slot-value labels in a non-parametric way. From our experiments on two dialog corpora, WOZ 2.0 and MultiWOZ, the proposed model showed performance improvement in comparison with slot-dependent methods and achieved the state-of-the-art joint accuracy.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Dialog State Tracking | MultiWOZ 2.1 (test) | Joint Goal Accuracy66.5 | 88 | |
| Dialogue State Tracking | MultiWOZ 2.1 (test) | Joint Goal Accuracy49.2 | 85 | |
| Dialogue State Tracking | MultiWOZ 2.2 (test) | Joint Goal Accuracy49.7 | 80 | |
| Dialogue State Tracking | WOZ 2.0 (test) | Joint Goal Accuracy91 | 65 | |
| Dialog State Tracking | MultiWOZ 2.0 (test) | Joint Goal Accuracy46.65 | 47 | |
| Dialogue State Tracking | MultiWOZ 2.4 (test) | Joint Goal Acc61.9 | 45 | |
| Dialogue State Tracking | MultiWOZ 2.0 (test) | Joint Goal Accuracy42.4 | 13 | |
| Dialogue State Tracking | MultiWOZ 2.3 (test) | JGA52.9 | 11 |