Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Towards Zero-Shot Frame Semantic Parsing for Domain Scaling

About

State-of-the-art slot filling models for goal-oriented human/machine conversational language understanding systems rely on deep learning methods. While multi-task training of such models alleviates the need for large in-domain annotated datasets, bootstrapping a semantic parsing model for a new domain using only the semantic frame, such as the back-end API or knowledge graph schema, is still one of the holy grail tasks of language understanding for dialogue systems. This paper proposes a deep learning based approach that can utilize only the slot description in context without the need for any labeled or unlabeled in-domain examples, to quickly bootstrap a new domain. The main idea of this paper is to leverage the encoding of the slot names and descriptions within a multi-task deep learned slot filling model, to implicitly align slots across domains. The proposed approach is promising for solving the domain scaling problem and eliminating the need for any manually annotated data or explicit schema alignment. Furthermore, our experiments on multiple domains show that this approach results in significantly better slot-filling performance when compared to using only in-domain data, especially in the low data regime.

Ankur Bapna, Gokhan Tur, Dilek Hakkani-Tur, Larry Heck• 2017

Related benchmarks

TaskDatasetResultRank
Named Entity RecognitionSciTech News Target Domain
F1 Score67.14
19
Slot FillingSNIPS zero-shot (SearchScreeningEvent)
F1 Score13.83
11
Slot FillingSNIPS PlayMusic zero-shot (test)
F1 Score32.86
11
Slot FillingSNIPS zero-shot AddToPlaylist
F1 Score38.82
11
Slot FillingSNIPS BookRestaurant zero-shot
F1 Score27.54
11
Slot FillingSNIPS GetWeather zero-shot
F1 Score46.45
11
Slot FillingSNIPS zero-shot Average
F1 Score30.55
11
Slot FillingSNIPS RateBook zero-shot (test)
F1 Score14.54
11
Slot FillingSNIPS SearchCreativeWork zero-shot (test)
F1-score39.79
11
Showing 9 of 9 rows

Other info

Follow for update