Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Towards Open-Domain Topic Classification

About

We introduce an open-domain topic classification system that accepts user-defined taxonomy in real time. Users will be able to classify a text snippet with respect to any candidate labels they want, and get instant response from our web interface. To obtain such flexibility, we build the backend model in a zero-shot way. By training on a new dataset constructed from Wikipedia, our label-aware text classifier can effectively utilize implicit knowledge in the pretrained language model to handle labels it has never seen before. We evaluate our model across four datasets from various domains with different label sets. Experiments show that the model significantly improves over existing zero-shot baselines in open-domain scenarios, and performs competitively with weakly-supervised models trained on in-domain data.

Hantian Ding, Jinrui Yang, Yuqian Deng, Hongming Zhang, Dan Roth• 2023

Related benchmarks

TaskDatasetResultRank
Sentiment AnalysisIMDB (test)
Accuracy62
248
Topic ClassificationAG-News
Accuracy79.6
173
Sentiment AnalysisSST-5 (test)
Accuracy24.5
173
Topic ClassificationYahoo (test)
Accuracy56.5
36
Topic ClassificationDBpedia original (test)
Accuracy90.4
11
Topic ClassificationAG News original (test)
Accuracy79.4
11
Sentiment AnalysisSST-2 original (test)
Accuracy57.3
11
Sentiment AnalysisYelp original (test)
Accuracy58.5
10
Sentiment AnalysisMovie Review mr original (test)
Accuracy56.2
10
Sentiment AnalysisAmazon (amz) original (test)
Accuracy55.8
10
Showing 10 of 11 rows

Other info

Follow for update