Intent Detection

Benchmarks

Dataset Name	SOTA Method	Metric
All Posts and Comment Mean	BOT-MOD	Mean Score68.51	42	20d ago
ATIS		ID Accuracy99.43	32	1mo ago
Comment Dataset Out-of-Distribution	BOT-MOD	F1 Score (val)65.6	21	20d ago
Posts Dataset Out-of-Distribution	self-consistency	F1 Score (val)70.37	21	20d ago
Comment Dataset In-Distribution	BOT-MOD	F1 Score (Validation)66.47	21	20d ago
Posts Dataset In-Distribution		F1 Score (Validation)76.46	21	20d ago
M2M	MIDAS	Accuracy94.27	18	3mo ago
MWOZ	MIDAS	Accuracy85.02	18	3mo ago
SLURP	ML-LMCL	Accuracy89.16	16	3mo ago
CLINC 10-shot (test)	ROBERTa-Large + Classifier + ICDA	Accuracy94.84	16	3mo ago
HWU 10-shot (test)	ROBERTa-Large + Classifier + ICDA	Accuracy87.41	16	3mo ago
BANKING 10-shot (test)	ROBERTa-Large + Classifier + ICDA	Accuracy89.79	16	3mo ago
SNIPS Type 3 (test)	ID-ALC	Accuracy96.2	15	2mo ago
SNIPS Type 2 (test)	LLaMA2-FT+ALC	Accuracy98.9	15	2mo ago
SNIPS Type 1 (test)	ID-ALC	Accuracy98.6	15	2mo ago
ATIS (test)	Joint model with recurrent intent + slot label context	Intent Error1.57	13	3mo ago
CLINC 5-shot (test)	ROBERTa-Large + Classifier + ICDA	Accuracy92.62	12	3mo ago
HWU 5-shot (test)	ROBERTa-Large + Classifier + ICDA	Accuracy0.8245	12	3mo ago
BANKING 5-shot (test)	ROBERTa-Large + Classifier + ICDA	Accuracy84.01	12	3mo ago
CLINC Full (test)	CONVBERT	Accuracy97.31	11	3mo ago
HWU Full (test)	CONVBERT	Accuracy93.03	11	3mo ago
BANKING Full (test)	ROBERTa-Large + Classifier + ICDA	Accuracy94.42	11	3mo ago
TREC6	ML-LMCL	Accuracy89.96	10	3mo ago
Chile es	Self-Debias	Macro F172.4	9	2mo ago
Haiti ht	Goal-based strategy	Macro-F169.8	9	2mo ago

Showing 25 of 36 rows