Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

A large annotated corpus for learning natural language inference

About

Understanding entailment and contradiction is fundamental to understanding natural language, and inference about entailment and contradiction is a valuable testing ground for the development of semantic representations. However, machine learning research in this area has been dramatically limited by the lack of large-scale resources. To address this, we introduce the Stanford Natural Language Inference corpus, a new, freely available collection of labeled sentence pairs, written by humans doing a novel grounded task based on image captioning. At 570K pairs, it is two orders of magnitude larger than all other resources of its type. This increase in scale allows lexicalized classifiers to outperform some sophisticated existing entailment models, and it allows a neural network-based model to perform competitively on natural language inference benchmarks for the first time.

Samuel R. Bowman, Gabor Angeli, Christopher Potts, Christopher D. Manning• 2015

Related benchmarks

TaskDatasetResultRank
Natural Language InferenceSNLI (test)
Accuracy83.2
681
Language ModelingPenn Treebank (test)
Perplexity115
411
Natural Language InferenceSNLI
Accuracy77.6
174
Sentiment AnalysisSST-5 (test)
Accuracy46.4
173
Natural Language InferenceSNLI (train)
Accuracy99.7
154
Sentiment ClassificationStanford Sentiment Treebank SST-2 (test)
Accuracy84.9
99
Matching Question and AnswerYahoo! Answers (test)
Precision@1 (Top 5)66.9
11
Hallucination DetectionMetaQA 1hop (Qwen2.5-7B)
AUC50.44
7
Hallucination DetectionMetaQA 1hop (LLaMA2-7B)
AUC57.41
7
Showing 9 of 9 rows

Other info

Code

Follow for update