Contrastive Learning of Sociopragmatic Meaning in Social Media
About
Recent progress in representation and contrastive learning in NLP has not widely considered the class of \textit{sociopragmatic meaning} (i.e., meaning in interaction within different language communities). To bridge this gap, we propose a novel framework for learning task-agnostic representations transferable to a wide range of sociopragmatic tasks (e.g., emotion, hate speech, humor, sarcasm). Our framework outperforms other contrastive learning frameworks for both in-domain and out-of-domain data, across both the general and few-shot settings. For example, compared to two popular pre-trained language models, our method obtains an improvement of $11.66$ average $F_1$ on $16$ datasets when fine-tuned on only $20$ training samples per dataset.Our code is available at: https://github.com/UBC-NLP/infodcl
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Semantic Textual Similarity | STS tasks (STS12, STS13, STS14, STS15, STS16, STS-B, SICK-R) various (test) | STS12 Score48.13 | 393 | |
| Transfer Learning | SentEval Transfer tasks (test) | MR86.83 | 23 | |
| Crisis Classification | CrisisOltea v1 (test) | Macro F196.01 | 14 | |
| Emotion Detection | EmoMoham v1 (test) | Macro F1 Score81.96 | 14 | |
| Hate Speech Detection | HateWas v1 (test) | Macro F157.65 | 14 | |
| Out-of-domain performance average | Average Out-of-Domain | Macro F175.54 | 14 | |
| Twitter dataset performance average | Average In-Domain | Macro F178.58 | 14 | |
| Sociopragmatic classification | 16 in-domain sociopragmatic datasets (test) | F1 (N=20)46.88 | 6 | |
| Sociopragmatic classification | 8 out-of-domain sociopragmatic datasets (test) | Avg F1 (N=20)42.19 | 6 | |
| Emotion Classification | EmoBian Italian (test) | Macro F1 Score74.07 | 5 |