InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis
About
We introduce InstructABSA, an instruction learning paradigm for Aspect-Based Sentiment Analysis (ABSA) subtasks. Our method introduces positive, negative, and neutral examples to each training sample, and instruction tune the model (Tk-Instruct) for ABSA subtasks, yielding significant performance improvements. Experimental results on the Sem Eval 2014, 15, and 16 datasets demonstrate that InstructABSA outperforms the previous state-of-the-art (SOTA) approaches on Term Extraction (ATE), Sentiment Classification(ATSC) and Sentiment Pair Extraction (ASPE) subtasks. In particular, InstructABSA outperforms the previous state-of-the-art (SOTA) on the Rest14 ATE subtask by 5.69% points, the Rest15 ATSC subtask by 9.59% points, and the Lapt14 AOPE subtask by 3.37% points, surpassing 7x larger models. We also get competitive results on AOOE, AOPE, and AOSTE subtasks indicating strong generalization ability to all subtasks. Exploring sample efficiency reveals that just 50% train data is required to get competitive results with other instruction tuning approaches. Lastly, we assess the quality of instructions and observe that InstructABSA's performance experiences a decline of ~10% when adding misleading examples.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Aspect-level sentiment classification | SemEval Restaurant 2014 (test) | -- | 67 | |
| Aspect-level sentiment classification | SemEval Laptop 2014 (test) | -- | 59 | |
| aspect sentiment triplet extraction | Rest SemEval 2014 (test) | F1 Score71.17 | 40 | |
| Aspect-based Sentiment Classification | Lap14 | Accuracy81.56 | 37 | |
| Aspect Sentiment Classification | Restaurant SemEval 2015 (test) | Accuracy84.5 | 32 | |
| Aspect-based Sentiment Analysis | SemEval Task 4 Subtask 2 Restaurant domain 2014 (test) | Accuracy88.03 | 30 | |
| Aspect Extraction | LAPTOP SemEval 2014 (test) | F1 Score92.3 | 28 | |
| Aspect Sentiment Classification | Restaurant SemEval 2016 (test) | F1 Score93.01 | 27 | |
| aspect sentiment triplet extraction | Rest 15 (test) | F1 Score60.63 | 26 | |
| Aspect-level Sentiment Analysis | Rest 14 | Accuracy86.25 | 25 |