Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Instruction Tuning with Retrieval-based Examples Ranking for Aspect-based Sentiment Analysis

About

Aspect-based sentiment analysis (ABSA) identifies sentiment information related to specific aspects and provides deeper market insights to businesses and organizations. With the emergence of large language models (LMs), recent studies have proposed using fixed examples for instruction tuning to reformulate ABSA as a generation task. However, the performance is sensitive to the selection of in-context examples; several retrieval methods are based on surface similarity and are independent of the LM generative objective. This study proposes an instruction learning method with retrieval-based example ranking for ABSA tasks. For each target sample, an LM was applied as a scorer to estimate the likelihood of the output given the input and a candidate example as the prompt, and training examples were labeled as positive or negative by ranking the scores. An alternating training schema is proposed to train both the retriever and LM. Instructional prompts can be constructed using high-quality examples. The LM is used for both scoring and inference, improving the generation efficiency without incurring additional computational costs or training difficulties. Extensive experiments on three ABSA subtasks verified the effectiveness of the proposed method, demonstrating its superiority over various strong baseline models. Code and data are released at https://github.com/zgMin/IT-RER-ABSA.

Guangmin Zheng, Jin Wang, Liang-Chih Yu, Xuejie Zhang• 2024

Related benchmarks

TaskDatasetResultRank
Aspect Sentiment ClassificationRestaurant SemEval 2015 (test)
Accuracy94.31
32
Aspect ExtractionLAPTOP SemEval 2014 (test)
F1 Score90.05
28
Aspect-based Sentiment AnalysisSemEval Laptop 2014--
19
Aspect Sentiment Pair ExtractionLapt14 SemEval-2014 (test)
F1 Score81.49
15
Aspect Sentiment Pair ExtractionRest SemEval 2014 (test)
F1 Score80.73
13
Aspect ExtractionRest SemEval 2016 (test)
F1 Score83.85
12
Aspect Sentiment Pair ExtractionRest SemEval 2015 (test)
ASP F1 Score73.41
11
Aspect Sentiment Pair ExtractionRest SemEval 2016 (test)
F1 Score78.64
9
Aspect Term Sentiment ClassificationSemEval Restaurant 2014
Accuracy90.44
8
Aspect Term ExtractionRest SemEval 2014 (test)
F1 Score90.72
7
Showing 10 of 12 rows

Other info

Code

Follow for update