Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection
About
Our submission presented in this paper is for SemEval-2026 Task 9: Multilingual Text Classification Challenge - Polarization Detection and it covers all three subtasks: (1) binary polarization detection, (2) polarization type classification and (3) polarization manifestation identification. We adopt a systematic approach of research on short designed prompts by considering twelve designed prompts that are different in terminology clarity, detail of the definition, guidance of reasoning and in-context examples use. The experiments are conducted using aya-101 and Gemma3-27B, with the latter chosen for the submission at the end of the development through performance considerations. Our system has an average macro level F1-score of 0.762 on Subtask 1, 0.587 on Subtask 2 and 0.444 on Subtask 3 with the average accuracy of 0.819, 0.678 and 0.498, respectively, on the official test set averaged among 22 languages, respectively. With cross-task and cross-lingual analysis, we demonstrate that prompt-based approaches can be used effectively to detect coarse grained polarization but encounter more and more difficulties as far as fine-grained and multi-label sociolinguistic classification is concerned.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Multilingual Taxonomy Enrichment | SemEval Subtask 1 | F1 Score92.11 | 44 | |
| Subtask 1 | SemEval | Total Count5 | 22 | |
| Subtask 2 | SemEval | Total Count7 | 15 | |
| Subtask 3 | SemEval | Total Count1 | 11 | |
| Subtask 2 | SemEval Subtask 2 amh | F1 Score54.58 | 2 | |
| Subtask 2 | SemEval Subtask 2 arb | F1 Score65.17 | 2 | |
| Subtask 2 | SemEval Subtask 2 ben | F1 Score42.16 | 2 | |
| Subtask 2 | SemEval Subtask 2 deu | F1 Score59.94 | 2 | |
| Subtask 2 | SemEval Subtask 2 eng | F1 Score50.27 | 2 | |
| Subtask 2 | SemEval Subtask 2 fas | F1 Score57.57 | 2 |