SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection
About
Lexical Semantic Change detection, i.e., the task of identifying words that change meaning over time, is a very active research area, with applications in NLP, lexicography, and linguistics. Evaluation is currently the most pressing problem in Lexical Semantic Change detection, as no gold standards are available to the community, which hinders progress. We present the results of the first shared task that addresses this gap by providing researchers with an evaluation framework and manually annotated, high-quality datasets for English, German, Latin, and Swedish. 33 teams submitted 186 systems, which were evaluated on two subtasks.
Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky, Nina Tahmasebi• 2020
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Lexical Semantic Change Detection | SemEval Task 1 Subtask 2 English 2020 | Spearman Correlation0.436 | 54 | |
| Binary Lexical Semantic Change Detection | SemEval Subtask 1 (English) 2020 | Accuracy0.73 | 28 | |
| Binary semantic change detection | SemEval Subtask 1 2020 | Average Accuracy61.3 | 11 | |
| Graded semantic change detection | SemEval Subtask 2 2020 | Average Score0.144 | 10 |
Showing 4 of 4 rows