Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Rethinking Negative Instances for Generative Named Entity Recognition

About

Large Language Models (LLMs) have demonstrated impressive capabilities for generalizing in unseen tasks. In the Named Entity Recognition (NER) task, recent advancements have seen the remarkable improvement of LLMs in a broad range of entity domains via instruction tuning, by adopting entity-centric schema. In this work, we explore the potential enhancement of the existing methods by incorporating negative instances into training. Our experiments reveal that negative instances contribute to remarkable improvements by (1) introducing contextual information, and (2) clearly delineating label boundaries. Furthermore, we introduce an efficient longest common subsequence (LCS) matching algorithm, which is tailored to transform unstructured predictions into structured entities. By integrating these components, we present GNER, a Generative NER system that shows improved zero-shot performance across unseen entity domains. Our comprehensive evaluation illustrates our system's superiority, surpassing state-of-the-art (SoTA) methods by 9 $F_1$ score in zero-shot evaluation.

Yuyang Ding, Juntao Li, Pinzheng Wang, Zecheng Tang, Bowen Yan, Min Zhang• 2024

Related benchmarks

TaskDatasetResultRank
Named Entity RecognitionCoNLL 03
F1 (Entity)93.6
135
Named Entity RecognitionOntoNotes
F1-score91.8
121
Named Entity RecognitionBC5CDR
F1 Score90.3
102
Named Entity RecognitionConll 2003
F1 Score93.6
86
Named Entity RecognitionMIT Movie
Entity F190.2
71
Named Entity RecognitionMIT Restaurant--
57
Named Entity RecognitionmultiNERD
Entity F194.4
50
Named Entity Recognitionbc2gm
Entity F184.3
48
Named Entity RecognitionFabNER
Entity F185.4
45
Named Entity RecognitionBroad Twitter Corpus
Entity F10.813
42
Showing 10 of 12 rows

Other info

Follow for update