CodeNER: Code Prompting for Named Entity Recognition
About
Recent studies have explored various approaches for treating candidate named entity spans as both source and target sequences in named entity recognition (NER) by leveraging large language models (LLMs). Although previous approaches have successfully generated candidate named entity spans with suitable labels, they rely solely on input context information when using LLMs, particularly, ChatGPT. However, NER inherently requires capturing detailed labeling requirements with input context information. To address this issue, we propose a novel method that leverages code-based prompting to improve the capabilities of LLMs in understanding and performing NER. By embedding code within prompts, we provide detailed BIO schema instructions for labeling, thereby exploiting the ability of LLMs to comprehend long-range scopes in programming languages. Experimental results demonstrate that the proposed code-based prompting method outperforms conventional text-based prompting on ten benchmarks across English, Arabic, Finnish, Danish, and German datasets, indicating the effectiveness of explicitly structuring NER instructions. We also verify that combining the proposed code-based prompting method with the chain-of-thought prompting further improves performance.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Named Entity Recognition | CoNLL English 2003 | -- | 19 | |
| Named Entity Recognition | DaNE (test) | -- | 15 | |
| Named Entity Recognition | FIN English | Macro-F141.13 | 4 | |
| Named Entity Recognition | MIT restaurant English | Macro-F149.95 | 4 | |
| Named Entity Recognition | WNUT English 17 | Macro-F152.09 | 4 | |
| Named Entity Recognition | Arabic A | Macro F1 Score44.52 | 4 | |
| Named Entity Recognition | Arabic B | Macro F146.57 | 4 | |
| Named Entity Recognition | Finnish A | Macro F162.98 | 4 | |
| Named Entity Recognition | SwissNER German | Macro F1 Score61.74 | 4 | |
| Named Entity Recognition | MIT movie English | Macro F1 Score48.38 | 4 |