Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

SMADE-IE: Sparse Multi-Agent Framework with Evidence-Driven Debate for Zero-Shot Information Extraction

About

Zero-shot information extraction (IE) with large language models (LLMs) has attracted increasing attention due to its flexibility in adapting to new schemas and domains without task-specific training. Existing approaches mainly rely on monolithic prompting, each-type prompting, or multi-agent debate. However, monolithic prompting often suffers from boundary and type errors, while each-type prompting and multi-agent debate introduce cross-type conflicts, redundant agent interactions, and substantial token overhead. To address these challenges, we propose SMADE-IE, a sparse and evidence-driven multi-agent framework for zero-shot IE. SMADE-IE first employs an Adaptive Mode Selector to dynamically route inputs into either a lightweight Global Extraction Mode or a Type-Centric Extraction Mode, reducing unnecessary type selection and reasoning noise. For conflicting predictions, we further introduce an Evidence-Driven Debate mechanism that structures arguments into Toulmin-style components and performs confidence aggregation through external evidence scoring and Bayesian updates. Experimental results on 9 benchmark datasets across NER, RE, and JERE tasks show that SMADE-IE consistently outperforms existing zero-shot IE baselines while also improving token efficiency through sparse agent selection and early-stopping debate.

Kenfeng Huang, Yi Cai, Xin Wu, Zikun Deng, Li Yuan• 2026

Related benchmarks

TaskDatasetResultRank
Named Entity RecognitionCoNLL 2003 (test)--
556
Relation ExtractionSciERC
Relation Strict F111.2
99
Named Entity RecognitionOntoNotes 5.0 (test)--
90
Joint Entity and Relation ExtractionNYT--
38
Joint Entity and Relation ExtractionCONLL04
Entity F162.09
35
Named Entity RecognitionCrossNER (test)--
13
Relation ExtractionDocRED
F1 Score30.78
11
Relation ExtractionSemEval 2010
F1 Precision30.6
10
Named Entity RecognitionREDFM (test)
F1p Score31.12
5
Relation ExtractionCrossRE
F1 Score (p)16.22
5
Showing 10 of 11 rows

Other info

Follow for update