Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

From Query to Counsel: Structured Reasoning with a Multi-Agent Framework and Dataset for Legal Consultation

About

Legal consultation question answering (Legal CQA) presents unique challenges compared to traditional legal QA tasks, including the scarcity of high-quality training data, complex task composition, and strong contextual dependencies. To address these, we construct JurisCQAD, a large-scale dataset of over 43,000 real-world Chinese legal queries annotated with expert-validated positive and negative responses, and design a structured task decomposition that converts each query into a legal element graph integrating entities, events, intents, and legal issues. We further propose JurisMA, a modular multi-agent framework supporting dynamic routing, statutory grounding, and stylistic optimization. Combined with the element graph, the framework enables strong context-aware reasoning, effectively capturing dependencies across legal facts, norms, and procedural logic. Trained on JurisCQAD and evaluated on a refined LawBench, our system significantly outperforms both general-purpose and legal-domain LLMs across multiple lexical and semantic metrics, demonstrating the benefits of interpretable decomposition and modular collaboration in Legal CQA.

Mingfei Lu, Yi Zhang, Mengjia Wu, Yue Feng• 2026

Related benchmarks

TaskDatasetResultRank
Legal Question AnsweringLawBench revised (test)
ROUGE-144.68
17
Cross-jurisdiction Legal ReasoningKoBLEX
Token F1 Score34.88
11
Reading ComprehensionLawBench 2-5
RC-F146.62
8
Cross-lingual Question AnsweringLEGALBENCH RuleQA English (test)
ROUGE-120.25
3
Legal ConsultationLawBench Revised
Legal Soundness4.14
3
Showing 5 of 5 rows

Other info

Follow for update