Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Chatlaw: A Multi-Agent Collaborative Legal Assistant with Knowledge Graph Enhanced Mixture-of-Experts Large Language Model

About

AI legal assistants based on Large Language Models (LLMs) can provide accessible legal consulting services, but the hallucination problem poses potential legal risks. This paper presents Chatlaw, an innovative legal assistant utilizing a Mixture-of-Experts (MoE) model and a multi-agent system to enhance the reliability and accuracy of AI-driven legal services. By integrating knowledge graphs with artificial screening, we construct a high-quality legal dataset to train the MoE model. This model utilizes different experts to address various legal issues, optimizing the accuracy of legal responses. Additionally, Standardized Operating Procedures (SOP), modeled after real law firm workflows, significantly reduce errors and hallucinations in legal services. Our MoE model outperforms GPT-4 in the Lawbench and Unified Qualification Exam for Legal Professionals by 7.73% in accuracy and 11 points, respectively, and also surpasses other models in multiple dimensions during real-case consultations, demonstrating our robust capability for legal consultation.

Jiaxi Cui, Munan Ning, Zongjian Li, Bohua Chen, Yang Yan, Hao Li, Bin Ling, Yonghong Tian, Li Yuan• 2023

Related benchmarks

TaskDatasetResultRank
Knowledge QuestioningJ1 EVAL
Average Score52.9
14
Legal ConsultationJ1 EVAL
Average Score36.5
14
Defence DraftingJ1 EVAL
FOR15.8
14
Civil CourtJ1 EVAL
PFS Score3.7
14
Complaint DraftingJ1 EVAL
FOR Score30.3
14
Criminal CourtJ1 EVAL
PFS (Procedural Fairness Score)3.7
14
Showing 6 of 6 rows

Other info

Follow for update