DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services
About
We propose DISC-LawLLM, an intelligent legal system utilizing large language models (LLMs) to provide a wide range of legal services. We adopt legal syllogism prompting strategies to construct supervised fine-tuning datasets in the Chinese Judicial domain and fine-tune LLMs with legal reasoning capability. We augment LLMs with a retrieval module to enhance models' ability to access and utilize external legal knowledge. A comprehensive legal benchmark, DISC-Law-Eval, is presented to evaluate intelligent legal systems from both objective and subjective dimensions. Quantitative and qualitative results on DISC-Law-Eval demonstrate the effectiveness of our system in serving various users across diverse legal scenarios. The detailed resources are available at https://github.com/FudanDISC/DISC-LawLLM.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Criminal Sentencing and Legal Validity Evaluation | LeCaRD v2 | RMSE37.14 | 32 | |
| Criminal Sentencing, Legal Validity, and Suspect-Level Performance Evaluation | LEEC | -- | 28 | |
| General Provision Prediction | LeCaRD v2 | Precision66.67 | 24 | |
| Specific Provision Prediction | LeCaRD v2 | Precision64.22 | 24 | |
| Legal Judgment Prediction | CAIL 2018 | Public Safety Accuracy40.1 | 15 | |
| Legal Judgment Prediction | CMDL | Public Safety Accuracy49.7 | 15 | |
| Legal Judgment Prediction | CAIL & CMDL Average | Average Score30.3 | 15 | |
| Complaint Drafting | J1 EVAL | FOR Score74.2 | 14 | |
| Defence Drafting | J1 EVAL | FOR26.5 | 14 | |
| Legal Consultation | J1 EVAL | Average Score35.3 | 14 |