DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services
About
We propose DISC-LawLLM, an intelligent legal system utilizing large language models (LLMs) to provide a wide range of legal services. We adopt legal syllogism prompting strategies to construct supervised fine-tuning datasets in the Chinese Judicial domain and fine-tune LLMs with legal reasoning capability. We augment LLMs with a retrieval module to enhance models' ability to access and utilize external legal knowledge. A comprehensive legal benchmark, DISC-Law-Eval, is presented to evaluate intelligent legal systems from both objective and subjective dimensions. Quantitative and qualitative results on DISC-Law-Eval demonstrate the effectiveness of our system in serving various users across diverse legal scenarios. The detailed resources are available at https://github.com/FudanDISC/DISC-LawLLM.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Complaint Drafting | J1 EVAL | FOR Score74.2 | 14 | |
| Defence Drafting | J1 EVAL | FOR26.5 | 14 | |
| Legal Consultation | J1 EVAL | Average Score35.3 | 14 | |
| Civil Court | J1 EVAL | PFS Score3.4 | 14 | |
| Criminal Court | J1 EVAL | PFS (Procedural Fairness Score)2.2 | 14 | |
| Knowledge Questioning | J1 EVAL | Average Score48.7 | 14 |