ChatCounselor: A Large Language Models for Mental Health Support
About
This paper presents ChatCounselor, a large language model (LLM) solution designed to provide mental health support. Unlike generic chatbots, ChatCounselor is distinguished by its foundation in real conversations between consulting clients and professional psychologists, enabling it to possess specialized knowledge and counseling skills in the field of psychology. The training dataset, Psych8k, was constructed from 260 in-depth interviews, each spanning an hour. To assess the quality of counseling responses, the counseling Bench was devised. Leveraging GPT-4 and meticulously crafted prompts based on seven metrics of psychological counseling assessment, the model underwent evaluation using a set of real-world counseling questions. Impressively, ChatCounselor surpasses existing open-source models in the counseling Bench and approaches the performance level of ChatGPT, showcasing the remarkable enhancement in model capability attained through high-quality domain-specific data.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Emotional Support Conversation | ESConv (test) | -- | 50 | |
| Clinical Counseling Dialogue Evaluation | Session Rating Scale | Helpful Score (SRS)3.41 | 18 | |
| Psychological Counseling | CPsyCounR Single-Session | T.Alli0.237 | 12 | |
| Psychological Counseling | CPsyCounR Multi-Session | Coherence (Coh)0.472 | 12 | |
| Clinical Counseling Dialogue Evaluation | Session Rating Scale Engaged | Helpful Rating3.74 | 9 | |
| Clinical Counseling Dialogue Evaluation | Session Rating Scale Resistant | Helpful Rating3.12 | 9 | |
| Counselor Competence Assessment | Counseling Dialogues (test) | Understanding4.01 | 9 |