Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions

About

The growing demand for scalable psychological counseling highlights the need for high-quality, privacy-compliant data, yet such data remains scarce. Here we introduce MAGneT, a novel multi-agent framework for synthetic psychological counseling session generation that decomposes counselor response generation into coordinated sub-tasks handled by specialized LLM agents, each modeling a key psychological technique. Unlike prior single-agent approaches, MAGneT better captures the structure and nuance of real counseling. We further propose a unified evaluation framework that consolidates diverse automatic metrics and expands expert assessment from four to nine counseling dimensions, thus addressing inconsistencies in prior evaluation protocols. Empirically, MAGneT substantially outperforms existing methods: experts prefer MAGneT-generated sessions in 77.2% of cases, and sessions generated by MAGneT yield 3.2% higher general counseling skills and 4.3% higher CBT-specific skills on cognitive therapy rating scale (CTRS). A open source Llama3-8B-Instruct model fine-tuned on MAGneT-generated data also outperforms models fine-tuned using baseline synthetic datasets by 6.9% on average on CTRS.We also make our code and data public.

Aishik Mandal, Tanmoy Chakraborty, Iryna Gurevych• 2025

Related benchmarks

TaskDatasetResultRank
Counseling Accuracy EvaluationCounselingBench
Accuracy61.6
12
Counseling Dialogue EvaluationExpert Evaluation
Specificity3.97
4
Counseling Dialogue GenerationCounselBench Eval (test)
Overall Score4.12
4
Showing 3 of 3 rows

Other info

Follow for update