Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

You need to MIMIC to get FAME: Solving Meeting Transcript Scarcity with a Multi-Agent Conversations

About

Meeting summarization suffers from limited high-quality data, mainly due to privacy restrictions and expensive collection processes. We address this gap with FAME, a dataset of 500 meetings in English and 300 in German produced by MIMIC, our new multi-agent meeting synthesis framework that generates meeting transcripts on a given knowledge source by defining psychologically grounded participant profiles, outlining the conversation, and orchestrating a large language model (LLM) debate. A modular post-processing step refines these outputs, mitigating potential repetitiveness and overly formal tones, ensuring coherent, credible dialogues at scale. We also propose a psychologically grounded evaluation framework assessing naturalness, social behavior authenticity, and transcript difficulties. Human assessments show that FAME approximates real-meeting spontaneity (4.5/5 in naturalness), preserves speaker-centric challenges (3/5 in spoken language), and introduces richer information-oriented difficulty (4/5 in difficulty). These findings highlight that FAME is a good and scalable proxy for real-world meeting conditions. It enables new test scenarios for meeting summarization research and other conversation-centric applications in tasks requiring conversation data or simulating social scenarios under behavioral constraints.

Frederic Kirstein, Muneeb Khan, Jan Philip Wahle, Terry Ruas, Bela Gipp• 2025

Related benchmarks

TaskDatasetResultRank
Abstractive Meeting SummarizationQMSum--
4
Abstractive Meeting SummarizationFAME EN--
4
Abstractive Meeting SummarizationFAME [GER]--
4
Meeting Transcript EvaluationFAME English (EN)
Coherence4.5
1
Meeting Transcript EvaluationFAME German
Coherence (COH)4.5
1
Transcript Challenge AssessmentFAME English 1.0 (test)
Spoken Language Score3
1
Transcript Challenge AssessmentFAME German 1.0 (test)
Spoken Language Score3
1
Meeting Transcript EvaluationAMI--
1
Meeting Transcript EvaluationICSI--
1
Meeting Transcript EvaluationWPCP--
1
Showing 10 of 16 rows

Other info

Code

Follow for update