Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

QMSum

Benchmarks

Task NameDataset NameSOTA ResultTrend
Next Token PredictionQMSum
Next Token Accuracy47
32
Query-based meeting summarizationQMSum (test)
ROUGE-143.8
26
SummarizationQMSum (val)
ROUGE-L0.2378
17
Abstractive SummarizationQMSum
BLEU6.75
11
Synthetic Text GenerationQMSum
Mean Embedding Similarity52
10
Document SummarizationQMSum (test)
ROUGE-138.9
10
SummarizationQMSum
Std Dev ROUGE-10.3
8
Query-focused SummarizationQMSum (test)
ROUGE-138.06
7
Query-based Meeting SummarizationQMSum
ROUGE-L10
6
Query-focused Meeting SummarizationQMSum 50 samples
Fluency4.88
6
SummarizationQMSum (test)
Fluency4.93
5
Next Token PredictionQMSum
Acc (BERT-Small, Epsilon=Inf)32.82
4
Abstractive Meeting SummarizationQMSum
Coreference1.67
4
Meeting SummarizationQMSum (all turns)
ROUGE-134.03
4
Meeting SummarizationQMSum Gold turns only
ROUGE-140.2
3
Query-based SummarizationQMSum SCROLLS (val)
ROUGE-130.9
2
Transcript Challenge AssessmentQMSum (test)
Spoken Language Score3
1
Meeting Transcript EvaluationQMSum
Coherence4.5
1
Showing 18 of 18 rows