Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

QMSum

Benchmarks

Task NameDataset NameSOTA ResultTrend
Query-based meeting summarizationQMSum (test)
ROUGE-219.63
34
Next Token PredictionQMSum
Next Token Accuracy47
32
Long-context language generationQMSum
Average Acceptance Length (τ)3.15
25
Query-focused SummarizationQMSum (test)
ROUGE-138.67
23
SummarizationQMSum (val)
ROUGE-L0.2378
17
Traceback (Prompt Injection Attacks)QMSum
Precision99
13
Query-based SummarizationQMSum Golden v1 (test)
ROUGE-140.85
11
Abstractive SummarizationQMSum
BLEU6.75
11
Meeting SummarizationQMSum
Coherence4.47
10
Query-focused summarizationQMSum
ROUGE-135.8
10
Context TracebackQMSum LongBench
Precision99
10
Synthetic Text GenerationQMSum
Mean Embedding Similarity52
10
Query-based Meeting SummarizationQMSum
Overall Score14.734
10
Document SummarizationQMSum (test)
ROUGE-138.9
10
Long-context retrievalQMSum LongEmbed
nDCG@100.813
9
Document SummarizationQMSum
G-mean15.47
9
SummarizationQMSum
Std Dev ROUGE-10.3
8
Payload-splitting attack detectionQMSum
Precision (QMSum)81
6
Query-focused Meeting SummarizationQMSum 50 samples
Fluency4.88
6
SummarizationQMSum (test)
Fluency4.93
5
RetrievalQMSum
R@1058.82
4
Next Token PredictionQMSum
Acc (BERT-Small, Epsilon=Inf)32.82
4
Abstractive Meeting SummarizationQMSum
Coreference1.67
4
Meeting SummarizationQMSum (all turns)
ROUGE-134.03
4
SummarizationQMSum Golden (test)
Best Score65
3
Showing 25 of 31 rows