Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

MuST-C

Benchmarks

Task NameDataset NameSOTA ResultTrend
Speech Translation EvaluationMuST-C
Pearson Correlation0.9988
94
Speech TranslationMuST-C EN-DE (test-COMMON)
BLEU31.7
41
Simultaneous Speech TranslationMuST-C EN-DE (tst-COMMON)
BLEU23.8
39
Simultaneous Speech TranslationMuST-C en-de v1.0 (test)
BLEU27.38
36
Simultaneous Speech TranslationMuST-C en-ru v1.0 (test)
BLEU18.5
36
Simultaneous Speech TranslationMuST-C en-ro 1.0 (test)
BLEU26.44
36
Simultaneous Speech TranslationMuST-C en-pt v1.0 (test)
BLEU32.23
36
Simultaneous Speech TranslationMuST-C en-nl 1.0 (test)
BLEU31.41
36
Contrastive AlignmentMust-C (test)
Cosine Similarity1.37
36
Speech RecognitionMuST-C (test)
WER (Avg)6.54
30
Speech TranslationMuST-C (test)
BLEU Score31.54
29
Speech Question AnsweringMuST-C (test)
EM76.11
27
Automatic Speech RecognitionMuST-C
LASER0.8308
21
Speech TranslationMuST-C (tst-COMMON)
BLEU (De)31.2
20
Speech TranslationMuST-C En-De (test)
SacreBLEU28.7
18
Speech TranslationMuST-C EN-FR COMMON (test)
BLEU37.4
17
Speech-to-text TranslationMuST-C En-X (tst-COM)
BLEU (German)30.1
16
Automatic Speech RecognitionMuST-C En-De COMMON (test)
WER6.54
16
Speech TranslationMust-C
BLEU31.54
15
Simultaneous Speech TranslationMuST-C En-De (tst-HE)
BLEU22.3
14
Speech TranslationMuST-C en-de (dev)
BLEU26.92
14
Speech TranslationMuST-C EN-ES (tst-COMMON)
BLEU31
14
Simultaneous Machine TranslationMuST-C en-es v1.0 (test)
BLEU35.14
13
Simultaneous Machine TranslationMuST-C en-de v1.0 (test)
BLEU27.32
13
Speech TranslationMuST-C COMMON (tst)
WER (de)16.5
13
Showing 25 of 64 rows