Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FLEURS

Benchmarks

Task NameDataset NameSOTA ResultTrend
Automatic Speech RecognitionFleurs
WER0.1058
56
Automatic Speech RecognitionFLEURS en
WER2.72
49
Automatic Speech RecognitionFleurs zh
WER0.83
34
Automatic Speech RecognitionFLEURS (test)
Average Error Rate0.1745
30
Speech RecognitionFleurs En
WER2.4
24
Script confusion mitigationFLEURS zh-trad (test)
Accuracy91
21
Script confusion mitigationFLEURS zh-sim (test)
Normalized Edit Similarity Accuracy93
21
Script confusion mitigationFLEURS sr-cyrl (test)
Normalized Edit Similarity0.94
21
Script confusion mitigationFLEURS sr-latn (test)
Accuracy (Normalized Edit Similarity)96
21
Hallucination detectionFLEURS S2TT (test)
Hallucination Rate (%)89.66
18
Speech RecognitionFLEURS Kazakh (test)
WER (Oracle)15.4
17
Automatic Speech Recognitionfleurs Tamil
WER28.68
17
Automatic Speech RecognitionFleurs en (test)
WER0.336
17
Speech TranslationFLEURS X→En (test)
BLEU (Average)37.4
16
Noise RobustnessFLEURS (test)
Robustness Score (Gaussian Noise)84.46
15
Speech-to-Text TranslationFLEURS (test)
Avg. Score86.3
15
Speech TranslationFLEURS (test)
COMET (Low)0.853
15
Multilingual Speech RecognitionFLEURS
DE WER2.47
13
Automatic Speech RecognitionFLEURS
WER (en)2.71
13
Automatic Speech RecognitionFLEURS
AMH12.77
13
Document RetrievalFleurs Retrieval-Eval (test)
nDCG@557.89
13
Speech Translation (ST)FLEURS Vi-En wav2vec2-base ASR transcripts (test)
BLEU23.65
10
Speech Translation (ST)FLEURS Vi-En PhoWhisper-large ASR transcripts (test)
BLEU28.29
10
Machine Translation (MT)FLEURS Vi-En (test)
BLEU33.72
10
Speech-to-Text TranslationFLEURS 9×27 directions (test)
ara78.7
10
Showing 25 of 129 rows