MCIF
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Speech Translation | MCIF long track | COMET Score83.75 | 20 | |
| Automatic Speech Recognition | MCIF long track | WER5.88 | 20 | |
| Spoken Summarization | MCIF long track | BERTScore29.74 | 20 | |
| Spoken Question Answering | MCIF long track | BERTScore40.86 | 20 | |
| Speech Question Answering | MCIF short-form | BERTScore (SQA)45.13 | 10 | |
| Machine Translation | MCIF en-it | BLEU38.14 | 9 | |
| Machine Translation | MCIF en-de | BLEU30.88 | 9 | |
| Offline Speech Translation | MCIF (test) | XCOMET-XL86.59 | 9 | |
| Speech Translation | MCIF short-form | ST COMET78.69 | 8 | |
| Simultaneous Speech Translation | MCIF En-It (dev) | BLEU43.56 | 7 | |
| Simultaneous Speech Translation | MCIF En-De (dev) | BLEU31.73 | 7 | |
| Speech Question Answering | MCIF constrained short audio | BERTScore-F1 (EN)34.6 | 4 | |
| Automatic Speech Recognition | MCIF constrained, short audio | WER21.49 | 4 | |
| Automatic Speech Recognition | MCIF short-form | ASR Accuracy88.77 | 4 | |
| Speech Instruction Following | MCIF short-form | SIFS Score2.0708 | 2 |
Showing 15 of 15 rows