MMT
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multimodal Understanding | MMT | Speedup Ratio3.85 | 60 | |
| Multimodal Understanding | MMT emo | Accuracy60.8 | 20 | |
| Multimodal Understanding | MMT | Accuracy77.2 | 19 | |
| Action Understanding | MMT-47 Action Understanding | Accuracy53.48 | 17 | |
| High Level Reasoning | MMT-47 High Level Reasoning | Accuracy45.92 | 17 | |
| Commonsense Reasoning | MMT-47 Commonsense Reasoning | Accuracy84.98 | 17 | |
| Image Classification | MMT-47 | Accuracy94.74 | 17 | |
| Multi-modal Multi-image Reasoning | MMT (val) | Accuracy67.4 | 14 | |
| Machine Translation | MMT eng-xx very-low-resource | chrF++49.2 | 12 | |
| Machine Translation | MMT eng-xx high-resource | chrF++64.7 | 12 | |
| Machine Translation | MMT eng-xx (all) | chrF++55.1 | 12 | |
| Multi-image Understanding | MMT (val) | Accuracy67.4 | 11 | |
| Visual Understanding | MMT | Score1,075.5 | 8 | |
| Machine Translation | MMT xx-yy (all) | chrF++42.8 | 6 | |
| Machine Translation | MMT xx-eng low-resource | chrF++51.5 | 6 | |
| Machine Translation | MMT eng-xx low-resource | chrF++41.8 | 6 |
Showing 16 of 16 rows