MM-MergeBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multimodal Generative Tasks | MM-MergeBench (test) | Accuracy56.65 | 12 | |
| Multi-modal Task Evaluation | MM-MergeBench Seen Tasks | SciQA Score83.74 | 12 | |
| Multi-modal Task Evaluation | MM-MergeBench Unseen Tasks | AVQA Score79.3 | 11 | |
| Vision-language model merging | MM-MergeBench seen tasks | Average Absolute Accuracy69.23 | 9 | |
| Vision-language model merging | MM-MergeBench unseen tasks | Average Absolute Accuracy39.84 | 8 | |
| Multi-task learning evaluation | MM-MergeBench 20 tasks 1.0 (test) | Average Absolute Accuracy91.37 | 6 | |
| Multi-task learning evaluation | MM-MergeBench 14 tasks 1.0 (test) | Average Absolute Accuracy90.88 | 6 | |
| Multi-task learning evaluation | MM-MergeBench 8 tasks 1.0 (test) | Avg Absolute Accuracy92.83 | 6 |
Showing 8 of 8 rows