MLLMU-Bench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multimodal Machine Unlearning | MLLMU-Bench LLaVA-1.5-7B (test 2) | Forget Rate62.8 | 24 | |
| Multimodal Machine Unlearning | MLLMU-Bench LLaVA-1.5-7B (test 1) | Forget Rate65.4 | 24 | |
| Classification | MLLMU-Bench Forget Set | Accuracy0.5187 | 21 | |
| Question Answering | MLLMU-Bench Real-world 1.0 | Accuracy78.3 | 16 | |
| Visual Question Answering | MLLMU-Bench Real-world 1.0 | Accuracy77.3 | 16 | |
| Question Answering | MLLMU-Bench 1.0 (Retain) | Accuracy58.6 | 16 | |
| Visual Question Answering | MLLMU-Bench Retain 1.0 | Accuracy56 | 16 | |
| Question Answering | MLLMU-Bench Forget 1.0 | Accuracy55 | 16 | |
| Visual Question Answering | MLLMU-Bench Forget 1.0 | Accuracy29.8 | 16 | |
| Multimodal Machine Unlearning Evaluation | MLLMU-Bench Forget Set | Classification Accuracy54.67 | 12 | |
| Generation | MLLMU-Bench (Forget Set) | Rouge Score64.5 | 7 | |
| Multimodal Machine Unlearning Evaluation | MLLMU-Bench Real Celebrity | Class Acc52.75 | 4 | |
| Multimodal Machine Unlearning Evaluation | MLLMU-Bench (test) | Classification Accuracy39.75 | 3 | |
| Classification | MLLMU-Bench Real Celebrity | Accuracy51.8 | 2 | |
| Classification | MLLMU-Bench (Retain Set) | Accuracy46.11 | 2 | |
| Classification | MLLMU-Bench (test) | Accuracy47.86 | 2 | |
| Cloze | MLLMU-Bench (Forget Set) | Cloze Accuracy25.81 | 2 |
Showing 17 of 17 rows