MUSTARD
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Table Structure Recognition | MUSTARD | S-TEDS95.3 | 20 | |
| Multimodal Sarcasm Detection | MUStARD original (speaker-dependent) | Precision86.8 | 15 | |
| Multimodal Sarcasm Detection | MUStARD speaker-independent original | F1 Score75.6 | 13 | |
| Sarcasm Detection | MUStARD | Accuracy76.62 | 13 | |
| Multimodal Classification | MUSTARD | Accuracy69.86 | 13 | |
| Sarcasm Detection | MUSTARD++ (test) | F1 Score83.2 | 13 | |
| Multi-modal Classification | MUSTARD (test) | Accuracy72.5 | 12 | |
| Sarcasm Understanding | MuSTARD++ | F1 Score83.2 | 10 | |
| Multimodal Classification | MUStARD Synergy | Accuracy52.9 | 8 | |
| Sarcasm Detection | MUStARD (held-out) | F1 Score65.8 | 8 | |
| Sarcasm Detection (SAR) | MUStARD | Weighted F10.795 | 7 | |
| Sarcasm Understanding | MuSTARD | Precision86.8 | 7 | |
| Multimodal Sarcasm Detection | MUStARD | Accuracy80.6 | 6 | |
| Multimodal Sarcasm Explanation | MUStARD (speaker dependent) | ROUGE-138.4 | 6 | |
| Multimodal Sarcasm Detection | MUStARD speaker dependent | Precision86.8 | 6 | |
| Sarcasm Detection | MUSTARD (test) | F1 Score78.7 | 5 | |
| Multimodal Retrieval | MUStARD | Recall@10 (Q: M23, T: M1)81 | 4 | |
| Sarcasm Detection | MUStARD (5-fold CV) | Accuracy74.42 | 2 | |
| Sarcasm Detection | MUStARD++ (5-fold CV) | Accuracy73.29 | 1 | |
| Model Selection | MUSTARD (unseen) | Performance95.15 | 1 |
Showing 20 of 20 rows