MovieChat
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Video Question Answering | MovieChat | Throughput (Tokens/s)34.04 | 24 | |
| Video Question Answering | MovieChat-1k Breakpoint | Accuracy74.72 | 23 | |
| Video Question Answering | MovieChat Global Breakpoint | Breakpoint Accuracy74.72 | 14 | |
| Video Question Answering | MovieChat long video 1K (test) | CI Score3.11 | 11 | |
| Open-ended Video Question Answering | MovieChat-1K Global mode | Accuracy93.1 | 10 | |
| Video Question Answering | MovieChat 1K (test) | Accuracy86.8 | 7 | |
| Video Question Answering | MovieChat-1K Global Mode | Correctness of Information3.28 | 7 | |
| Video Question Answering | MovieChat1k | Accuracy79 | 6 | |
| Video Question Answering | MovieChat-1k Global | Accuracy84.9 | 6 | |
| Video Understanding | MovieChat-1K Breakpoint mode | Accuracy48.3 | 5 | |
| Video Understanding | MovieChat-1K Global mode | Accuracy62.3 | 5 | |
| Long Video Generative Performance | MovieChat-1K breakpoint mode 1.0 | CI2.64 | 4 | |
| Video Summarization | MovieChat-1K | ROUGE-24.27 | 3 | |
| Proxy multimodal evaluation | MovieChat-1K | Accuracy55.51 | 2 | |
| Video Understanding | MovieChat-1K Global mode (test) | Accuracy- | 0 |
Showing 15 of 15 rows