OmniBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Vision-Audio-Text | Omnibench | Accuracy47.6 | 34 | |
| Audio-Visual Reasoning | OmniBench | Accuracy63.6 | 16 | |
| Humanoid Motion Tracking | OmniBench Walk (Slow) | Success Rate (SR)100 | 14 | |
| Loco-Manip | OmniBench Low 1.0 (test) | Success Rate (SR)100 | 14 | |
| Loco-Manip | OmniBench Medium 1.0 (test) | Success Rate (SR)100 | 14 | |
| Loco-Manip | OmniBench High 1.0 (test) | Success Rate (SR)100 | 14 | |
| Omni-modal reasoning | OmniBench (test) | Score43.6 | 12 | |
| Safety Evaluation | OmniBench | Accuracy42.47 | 12 | |
| Integrated Multimodal Reasoning | OmniBench (held-out) | Causal Score55.2 | 11 | |
| Audio-video understanding | OmniBench | Score49.1 | 10 | |
| Multimodal Reasoning | OmniBench | Causal Score54.9 | 9 | |
| Omni-modal Understanding | OmniBench | Overall Score58.41 | 8 | |
| Humanoid Motion Tracking | OmniBench Jump (Low) | Success Rate100 | 7 | |
| Humanoid Motion Tracking | OmniBench Jump Medium | Success Rate100 | 7 | |
| Humanoid Motion Tracking | OmniBench Jump (High) | Success Rate (SR)90 | 7 | |
| Humanoid Motion Tracking | OmniBench Run Medium | Success Rate (SR)100 | 7 | |
| Humanoid Motion Tracking | OmniBench Run (Fast) | Success Rate (SR)100 | 7 | |
| Humanoid Motion Tracking | OmniBench Walk (Fast) | Success Rate100 | 7 | |
| Squat | OmniBench Low 1.0 (test) | Success Rate (SR)100 | 7 | |
| Squat | OmniBench Medium 1.0 (test) | SR100 | 7 | |
| Squat | OmniBench High 1.0 (test) | Success Rate (SR)100 | 7 | |
| Audio-Visual QA | OmniBench | Accuracy48.25 | 6 | |
| Image Question Answering | OmniBench | Metric- | 0 |
Showing 23 of 23 rows