Evaluation dataset
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Tool-use | Evaluation dataset | Accuracy51.98 | 20 | |
| Compositional Generalization | Evaluation Dataset (Unseen Average) | Score42.86 | 18 | |
| Compositional Generalization | Evaluation Dataset Seen Average | Score62.34 | 18 | |
| Compositional Generalization | Evaluation Dataset Unseen (Fold 3) | Score0.4022 | 18 | |
| Compositional Generalization | Evaluation Dataset (Fold 3 Seen) | Score66.69 | 18 | |
| Compositional Generalization | Evaluation Dataset Unseen (Fold 2) | Score50 | 18 | |
| Compositional Generalization | Evaluation Dataset (Fold 2 Seen) | Score63.63 | 18 | |
| Compositional Generalization | Evaluation Dataset Unseen (Fold 1) | Score0.4818 | 18 | |
| Compositional Generalization | Evaluation Dataset (Fold 1 Seen) | Score0.6191 | 18 | |
| Compositional Generalization | Evaluation Dataset (Full) | Score0.6379 | 18 | |
| Malicious Package Detection | Evaluation Dataset | Accuracy99.5 | 11 | |
| Correlation analysis with ground truth | Evaluation Dataset 2000 samples | Pearson Correlation Coefficient0.754 | 7 | |
| Global 3D Editing | Evaluation dataset unseen 3D assets (test) | CLIP Similarity0.272 | 6 | |
| Local 3D Editing | Evaluation dataset unseen 3D assets (test) | CLIP Similarity0.292 | 6 | |
| Image-to-3D Generation | Evaluation Dataset | FID34.251 | 2 | |
| Inconsistency detection | Evaluation dataset Full (4,556 skills) | Total Flagged Count487 | 1 |
Showing 16 of 16 rows