Polaris
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Mathematical Reasoning | Polaris (test) | Cumulative Rollouts (M)4.92 | 16 | |
| Human Correlation for Image Captioning Evaluation | Polaris | Kendall's Tau-c63.2 | 16 | |
| Correlation with human judgments | Polaris (test) | Kendall's Tau-c0.578 | 16 | |
| Clear Table | PolaRiS | Success Rate40 | 10 | |
| Marker in Cup | PolaRiS | Success Rate92 | 10 | |
| Cup in Bowl | PolaRiS | Success Rate100 | 10 | |
| Serve Fruits | PolaRiS | Success Rate72 | 10 | |
| Multimodal Preference Evaluation | Polaris | tau_c57.8 | 10 | |
| Throw Away Trash | PolaRiS | Success Rate48 | 6 | |
| Store Marker | PolaRiS | Success Rate60 | 6 | |
| Stack Dishware | PolaRiS | Success Rate100 | 6 | |
| Mathematical Reasoning and Coding | Polaris | Peak Accuracy@848.79 | 6 | |
| ADME Prediction | Polaris adme-fang (test) | Aggregate Normalised Improvement0.042 | 4 | |
| ADME Prediction | Polaris adme-fang (val) | Aggregate Normalized Improvement6.6 | 4 | |
| Jailbreak Attack Evaluation | POLARIS | Attack Success Count520 | 2 |
Showing 15 of 15 rows