P3
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Tool Integration and Execution | P3 Real-world Deployment (Tasks 1-6) | Success Rate100 | 15 | |
| Image Restoration | P3 Pepper | SNR20.48 | 12 | |
| General Language Understanding | P3 v1 (unseen) | RTE Accuracy80.83 | 11 | |
| Competing-task Planning and Coordination | P3 Real-world Deployment (Tasks 7-11) | Success Rate92.86 | 10 | |
| Constrained Bayesian Optimization | P3 | Log10 Median Utility Gap1.28 | 10 | |
| Evolutionary Multi-Task Optimization | P3 | Normalized Fitness92.7 | 6 | |
| Investment decision alignment | P3 v1 (test) | Overall MSE1.59 | 6 | |
| Word Sense Disambiguation | P3 | WiC Score53.3 | 5 | |
| Coreference Resolution | P3 | Winogrades Score61.6 | 5 | |
| Sentence Completion | P3 | COPA Accuracy85.3 | 5 | |
| Natural Language Inference | P3 | RTE81.3 | 5 | |
| Multiple-Choice Question Answering | P3 | Dream77.6 | 5 | |
| Summarization | P3 | Mul. News Score7.8 | 5 | |
| Sentiment Analysis | P3 | Emotion Accuracy49.4 | 5 | |
| Minimal Problem Solving | P3.5P focal | Template Size (R×C)2,043 | 4 |
Showing 15 of 15 rows