| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| AppWorld (test challenge) | Goal Completion Score32 | 35 | 1mo ago | ||
| AppWorld | HINT-SD-Multi | Average Completion Score @418.46 | 7 | 2mo ago | |
| AppWorld Hard 63 tasks (test_normal) | Accuracy31.7 | 4 | 1mo ago | ||
| AppWorld Medium 48 tasks (test_normal) | Accuracy43.8 | 4 | 1mo ago | ||
| AppWorld Easy normal 57 tasks (test) | LRE | Accuracy73.7 | 4 | 1mo ago | |
| AppWorld Average 168 tasks (test normal) | Accuracy44 | 4 | 1mo ago | ||
| AppWorld Challenge Qwen-2.5-32B (test) | ADAPO | Average Task Completion Score51 | 4 | 4mo ago | |
| AppWorld Normal Qwen-2.5-32B (test) | LOOP | Average Task Completion Score75 | 4 | 4mo ago |
