SHADE-Arena
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| LLM behavior monitoring | SHADE-Arena (test) | Accuracy75.2 | 8 | |
| Sabotage detection | SHADE-Arena | log-AUROC78 | 8 | |
| Sabotage Detection | SHADE-Arena Total | Accuracy70.6 | 3 | |
| Sabotage Detection | SHADE-Arena Charity Allocation | Accuracy91.4 | 3 | |
| Sabotage Detection | SHADE-Arena Performance Reviews | Accuracy67.3 | 3 | |
| Sabotage Detection | SHADE-Arena Expenditure Accounting | Accuracy87.9 | 3 | |
| Sabotage Detection | SHADE-Arena API Key Calendar | Accuracy88.5 | 3 | |
| Sabotage Detection | SHADE-Arena Jira Tickets | Accuracy72.5 | 3 | |
| Sabotage Detection | SHADE-Arena Election Tampering | Accuracy44.4 | 3 | |
| Sabotage Detection | SHADE-Arena Spam Filter | Accuracy69.5 | 3 | |
| Sabotage Detection | SHADE-Arena American Airlines | Accuracy83 | 3 | |
| Sabotage Detection | SHADE-Arena Bank Transfer | Accuracy85.7 | 3 | |
| Sabotage Detection | SHADE-Arena SSN Task | Accuracy96.3 | 3 |
Showing 13 of 13 rows