GITHUB
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Language Modeling | GitHub (test) | Perplexity2.42 | 113 | |
| Membership Inference Attack | GitHub Pythia | ROC AUC1 | 36 | |
| Membership Inference | GitHub Pythia (train) | TPR@1%FPR95.6 | 36 | |
| Membership Inference Attack | GitHub | AUC0.876 | 32 | |
| Semi-supervised graph classification | GITHUB 10-fold cross-validation | Accuracy0.6996 | 21 | |
| Graph Classification | GITHUB | Accuracy71.06 | 18 | |
| Language Modeling | GitHub (val) | Perplexity1.83 | 13 | |
| Language Modeling | GitHub tokens (test) | Bits Per Token (BPT)0.976 | 11 | |
| Tokenization efficiency | GitHub | Token Count688 | 6 | |
| Autonomous Task Completion | GitHub | Success Rate84 | 6 | |
| Token Efficiency | GitHub Events | JSON Compact Token Count968 | 1 | |
| Graph Explanation | Github (test) | FID65.01 | 1 | |
| Website Navigation | GitHub (test) | Metric- | 0 |
Showing 13 of 13 rows