CoLA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Text Reconstruction Attack | CoLA | Total Runtime (hours)0.1 | 36 | |
| Membership Inference Attack | CoLA | AUC100 | 32 | |
| Sequence reconstruction | CoLA | ROUGE-199.6 | 32 | |
| Training Data Reconstruction | CoLA | ROUGE-1100 | 32 | |
| Misclassification Detection | Cola | ROC-AUC81.6 | 31 | |
| Linguistic Acceptability | COLA | Accuracy83.43 | 31 | |
| Text reconstruction from gradients | CoLA | ROUGE-1100 | 24 | |
| Acceptability Classification | CoLA out-of-domain (dev test) | Accuracy84.9 | 24 | |
| Linguistic Acceptability | CoLA | Accuracy (CoLA)85.91 | 21 | |
| Text Reconstruction from Gradients | CoLA | Average per-batch runtime (seconds)3 | 20 | |
| Text Classification | CoLA (test) | MCC61.21 | 18 | |
| Natural Language Understanding | CoLA | Exact Match77.7 | 14 | |
| Grammatical Acceptability | CoLA (held-out) | F1 Score46.8 | 14 | |
| Text-to-Image retrieval | Cola (test) | Multi-Obj Acc83.88 | 14 | |
| Dependency tree recovery | CoLA max length 14 (val) | ARR75.7 | 14 | |
| Text Reconstruction | CoLA | ROUGE-L96.3 | 12 | |
| Acceptability Classification | CoLA English (IDD) | Accuracy88.2 | 12 | |
| Acceptability Classification | CoLA in-domain (dev) | Accuracy88.6 | 12 | |
| Natural Language Inference | CoLA | Accuracy86.26 | 10 | |
| Compositional Reasoning | Cola | Txt2Img Score33.33 | 10 | |
| Linguistic Acceptability | CoLA GLUE (val) | Accuracy84.76 | 9 | |
| Sentence Classification | CoLA full (test) | Accuracy86.4 | 9 | |
| Linguistic Acceptability | CoLA (dev) | Matthews Correlation68.5 | 8 | |
| Syntax | CoLA | MCC59.71 | 8 | |
| Classification | COLA | ASR Score0.175 | 8 |
Showing 25 of 35 rows