BLiMP
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Linguistic Minimal Pair Scoring | BLiMP | Overall Accuracy88.9 | 49 | |
| Linguistic Minimal Pair Evaluation | BLiMP (test) | NPI lic. (2)100 | 28 | |
| Linguistic Probing | BLiMP | Performance64.8 | 10 | |
| Linguistic Acceptability | BLiMP English (all) | Accuracy81.6 | 9 | |
| Syntax | BLiMP | Accuracy84.61 | 8 | |
| Audio Language Modeling | sBLIMP | Accuracy64.7 | 8 | |
| Zero-shot Language Modeling | BLiMP (test) | Accuracy79.6 | 8 | |
| Syntactic Generalization | BLiMP (test) | BLiMP Accuracy0.773 | 8 | |
| Semantic Anomaly Detection | BLIMP Animacy | Accuracy78.7 | 6 | |
| Morphosyntax Anomaly Detection | BLIMP Det-Noun | Accuracy99.9 | 6 | |
| Morphosyntax Anomaly Detection | BLIMP Subject-Verb | Accuracy97.1 | 6 | |
| Linguistic Analysis | BLiMP | Accuracy60.5 | 4 | |
| Computation Time Analysis | Blimp Circle C trajectory | Computation Time (ms)0.96 | 3 | |
| Computation Time Analysis | Blimp Helix B trajectory | Computation Time (ms)0.95 | 3 | |
| Computation Time Analysis | Blimp Helix A trajectory | Computation Time (ms)0.94 | 3 | |
| Computation Time Analysis | Blimp Lemniscate C trajectory | Computation Time (ms)0.86 | 3 | |
| Computation Time Analysis | Blimp Lemniscate B trajectory | Computation Time (ms)0.87 | 3 | |
| Computation Time Analysis | Blimp Lemniscate A trajectory | Computation Time (ms)0.82 | 3 | |
| Computation Time Analysis | Blimp Circle B trajectory | Computation Time (ms)0.82 | 3 | |
| Computation Time Analysis | Blimp Circle A trajectory | Computation Time (ms)0.84 | 3 | |
| Trajectory Tracking | Blimp (experimental trials) | Avg CPU Energy Expenditure (µJ)1.25 | 3 | |
| Linguistic Competence Classification | BLiMP | Accuracy83 | 3 |
Showing 22 of 22 rows