Shakespeare
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Federated Learning | Shakespeare (val) | Perplexity3.721 | 73 | |
| Character-level Language Modeling | Shakespeare modern | Accuracy55.63 | 48 | |
| Language Modeling | Shakespeare's text | Cross-Entropy0.16 | 42 | |
| Language Modeling | Shakespeare 1M params (val) | NLL4.79 | 35 | |
| Federated Learning | Shakespeare | Accuracy50.611 | 33 | |
| Next Character Prediction | Shakespeare (test) | Accuracy47.55 | 31 | |
| Character-Level Language Modeling | Shakespeare (val) | Perplexity10.08 | 27 | |
| Language Modeling | Shakespeare | Accuracy (Mean)55.9 | 25 | |
| Language Modeling | Shakespeare Low Resource | PPL10.46 | 15 | |
| Language Modeling | Shakespeare High Resource | PPL5.01 | 15 | |
| Character-level language modeling | Shakespeare (train) | Accuracy59.8 | 12 | |
| Character-level language modeling | Shakespeare (test) | Accuracy50.2 | 12 | |
| Next character prediction | Shakespeare realistic non-IID LEAF benchmark (test) | Top-1 Acc54.52 | 11 | |
| Language Modeling | Tiny-Shakespeare (o.o.d) | Perplexity569.3 | 9 | |
| Next Word Prediction | ShakeSpeare LEAF (non-i.i.d.) | Accuracy (500R)46.36 | 9 | |
| Next Word Prediction | Shakespeare (test) | Percent Clients Acc_p Greater Than Acc_g89.77 | 8 | |
| Classification | Shakespeare (test) | Accp > Accg %89.77 | 8 | |
| Federated Character Prediction | Shakespeare (non-i.i.d.) | Test Error45.24 | 8 | |
| Federated Character Prediction | Shakespeare i.i.d. | Test Error44.52 | 8 | |
| Language Modeling | Shakespeare (non-i.i.d.) | Test Error45.24 | 8 | |
| Language Modeling | Shakespeare (i.i.d.) | Test Error44.52 | 8 | |
| Next-Character Prediction | Shakespeare 7, 47 (test) | Average Accuracy46.7 | 7 | |
| Character-level language modeling | Shakespeare | Cross-entropy Loss1.441 | 6 | |
| Language Modeling | Shakespeare | Perplexity283 | 5 | |
| Natural Language Generation | Shakespeare 7K | BLEU25.7 | 4 |
Showing 25 of 29 rows