Shakespeare
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Character-level Language Modeling | Shakespeare modern | Accuracy55.63 | 48 | |
| Federated Learning | Shakespeare | Accuracy50.611 | 33 | |
| Next Character Prediction | Shakespeare (test) | Accuracy47.55 | 31 | |
| Language Modeling | Shakespeare | Accuracy (Mean)55.9 | 25 | |
| Language Modeling | Shakespeare Low Resource | PPL10.46 | 15 | |
| Language Modeling | Shakespeare High Resource | PPL5.01 | 15 | |
| Next character prediction | Shakespeare realistic non-IID LEAF benchmark (test) | Top-1 Acc54.52 | 11 | |
| Language Modeling | Tiny-Shakespeare (o.o.d) | Perplexity569.3 | 9 | |
| Next Word Prediction | ShakeSpeare LEAF (non-i.i.d.) | Accuracy (500R)46.36 | 9 | |
| Next Word Prediction | Shakespeare (test) | Percent Clients Acc_p Greater Than Acc_g89.77 | 8 | |
| Classification | Shakespeare (test) | Accp > Accg %89.77 | 8 | |
| Federated Character Prediction | Shakespeare (non-i.i.d.) | Test Error45.24 | 8 | |
| Federated Character Prediction | Shakespeare i.i.d. | Test Error44.52 | 8 | |
| Language Modeling | Shakespeare (non-i.i.d.) | Test Error45.24 | 8 | |
| Language Modeling | Shakespeare (i.i.d.) | Test Error44.52 | 8 | |
| Next-Character Prediction | Shakespeare 7, 47 (test) | Average Accuracy46.7 | 7 | |
| Language Modeling | Shakespeare | Perplexity283 | 5 | |
| Natural Language Generation | Shakespeare 7K | BLEU25.7 | 4 | |
| Next-character Prediction | Shakespeare (val) | Rounds to 20.0% Acc15 | 3 | |
| Next-word prediction | SHAKESPEARE LEAF (test) | Prediction Accuracy (PA)53.63 | 3 | |
| Federated Next-Character Prediction | Shakespeare unseen clients (test) | Test Accuracy46.7 | 3 | |
| Language Modeling | Tiny-Shakespeare | Perplexity- | 0 |
Showing 22 of 22 rows