BERT
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Weak scaling | BERT-Base (train) | Memory (MB)3,707.01 | 15 | |
| Masked Language Modelling | BERT (val) | Accuracy65.54 | 14 | |
| End-to-end inference tuning | BERT-Large | Tuning Time (s)22.6 | 9 | |
| End-to-end inference tuning | BERT Base | Tuning Time (s)23.3 | 9 | |
| Inference Latency | BERT-base | Attention Layer Latency (s)40.54 | 6 | |
| Masked Language Modeling | BERT large | vNMSE0.0022 | 6 | |
| Device Placement | BERT | Latency per G0.0027 | 6 | |
| Language Model Pre-training | BERT-Large NVIDIA V100 (train) | Max Batch Size96 | 6 | |
| Language Model Pre-training | BERT-Large NVIDIA 2080 Ti (train) | Max Batch Size50 | 6 | |
| Inference | BERT base | Speedup21.5 | 5 | |
| Secure Transformer Inference | BERT-base | Online Overhead (GB)2.2 | 4 | |
| Privacy-Preserving Inference | BERT Large (inference) | GeLU Time (s)0.351 | 4 | |
| Privacy-Preserving Inference | BERT Base (inference) | GeLU Time (s)0.351 | 4 | |
| Recursive circuit gate count analysis | BERT base | Nova589,824 | 1 | |
| Masked Language Modeling | BERT MLM small (val) | Validation Loss6.9412 | 1 |
Showing 15 of 15 rows