Inference Speed Evaluation on generation 128 tokens
2,747.88Inference Time (ms)
BlockPruner
Updated 3mo ago
Evaluation Results
| Method | Links | ||
|---|---|---|---|
2024.06 | 2,747.88 | 1.47 | |
2024.06 | 3,094.36 | 1.31 | |
2024.06 | 3,226.68 | 1.25 | |
2024.06 | 3,873.2 | 1.88 | |
2024.06 | 4,044.3 | 1 | |
2024.06 | 4,099.08 | 1.78 | |
2024.06 | 4,111.65 | 1.77 | |
2024.06 | 7,285.73 | 1 |