LLM Serving on Alpaca (Average Latency)
111.46Average Latency (ms/token)
Oracle SJF
Updated 26d ago
Evaluation Results
| Method | Links | |
|---|---|---|
2025.09 | 111.46 | |
2025.09 | 124.92 | |
2025.09 | 137.32 | |
2025.09 | 138.53 | |
2025.09 | 148.97 | |
2025.09 | 152.3 | |
2025.09 | 157.07 | |
2025.09 | 178.07 | |
2025.09 | 187.41 | |
2025.09 | 283.39 | |
2025.09 | 372.03 | |
2025.09 | 962.53 |