Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Text Generation on benchmark suite 20-task
Loading...
2,520
Average End-to-End Latency (ms)
Remote Speculate (RLM-Cascade)
2,417.92
3,106.96
3,796
4,485.04
Jun 22, 2026
Average End-to-End Latency (ms)
p50 Latency (ms)
Average TTFT (ms)
Throughput (req/s)
Wall Time (ms)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Average End-to-End Latency (ms)
p50 Latency (ms)
Average TTFT (ms)
Throughput (req/s)
Wall Time (ms)
Remote Speculate (RLM-Cascade)
Serving Endpoint=Remot...
2026.06
2,520
2,026
2,520
2.5
2,000
Remote Native Opus
Serving Endpoint=Remot...
2026.06
4,159
3,698
1,213
2.53
1,976
Local vLLM (4-bit 7B)
Serving Endpoint=Local...
2026.06
5,072
5,234
165
1.66
3,004
Feedback
Search any
task
Search any
task