Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Question Answering on RAG QA 5 retrieved documents
Loading...
191.7
TTFT (ms)
Lazy-Attn
149.248
435.799
722.35
1,008.901
Jun 3, 2026
TTFT (ms)
Speedup
Updated 1mo ago
Evaluation Results
Method
Method
Links
TTFT (ms)
Speedup
Lazy-Attn
Model=Tulu3-Block-FT 8B
2026.06
191.7
6.2
Lazy-Attn
Model=Llama-3.1-70B, T...
2026.06
238.4
5.2
CacheBlend
Model=Tulu3-Block-FT 8B
2026.06
274.8
4.4
CacheBlend
Model=Llama-3.1-70B, T...
2026.06
365.5
3.4
Standard RAG
Model=Tulu3-Block-FT 8B
2026.06
1,196.8
1
Standard RAG
Model=Llama-3.1-70B, T...
2026.06
1,253
1
Feedback
Search any
task
Search any
task