Why my long prompts were slow: a KV-cache detour

Decode speed looked fine, but time-to-first-token grew linearly with prompt length. That is prefill, not decode — and it was redoing the same 3,000-token system prompt every request.

kv_bytes = 2 * n_layers * n_kv_heads * head_dim * seq_len * bytes_per_elem
# 32 layers, 8 kv heads, 128 dim, 4096 tokens, fp16  ->  ~0.5 GB

Reusing the cached prefix across requests cut median time-to-first-token from 1.9 s to 0.6 s. The lesson: measure prefill and decode separately.