Tech Radar: Disaggregated Prefill-Decode Architecture: Decoupling Compute & Memory Bandwidth via RoCEv2 KV-Transfer
Tech Radar: Disaggregated Prefill-Decode Architecture: Decoupling Compute & Memory Bandwidth via RoCEv2 KV-Transfer Answer-First: Disaggregated Prefill-Decode serving defines 2026 enterprise LLM infrastructure, resolving the tension between compute-heavy prefill and memory-bound decode. By streaming KV caches across 400Gbps RoCEv2 fabrics, it cuts P99 TTFT by 11x (420ms to 38ms) and eliminates decode latency jitter on NVIDIA H100 clusters. name: "Disaggregated Prefill-Decode Serving" ring: "Adopt" quadrant: "AI Infrastructure & Large Language Models" rationale: "Decouples compute-bound prompt prefill from memory-bandwidth-bound token decode, eliminating head-of-line blocking and slashing P99 TTFT by 11x via zero-copy RoCEv2 KV transfer." adr_link: "/radar/2026-09/disaggregated-prefill-decode/" justification: "Empirically verified across 64x NVIDIA H100 SXM5 GPUs on DeepSeek-V3 and Llama-3.1-70B; production-ready in vLLM v1 and Mooncake architectures with 2.8x higher throughput per dollar." 1. The Compute vs. Memory-Bandwidth Dichotomy in Autoregressive Serving Autoregressive large language model serving is governed by two radically divergent computational regimes, creating an insurmountable structural tension within traditional monolithic GPU deployments: ...