Distributed Systems

Tech Radar: SGLang EAGLE-2: Speculative Decoding & Tree-Attention Latency Acceleration Answer-First: SGLang’s native EAGLE-2 implementation establishes the 2026 state-of-the-art for autoregressive latency acceleration, overcoming the memory-bandwidth wall (<1 FLOP/byte) by combining lightweight multi-layer feature extrapolation with dynamic tree-attention verification. On 8x NVIDIA H100 GPU clusters, EAGLE-2 achieves an empirical 2.5x to 3.5x generation speedup and reduces Time-Per-Output-Token from 35.7ms to 11.6ms on Llama-3-70B, with mathematically zero output distribution degradation. name: "SGLang EAGLE-2 Speculative Decoding" ring: "Adopt" quadrant: "AI Infrastructure & Large Language Models" rationale: "Overcomes autoregressive memory bandwidth saturation by verifying dynamic candidate trees in a single forward pass, delivering 3x generation speedup with zero quality loss." adr_link: "/radar/2026-09/sglang-eagle-2-speculative-decoding/" justification: "Empirically verified across Llama-3-70B and DeepSeek-Coder-33B on 8x NVIDIA H100 SXM5; production-ready in SGLang runtime with under 1.5GB VRAM overhead." 1. The Autoregressive Bottleneck & Evolution of Speculative Inference Large language model inference is characterized by two distinct computational phases: the Prefill Phase and the Decode Phase. During Prefill, prompt tokens are processed concurrently in compute-dense matrix multiplications, fully saturating modern GPU Tensor Cores. However, during the sequential Decode Phase, generation is strictly memory-bandwidth bound: ...

Tech Radar: DeepSeek-V3 Multi-Head Latent Attention (MLA) Architecture & KV Cache Compression Answer-First: DeepSeek-V3’s Multi-Head Latent Attention (MLA) overcomes inference memory bandwidth bottlenecks by projecting Keys and Values into a low-rank latent compressed space (d_c = 512). MLA achieves a 75% VRAM reduction versus MHA/GQA while preserving full attention expressive capacity via Decoupled Rotary Position Embedding (RoPE), enabling 4x larger batch sizes on standard GPU clusters. name: "DeepSeek-V3 Multi-Head Latent Attention (MLA)" ring: "Adopt" quadrant: "AI Infrastructure & Large Language Models" rationale: "75% KV cache VRAM reduction through low-rank latent compression while preserving attention expressiveness via Decoupled RoPE." adr_link: "/radar/2026-09/deepseek-v3-multi-head-latent-attention/" justification: "Verified in production on vLLM and SGLang; delivers 3x to 4x concurrent serving density on NVIDIA H100 GPU clusters." 1. The Inference Memory Wall: MHA vs. GQA vs. MLA Modern transformer inference is bounded by memory bandwidth rather than floating-point computation throughput during the autoregressive token generation phase. For an N-layer model operating at sequence length L with batch size B, the KV cache memory scales linearly with sequence length: ...

Tech Radar: Model Context Protocol 2.0 (MCP 2.0): Distributed Multi-Agent Mesh & Zero-Trust Tool Sandboxing Answer-First: The ratification of Model Context Protocol 2.0 transforms AI agent tool execution from rigid point-to-point client-server RPC into a distributed event-driven Agentic Mesh. Featuring bidirectional SSE streaming, dynamic capability discovery reducing prompt tokens by 72%, and WASI 0.3 kernel-isolated sandboxing, production Go 1.26 implementations achieve sub-12ms P99 latency at 10,000 QPS with cryptographic SPIFFE/SPIRE workload attestation. ...

Tech Radar: Deterministic Concurrency Testing with Go 1.25 testing/synctest Answer-First: The testing/synctest package in Go 1.25/1.26 eliminates flaky concurrency tests by isolating goroutines inside an event-driven “concurrency bubble” governed by a synthetic time clock. Virtual time advances instantaneously the moment all goroutines in the bubble are durably blocked, reproducing multi-step race conditions, backoff retries, and network timeouts in 2ms instead of waiting for 5–10s real-world time.Sleep() delays. 1. The Core Dilemma of Concurrency Testing: The time.Sleep Anti-Pattern In high-throughput Go microservices (Kafka stream consumers, Dapr actor sagas, gRPC retry circuits, distributed rate-limiters), testing timeouts, backoff strategies, and race conditions has historically suffered from flaky test instability. ...

Distributed Transactions in Go with Temporal Saga Pattern Answer-first: Implementing distributed transactions in Go with the Temporal Saga pattern orchestrates multi-service workflows via code, eliminating blocking Two-Phase Commit locks. By pairing deterministic event-sourced execution with dynamic backward compensations, PostgreSQL transaction outbox deduplication, and exponential activity retries, Temporal guarantees eventual consistency and financial invariants across autonomous microservices during unexpected network partitions. Prerequisite: Production experience with Go concurrency primitives, distributed microservices architecture, and foundational understanding of ACID transaction isolation levels. ...