High Concurrency

← Previous Chapter: Part 1: CAP, PACELC & Clean Architecture | Series Hub: System Design Masterclass | Next Chapter: Part 3: Caching Strategies, Redis/Valkey & Stampede Prevention → Prerequisite: Read Part 1: CAP, PACELC & Clean Architecture Primer to understand distributed trade-offs and composite availability foundations. Answer-first: Layer 4 load balancers route packets via eBPF and Direct Server Return to achieve sub-millisecond wire speed, while Layer 7 API gateways inspect HTTP headers and enforce token bucket rate limits. Combining kernel-bypass XDP packet filtering with Go reverse proxy buffer pools sustains 100,000 requests per second with sub-5ms P99 latency bounds across distributed clusters. ...

Answer-first: Mitigating caching vulnerabilities at scale requires a multi-layered defense against three fatal failure modes: cache penetration is eliminated using Bloom filters and null-object caching; cache avalanche is prevented by injecting randomized TTL jitter and asynchronous background warming; and cache breakdown is solved using Go singleflight to coalesce thousands of duplicate concurrent requests into a single database query. Prerequisite: Advanced understanding of memory caching hierarchies (L1 in-process vs L2 distributed clusters), probabilistic data structures (Bloom and Cuckoo filters), Go synchronization primitives, and database connection pool behavior is assumed for this chapter. ...

Answer-first: Local in-memory rate limiters fail in autoscaled microservices because client requests scatter across dynamic pods. Distributed rate limiting requires an atomic, single-variable algorithm: the Generic Cell Rate Algorithm executed within a Redis Lua script. GCRA tracks a single Theoretical Arrival Time per client, reducing network round-trips and memory consumption by seventy percent compared to classical sliding window counters. Prerequisite: Advanced understanding of distributed rate limiting concepts, token bucket mathematics, Redis single-threaded execution models, Lua script atomicity, and HTTP traffic shaping semantics is assumed for this chapter. ...

Answer-first: Publishing messages to Kafka directly after database commits triggers catastrophic dual-write divergences during network timeouts or process crashes. The production standard is the Transactional Outbox Pattern powered by Log-based Change Data Capture: events are inserted atomically into an outbox table within the business transaction, and an external Debezium connector streams database write-ahead logs to Kafka with zero polling overhead. Prerequisite: Advanced understanding of database ACID transaction guarantees, distributed consistency anomalies, message broker delivery semantics (at-least-once vs exactly-once), and database replication mechanics is required. ...

Previous Chapter: Part 4 — SRE Practices & Chaos Engineering | Series Hub | Next Chapter: Part 6 — AI Platform: Real-Time Fraud & LLM Hub Answer-First: Handling viral promotional spikes like the historic “10-Billion Yen Campaign” requires safeguarding core payment processing from promotional logic overload. PayPay achieves this through a multi-tier defense: Edge Virtual Waiting Rooms buffer traffic surges at CloudFront, admitting users only at backend processing capacity; Atomic Redis Lua scripts track finite campaign budgets in sub-millisecond memory to prevent budget overruns; and Two-Phase Reward Decoupling isolates the synchronous payment checkout from deferred cashback calculations via Kafka, verified by automated end-of-day three-way reconciliation. ...

Answer-First: The Shopee Architecture series details how Go microservices, Redis Lua inventory reservation, Apache Kafka peak shaving, TiDB distributed SQL, and OpenTelemetry/ClickHouse observability handle 10M+ QPS and millions of concurrent buyers during 11.11 flash sales without overselling or database connection starvation. Masterclass Overview: The Southeast Asian E-Commerce Engine Shopee is the leading e-commerce platform in Southeast Asia and Taiwan, operating across Singapore, Indonesia, Vietnam, Thailand, Philippines, and Malaysia. During annual shopping festivals (9.9, 11.11, 12.12), platform traffic surges by more than 10x within seconds at midnight, creating catastrophic load spikes that break traditional web architectures. ...