Rate Limiting

← Previous Chapter: Part 1: CAP, PACELC & Clean Architecture | Series Hub: System Design Masterclass | Next Chapter: Part 3: Caching Strategies, Redis/Valkey & Stampede Prevention → Prerequisite: Read Part 1: CAP, PACELC & Clean Architecture Primer to understand distributed trade-offs and composite availability foundations. Answer-first: Layer 4 load balancers route packets via eBPF and Direct Server Return to achieve sub-millisecond wire speed, while Layer 7 API gateways inspect HTTP headers and enforce token bucket rate limits. Combining kernel-bypass XDP packet filtering with Go reverse proxy buffer pools sustains 100,000 requests per second with sub-5ms P99 latency bounds across distributed clusters. ...

Previous Chapter: Chapter 2 — Flash Sale Engine & Zero Overselling | Series Hub | Next Chapter: Chapter 4 — Database Scalability: From MySQL to TiDB Answer-first: Shopee defends its e-commerce infrastructure during mega shopping surges using a multi-layered traffic shield combining WAF rate limiting, virtual waiting rooms, and Apache Kafka asynchronous peak shaving. Decoupling order creation from relational persistence flattens extreme traffic spikes, preserving database stability while ensuring sub-fifty-millisecond checkout response times and zero message loss across millions of concurrent users. ...

Answer-first: Local in-memory rate limiters fail in autoscaled microservices because client requests scatter across dynamic pods. Distributed rate limiting requires an atomic, single-variable algorithm: the Generic Cell Rate Algorithm executed within a Redis Lua script. GCRA tracks a single Theoretical Arrival Time per client, reducing network round-trips and memory consumption by seventy percent compared to classical sliding window counters. Prerequisite: Advanced understanding of distributed rate limiting concepts, token bucket mathematics, Redis single-threaded execution models, Lua script atomicity, and HTTP traffic shaping semantics is assumed for this chapter. ...

Previous Chapter: Part 4 — SRE Practices & Chaos Engineering | Series Hub | Next Chapter: Part 6 — AI Platform: Real-Time Fraud & LLM Hub Answer-first: Handling viral traffic spikes during nationwide cashback promotions without compromising core payment reliability requires decoupling promotional logic from financial checkouts. PayPay accomplishes this via Edge Virtual Waiting Rooms to throttle traffic bursts, single-threaded atomic Redis Lua scripts that prevent budget overruns in sub-millisecond memory, and asynchronous reward crediting reconciled via daily three-way automated audit pipelines. ...

Prerequisite: Strong understanding of distributed systems fundamentals, CAP and PACELC theorems, concurrency race conditions, and atomic state synchronization. Answer-first: High-level distributed systems design, data consistency modeling, and network partition resilience remain the irreplaceable domain of human software engineers. Large language models fundamentally fail at non-local reasoning, subtle concurrency race conditions, and CAP theorem trade-offs. Mastering storage engine internals, distributed transactions, and failure domain isolation guarantees technical leadership and long-term career durability. ...

← Previous Chapter: Part 10: Observability, Continuous Profiling & Pprof in Go | Series Hub: System Design Masterclass | Next Chapter: Part 12: High-Performance Transport Protocols & Serialization in Go → Prerequisite: Read Part 10: Observability, Continuous Profiling & Pprof in Go to master deep runtime forensics and metric instrumentation before hardening network perimeters and throttling abusive traffic. Answer-first: Securing modern cloud-native Go microservices requires a defense-in-depth Zero Trust architecture uniting SPIFFE/SPIRE mutual TLS, cryptographic PASETO v4 tokens, and multi-tier sliding window rate limiters. Enforcing token-bucket throttles via atomic Redis Lua scripts blocks credential stuffing attacks and BOLA vulnerabilities, preventing denial-of-service degradation while sustaining sub-millisecond API authorization latency across multi-tenant clusters. ...

Flash Sale Architecture: Rate Limiting & Redis Answer-first: High-concurrency flash sale systems absorb millions of synchronized user requests using a 5-Tier Traffic Shedding Architecture: Cloudflare CDN edge static asset caching, Envoy API Gateway atomic Token Bucket rate limiting, Redis Cluster Lua inventory reservations with hotkey slot splitting, partitioned Kafka queue buffering, and asynchronous Go worker pools executing batch upserts into TiDB/MySQL. [!NOTE] On sourcing: This article describes flash-sale architecture patterns for C10M-scale events; it is not a disclosure of Shopee’s internal systems, and the figures here are engineering targets rather than published Shopee metrics. Shopee has not publicly documented its flash-sale internals in detail. What is public is its database platform choice — Shopee’s adoption of TiDB is documented in PingCAP’s case studies (How Shopee Chose the Right Database, Shopping on Shopee, the TiDB Way). Treat everything else as a reference pattern to validate against your own workload. ...