Rate Limiting

← Previous Chapter: Part 1: CAP, PACELC & Clean Architecture | Series Hub: System Design Masterclass | Next Chapter: Part 3: Caching Strategies, Redis/Valkey & Stampede Prevention → Prerequisite: Read Part 1: CAP, PACELC & Clean Architecture Primer to understand distributed trade-offs and composite availability foundations. Answer-first: Layer 4 load balancers route packets via eBPF and Direct Server Return to achieve sub-millisecond wire speed, while Layer 7 API gateways inspect HTTP headers and enforce token bucket rate limits. Combining kernel-bypass XDP packet filtering with Go reverse proxy buffer pools sustains 100,000 requests per second with sub-5ms P99 latency bounds across distributed clusters. ...

Multi-Language Edition: This chapter is also available in Vietnamese at 📖 Bản tiếng Việt (Vietnamese Edition). Previous Chapter: Chapter 2 — Flash Sale Engine & Zero Overselling | Series Hub | Next Chapter: Chapter 4 — Database Scalability: From MySQL to TiDB Answer-First: Shopee utilizes Apache Kafka queues for asynchronous peak shaving during 11.11 mega-campaigns. Decoupling user order submission from relational database persistence guarantees sub-50ms API responses while downstream consumer workers insert orders at a flat, controlled rate. Combined with Alibaba Sentinel adaptive load shedding, virtual waiting rooms, and strict Dead Letter Queue (DLQ) isolation, this traffic shield absorbs 10x traffic surges without database connection pool exhaustion or cascaded outages. ...

Multi-Language Edition: This chapter is also available in Vietnamese at 📖 Bản tiếng Việt (Vietnamese Edition). Previous: Chapter 2 — Caching Vulnerabilities & Go Singleflight | Series Hub | Next: Chapter 4 — Dual-Write Prevention via Transactional Outbox Answer-First: Local in-memory rate limiters (e.g., golang.org/x/time/rate) fail in horizontally autoscaled microservices because client traffic is scattered across dynamic nodes. Distributed rate limiting requires an atomic, single-variable algorithm: the Generic Cell Rate Algorithm (GCRA) executed within a single Redis Lua script. GCRA tracks a single Theoretical Arrival Time (TAT) per client, reducing network round-trips and memory footprint by 70% compared to classical sliding window counters. ...

Multi-Language Edition: This chapter is also available in Vietnamese at 📖 Bản tiếng Việt (Vietnamese Edition). Previous Chapter: Part 4 — SRE Practices & Chaos Engineering | Series Hub | Next Chapter: Part 6 — AI Platform: Real-Time Fraud & LLM Hub Answer-First: Handling viral promotional spikes like the historic “10-Billion Yen Campaign” requires safeguarding core payment processing from promotional logic overload. PayPay achieves this through a multi-tier defense: Edge Virtual Waiting Rooms buffer traffic surges at CloudFront, admitting users only at backend processing capacity; Atomic Redis Lua scripts track finite campaign budgets in sub-millisecond memory to prevent budget overruns; and Two-Phase Reward Decoupling isolates the synchronous payment checkout from deferred cashback calculations via Kafka, verified by automated end-of-day three-way reconciliation. ...

📖 Bản tiếng Việt (Vietnamese Edition) Prerequisite: Familiarity with the concepts introduced in Part 6 — From Coder To Orchestrator. Review it first if the terminology in this part is unfamiliar. Answer-first: While AI assistants excel at generating localized code functions, they remain blind to holistic distributed system failures, network partition handling, and cascading degradation. System design—encompassing Circuit Breakers, Rate Limiters, Distributed Locks, and CAP theorem trade-offs—serves as the ultimate career survival shield for software engineers. Mastering distributed resilience primitives—Circuit Breakers, Sliding-Window Rate Limiters, Distributed Mutexes, and CAP theorem consistency boundaries—protects production platforms against probabilistic AI hallucination failures. ...

← Previous Chapter: Part 10: Observability, Continuous Profiling & Pprof in Go | Series Hub: System Design Masterclass | Next Chapter: Part 12: High-Performance Transport Protocols & Serialization in Go → Prerequisite: Read Part 10: Observability, Continuous Profiling & Pprof in Go to master deep runtime forensics and metric instrumentation before hardening network perimeters and throttling abusive traffic. Answer-first: Securing modern cloud-native Go microservices requires a defense-in-depth Zero Trust architecture uniting SPIFFE/SPIRE mutual TLS, cryptographic PASETO v4 tokens, and multi-tier sliding window rate limiters. Enforcing token-bucket throttles via atomic Redis Lua scripts blocks credential stuffing attacks and BOLA vulnerabilities, preventing denial-of-service degradation while sustaining sub-millisecond API authorization latency across multi-tenant clusters. ...

Flash Sale Architecture: Rate Limiting & Redis Answer-first: High-concurrency flash sale systems absorb millions of synchronized user requests using a 5-Tier Traffic Shedding Architecture: Cloudflare CDN edge static asset caching, Envoy API Gateway atomic Token Bucket rate limiting, Redis Cluster Lua inventory reservations with hotkey slot splitting, partitioned Kafka queue buffering, and asynchronous Go worker pools executing batch upserts into TiDB/MySQL. [!NOTE] On sourcing: This article describes flash-sale architecture patterns for C10M-scale events; it is not a disclosure of Shopee’s internal systems, and the figures here are engineering targets rather than published Shopee metrics. Shopee has not publicly documented its flash-sale internals in detail. What is public is its database platform choice — Shopee’s adoption of TiDB is documented in PingCAP’s case studies (How Shopee Chose the Right Database, Shopping on Shopee, the TiDB Way). Treat everything else as a reference pattern to validate against your own workload. ...