Rate Limiting

Prerequisite: Familiarity with the concepts introduced in Part 4 — Sre Chaos Engineering. Review it first if the terminology in this part is unfamiliar. Answer-first: Scaling for billion-yen cashback campaigns requires pre-warmed Redis cluster caching, token-bucket rate limiting at the API gateway, and async queue-based payment processing to shave peak traffic spikes. Answer-first: The PayPay campaign architecture isolates high-throughput reward campaigns from core payment processing. By evaluating campaign eligibility out-of-band and writing reward points asynchronously using event queues, PayPay prevents promotional traffic spikes from impacting critical credit card processing pipelines. ...

API rate limiting defends backend services by restricting request volume. Security requires a layered defense: Web Application Firewalls (WAF) block edge-level volumetric spikes, API Gateways manage L7 credentials and quotas, and application middleware enforces fine-grained business limits. Client identification must rely on validated, secure IP parsing (using the PROXY protocol or rightmost X-Forwarded-For checks). Prerequisite: This is Part 11 of the System Design Masterclass. Previous parts built the core components — this part covers securing APIs and managing client traffic spikes at scale. ...

Answer-first: Building a Go API gateway with Envoy and NGINX enables L7 load balancing, JWT authentication, and token-bucket rate limiting at the ingress layer. Prerequisite: Part 2 of the System Design Masterclass. Read Part 1: System Design Thinking first. Load Balancing L4/L7 in Go — DSR, Rate Limiting & API Gateway Answer-first: L4 load balancing routes traffic at the transport layer using IP/TCP metadata with minimal CPU overhead, whereas L7 load balancing inspects HTTP headers, cookies, and URLs for intelligent content-based routing. Combining L4 Direct Server Return (DSR) with L7 Envoy API Gateways and Go token-bucket rate limiters handles peak traffic spikes smoothly. ...

Prerequisite: Before reading this chapter, review Chapter 2: The 3 Caching Vulnerabilities. Chapter 3: Distributed Rate Limiting with Redis & GCRA Algorithm Answer-first: Distributed rate limiting in microservice architectures requires centralized state management in Redis to avoid load-balancer bypasses. Implementing the Generic Cell Rate Algorithm (GCRA) via atomic Lua scripts tracks Theoretical Arrival Times (TAT) using a single 64-bit integer per user key, guaranteeing sub-millisecond execution. Key Takeaways: Local Limiter Flaws: Local in-memory limiters fail under multi-node load balancers because traffic distribution allows clients to multiply effective throughput limits. GCRA Efficiency: GCRA tracks arrival time deltas rather than token counts, requiring only one Redis key lookup per request. Lua Atomicity: Executing GCRA calculations inside Redis Lua scripts eliminates race conditions between concurrent API Gateway nodes. What You’ll Learn GCRA TAT Mathematics: How Theoretical Arrival Time formulas ($TAT = \max(now, TAT) + \tau$) calculate exact retry delays. Lua Script Race Conditions: Why atomic execution in Redis single-threaded engine is mandatory for rate limit precision. Memory Footprint Math: Comparing GCRA (1 key/user) against Token Bucket and Sliding Window Log memory overheads. If caching is the shield protecting your database, Rate Limiting is the armor guarding your API servers from DDoS attacks and resource exhaustion caused by abusive clients. ...

Flash Sale Architecture: Rate Limiting & Redis [!NOTE] On sourcing: This article describes flash-sale architecture patterns for C10M-scale events; it is not a disclosure of Shopee’s internal systems, and the figures here are engineering targets rather than published Shopee metrics. Shopee has not publicly documented its flash-sale internals in detail. What is public is its database platform choice — Shopee’s adoption of TiDB is documented in PingCAP’s case studies (How Shopee Chose the Right Database, Shopping on Shopee, the TiDB Way). Treat everything else as a reference pattern to validate against your own workload. ...

Prerequisite: Familiarity with the concepts introduced in Part 6 — From Coder To Orchestrator. Review it first if the terminology in this part is unfamiliar. Answer-first: While AI assistants excel at generating localized code functions, they remain blind to holistic distributed system failures, network partition handling, and cascading degradation. System design—encompassing Circuit Breakers, Rate Limiters, Distributed Locks, and CAP theorem trade-offs—serves as the ultimate career survival shield for software engineers. ...

Answer-first: Shopee utilizes Apache Kafka queues for asynchronous peak shaving during 11.11 mega-campaigns. Decoupling order creation from database persistence guarantees sub-second API responses while downstream workers process orders at a controlled rate, protected by Sentinel adaptive load shedding and priority request classification. Chapter 3: Peak Shaving - The Power of Apache Kafka and Graceful Degradation ← Series hub | ← Prev | Next → Prerequisite: Read the previous article: Chapter 2: Flash Sale Engine - Solving Overselling and Hot Keys. ...