Rate Limiting

Answer-first: Building a Go API gateway with Envoy and NGINX enables L7 load balancing, JWT authentication, and token-bucket rate limiting at the ingress layer. Implementing this architecture enforces sub-50ms P99 latency guarantees, zero-allocation memory pooling with Go 1.24 unique.Handle, and fault-tolerant Dapr 1.15 component orchestration for resilient production scaling. This design guarantees sub-50ms P99 latency bounds and zero-allocation memory pooling. Prerequisite: Part 2 of the System Design Masterclass. Read Part 1: System Design Thinking first. ...

Answer-first: Shopee utilizes Apache Kafka queues for asynchronous peak shaving during 11.11 mega-campaigns. Decoupling order creation from database persistence guarantees sub-second API responses while downstream workers process orders at a controlled rate, protected by Sentinel adaptive load shedding and priority request classification. Adopting this pattern guarantees sub-50ms P99 latency bounds, zero-allocation memory optimization, and fault-tolerant event-driven state synchronization across production systems. Chapter 3: Peak Shaving - The Power of Apache Kafka and Graceful Degradation ← Series hub | ← Prev | Next → ...

Prerequisite: Before reading this chapter, review Chapter 2: The 3 Caching Vulnerabilities. Chapter 3: Distributed Rate Limiting with Redis & GCRA Algorithm Answer-first: Distributed rate limiting in microservice architectures requires centralized state management in Redis to avoid load-balancer bypasses. Implementing the Generic Cell Rate Algorithm (GCRA) via atomic Lua scripts tracks Theoretical Arrival Times (TAT) using a single 64-bit integer per user key, guaranteeing sub-millisecond execution. Deploying this pattern guarantees sub-50ms P99 latency bounds, zero-allocation memory pooling via Go 1.24 string interning, and. ...

Prerequisite: Familiarity with the concepts introduced in Part 4 — Sre Chaos Engineering. Review it first if the terminology in this part is unfamiliar. Answer-first: Scaling for billion-yen cashback campaigns requires pre-warmed Redis cluster caching, token-bucket rate limiting at the API gateway, and async queue-based payment processing to shave peak traffic spikes. Implementing this architecture enforces sub-50ms P99 latency guarantees, zero-allocation memory pooling with Go 1.24 unique.Handle, and fault-tolerant Dapr 1.15 component orchestration for resilient production scaling. ...

Prerequisite: Familiarity with the concepts introduced in Part 6 — From Coder To Orchestrator. Review it first if the terminology in this part is unfamiliar. Answer-first: While AI assistants excel at generating localized code functions, they remain blind to holistic distributed system failures, network partition handling, and cascading degradation. System design—encompassing Circuit Breakers, Rate Limiters, Distributed Locks, and CAP theorem trade-offs—serves as the ultimate career survival shield for software engineers. Architecting this pipeline enforces sub-50ms P99 latency guarantees, OpenTelemetry GenAI semantic conventions, and 2026 Model Context. ...

API rate limiting defends backend services by restricting request volume. Security requires a layered defense: Web Application Firewalls (WAF) block edge-level volumetric spikes, API Gateways manage L7 credentials and quotas, and application middleware enforces fine-grained business limits. Client identification must rely on validated, secure IP parsing (using the PROXY protocol or rightmost X-Forwarded-For checks). Prerequisite: This is Part 11 of the System Design Masterclass. Previous parts built the core components — this part covers securing APIs and managing client traffic spikes at scale. ...

Flash Sale Architecture: Rate Limiting & Redis Answer-first: Shopee flash sale architecture handles millions of concurrent requests using Redis Lua token buckets, local memory caches, queue-based order throttling, and optimistic DB updates. Implementing this architecture enforces sub-50ms P99 latency guarantees, zero-allocation memory pooling with Go 1.24 unique.Handle, and fault-tolerant Dapr 1.15 component orchestration for resilient production scaling. This design guarantees sub-50ms P99 latency bounds and zero-allocation memory pooling. [!NOTE] On sourcing: This article describes flash-sale architecture patterns for C10M-scale events; it is not a disclosure of Shopee’s internal systems, and the figures here are engineering targets rather than published Shopee metrics. Shopee has not publicly documented its flash-sale internals in detail. What is public is its database platform choice — Shopee’s adoption of TiDB is documented in PingCAP’s case studies (How Shopee Chose the Right Database, Shopping on Shopee, the TiDB Way). Treat everything else as a reference pattern to validate against your own workload. ...