Answer-first: The Shopee Architecture series details how Go microservices, Redis Lua inventory reservation, Apache Kafka peak shaving, TiDB distributed SQL, and OpenTelemetry/ClickHouse observability handle 10M+ QPS and millions of concurrent buyers during 11.11 flash sales without overselling or database connection starvation.
This series explores the core architectural patterns and technologies Shopee uses to handle millions of concurrent users, specifically focusing on extreme traffic spikes during Flash Sales and mega-campaigns like 11.11.
Series Contents#
The following five chapters break down Shopee’s high-concurrency production stack step-by-step, tracing requests from gateway routing down to storage engine persistence and observability:
Looking for an architectural guide to migrating a legacy e-commerce platform to a microservices architecture similar to Shopee’s? See our Composable Commerce Migration Series for a step-by-step production case study.
Architectural Scope & Technical Pillars#
The matrix below maps each architectural module to its core engineering focus, primary technology stack, and target production Service Level Agreement (SLA):
| Module | Focus Area | Core Technologies | Target SLA / Metric |
|---|
| Chapter 1: Microservices Foundation | Domain decomposition & RPC dispatch | Go, gRPC, Protobuf, Kratos | Sub-5ms RPC latency |
| Chapter 2: Flash Sale Engine | Hot key prevention & inventory isolation | Redis, Lua Scripts, Atomic Decr | 100k+ QPS write throughput |
| Chapter 3: Traffic Shield | Peak shaving & graceful degradation | Apache Kafka, Token Bucket Rate Limiting | 99.99% availability under traffic spikes |
| Chapter 4: Database Scale | Distributed SQL & Sharding migration | MySQL Sharding, TiDB, Debezium CDC | Zero-downtime schema migrations |
| Chapter 5: Observability | Tracing & telemetry instrumentation | OpenTelemetry, Prometheus, Grafana | Unified trace context across microservices |
Target Audience & Technical Prerequisites#
This masterclass is engineered for Senior Backend Architects, Systems Engineers, and Go Developers building ultra-high-throughput systems. The guidelines assume practical experience with distributed systems design principles:
- Strong familiarity with Go concurrency patterns (goroutines, channels, worker pools).
- Understanding of distributed caching (Redis) and event messaging (Kafka).
- Fundamental knowledge of relational database indexing and sharding principles.
Key System Invariants#
Every architectural pattern in this series adheres to five core system invariants that guarantee system stability during 10M+ QPS traffic surges:
- Zero Overselling Guarantee: Redis atomic decrements (
DECR) coupled with Lua scripts prevent inventory balance from going negative under extreme peak loads. - Asynchronous Write Offloading: Order creation events flow into Apache Kafka topics before relational database persistence, absorbing 100k+ QPS traffic surges.
- Graceful Degradation Shields: Token bucket rate limiters reject invalid request bursts at the API gateway before hitting backend microservices.
- Distributed Telemetry Correlation: Unified OpenTelemetry trace context propagates through gRPC headers, providing distributed observability across all microservice layers.
- Storage Decoupling: LSM-Tree storage models and distributed SQL (TiDB) decouple write transactions from physical disk latency bottlenecks.
Frequently Asked Questions (FAQ)#
How does Shopee's architecture prevent inventory overselling during 11.11 flash sales?#
Shopee isolates inventory in Redis memory shards and executes atomic Lua scripts to evaluate user eligibility and decrement stock in a single thread-safe step. By decoupling stock pre-allocation from SQL database writes, inventory deductions execute in sub-milliseconds without acquiring relational database locks.
Why is Apache Kafka used for asynchronous peak shaving instead of synchronous gRPC processing?#
Synchronous gRPC calls to relational databases during 10M+ QPS flash sale spikes exhaust database connection pools and cause severe row-lock deadlocks. Pushing order events into Kafka queues absorbs traffic bursts immediately, allowing write-behind consumer workers to persist orders into storage at a steady, controlled rate.
What architectural advantages does TiDB NewSQL provide over traditional MySQL sharding?#
TiDB separates stateless SQL compute nodes from distributed TiKV storage nodes, eliminating manual database sharding maintenance and complex application routing proxies. It uses Multi-Raft consensus to automatically split and rebalance data regions across storage nodes while preserving strict ACID transactions.
Answer-first: Shopee isolates latency bottlenecks across 30+ microservice call hops by combining OpenTelemetry distributed tracing, ClickHouse columnar log storage, and Apache Flink real-time stream processing. Injecting W3C trace contexts through gRPC headers enables SREs to reconstruct waterfall traces and diagnose microservice failures in sub-seconds.
Chapter 5: Observability - Finding Bugs in the Microservices Jungle ← Series hub | ← Prev
Prerequisite: Read the previous article: Chapter 4: Shopee DB: MySQL Sharding to TiDB NewSQL Migration.
...
Answer-first: Shopee scales its relational database layer past single-node MySQL limits by migrating to TiDB Distributed SQL. By separating stateless SQL compute (TiDB) from stateful key-value storage (TiKV) and columnar analytics (TiFlash), TiDB delivers transparent horizontal auto-sharding and ACID transactions without application-level sharding logic.
Chapter 4: Database Scale - The Rise of TiDB and NewSQL ← Series hub | ← Prev | Next →
Prerequisite: Read the previous article: Chapter 3: Traffic Shield - Peak Shaving with Kafka and Graceful Degradation.
...
Answer-first: Shopee utilizes Apache Kafka queues for asynchronous peak shaving during 11.11 mega-campaigns. Decoupling order creation from database persistence guarantees sub-second API responses while downstream workers process orders at a controlled rate, protected by Sentinel adaptive load shedding and priority request classification.
Chapter 3: Peak Shaving - The Power of Apache Kafka and Graceful Degradation ← Series hub | ← Prev | Next →
Prerequisite: Read the previous article: Chapter 2: Flash Sale Engine - Solving Overselling and Hot Keys.
...
Answer-first: Shopee prevents overselling during high-concurrency flash sales by combining local memory caching, Redis inventory sharding, and atomic Lua script decrements. This multi-tier architecture isolates hot keys in Redis memory shards and evaluates stock availability in sub-milliseconds without acquiring relational database locks.
Chapter 2: Flash Sale Engine - The Mystery Behind Redis and Hot Keys ← Series hub | ← Prev | Next →
Prerequisite: Read the previous article: Chapter 1: Microservices Foundation - The Power of Go, gRPC, and API Gateway.
...
Answer-first: Shopee handles millions of concurrent users by migrating from monolithic systems to high-performance Go microservices. Inter-service gRPC Protobuf communication and Istio/Envoy service mesh sidecars enforce strict SLAs and sub-millisecond RPC latencies across thousands of internal microservice nodes.
Chapter 1: Building a Massive Foundation with Microservices, Golang, and gRPC ← Series hub | Next →
Prerequisite: This is the first chapter of the Shopee Architecture series. No prior reading is required to start here. You can view the full series roadmap at the Series Hub.
...