API Gateway

← Previous Chapter: Part 1: CAP, PACELC & Clean Architecture | Series Hub: System Design Masterclass | Next Chapter: Part 3: Caching Strategies, Redis/Valkey & Stampede Prevention → Prerequisite: Read Part 1: CAP, PACELC & Clean Architecture Primer to understand distributed trade-offs and composite availability foundations. Answer-first: Layer 4 load balancers route packets via eBPF and Direct Server Return to achieve sub-millisecond wire speed, while Layer 7 API gateways inspect HTTP headers and enforce token bucket rate limits. Combining kernel-bypass XDP packet filtering with Go reverse proxy buffer pools sustains 100,000 requests per second with sub-5ms P99 latency bounds across distributed clusters. ...

Multi-Language Edition: This chapter is also available in Vietnamese at 📖 Bản tiếng Việt (Vietnamese Edition). Previous: Chapter 5 — Optimizing Golang Database Connection Pools | Series Hub | Next: Chapter 7 — Designing Idempotency APIs for Payment Systems Answer-First: The long-standing debate of “API Gateway vs. Service Mesh” is resolved by strict traffic topology boundaries: North-South Traffic (external untrusted clients entering the cluster) is exclusively governed by an API Gateway (Envoy, Kong, or K8s Gateway API) focusing on edge SSL termination, WAF scrubbing, OAuth2/OIDC token exchange, and API monetization. In contrast, East-West Traffic (inter-service communication within the private cluster) is handled by a Service Mesh (Istio Ambient Mesh or Cilium eBPF) delivering zero-trust mTLS via SPIFFE/SPIRE, dynamic circuit breaking, outlier detection, and distributed tracing without application code changes. ...

High-throughput local LLM architecture guide combining vLLM PagedAttention virtual memory, Prefill-Decode disaggregation over RoCE v2/NVLink, and a custom Go API Gateway with SHA256 prompt prefix context-affinity routing, zero-allocation SSE streaming, and 71% cost savings over SaaS APIs.

Flash Sale Architecture: Rate Limiting & Redis Answer-first: High-concurrency flash sale systems absorb millions of synchronized user requests using a 5-Tier Traffic Shedding Architecture: Cloudflare CDN edge static asset caching, Envoy API Gateway atomic Token Bucket rate limiting, Redis Cluster Lua inventory reservations with hotkey slot splitting, partitioned Kafka queue buffering, and asynchronous Go worker pools executing batch upserts into TiDB/MySQL. [!NOTE] On sourcing: This article describes flash-sale architecture patterns for C10M-scale events; it is not a disclosure of Shopee’s internal systems, and the figures here are engineering targets rather than published Shopee metrics. Shopee has not publicly documented its flash-sale internals in detail. What is public is its database platform choice — Shopee’s adoption of TiDB is documented in PingCAP’s case studies (How Shopee Chose the Right Database, Shopping on Shopee, the TiDB Way). Treat everything else as a reference pattern to validate against your own workload. ...

21-Service Go Ecommerce Microservices Diagram E-Commerce Architecture Patterns: Monolith vs Microservices Answer-first: An ecommerce microservices architecture diagram structures enterprise retail platforms into 6 bounded domains—Commerce Flow, Product & Content, Logistics, Post-Purchase, Identity & Access, and Platform Operations—powering 21 Go microservices. Orchestrated via gRPC contracts and Dapr Pub/Sub event meshes, this design delivers sub-50ms P99 latency, isolates database-per-service failures, and automates rollbacks via distributed Saga workflows. Monolithic vs Microservices E-Commerce Comparison Dimension Monolithic E-Commerce Microservices E-Commerce Scaling Vertical scaling of entire monolith application Independent horizontal scaling per domain (e.g., Catalog 10x Cart) Database Architecture Single shared database with cross-table SQL joins Database-per-service (PostgreSQL, Redis, Elasticsearch) with zero cross-domain access Deployment Frequency Low frequency; all-or-nothing monolithic releases High frequency; independent CI/CD pipelines per microservice Fault Tolerance Low; a single bug or memory leak crashes the entire store High; failure in one domain (e.g., Reviews) does not block Checkout Complexity Low initial architectural and operational complexity High distributed complexity (Saga pattern, gRPC contracts, Dapr mesh) Operational Cost Lower initial cost; scales expensively at high traffic Higher initial infrastructure setup; cost-effective at high scale Practical latency and memory metrics comparing an Envoy-based API Gateway to a custom Go reverse proxy under 100k concurrent connections. How to tune circuit breaker thresholds (go-resiliency/breaker) to prevent premature service isolation during temporary network jitters. When transitioning from a monolithic platform to a distributed microservice setup, the hardest question isn’t “How do we write the code?” — it’s “How do these moving parts talk to each other safely, and why is each boundary drawn exactly where it is?” ...