High Concurrency

Executive Overview: High-concurrency system design requires lock-free data structures, asynchronous I/O multiplexing, and zero-copy memory buffers to scale to millions of requests. Pillar Architecture Guide: This article is part of the High-throughput Go Framework Benchmarks: Gin, Fiber, Kratos series. Please refer to the original article for a comprehensive overview of the architecture. Answer-First: Handling millions of requests per second (the C10M problem) requires eliminating kernel-space context switching overhead through asynchronous event loops (epoll/kqueue) or kernel-bypass networking (DPDK, io_uring), paired with zero-copy I/O memory buffers, L4 DSR (Direct Server Return) load balancing, and lock-free concurrency structures in Go. ...

Pillar Architecture Guide: This article is part of the High-throughput Go Framework Benchmarks: Gin, Fiber, Kratos series. Please refer to the original article for a comprehensive overview of the architecture. Prerequisite: Read the previous article: Chapter 3: Distributed Rate Limiting with Redis & GCRA Algorithm. When your Golang application migrates from a Monolith to event-driven Microservices, you will immediately face an architectural nightmare: the Dual-Write Problem. 1. What is the Dual-Write Problem? Dual-Write occurs when an app attempts to write to a Database and publish to a Message Broker (Kafka) simultaneously. Without a distributed transaction, network failures will cause the two systems to fall out of sync. ...

Pillar Architecture Guide: This article is part of the High-throughput Go Framework Benchmarks: Gin, Fiber, Kratos series. Please refer to the original article for a comprehensive overview of the architecture. Prerequisite: Before reading this chapter, review Chapter 2: The 3 Caching Vulnerabilities. Chapter 3: Distributed Rate Limiting with Redis & GCRA Algorithm Executive Summary & Quick Answer: Distributed rate limiting in microservice architectures requires centralized state management in Redis to avoid load-balancer bypasses. Implementing the Generic Cell Rate Algorithm (GCRA) via atomic Lua scripts tracks Theoretical Arrival Times (TAT) using a single 64-bit integer per user key, guaranteeing sub-millisecond execution. ...

Multi-tier distributed caching using Redis clusters and in-memory LRU buffers prevents database thundering herd and reduces read latency to sub-millisecond ranges. Pillar Architecture Guide: This article is part of the High-throughput Go Framework Benchmarks: Gin, Fiber, Kratos series. Please refer to the original article for a comprehensive overview of the architecture. Prerequisite: Before reading this chapter, review Chapter 1: How Systems Handle Millions of Requests/s. What You’ll Learn That AI Won’t Tell You Bloom Filter Math: How to calculate bit array sizes ($m$) and hash function counts ($k$) for <1% false positive rates. XFetch Beta Tuning: Adjusting the scaling factor ($\beta$) to force probabilistic background recomputation before TTL expiration. Singleflight Timeout Leaks: Guarding singleflight calls with Go context deadlines to prevent goroutine hangs. Caching is the ultimate shield for databases in distributed systems. However, poorly implemented caches can become the exact reason your system crashes. In this chapter, we dissect three classic caching phenomenons and how to defend against them using Golang. ...

Answer-First: High-concurrency B2B commerce platforms achieve 25M monthly throughput by coupling Go microservices, distributed queues, and resilient database connection pooling. Pillar Architecture Guide: This article is part of the High-throughput Go Framework Benchmarks: Gin, Fiber, Kratos series. Please refer to the original article for a comprehensive overview of the architecture. Prerequisite: This is the executive summary and introductory overview of the High Concurrency Systems series. No prior reading is required to start here. You can view the full series roadmap at the Series Hub. ...

Answer-First: Shopee handles millions of concurrent users by migrating from monolithic systems to high-performance Go microservices. Inter-service gRPC Protobuf communication and Istio/Envoy service mesh sidecars enforce strict SLAs and sub-millisecond RPC latencies across thousands of internal microservice nodes. Chapter 1: Building a Massive Foundation with Microservices, Golang, and gRPC ← Series hub | Next → Prerequisite: This is the first chapter of the Shopee Architecture series. No prior reading is required to start here. You can view the full series roadmap at the Series Hub. ...