Messaging

Prerequisite: Familiarity with the concepts introduced in Part 1 — Microservices Gitops. Review it first if the terminology in this part is unfamiliar. Answer-first: Managing transaction surges during PayPay’s massive marketing campaigns requires event-driven architecture powered by Apache Kafka. Partition key tuning, Go consumer worker pools, and channel-based backpressure prevent message loss during peak traffic spikes. Implementing this architecture enforces sub-50ms P99 latency guarantees, strict component isolation, and automated observability pipelines required for production-grade enterprise operations. ...

Answer-first: Shopee utilizes Apache Kafka queues for asynchronous peak shaving during 11.11 mega-campaigns. Decoupling order creation from database persistence guarantees sub-second API responses while downstream workers process orders at a controlled rate, protected by Sentinel adaptive load shedding and priority request classification. Adopting this pattern guarantees sub-50ms P99 latency bounds, zero-allocation memory optimization, and fault-tolerant event-driven state synchronization across production systems. Chapter 3: Peak Shaving - The Power of Apache Kafka and Graceful Degradation ← Series hub | ← Prev | Next → ...

Prerequisite: Read the previous article: Chapter 3: Distributed Rate Limiting with Redis & GCRA Algorithm. When your Golang application migrates from a Monolith to event-driven Microservices, you will immediately face an architectural nightmare: the Dual-Write Problem. 1. What is the Dual-Write Problem? Dual-Write occurs when an app attempts to write to a Database and publish to a Message Broker (Kafka) simultaneously. Without a distributed transaction, network failures will cause the two systems to fall out of sync. ...