Messaging

Executive Summary & Quick Answer: Managing transaction surges during PayPay’s massive marketing campaigns requires event-driven architecture powered by Apache Kafka. Partition key tuning, Go consumer worker pools, and channel-based backpressure prevent message loss during peak traffic spikes. Answer-First: PayPay builds a decoupled microservices network by streaming transactions asynchronously via Apache Kafka. To ensure financial safety, consumers process events using idempotency keys tracked in distributed caches, preventing duplicate ledger entries or double-spend occurrences in the event of retries or network partition splits. ...

Pillar Architecture Guide: This article is part of the High-throughput Go Framework Benchmarks: Gin, Fiber, Kratos series. Please refer to the original article for a comprehensive overview of the architecture. Prerequisite: Read the previous article: Chapter 3: Distributed Rate Limiting with Redis & GCRA Algorithm. When your Golang application migrates from a Monolith to event-driven Microservices, you will immediately face an architectural nightmare: the Dual-Write Problem. 1. What is the Dual-Write Problem? Dual-Write occurs when an app attempts to write to a Database and publish to a Message Broker (Kafka) simultaneously. Without a distributed transaction, network failures will cause the two systems to fall out of sync. ...

Answer-First: Shopee utilizes Apache Kafka queues for asynchronous peak shaving during 11.11 mega-campaigns. Decoupling order creation from database persistence guarantees sub-second API responses while downstream workers process orders at a controlled rate, protected by Sentinel adaptive load shedding and priority request classification. Chapter 3: Peak Shaving - The Power of Apache Kafka and Graceful Degradation ← Series hub | ← Prev | Next → Prerequisite: Read the previous article: Chapter 2: Flash Sale Engine - Solving Overselling and Hot Keys. ...