Kafka

In Part 1: The Paradigm Shift - Agentic Architecture & Golang Orchestration Power, we established the Orchestration Engine using Golang and Eino. However, no matter how smart a brain is, it becomes useless if fed with misleading, unstructured, or fragmented information. In the e-commerce domain, product catalog data changes continuously every second: prices fluctuate, inventory is updated, new products are added. Meanwhile, chunking product data to feed into a Vector Database (Qdrant) is entirely different from chunking a PDF document or a news article. ...

1. “Yesterday’s Data” is a Disaster If a customer asks a banking Chatbot about savings interest rates, and the Chatbot answers based on a PDF policy file that was changed… 2 hours ago. What happens? In Enterprise environments like Finance, Healthcare, or E-commerce, Yesterday’s data is a legal liability. Legacy data pipelines (ETL Batch Jobs running at midnight) no longer meet the demands of 2026. If the Core Database changes, your Vector Database must be updated immediately. Data Freshness must be measured in seconds. ...

Answer-first: Monolith decoupling succeeds only when solving eventual consistency and distributed tracing overhead early. Mitigate inventory overselling via Redis-based BFF locking, stream database sync in real-time via Debezium CDC and Kafka, and build distributed tracing via OpenTelemetry from day one to avoid system blindness. What You’ll Learn That AI Won’t Tell You Strangler Fig routing configurations for Envoy that migrate traffic path-by-path from Magento to Go microservices without dropping active sessions. How to implement a double-write database sync listener in Go to prevent data drift during the multi-month migration window. In theory, MACH (Microservices, API-first, Cloud-native, Headless) and Composable Commerce are the “holy grail” of the ecommerce industry. However, when systems scale to process millions of transactions, issues regarding data consistency and Observability costs truly surface. This article outlines the hard-learned lessons from our Chief Architects when migrating a monolithic system to a Composable architecture. ...

Prerequisite: Part 5 of the System Design Masterclass. Read Part 4: Database Scaling to understand the storage tier that persisted events are written to. Answer-first: Event-Driven Architecture decouples services through asynchronous communication via a durable message log. In Go, goroutines and buffered channels implement natural backpressure — when consumers fall behind producers, the channel fills up and blocks the producer, throttling the ingest rate automatically. Kafka vs RabbitMQ — When to Use Each? Answer-first: Kafka is a distributed commit log — messages are retained indefinitely, consumers manage their own offsets, and replay is possible. RabbitMQ is a message broker — messages are deleted after acknowledgment, the broker handles routing complexity, push-based delivery. They solve different problems. ...

Answer-first: Composable banking replaces rigid legacy cores with modular Go microservices. The transition uses the Strangler Fig pattern to decouple domains, while distributed Sagas manage eventual consistency across transaction engines, and NewSQL databases provide horizontal scaling without sacrificing ACID compliance. What You’ll Learn That AI Won’t Tell You Strangler fig patterns for core banking systems that prevent data corruption. How to bridge legacy COBOL records into dynamic JSON streams using Go middleware. Legacy core banking systems were designed in a different era. Temenos T24, Finacle, and Flexcube shared one defining assumption: the bank’s entire product catalogue — deposits, lending, payments, trade finance — would live inside a single, tightly coupled application and a single, shared database. That assumption held when banking moved at human speed. It breaks completely when product releases need to go from months to days, when a single fraud engine update must not risk a payments outage, and when engineers on a COBOL codebase are retiring faster than they can be replaced. ...

← Previous | Series hub | Next → Chapter 4: Eliminating the Dual-Write Nightmare When your Golang application migrates from a Monolith to Event-Driven Microservices, you will immediately face an architectural nightmare: the Dual-Write Problem. 1. What is the Dual-Write Problem? Answer-first: Dual-Write occurs when an app attempts to write to a Database and publish to a Message Broker (Kafka) simultaneously. Without a distributed transaction, network failures will cause the two systems to fall out of sync. ...

Answer-first: Attempting to simultaneously write inventory updates to a fast cache (Redis) and a relational database (PostgreSQL) creates the dual-write problem. If one system fails, data diverges. Furthermore, synchronous SELECT FOR UPDATE queries in SQL cause massive lock queues and API timeouts. What You’ll Learn That AI Won’t Tell You Write-through caches configuration in Redis to prevent inventory drift. Lua scripting implementations in Redis that prevent double-reservations under peak load. What Is Real-Time Inventory Synchronization? Real-time inventory synchronization is the process of propagating stock count changes from the system of record (database) to all sales channels — web storefront, mobile app, WMS, ERP — in sub-second time. Instead of batch ETL jobs that run every hour, a CDC + Kafka pipeline streams every committed stock change as an event, eliminating overselling and stale stock displays. ...

Answer-first: Solve observability blind spots across distributed Go microservices by implementing an OpenTelemetry pipeline. Propagate W3C trace context across HTTP/gRPC boundaries and Kafka streams, batch metrics at the local agent level, and use tail-based sampling at the collector gateway to filter noise before ingestion. What You’ll Learn That AI Won’t Tell You OpenTelemetry collector tuning for low-overhead distributed tracing. Propagating span contexts over asynchronous Kafka messaging systems without breaking tracing chains. Monitoring complex Go microservices requires more than isolated logs. When a request traverses HTTP APIs, Kafka event streams, and asynchronous worker pools, you need absolute visibility to pinpoint latency bottlenecks and failures. ...

Answer-first: PayPay handles 7.8B annual transactions using a cloud-native architecture centered on TiDB for distributed ACID transactions, Kafka for event streaming, and Kotlin/Go microservices. GitOps-driven deployments and continuous chaos engineering ensure high availability and disaster recovery. What You’ll Learn That AI Won’t Tell You Running chaos engineering scripts in TiDB payment systems. How event sourcing with Kafka isolates PayPay checkout routes from legacy bank outages. PayPay launched in October 2018 and grew to 10 million users in just 3 months — a growth rate that no Japanese fintech had ever seen. By 2025, the platform had crossed 70 million registered users and processed 7.8 billion payments per year. Behind this growth is an engineering team that has had to scale not just their infrastructure, but their entire engineering culture: from service standardization and GitOps-driven deployments to chaos engineering and AI-powered fraud detection. ...

Answer-first: Ride-hailing architectures ingest millions of GPS pings per second using Uber’s H3 spatial index for geofencing. Kafka streams location updates to matching engines for driver allocation, while Flink processes real-time pricing and push gateways notify users. What You’ll Learn That AI Won’t Tell You Scaling matching engines to millions of geographic updates using H3 indexing. Designing low-latency push notification gateways to dispatch driver routes. The moment you open the Uber or Grab app, a cascade of real-time systems activates simultaneously: your phone begins transmitting GPS coordinates, a geospatial index updates your location, a matching engine re-evaluates nearby driver availability, a pricing model recalculates the fare based on supply-demand ratios, and a push notification pipeline prepares to deliver your match confirmation in under 3 seconds. ...

Answer-first: Build resilient event-driven microservices by using Dapr’s Pub/Sub APIs to decouple message transport. Ensuring eventual consistency requires implementing the Transactional Outbox pattern on writes, utilizing dead-letter queues (DLQs) for failed runs, and designing idempotent message handlers. What You’ll Learn That AI Won’t Tell You How to configure dead-letter queues in Dapr to handle poison messages. Designing idempotent message handlers that process duplicate events safely. In my previous post, we explored how abandoning monolithic architecture in favor of strict Domain-Driven Design (DDD) bounded contexts allowed an e-commerce platform to scale beyond 10,000+ orders per day. However, splitting one big database into 20+ isolated Postgres databases introduces a terrifying new problem: How do we maintain data consistency across disconnected services? ...