Database

Prerequisite: This is the starting part of the series — no prior part is required. Later parts assume the concepts introduced here. Answer-first: A production-grade double-entry ledger enforces immutable, append-only transaction logs decoupled from balance state updates. By using fixed-size C-aligned memory structs or PostgreSQL check constraints and triggers, the schema guarantees strict debit-credit mathematical invariants, prevents hot-row lock contention, and eliminates double-spend risks in high-concurrency core banking architectures. Implementing this architecture enforces sub-50ms P99 latency guarantees, strict component isolation, and automated observability. ...

Prerequisite: Familiarity with the concepts introduced in Part 1 — Double Entry Ledger Schema. Review it first if the terminology in this part is unfamiliar. Answer-first: Distributed SQL engines preserve multi-region ACID serializability by combining Raft/Paxos consensus with bounded clock synchronization protocols such as Spanner TrueTime, CockroachDB HLC, or TiDB Percolator TSO. Selecting optimal commit-wait delays and timestamp allocation strategies minimizes two-phase commit overhead, achieving low transaction latencies across cross-region core banking nodes. Implementing this architecture enforces sub-50ms P99 latency guarantees, strict component isolation, and automated. ...

Prerequisite: Familiarity with the concepts introduced in Part 4 — Grpc Rest Gateway. Review it first if the terminology in this part is unfamiliar. The EAV schema is why most Magento migrations fail. It looks manageable from the outside: products stored across catalog_product_entity, catalog_product_entity_varchar, catalog_product_entity_int, catalog_product_entity_decimal, catalog_product_entity_datetime, and catalog_product_entity_text. Six tables, straightforward ETL job, done in a weekend. Then you discover that attribute_id = 75 means “product name” in your Magento instance and “color” in your staging instance. Every attribute ID is generated at install time and differs between environments. Any ETL script that hardcodes attribute IDs will produce corrupted data in production. ...

← Previous Chapter: Part 2 — Golang vs. PHP/Laravel | Series hub | Next Chapter: Part 4 — Apache Kafka vs. NATS JetStream → Answer-first: For distributed write-heavy architectures (≥10,000 writes/s) on MySQL/InnoDB, Snowflake ID (64-bit) is optimal, eliminating the 50% secondary index multiplier tax while preserving B-tree locality. For PostgreSQL, client-generated keys, or coordinate-free distributed topologies, UUIDv7 (RFC 9562) delivers 98% sequential page packing without dedicated coordinator nodes, overcoming random UUIDv4 page thrashing and IOPS cliff failures. ...

Prerequisite: Familiarity with the concepts introduced in Part 2 — Event Driven Kafka. Review it first if the terminology in this part is unfamiliar. Answer-first: PayPay migrated its database layer from AWS Aurora MySQL to TiDB Distributed SQL to overcome vertical scaling limitations. TiDB’s Raft-based auto-sharding and horizontal compute/storage separation deliver linear scaling under billion-row transaction tables. Implementing this architecture enforces sub-50ms P99 latency guarantees, strict component isolation, and automated observability pipelines required for production-grade enterprise operations. ...

Answer-first: Spatial indexing serves as a high-performance pre-filtering layer that prevents heavy routing engines from collapsing under load. By using Uber H3 hexagonal cells and Redis GEO to narrow down 10,000 active drivers to the 50 closest candidates in RAM (<2ms), systems reduce routing engine CPU overhead by up to 95%. Prerequisite: Before reading this part, review Part 2: Zero to Hero Environment Setup. Part 3: Spatial Indexing — Uber H3, PostGIS & Redis GEO Answer-first: Spatial indexing serves as a high-performance pre-filtering layer that prevents heavy routing engines from collapsing under load. By using Uber H3 hexagonal cells and Redis GEO to narrow down 10,000 active drivers to the 50 closest candidates in RAM (<2ms), systems reduce routing engine CPU overhead by up to 95%. ...

Answer-first: Enforcing ACID isolation levels in core banking prevents lost updates and dirty reads during high-concurrency transfers. Using PostgreSQL REPEATABLE READ or pessimistic row locking (SELECT FOR UPDATE) combined with Go connection pooling guarantees transactional integrity. Spanner and CockroachDB provide linearizable distributed ACID transactions across microservices using Paxos consensus and Hybrid Logical Clocks. Prerequisite: Part 2: CASA & Lending Domain Logic on transaction parameters. The Core Problem: Concurrency Answer-first: High-concurrency banking transfers risking race conditions and lost updates require strict database lock isolation to protect ledger state. ...

Answer-first: Shopee scales its relational database layer past single-node MySQL limits by migrating to TiDB Distributed SQL. By separating stateless SQL compute (TiDB) from stateful key-value storage (TiKV) and columnar analytics (TiFlash), TiDB delivers transparent horizontal auto-sharding and ACID transactions without application-level sharding logic. Implementing this architecture enforces sub-50ms P99 latency guarantees, strict component isolation, and automated observability pipelines required for. Chapter 4: Database Scale - The Rise of TiDB and NewSQL ← Series hub | ← Prev | Next → ...

← Series hub ← Prev • Next → Answer-first: Alipay’s Double 11 technology deep dive reveals high-performance internals: binary Bolt RPC protocol multiplexing over single TCP streams, RocketMQ 2PC transactional messaging for async decoupling, OceanBase LSM-tree compaction tuning, and multi-zone Paxos quorum consensus to achieve 544,000 TPS payment processing. Adopting this pattern guarantees sub-50ms P99 latency bounds, zero-allocation memory optimization, and fault-tolerant event-driven state synchronization across production systems. Prerequisite: Phase 4: Technology Overview ...

MySQL Scalability Guide: Read Replicas, Sharding, and Distributed SQL Answer-first: Scaling MySQL for high-traffic applications involves a phased progression: tuning InnoDB buffer pools and slow queries (0–500 TPS), offloading reads via ProxySQL and read replicas (500–3,000 TPS), and adopting horizontal write scaling (3,000+ TPS) via Vitess sharding or TiDB Distributed SQL to maintain sub-50ms P99 query latencies. MySQL scalability is the ability to increase database throughput — reads per second, writes per second, or data volume — without rewriting your application. The critical distinction: read scaling (adding replicas) and write scaling (sharding or distributed SQL) require completely different architectural approaches. Choosing the wrong path creates technical debt that takes months to unwind. ...

Vitess vs GORM Sharding: MySQL Write Scaling in Go Answer-first: Horizontal MySQL write scaling uses Vitess proxy sharding or application-level GORM shard key routing to distribute table data across independent database master instances. While Vitess provides transparent VTGate query routing and zero-downtime resharding for large teams, GORM Sharding offers a lightweight Go-native AST-parsing solution for single-service architectures. When your application reaches millions of users, a single database instance will inevitably become the biggest bottleneck in your entire architecture. To solve this, MySQL database scaling becomes mandatory. You must Scale DB for Microservices using Horizontal Scaling techniques. ...

Replace MySQL Sharding with TiDB: Distributed SQL Architecture Answer-first: TiDB is the leading open-source MySQL sharding alternative, replacing fragile application-level sharding logic (Vitess, GORM Sharding) with an auto-partitioned Distributed SQL architecture. By distributing 96MB Raft Regions across TiKV storage nodes and utilizing the Percolator distributed transaction protocol, TiDB delivers horizontal write scaling, cross-node ACID transactions, and zero-downtime online DDL while maintaining 100% MySQL wire compatibility. Scaling a relational database is one of the most demanding challenges in system design. As applications grow from thousands to millions of active users, the database ceases to be a simple storage engine and becomes the primary bottleneck of the entire system architecture. In this technical guide, we explore the architectural progression of scaling MySQL—beginning with replication topologies, stepping through the complexities and operational hazards of manual database sharding (including proxy middleware like Vitess), and evaluating NewSQL alternatives, specifically the distributed architecture of TiDB. ...