Connection Pool

Answer-first: Database sharding distributes data horizontally across independent partitions (shards) based on a shard key, reducing write contention and enabling linear storage growth. Choosing the wrong shard key leads to hot spots that can be worse than no sharding at all. Prerequisite: Part 4 of the System Design Masterclass. Read Part 3: Caching Strategies to understand the cache layer before examining storage. What You’ll Learn That AI Won’t Tell You PostgreSQL Connection Memory Math: Why each PostgreSQL connection eats 5–10MB of server RAM, and why a naive database/sql pool configuration crashes databases. TiDB Percolator Failures: The exact edge cases where primary lock failures leave secondary locks orphaned, and how TiDB’s async lock resolver cleans them up. LSM-Tree Write Amplification: The performance penalty of Cassandra/TiKV compaction cycles on SSD disk lifespan. Vertical vs Horizontal Scaling — When to Switch? Key Concept: Vertical scaling (scale-up) increases resources on a single server — simple but has a hard physical ceiling and non-linear cost growth. Horizontal scaling (scale-out) adds more servers — no theoretical ceiling, linear cost, but significantly higher operational complexity. ...

Prerequisite: Before reading this chapter, please ensure you have read the previous article in this series: Chapter 4: Solving the Dual-Write Problem with Transactional Outbox Pattern. If your Golang system processes business logic blazingly fast but chokes at the Database layer, 90% of the time, it is due to an incorrectly configured *sql.DB. 1. Understanding *sql.DB In Golang, sql.Open() does NOT create a direct database connection. It instantiates a thread-safe Connection Pool manager. You must initialize the db variable only once during app startup. ...