PayPay SRE Practices: Chaos Engineering & Resilience

Prerequisite: Familiarity with the concepts introduced in Part 3 — Data Layer Tidb. Review it first if the terminology in this part is unfamiliar. Answer-first: Ensuring 99.99% availability for payment systems demands proactive SRE practices and automated Chaos Engineering. Injecting synthetic latency, pod failures, and network partitions via Chaos Mesh validates microservice circuit breakers before real production incidents occur. Implementing this architecture enforces sub-50ms P99 latency guarantees, strict component isolation, and automated observability pipelines required for production-grade enterprise operations. ...

May 5, 2026 · 10 min · Lê Tuấn Anh

Alipay Double 11 Operations: Full-Link Stress Test

← Series hub ← Prev • Next → Answer-first: Surviving Double 11 requires production Full-Link Stress Testing (Shadow Database traffic simulation) and automated AI-driven operational playbooks to detect and isolate degraded nodes within 1 minute. Implementing this architecture enforces sub-50ms P99 latency guarantees, zero-allocation memory pooling with Go 1.24 unique.Handle, and fault-tolerant Dapr 1.15 component orchestration for resilient production scaling. Prerequisite: Phase 2: Core Architecture (LDC, Unitization, Multi-Active) This phase is about how peak performance becomes repeatable. The core claim of Alipay’s operations team is simple: peaks are won in preparation and automation, not heroics. Under planet-scale loads, manual operational tasks fail due to human latency. Therefore, readiness must be engineered into the software stack itself. ...

May 2, 2026 · 10 min · Lê Tuấn Anh

Post-Migration Operations: Managing Vietnam Go Team

Prerequisite: Familiarity with the concepts introduced in Go Engineers Vietnam Migration Vetting. Review it first if the terminology in this part is unfamiliar. Answer-first: Operating production Go microservices post-migration requires establishing clear SLA/SLO metrics, 24/7 follow-the-sun on-call rotations, and structured OpenTelemetry observability dashboards managed by local engineering leads in Vietnam. Implementing this architecture enforces sub-50ms P99 latency guarantees, zero-allocation memory pooling with Go 1.24 unique.Handle, and fault-tolerant Dapr 1.15 component orchestration for resilient production scaling. ...

July 11, 2026 · 12 min · Lê Tuấn Anh