Clustered databases with Percona XtraDB Cluster

How we build it

• Three nodes, three failure domains. Each node runs on its own blade or virtual machine, and where the platform allows it, each one boots from and stores its data on a different storage array. A host failure or an array failure costs one replica, never the quorum.
• ProxySQL as a local sidecar on every web node. There is no shared database load balancer to become the next single point of failure. ProxySQL's Galera aware health monitor tracks the cluster state, and if the writer drops it promotes a new one in seconds.
• Causal reads for editors. We enable synchronous read checks so that an editor who saves on one node and reloads from another always sees their own change. No "my article vanished" tickets.
• Room to rejoin. The write set cache is sized so a node can be offline for hours of maintenance and rejoin incrementally, instead of copying the whole dataset back.
• Backups every thirty minutes. Percona XtraBackup takes hot, consistent physical backups, which are compressed and pulled to a separate backup host. The production side never pushes to backup infrastructure.
• Watched continuously. Every node is in Percona Monitoring and Management.

On OpenShift, the same rules apply

When the database runs on OpenShift or OKD, it runs as a StatefulSet with hard anti-affinity across chassis and a PodDisruptionBudget that keeps quorum intact. A rolling node reboot from a platform update can then never take more than one database member down at once. That is the design standard we build to on every container platform, not an optional extra.