Advanced: Replication and Partitioning
Replication copies data across machines. Partitioning splits data across machines. Most large systems use both.
Why Replicate?
Replication can improve availability, reduce read latency, and protect against machine failure. It also creates the central problem of keeping copies consistent enough for the application.
Leader-Based Replication
In leader-based replication, writes go to a leader and are copied to followers. Reads may go to the leader or followers.
Synchronous replication gives stronger durability or freshness but increases latency and reduces availability. Asynchronous replication is faster and more available, but followers can lag and acknowledged writes may be lost if the leader fails before replication catches up.
Replication Lag
Replication lag creates user-visible anomalies:
- reading stale data after your own write
- seeing time move backward when reading from different replicas
- inconsistent results across pages or requests
Common mitigations include read-your-writes routing, monotonic reads, sticky sessions, version checks, and careful use of leader reads.
Multi-Leader and Leaderless Replication
Multi-leader replication accepts writes in multiple places, which helps with multi-region availability and offline clients. It also creates conflict resolution problems.
Leaderless replication lets clients write/read several replicas directly. Quorums can improve fault tolerance, but they do not remove all race conditions or consistency anomalies.
Partitioning
Partitioning divides a dataset so each node handles only part of it. The goal is to spread storage and load.
Key-range partitioning supports efficient range scans but can create hot spots. Hash partitioning spreads keys more evenly but loses natural ordering. Secondary indexes are harder because the lookup may need to touch many partitions unless the index is also partitioned carefully.
Rebalancing and Routing
As data or load changes, partitions need to move. Good rebalancing avoids moving too much data at once and avoids disrupting active traffic. Clients also need a way to find the node responsible for a key, either through a routing tier, cluster metadata, or smart clients.
Practical Takeaway
Replication and partitioning improve scale and availability, but they weaken simple mental models. Always ask:
- where can writes be accepted?
- how stale can reads be?
- what conflict resolution is used?
- what happens during failover?
- how are hot keys and large partitions handled?
- does an operation touch one partition or many?