Distributed Systems - Quorum vs Raft vs 2PC
Quorum vs. Raft: The Hierarchy of Distributed Systems
In distributed systems, we often confuse "how we store data" with "how we govern it." To build reliable systems, we must distinguish between a mathematical property (Quorum), an orchestration protocol (Raft), and a logical contract (ACID).
1. The Functional Split
The primary difference lies in which "Plane" the logic occupies:
- Quorum (The Data Plane Rule): This is a mathematical requirement for Durability. It defines how many nodes must acknowledge a piece of data to ensure it isn't lost ($W + R > N$). It is "blind"—it doesn't care who sends the data, only that it is stored safely.
- Raft (The Full System Protocol): Raft provides Ordering. It tightly couples the Control Plane (Who is the leader?) with the Data Plane (Replicating the log). It ensures that every node sees the exact same sequence of events.
2. The "Fencing" Problem
These approaches differ most during a network failure:
- Raft (Built-in Protection): Because Raft handles leadership and data, it has an internal fencing mechanism. If a leader is partitioned, the protocol's "Terms" prevent it from committing more data.
- Quorum (External Dependency): In a pure Quorum system (like Amazon Aurora storage), the storage doesn't know who the leader is. You need an external Control Plane (AWS, Kubernetes, or a human) to "fence" or kill an old writer before they accidentally overwrite new data.
3. The Hierarchy of Guarantees
We can visualize these concepts as a stack. Each layer solves a progressively harder problem:
| Layer | Primary Guarantee | What it Solves |
|---|---|---|
| ACID (The Peak) | Correctness | Logical validity (e.g., bank transfers are safe and private). |
| Consensus (The Middle) | Ordering | Agreement on a single, linear sequence of events. |
| Quorum (The Base) | Durability | Survival of data even if a minority of nodes fail. |
4. Scaling: Partitioned Consensus
A single Raft group is a bottleneck. Modern systems like CockroachDB, TiDB, and Kafka scale by breaking data into shards, where each shard runs its own independent consensus.
- Multi-Raft: Every 64MB of data is its own "City-State" with its own leader. This minimizes the "blast radius" of a failure.
- Two-Phase Commit (2PC): When a transaction touches multiple shards, we use 2PC as the "Diplomatic Treaty." It ensures that Shard A and Shard B either commit together or fail together, providing Distributed ACID on top of the Raft layer.
5. Case Study: Kafka’s Hybrid Approach
Kafka (KRaft mode) uses a clever "Brain vs. Muscle" distinction:
- The Metadata Quorum (The Brain): A single Raft group for high-level cluster state.
- Data Partitions (The Muscle): A leaner protocol called ISR (In-Sync Replicas) for the messages. This keeps the data plane fast by avoiding the "chatter" of thousands of independent Raft heartbeats.
Summary Comparison
| System | Control Plane (The Brain) | Data Plane (The Muscle) |
|---|---|---|
| CockroachDB | Multi-Raft (Distributed) | Multi-Raft (Distributed) |
| Kafka (KRaft) | Raft Quorum | ISR Replication |
| AWS Aurora | External Cluster Manager | Storage Quorum ($W + R > N$) |
The Bottom Line: The Franchise Model
Think of a global business:
- Quorum is the requirement for a sale (Customer pays + Register logs it).
- Raft is the Store Manager. They ensure the local shop is running in order.
- Partitioning is opening 1,000 different stores so no one shop is too crowded.
- ACID is Corporate HQ ensuring a "Gift Card" used in Store A is correctly deducted and accepted in Store B.
Quorum gives you durability, Consensus gives you ordering, and ACID gives you correctness.