Phase 5: Fast Leader Election (FLE) & Peer State Machine¶

Audience: Principal Distributed Systems Architect & Consensus Engine Specialist
Source References: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/FastLeaderElection.java

1. The Four Quorum Peer States¶

Every node in an ensemble runs a state machine defined in QuorumPeer.ServerState:

┌──────────────┬─────────────────────────────────────────────────────────────────┐
│ State        │ Behavioral Mode                                                 │
├──────────────┼─────────────────────────────────────────────────────────────────┤
│ LOOKING      │ Leader has failed; node is actively campaigning or voting      │
│ FOLLOWING    │ Node follows the active leader, replicating state & voting      │
│ LEADING      │ Node is the authoritative leader coordinating ZAB proposals     │
│ OBSERVING    │ Non-voting read replica; does not participate in elections      │
└──────────────┴─────────────────────────────────────────────────────────────────┘

Peer State Transitions:
        ┌───────────────► [ LOOKING ] ◄───────────────┐
        │                        │                    │
 (Leader Crashes)         (Election Won)       (Election Lost)
        │                        ▼                    │
 [ LEADING ]               [ FOLLOWING ] ─────────────┘

2. Fast Leader Election (FLE) Mathematical Rules¶

When an active leader crashes, all followers transition to LOOKING and execute Fast Leader Election:

Vote Tuple Structure:
Vote = (proposedLeader, proposedZxid, electionEpoch)

FLE Vote Comparison Algorithm:
When Node A (holding Vote l) receives Vote v from Node B:

1. Epoch Comparison (Highest Epoch Wins):
   if (v.electionEpoch > l.electionEpoch) {
       l.electionEpoch = v.electionEpoch;
       l.proposedLeader = v.proposedLeader;
       l.proposedZxid = v.proposedZxid;
       broadcastVote();
       return;
   } else if (v.electionEpoch < l.electionEpoch) {
       // Ignore vote from stale election epoch!
       return;
   }

2. Zxid Comparison (Most Up-To-Date Data Wins):
   if (v.proposedZxid > l.proposedZxid) {
       l.proposedLeader = v.proposedLeader;
       l.proposedZxid = v.proposedZxid;
       broadcastVote();
       return;
   } else if (v.proposedZxid < l.proposedZxid) {
       // Keep local vote; our data is fresher!
       return;
   }

3. MyID Tie-Breaker (Highest Node ID Wins):
   if (v.proposedLeader > l.proposedLeader) {
       l.proposedLeader = v.proposedLeader;
       broadcastVote();
   }

2.1 The Data-Freshness Guarantee¶

Because Rule 2 prioritizes zxid, it is mathematically impossible for a stale node that missed recent transactions to become Leader. The candidate with the highest committed transaction log always wins!

3. Strict Quorum Convergence & Split-Brain Prevention¶

Once a candidate observes that a majority of ensemble peers have adopted its vote: $$\text{Votes Received} \ge \left\lfloor \frac{N}{2} \right\rfloor + 1$$

It enters a brief verification window (finalizeWait, default: 200ms) to ensure no newer epoch notifications arrive, then claims leadership.

3.1 Why ZooKeeper CANNOT Suffer a Split-Brain¶

Consider a 5-node cluster partitioned by a network switch failure: $$\text{Partition 1: } \{ \text{Node 1}, \text{Node 2} \} \quad (2 \text{ nodes})$$ $$\text{Partition 2: } \{ \text{Node 3}, \text{Node 4}, \text{Node 5} \} \quad (3 \text{ nodes})$$

Quorum requirement for 5 nodes: $\lfloor 5/2 \rfloor + 1 = \mathbf{3 \text{ votes}}$.
Partition 1: Cannot form a quorum ($2 < 3$). Nodes remain stuck in LOOKING state and reject all client writes.
Partition 2: Owns 3 nodes ($3 \ge 3$). Successfully elects a leader and continues operating.
Mathematical Invariant: Any two majorities of size $\lfloor N/2 \rfloor + 1$ must intersect by at least one member. Therefore, two leaders can NEVER be elected simultaneously!