Skip to content

Leader Election

1. What is it? (2-3 lines)

Leader Election is a distributed consensus process where a cluster of independent nodes algorithmically designates a single node as the authoritative "Leader" (or Master) responsible for coordinating state changes, sequencing writes, and orchestrating cluster tasks.

2. Why does it exist?

In distributed systems, multi-node writes without a central coordinator lead to race conditions, split-brain scenarios, and inconsistent data order. Designating a single Leader simplifies state management by centralizing write ordering and coordination while remaining fault-tolerant (if the leader crashes, a new leader is elected automatically).

3. Where is it used?

  • Distributed Coordination: ZooKeeper (ZAB), etcd (Raft), Consul (Raft).
  • Relational / NoSQL Databases: PostgreSQL HA (Patroni), MySQL (Orchestrator), MongoDB (Replica Sets), Kafka (KRAFT Controller / ZooKeeper).
  • Cluster Orchestration: Kubernetes (kube-controller-manager leader election via Leases).

4. How does it work? (simple flow)

  1. Heartbeat Monitoring: Follower nodes continuously listen for periodic heartbeats from the active Leader.
  2. Election Timeout Trigger: If a follower misses heartbeats within a randomized timeout window (e.g. 150-300ms), it transitions to Candidate state and increments the Election Term.
  3. Request Votes: Candidate node broadcasts a RequestVote RPC to all other cluster nodes.
  4. Majority Vote Check: Other nodes vote for the Candidate if its log is at least as up-to-date as theirs and they haven't voted for another candidate in this term.
  5. Leader Promotion: If the Candidate receives votes from a Majority Quorum (\(\lfloor N/2 \rfloor + 1\)), it becomes the new Leader and immediately broadcasts heartbeats to assert authority.

5. Advantages

  • Centralizes state updates and log ordering, eliminating multi-master write conflicts.
  • Automatic failover: System recovers authority within seconds of a leader node crash.
  • Prevents split-brain when combined with Majority Quorum rules.

6. Limitations

  • Single-leader bottleneck: All write traffic must route through 1 leader node (scalability limit).
  • Brief availability pause during election windows (100ms-5s) where write requests are rejected or blocked.
  • Misconfigured election timeouts cause Split-Brain or continuous candidate vote flapping.

7. Common interview questions (10)

  1. How does randomized election timing in Raft prevent split-vote scenarios?
  2. What is Split-Brain and how does Majority Quorum prevent two nodes from claiming to be Leader?
  3. How does Kubernetes implement Leader Election using Lease API objects?
  4. What is an Election Term (or Epoch) and how does it resolve stale leader claims?
  5. How does ZooKeeper implement leader election using Ephemeral Sequential Znodes?
  6. What happens if a partitioned former Leader continues receiving writes (Fencing Tokens)?
  7. How does MongoDB select a new Primary node during Replica Set elections?
  8. What is the Bully Algorithm and how does it differ from Raft leader election?
  9. How does Kafka KRaft mode elect controller nodes without external ZooKeeper dependencies?
  10. What is the impact of network latency spikes on leader election stability?

8. Common misconceptions

  • Misconception: Leader election requires a master authority outside the cluster. Fact: Cluster nodes elect a leader autonomously among themselves using consensus algorithms (Raft/Paxos).
  • Misconception: The node with the fastest CPU becomes the leader. Fact: The node that initiates an election first and holds the most up-to-date log history wins majority votes.

10. 30-second cheat sheet

  • Goal: Elect 1 authoritative Leader node to order writes and manage cluster coordination.
  • Trigger: Follower misses Leader heartbeat \(\implies\) Transitions to Candidate \(\implies\) Increments Term.
  • Winning Rule: Candidate receiving Majority Quorum votes (\(\lfloor N/2 \rfloor + 1\)) wins election.
  • Split-Brain Prevention: Enforce Majority Quorum (e.g., 3 of 5 nodes). A minority partition CANNOT elect a leader!