Multi-Paxos is the optimization that makes Paxos practical as a replicated log. The insight is that Phase 1 can be amortized across many slots once you have a stable leader.

What Phase 1 Actually Does

Recall basic Paxos. Phase 1 (Prepare/Promise) establishes that a proposer has the highest ballot number any acceptor will respond to. The acceptors promise to ignore any lower-numbered proposer. This is the “take control” phase – after it completes, the proposer has permission to drive proposals through Phase 2.

Phase 1 doesn’t say anything about which value gets proposed. It just establishes leadership over the acceptors for a given ballot number.

The Amortization

If the same leader runs Phase 1 once with a high ballot number, it doesn’t need to run Phase 1 again for subsequent log slots – as long as no other proposer challenges it with a higher ballot.

The leader runs Phase 1 once, gets promises from a majority, and then drives Phase 2 directly for slot 1, slot 2, slot 3, and so on. Each slot is just one round-trip (Accept/Accepted) instead of two. Throughput roughly doubles for steady-state operation under a stable leader.

Phase 1 (once):   Leader → Prepare(ballot=5) → Majority
                  Majority → Promise(ballot=5, ...) → Leader

Phase 2 (per slot):
  Slot 1:  Leader → Accept(ballot=5, slot=1, value=A) → Majority
           Majority → Accepted → Leader → Committed
  Slot 2:  Leader → Accept(ballot=5, slot=2, value=B) → Majority
           Majority → Accepted → Leader → Committed
  ...

This is Multi-Paxos. The Phase 1 cost is paid once per leader term, not once per log entry.

Leader Election

Basic Paxos has no notion of a leader – any node can propose at any time, which is why it’s correct but chaotic in practice. Multi-Paxos adds an informal leader election layer: one node is designated the distinguished proposer and the others voluntarily defer to it.

“Voluntarily” is the key word. If followers suspect the leader is dead (no heartbeats for a timeout period), any of them can run Phase 1 with a higher ballot to take over. The old leader doesn’t get a veto – it just gets its Accept messages rejected because the acceptors have now promised a higher ballot.

This is where Multi-Paxos implementations diverge significantly. The Paxos paper doesn’t specify how leader election works, what timeouts to use, or how to detect a live leader that’s just partitioned. Every real implementation makes different choices here, which is part of why Multi-Paxos implementations are hard to compare.

Log Holes

One complication basic Paxos avoids: with multiple slots running in parallel, you can get holes. The leader accepts slot 1 and slot 3 but slot 2 hasn’t been filled yet because a client request was lost. Followers can’t apply slot 3 to the state machine without knowing slot 2.

Multi-Paxos implementations handle this with a no-op: when a new leader wins an election, it runs Phase 1 for all uncommitted slots and fills any gaps with a no-op entry. This is what “leader reconciliation” looks like in practice – the new leader has to figure out what was committed before it took over and resolve any ambiguity.

Raft avoids holes entirely by requiring that the leader always send entries in order, with no gaps. This simplifies the implementation at the cost of some throughput (you can’t pipeline out-of-order).

Zab vs. Multi-Paxos

ZooKeeper uses Zab (ZooKeeper Atomic Broadcast), which is often described as “Paxos-like” but is actually closer to Multi-Paxos with some explicit differences.

The main one: Zab is designed around a primary-backup model where the primary sends updates in a strict FIFO order. The recovery protocol (Phase 1 equivalent in Zab) is designed to ensure the new primary has a superset of all committed entries before it starts serving. Zab also has an explicit synchronization phase where followers sync their logs to the new primary before it starts accepting writes.

Both tolerate the same number of failures and provide the same safety guarantees. The practical difference is that Zab is tuned for ZooKeeper’s specific write pattern (broadcast to all followers, wait for majority ack, apply in order) while Multi-Paxos is a more general framework.

Multi-Paxos vs. Raft

Multi-Paxos and Raft have the same fault tolerance and safety properties. The difference is that Raft specifies the things Multi-Paxos leaves up to the implementer – which is why Raft implementations tend to be more consistent across codebases.

Google’s Chubby uses a Paxos variant internally. Google Spanner uses a Paxos variant per shard. In both cases Google built their own leader election and recovery logic on top of the basic protocol – which is exactly the work Raft was designed to eliminate.

This is the consensus problem. Paxos is the foundational answer. It’s also notoriously difficult to implement correctly – which is exactly why Raft exists. But you can’t really understand Raft without understanding what Paxos is doing and why.

The Constraint

Two-phase commit solves distributed atomicity but requires the coordinator to stay up. If the coordinator crashes after phase 1, participants are stuck holding locks, unable to proceed. That’s not consensus – that’s a single point of failure with extra steps.

Paxos removes that constraint. It guarantees agreement as long as a majority of nodes are up, regardless of which specific nodes fail. Any node can propose a value. No permanent coordinator required.

Roles

In Paxos, nodes play three roles – and in practice, any node can play any of them:

• Proposer – initiates a round, tries to get a value accepted
• Acceptor – votes on proposals, must follow the protocol strictly
• Learner – learns the final chosen value once consensus is reached

The Protocol

Paxos runs in two phases.

Phase 1 – Prepare / Promise

The proposer picks a proposal number n – unique, higher than any it has seen before – and sends PREPARE(n) to a quorum of acceptors (any majority).

Each acceptor that receives this:

• Promises to ignore any future PREPARE with a lower number
• If it has already accepted a value in a prior round, includes that (old_n, old_value) in its reply

Phase 2 – Accept / Accepted

If the proposer receives promises from a quorum:

• If any promise included a prior accepted value, the proposer must use the highest one – it cannot choose freely
• Otherwise it can propose any value it wants

It sends ACCEPT(n, value) to the quorum. Acceptors that haven’t made a conflicting promise write the value and reply ACCEPTED. Once a quorum accepts, the value is chosen.

Proposer       Acceptor A    Acceptor B    Acceptor C
   |-- PREPARE(5) -->|             |             |
   |-- PREPARE(5) ---|------------>|             |
   |<-- PROMISE(5) --|             |             |
   |<-- PROMISE(5) --|-------------|             |
   |-- ACCEPT(5,v) ->|             |             |
   |-- ACCEPT(5,v) --|------------>|             |
   |<-- ACCEPTED ----|             |             |
   |<-- ACCEPTED ----|-------------|             |
      [value v is now chosen]

Why It’s Correct

The safety guarantee – that no two values are ever chosen – comes from the quorum overlap. Any two majorities in an N-node cluster share at least one node. That shared node is the bridge: if a value was accepted in round n, a new proposer in round n+1 will hear about it from the shared node and be forced to continue it rather than propose something different.

The constraint that proposers must adopt the highest prior accepted value is what makes this work. A new round can never silently override an old one.

The Gotchas

Livelock. Two competing proposers can keep preempting each other with higher proposal numbers indefinitely – neither ever reaching phase 2. The fix is to elect a single distinguished proposer (a leader) and have everyone else defer to it. This is what Multi-Paxos does.

Single-value vs. Multi-Paxos. Basic Paxos agrees on one value. Real systems need a log of values – a sequence of commands. Multi-Paxos runs Paxos repeatedly for each log slot with a stable leader, amortizing the phase 1 cost across many entries. Once a leader is established, phase 1 only needs to run once per leader lifetime.

Implementation gap. The original paper was intentionally abstract. “Paxos Made Simple” is cleaner but still leaves enormous implementation latitude – enough that teams have shipped subtly broken versions. The gap between the protocol description and a correct, production-safe implementation is one of the main reasons Raft was designed as an explicit replacement.

Where It Runs

• Google Chubby – distributed lock service, underpins Bigtable and GFS coordination
• Apache ZooKeeper – uses Zab (ZooKeeper Atomic Broadcast), a Paxos variant
• Google Spanner – Paxos per shard for replication across datacenters

Paxos is the theory. If you’re building something new, you’ll probably implement Raft instead – same guarantees, actually debuggable. But the mental model of quorum-based two-phase consensus is worth having regardless of which one you reach for.

Understanding Paxos is worth doing. It’s not just historical – Chubby, ZooKeeper, and Spanner are built on it or close variants, and many distributed systems interviews probe it. More importantly, every simplification Raft makes is a direct response to a specific pain point in Paxos. Understanding what Paxos gets wrong explains why Raft looks the way it does.

The Problem

Multiple nodes need to agree on a single value. The network is unreliable: messages can be dropped, reordered, or delayed arbitrarily. Nodes can crash at any time. There is no shared clock.

A minority of nodes can fail; the protocol must still make progress. The protocol must also be safe: once a value is chosen, no future round can choose a different value.

This is the consensus problem. Two-Phase Commit solves it only when the coordinator doesn’t fail. Paxos solves it even when any minority of nodes can fail, including the “coordinator.”

Roles

Any node can play any role (often all three simultaneously in practice):

• Proposer – initiates a round, trying to get a value agreed upon
• Acceptor – votes on proposals; the safety machinery lives here
• Learner – learns what value was finally chosen, acts on it

The Protocol – Two Phases

Phase 1: Prepare / Promise

1. The proposer picks a proposal number n. It must be unique across all proposers and higher than any proposal number the proposer has seen before. (Usually implemented with (timestamp, node_id) tuples or a monotonic counter stored durably.)

2. Sends PREPARE(n) to a quorum of acceptors (any majority).

3. Each acceptor that receives PREPARE(n):

   • If n is higher than any PREPARE it has responded to: replies PROMISE(n), committing to reject any future PREPARE or ACCEPT with a lower proposal number
   • If n is not higher: ignores or rejects the request
   • If it has already accepted a value in a prior round, it includes that (accepted_n, accepted_value) in the promise

Phase 2: Accept / Accepted

1. If the proposer receives PROMISE from a quorum:
   • If any promise included an (accepted_n, accepted_value), the proposer must use the value with the highest accepted_n – it cannot freely choose its own value
   • If no promise included an accepted value, the proposer can propose any value it wants
2. Sends ACCEPT(n, value) to the quorum
3. Each acceptor that receives ACCEPT(n, value):
   • If it hasn’t promised to ignore n: writes the value and replies ACCEPTED(n, value)
   • If it has made a promise with a higher number: ignores or rejects
4. Once a quorum of acceptors reply ACCEPTED with the same n → the value is chosen

Proposer          Acceptor A    Acceptor B    Acceptor C
   |-- PREPARE(5) -->|             |             |
   |-- PREPARE(5) ---|------------>|             |
   |<-- PROMISE(5) --|             |             |
   |<-- PROMISE(5) --|-------------|             |
   |-- ACCEPT(5,v) ->|             |             |
   |-- ACCEPT(5,v) --|------------>|             |
   |<-- ACCEPTED ---|              |             |
   |<-- ACCEPTED ---|--------------|             |
      [value v is chosen, Acceptor C was unreachable but quorum achieved]

Why It’s Correct

Safety (no two values can be chosen): Suppose value v was chosen at proposal n – meaning a quorum Q1 accepted it. Any later proposer n' > n that runs Phase 1 will receive promises from a quorum Q2. Q1 and Q2 must overlap (both are majorities). The overlapping acceptor already accepted v and includes (n, v) in its promise. The Phase 2 rule forces the new proposer to use v (since it’s the highest accepted value). So no different value can ever be chosen.

Liveness (the protocol makes progress): As long as a majority of nodes are reachable and a single stable proposer drives the protocol, a value will eventually be chosen. The protocol doesn’t guarantee progress if proposers keep competing – see the livelock problem below.

The Gotchas

Livelock with Competing Proposers

Two proposers P1 and P2 can keep preempting each other indefinitely:

P1 sends PREPARE(5), gets promises from quorum
P2 sends PREPARE(6) before P1 can reach ACCEPT -- quorum promises ignore n=5
P1's ACCEPT(5, v) is rejected -- quorum promised to ignore anything < 6
P1 increments to PREPARE(7), gets new promises
P2's ACCEPT(6, v) is now rejected
... loops forever

The fix is a distinguished proposer (leader): only one node proposes at a time. This is what Multi-Paxos formalizes – elect a stable leader, suppress all other proposers, and the livelock disappears. It’s not part of basic Paxos; it’s an add-on that every production implementation requires.

Single-Value vs. Log: Multi-Paxos

Basic Paxos agrees on one value. Real systems need to agree on a sequence of commands – a log. Multi-Paxos runs Paxos for each log slot, but with an optimization: once a stable leader exists, you can skip Phase 1 for subsequent slots. You pay the two-round-trip cost once per leader election, then only Phase 2 for every log entry.

The problem: Multi-Paxos is not a formal protocol. It’s described as an optimization on basic Paxos, and different implementations fill in the details differently. Log gaps are allowed – a slot can be accepted out of order. Cluster membership changes are not specified. Leader election is not specified. This leaves a large surface area for subtle bugs.

It’s Hard to Implement Correctly

“Paxos Made Simple” describes the algorithm clearly. Getting from there to a correct implementation in a production system requires handling: durability of acceptor state across crashes (promise/accept must be persisted before responding), leader election, log gap handling, log compaction, cluster membership changes, client request routing, and linearizable reads. None of these are in the paper.

Chubby’s authors wrote that their Paxos implementation took years to get right and required multiple rewrites. This is why Raft was created.

Paxos vs. Raft

The safety properties are equivalent. The differences are implementation surface:

Same fault tolerance: both tolerate ⌊(N-1)/2⌋ node failures.

Where Paxos Is Still Used

• Google Chubby – distributed lock service. Underpins Bigtable and GFS for leader election and metadata coordination.
• Apache ZooKeeper – uses Zab (ZooKeeper Atomic Broadcast), which is a Paxos variant with primary-order delivery guarantees added. Powers Kafka, HBase, and Hadoop ecosystem coordination.
• Google Spanner – Paxos per shard for cross-replica replication. Combined with TrueTime for external consistency.

New systems almost universally choose Raft. The guarantees are equivalent and the implementation surface is much smaller. Paxos matters because it’s what everything was built on, and because understanding what it leaves unspecified is the clearest way to understand why the distributed systems field spent a decade building on top of it before someone sat down and wrote a protocol that was complete enough to implement from scratch.