builds / raft / stage-5SHEET 5 / 6 · REV ASIGN IN
ASSEMBLY DIAGRAM — YOUR RAFT CLUSTERSCALE: LEARNING
clientthe outside worldTIMING CRYSTAL✓ builtELECTION CIRCUIT✓ builtREPLICATION BELT✓ builtCOMMIT INTERLOCK✓ builtBLACK BOX⚙ buildingSTORM ENCLOSUREstage 6
BUILTUNDER CONSTRUCTIONNOT YET IMAGINED INTO EXISTENCE
STAGE 5 · THE BLACK BOX RECORDER

The Black Box Recorder

Persist before you act, or a rebooted node will cheerfully break a promise it no longer remembers making.

What you're wiring up

Everything so far dies with the process. Three things must survive a crash for Raft's guarantees to mean anything: currentTerm, votedFor, and the log. Each one guards a specific promise. Forget votedFor and a rebooted node votes twice in the same term, giving you two leaders. Forget the term and time runs backwards. Forget the log and committed was a lie.

The rule is brutal and short: persist before you act on or acknowledge anything. Not after the reply, not on a timer — before the vote is counted, before the AppendEntries success goes back on the wire.

Then the harness starts pulling the crash lever, killing nodes mid-protocol and rebooting them out of their black box, including the cruel case where a node persists and then dies before its reply is sent. Notably absent from the box: commitIndex and lastApplied. A rebooted node is supposed to rediscover commitment from the next leader's heartbeat.

Assembly steps

[ 01 ]
Implement persist() and readPersist() against the harness's Persister, encoding term, votedFor, and the log with encoding/gob.
hint

The Persister is a crash-aware in-memory disk that gets swapped under you on reboot. Never write real files; the harness owns the disk contents so it can simulate crashes.

[ 02 ]
Audit every mutation site: anywhere the three persisted fields change, persist before releasing the lock and before the matching reply or vote count proceeds.
hint

Grep for every assignment to those three fields and treat the persist call as part of the assignment. The random-crash test exists to find the one site you missed.

[ 03 ]
Wire reboot: on construction, restore whatever the Persister holds and come up as a follower with the clock already running.
hint

Do not persist commitIndex or lastApplied. Rediscovering commitment from the next leader's leaderCommit is the intended path, and leaning on saved values hides bugs.

[ 04 ]
Re-run the Sheet 2 through 4 suites with persistence on and fix the fallout.
hint

What usually breaks is speed, not correctness — persisting on every heartbeat reply. Write only when one of the three fields actually changed.

Go deeper (after it passes)

Crash consistency is the whole subject of DDIA's Everything Fails chapter, and persist-before-acknowledge is the same contract a write-ahead log makes in the Redis track's flight recorder. If you built that stage, the symmetry is worth ten minutes of staring at both.