builds / redis / stage-6SHEET 6 / 6 · REV ASIGN IN
ASSEMBLY DIAGRAM — YOUR REDISSCALE: LEARNING
clientthe outside worldPORT JACK✓ builtRESP CODEC✓ builtMEMORY BANK✓ builtINTERLOCK✓ builtCLOCK UNIT✓ builtFLIGHT RECORDER⚙ building
BUILTUNDER CONSTRUCTIONNOT YET IMAGINED INTO EXISTENCE
STAGE 6 · THE FLIGHT RECORDER

The Flight Recorder

Before applying any write, append it to a file. At startup, replay the file.

What you're wiring up

Everything so far dies with the process. The fix is almost suspiciously simple, and it's the idea DDIA's storage chapter opens with: append every write to a log before replying; replay the log at boot. Redis calls this the AOF — and the lovely trick is that the log's format is RESP itself. The file is literally the SET/DEL commands as they arrived, so your Stage 2 parser reads your own log for free.

You'll confront the real durability dial (fsync always vs every-second vs never) and prove the recorder works the honest way: the harness kill -9s your server mid-traffic and checks what survives.

Assembly steps

[ 01 ]
Add --dir and --appendonly flags. On every successful write command, append its RESP bytes to appendonly.aof before replying.
hint

Re-serializing the parsed args is cleaner than keeping raw bytes around.

hint

One writer goroutine fed by a channel, or append under the store lock — either works; just never interleave two commands' bytes.

[ 02 ]
On startup, replay the AOF through your own ReadCommand and normal dispatch — with replies and the recorder disabled during replay.
hint

Reuse, don't re-implement: ReadCommand(bufio.NewReader(file)) until EOF. If replay writes to the AOF you've built a photocopier of doom.

[ 03 ]
Log an explicit DEL when a key is evicted (lazy or sweep) so replay doesn't resurrect dead keys.
hint

Simplest correct scheme: persist SETs without their TTL and log a DEL at eviction. Absolute-timestamp PEXPIREAT logging is the bonus rung.

[ 04 ]
Implement --appendfsync always|everysec|no, and survive a torn final entry: truncate the tail, don't crash the boot.
hint

Parse error at EOF during replay = torn tail: truncate and carry on. Parse error mid-file = corruption: refuse to start.

Go deeper (after it passes)

The blueprint is complete — solid ink. Victory lap: point a real redis-cli at port 6389. Then: build-your-own-database (from log to B-tree), build-your-own-raft (run three of these and make them agree).