STAGE 5 · THE INSTRUMENT PANEL

The Instrument Panel

You now have a machine you can't see into.

What you're wiring up

Operability means the balancer can answer what do you see right now without you grepping logs. Which backends are up, how many requests each has served, how many are in flight this second, how many probes have failed. Most of that data already exists inside your process from Stages 3 and 4 — this stage is mostly about giving it a door.

The one architectural rule: the admin surface lives on its own port, with its own mux, sharing zero handlers with the traffic port. Mix operator endpoints into the traffic port and one day /status stops being your dashboard and starts being a customer's proxied path — or worse, your internals get served to the internet.

The last piece closes a loop. Serve /healthz for the balancer itself: 200 while at least one backend is up, 503 otherwise. Now the thing that health-checks your backends is itself health-checkable by whatever sits in front of it. Turtles, but useful ones.

# GET :8081/status
{"strategy":"leastconn","uptime_seconds":412,
 "backends":[
  {"url":"http://127.0.0.1:9001","state":"up",
   "inflight":2,"requests_total":871,
   "failures_total":3,"checks_failed_total":0}
 ]}

Assembly steps

[ 01 ]

Start a second HTTP server on --admin 127.0.0.1:8081 with its own mux.

hint

Two http.Servers, two goroutines, one shared pool object. No handler may be registered on both.

[ 02 ]

Serve GET /status as JSON: strategy, uptime_seconds, and per backend the url, state, inflight, requests_total, failures_total and checks_failed_total.

hint

You already have inflight from Stage 4 and state from Stage 3 — most of this stage is wiring counters you nearly have.

[ 03 ]

Count requests_total per backend on completion and failures_total for 5xx-or-error outcomes in the proxy path.

hint

Increment on completion, not on dispatch, or the numbers won't reconcile with what clients actually got.

[ 04 ]

Serve GET /healthz on the admin port for the balancer itself: 200 if at least one backend is up, 503 otherwise.

Go deeper (after it passes)

Counters as first-class outputs is DDIA M1 (Measure Before You Build); the admin-versus-traffic split is Clean Code M6 (Taming the Edges: Boundaries). Try scraping /status on a loop and plotting it — that's a monitoring system in ten lines.