builds / load-balancer / stage-4SHEET 4 / 6 · REV ASIGN IN
ASSEMBLY DIAGRAM — YOUR LOAD BALANCERSCALE: LEARNING
browserthe outside worldFRONT DOOR✓ builtROTOR✓ builtPULSE MONITOR✓ builtSCALES⚙ buildingINSTRUMENT PANELstage 5RELIEF VALVEstage 6
BUILTUNDER CONSTRUCTIONNOT YET IMAGINED INTO EXISTENCE
STAGE 4 · THE SCALES

The Scales

Round-robin is blind: it hands a struggling server the same share as a healthy one.

What you're wiring up

A backend that's alive but slow is the nastiest failure mode, because health checks say it's fine. Round-robin keeps feeding it a third of your traffic and a third of your users wait. Least-connections fixes this without measuring latency at all: send each request to whoever is currently doing the least work. Count requests in flight per backend, pick the minimum, done. A slow backend accumulates in-flight requests and naturally stops being chosen.

The bookkeeping is where this stage bites. Increment when you dispatch, decrement when the response finishes — on every path, including errors, timeouts and panics. That's what defer is for. A counter that leaks on the error path silently pins all your traffic to one backend forever.

There's a design lesson riding along. Round-robin and least-connections are two interchangeable answers to one question: who's next? Pull that question behind an interface and you've arrived at the Strategy pattern because the code demanded it, not because a book told you to.

Assembly steps

[ 01 ]
Extract a Picker interface with a Pick method and move round-robin behind it. Behavior must not change.
hint

This is a pure refactor — run the Stage 3 checks straight after as your own regression gate.

[ 02 ]
Track in-flight count per backend: increment on dispatch, decrement on completion inside a defer.
hint

atomic.AddInt64(&b.inflight, 1) with the matching defer right where the request is handed to the transport.

hint

If the increment and the defer live in different functions, you'll eventually leak one.

[ 03 ]
Implement leastconn: among up backends pick the lowest in-flight, breaking ties by lowest index.
hint

Deterministic tie-breaking is what makes the behavior testable — don't reach for randomness.

[ 04 ]
Add --strategy roundrobin|leastconn (default roundrobin), selecting the picker at boot and rejecting unknown values.

Go deeper (after it passes)

The Picker interface is HFDP M1 (Ducks That Swap Their Wings, Strategy) arrived at by need. The counter discipline under load is Clean Code M10 again.