Find the Path With a Person Waiting on It

On the Friday of a nine-day fair, at 17:33, the identity service went down mid-event. Nobody could sign in.

Every door kept scanning.

That containment was design — though I'll say again what I said when I wrote up the event: the timing was also lucky. What the design got right is narrow and worth stating on its own, because it generalises further than access control does.

A badge scan doesn't ask "who is this reader?" The reader is already holding a signed token it was given at the start of the shift. The scan path reads a credential, checks a signature, writes a row. It never calls the thing that hands out tokens, so when that thing disappeared, the scan path didn't notice.

That's the whole trick, and the figure above shows it as an absence: no arrow crosses the line.

The question worth asking

Every system has one path where a human being is physically standing there, waiting on it. Not a browser tab someone will come back to — a person, in a queue, in a body, aware of how long this is taking.

Find that path. Then go through everything it touches and ask of each one: is this required, or merely convenient?

The two feel identical while everything is up. They are completely different at 17:33 on a Friday. Convenient dependencies are the ones you'll never notice until the day they're the reason a line stops moving — and they get added casually, because adding one call to a service that's always up costs nothing measurable in the moment.

It's the same shape everywhere once you look for it. Checkout doesn't need the recommendation engine. Scanning a boarding pass doesn't need the loyalty lookup. Unlocking a door doesn't need the reporting database. In each case someone put the dependency there for a good reason, and in each case it quietly extended the list of things that can stop a person from getting through.

What to actually do with it

You don't need a rearchitecture. You need a list.

Write down what has to be true for that one path to work — every service, every network hop, every credential. It's usually shorter than people expect, and about a third of it turns out to be convenience wearing the costume of a requirement.

Then delete those from the path. Cache the token. Read the stale copy. Write the event and reconcile later. Accept a degraded answer instead of waiting for a perfect one. Every one of those is a small compromise, and every one of them buys you a failure that a person standing in a queue never experiences.

Nobody at that fair knew the identity service was down. That's the goal — not that nothing breaks, but that what breaks isn't in front of anyone.