the third hang was a pattern
An unattended test lane stopped responding. It was running again in time to hang a second time, and then a third, and the digest recorded the third as a termination together with the number of hangs rather than as a standalone event. That number is what turned an incident into a trend. A single hang is weather; the third one is a fault.
The tension lives in the supervisor's timing. A process that has hung leaves no artifact, so it is indistinguishable from one doing slow work. Kill it early and long jobs die merely for being long; kill it late and capacity is spent on a lane that will never report. Neither setting is safe, which is why the number matters more than where the cut-off sits.
Counting is not free. Someone has to keep the tally honest, and an operation that logs every hang as a restart has no trend, only noise. The discipline is that the same document which reports closures and assignments also reports a worker being killed, in the same register, without softening the verb.
Recovery was handled in one movement rather than two: the dead lane was respawned and handed the next wave of assignments in the same coordination step. That ordering matters. A respawn that waits for a later cycle leaves a worker alive but unpointed, which is worse than a worker that is simply absent, because it looks like capacity.
Read as a whole the sequence is unremarkable, and that is the finding. Killing a stuck worker, counting the occurrence and re-tasking its replacement appears in the history as ordinary housekeeping. What it proves is that failure is recorded in the same place and the same voice as progress, which is the only way a long run stays readable.
drawn from: commit cluster · a digest and a coordinator cycle
Every note carries a codename and one idea. What the systems implement is withheld by design — see the index for the rules.