the retention policy that stopped deleting and said nothing
A snapshot policy is a promise about the future. It is also configuration that runs unattended, which means its failure mode is silence: nothing errors, nothing alerts, and storage grows.
The generational policy here was inspected at day zero and looked healthy — the keep ladder was applied, the prune step was present, and nothing had been removed, which is exactly what a policy that has not yet accumulated anything should report. Two drift risks were then found by reading the run configuration rather than the output. The grouping key included a per-run hostname, so snapshots taken by different instances were never comparable and the prune step could match nothing once the hostname changed — from a date the record names, not from the moment anyone noticed. And a one-off baseline snapshot carried its own group tag, so it would never be old enough in the group it sat in.
The mitigation made the grouping stable across instances, added a nightly cleanup for the one-off tag, and confirmed both with dry runs against the real repository, including the case where the tag is already gone and the cleanup correctly does nothing.
The deployed job was still the pre-fix version when the fix was verified, and that gap is recorded as deploy-pending rather than folded into the pass.
What it proves: retention was treated as a correctness property with a visible failure date, and the distance between fixed and running was reported instead of assumed.
drawn from: commit cluster
Every note carries a codename and one idea. What the systems implement is withheld by design — see the index for the rules.