the fast path has to reproduce the slow one

Rebuilding the whole record from scratch on every cycle is honest and expensive, and everything downstream waits on it. The alternative — extend the tail by reading only the new rows — has one property to recommend it and one large risk: any drift between the two paths is invisible, because both produce a file that looks exactly like a file.

So the incremental path was admitted against the full rebuild as an oracle, at every intermediate stage rather than only at the end. Each cut matched byte for byte, first on synthetic sequences built to be hostile and then on real data at several cut points. The final artifact had to be indistinguishable, not close.

None of that is a one-time event. The path keeps its own state, and that state is checked as it goes: any disagreement between the saved state, the file map and the watermark heals to a full rebuild rather than proceeding on a doubtful base. Separately, a shadow copy of the full rebuild runs continuously against the same inputs, and a mismatch flips the pipeline back to the slow lane while the discrepancy is inspected.

It is slower on paper, and cheaper in the only currency that matters here, which is being able to say what produced a number.

What it proves: an optimisation is admitted only when it can be shown equivalent to the thing it replaces, and the demonstration is kept running afterwards, not filed away.