A Python bug I fixed on 9 July caught a TypeScript bug on 26 July

A Python bug I fixed on 9 July caught a TypeScript bug on 26 July. Different project, different language, 17 days apart.

The mechanism is the interesting part, because it is boring on purpose.

The chain, in order

On 9 July a data pipeline of mine failed in a familiar way: a database write that could do nothing and still report success. I fixed it. Then I did the thing that turned out to matter more than the fix: I wrote the failure down as one question in the checklist my coding agents have to answer before anything merges.

On 26 July, in a different project, an agent finished a piece of work and a second agent verified it. The verifier reads only the original locked spec, never the builder’s summary of it. It works through the checklist. The item seeded by my July bug asked its question and found a database write using “do nothing on conflict” with no check of whether anything happened, while every neighbouring function in the same file throws on failure. My own spec’s acceptance criteria did not cover it. Fixed the same day.

Why I trust the result

Two details, both about restraint rather than detection.

The verifier downgraded two other checklist hits on its own, because the plan had already documented them as deliberate trade-offs. A reviewer that pads its list is a reviewer you stop reading after a week. Calibration in both directions is the only kind worth having.

And hedge language counts as unverified. “Should work” is not a pass. Two findings came back as pass-with-residual rather than a soft pass, because the evidence for them was thinner than the claim.

The honest part

The same day, in my own setup, I logged seven defects: wrong timestamps written by the agent itself, log lines that were not valid JSON because of unescaped Windows paths, a merge collision between a spec file written to the main tree and the same file committed on a branch, and a few more in that vein.

All seven sat in the seams between my harness and git. Zero were in the code the agent wrote. My scaffolding is buggier than its output, and that is worth saying before anyone asks whether a verify step this new can be trusted.

Scale, stated plainly

One checklist item, firing once, across two of my own projects. Not a study. What it shows is a mechanism, not a rate: a written failure travels across a language boundary that a test suite never crosses, because a test only checks what you thought to write, and a checklist item asks a question regardless of what language answers it.

What actually changed

Tests verify what you already thought of. A checklist remembers what burned you, and keeps asking after you have forgotten why.

So the question worth asking about any agentic setup is not which model writes the best code. It is where your failures go after you fix them. Mine go into a list the next agent has to answer, in whatever language it happens to be writing.

Source post: Post | LinkedIn