An amount in øre was displayed as kroner. A hundred times too much, nicely formatted, green checkmarks all the way through. The bug was caught by a reviewer who had never seen the code before — which is exactly why it was found.
The reviewer wasn't a colleague but a fresh AI model with no stake in the work. It's standing procedure in my workshop: nothing irreversible — no deploys, no database migrations, no deliverables of consequence — until a model with a completely blank memory has tried to shoot the work down.
Nobody grades their own exam
The reason isn't that the models are sloppy. It's that a model that has spent an hour building something is invested in it working — much like people. It hunts for confirmation. Ask a fresh model to refute a piece of work instead, and it hunts for holes. Same technology, opposite sign.
In a single month, that kind of fresh review caught 29 documented defects before anything went live. The øre-kroner blunder. A bug that let one user wipe another user's data — stopped before the app reached the App Store. And a string of smaller cases the building model itself had waved through.
This website is no exception. When its latest major rebuild was "done," the fresh review confirmed ten of twelve claims — and refuted two. Both refutations were real bugs that had slipped past both me and the model that built it.
29 documented defects caught in one month — one of them off by a factor of 100.
What the gate looks like
The recipe is deliberately inhospitable. The fresh model gets the code, the claims, and the means to build and run everything itself — but not the conversation history, and not the building model's explanations. The instruction is one word heavier than usual: refute. Confirmation is cheap. A refutation with receipts is worth gold.
The gate sits in one place: right before the irreversible. Everything before it is allowed to be fast and messy — that's where the speed comes from.
If you're buying AI-built work — or human-built, for that matter — the question is not whether the supplier declares it "done." They all do. The question is who has tried to prove the opposite. And what they found.