The first night my AI overseer was allowed to work alone, every safety gate did its job. It just didn't get anything done. The whole night's budget — two tasks — went to cleaning up the system's own test debris left over from the dress rehearsal.
It sounds like a failure. It was the opposite: proof that the limits held, and that this kind of blunder gets discovered over morning coffee instead of growing in the dark.
Trust in instalments
The system — a nightly overseer that picks up where the day's work left off — didn't get its independence in one go. First it ran dry: every decision made, none executed. Then its proposals were graded by a human for a while. Only after that was it allowed to execute anything itself, with a cap of two tasks per night and a full log for review the next morning. The cap goes up once several consecutive nights run clean — and comes back down by changing one number in a file.
Before the first real night, the plan had also survived two rounds of pointed pushback from a second AI model whose entire job was to find holes in it. That produced nine concrete tightenings. On top came a dress rehearsal with valid, stale, and outright pointless work mixed into the queue.
And then came the night where it all worked — except the leftovers from the dress rehearsal were still sitting in the queue. The system handled the debris correctly, spent its budget on it, and never touched the real work. The cleanup and one knock-on defect were fixed before the next night.
Night 1: 2 of 2 tasks spent on test debris · 0 real tasks touched · every gate held
Nothing ships on a robot's nod
The base contract is the same in everything I build: the machines are welcome to prepare — investigate, draft, propose, queue up. But nothing gets deployed, published, or deleted without a human sign-off. Autonomy is not a switch you flip. It is trust earned in instalments, and it can be rolled back.
For a business weighing up automation, this is probably the most usable model there is: start with proposals, not actions. Set a ridiculously low cap. Read the log in the morning. And only raise the cap the day the review has become boring — because that boredom is the proof.
That first night, my robot accomplished nothing of value. It was the most reassuring thing it could have done.