Shipping with coding agents without vibe coding: the gates in front of every merge

Fast was never the hard part

We are a small team building an AI travel planner for people who plan trips together. Agents let us build at a pace a team our size never could. The risk is the one the field keeps naming this year: vibe coding, accepting what the model writes because it runs.

An agent writes fluent code and fluent explanations of that code. Both sound right. Neither is evidence. So the question I care about is not how good the model is, but what has to be true before its work reaches a traveller, and what checks that for us.

For News, the answer was four gates. None of them is new. What is new is how much they carry once code arrives this fast.

Every merge, in order

Each box can stop the work. Only one of them is a person saying go to main, and everything before it exists so that the go is about the product, not about whether the code is right.

Gate 1: tests we have seen fail

A test passing tells you nothing until you have watched it fail. Agents are very good at writing tests that pass.

The same week, we rebuilt how our backend reads a traveller's trip list, to make it about ten times faster. The proof was a deep-equal test: the old read and the new one answer the same queries, and the output must be identical. An independent review found three gaps in it. The test data could not tell certain mistakes apart, so a wrong answer would have matched the right one.

So before we trusted it, we broke the code on purpose, one thing at a time: a cache key that ignores the city, invites attached to the wrong trip, journals in the wrong order, a lighter read that skips places, one that drops their images, a field that should have sent the request down the slow path but did not, and a check that does not look inside nested fields. Seven breaks, and each one turned a test red. Only then did the test count.

Gate 2: a validator that knows what is true

Every News story is a plain text file, and a validator reads it before anything is built. It is strict in ways a reviewer gets tired of being: no dashes, British spelling, no internal file paths in anything a reader sees, the right title length for search.

The rule that matters most is the truth gate. A story about a feature names the documents it rests on, and the validator refuses the story unless each one says status: shipped. An agent can write a convincing paragraph about a feature that is still on a branch. It cannot make that document say shipped.

The validator also decides what a failure means. In the hourly sync, a story with an error is held: its live version stays up, and the rest of the site carries on.

Gate 3: a review by someone who did not write it

Every slice was reviewed before it joined the rest, by a reviewer that had not written it. That separation is the point: a reviewer who has followed the author's reasoning tends to agree with it.

The review of the whole News build came back with five findings, and two of them overruled decisions the author had made and written down:

• A broken story would have stopped everything. In the first design, a validation error in one story stopped the whole hourly content sync, which also carries our city guides, places and articles. After the review, News runs in steps of its own and can stop nothing but News.
• A story could have gone live without its cover. After the review, a story is written only once all four sizes of its cover are uploaded.

Both were reasonable choices on the day. Both would have hurt in production. Neither would have been caught by a test, because the tests were written by the same author against the same design.

Gate 4: one branch, one sweep

All six slices merged into one integration branch, never into main. That branch took main in by merge and was never rebased, because other work was stacked on it. One preview of the whole thing got one QA sweep, and only then did one person give one go to main.

It costs something. A pull request of 55 commits is harder to read than six small ones, and the sweep waits for the slowest slice. It buys one picture of the whole system, tested as travellers will meet it, instead of six pictures that are each correct on their own.

The bug that got through anyway

News merged at 13:10. At 13:51 the hourly content sync failed at its very first step, before it synced anything. Every run after it would have failed the same way.

The cause was one line, a check that the optional slug of a single story to sync contains only safe characters:

An empty string

printf %s "$NEWS_SLUG" | grep -Eq '^[a-z0-9-]*$' looks as if it allows an empty slug, since the pattern matches an empty line. But given an empty string, grep reads no line at all, so it reports no match and the check fails. Every preview run had passed a slug. A scheduled run never sets one.

The fix tests the string itself, [[ "$NEWS_SLUG" =~ ^[a-z0-9-]*$ ]], and was merged at 14:47. Our alerting had flagged the failure, one hourly sync was lost, nothing was written and nothing was damaged. The first sync after the fix published News at 15:27.

The review had made sure News could stop nothing but News. Then a new check went into the shared first step of the sync, before any of News's own steps, and stopped everything. Our gates tested what the agents had run. Nothing tested what the schedule would run.

What I would do differently

Two things. The first test of any input check should be the empty input, before the happy path. And a change to a scheduled job should be run once in the exact shape the schedule uses, with no inputs at all, before it merges. Neither needs a smarter model. Both are checks.

The takeaway

You can ship agent code only as fast as your checks can vouch for it. Spend your time on the checks: tests you have watched fail, validators that know what is true, a reviewer who did not write the code, and one sweep of the whole thing before it ships.