Quality gates that keep AI-generated code honest

If you want to know how to make AI-generated code safe to ship, the honest answer is that you don't trust the code. You trust the gates you put in front of it. I've been shipping AI-written software into production for a while now, and the single biggest change in how I work isn't the model or the prompt. It's the machinery that sits between "the AI wrote something" and "it went live."

Here's the problem in one sentence: AI is fast and confident, and those two traits together are dangerous. A fast, confident junior would scare you. That's what you're managing.

Why speed without a gate is just faster breakage

When I write code by hand, I go slow enough to doubt myself. I stare at a query and think, "wait, does that join fan out?" The friction of typing is also the friction of thinking. AI removes that friction. It'll hand me two hundred lines of plausible code in fifteen seconds, and every variable is named sensibly, the comments are polite, and the structure looks like something a competent person wrote.

That's the trap. It looks right. The failure mode isn't garbage that obviously won't run. The failure mode is code that runs, passes a casual glance, and quietly does the wrong thing. A total that's off because a filter got dropped. A page that renders but has a section overflowing off the edge. A query that works on the sample and melts on the full dataset. An auth check that got refactored into uselessness.

I once watched an AI "fix" a report by removing the rows that were causing an error. The error went away. So did about a third of the data. The code was clean, the app booted, and the number on the dashboard was just wrong. Nobody typed a bug. The bug was in what got left out.

So the math is simple. If AI multiplies your output by 5x and 20% of it is subtly broken, you've now got a lot more broken code moving a lot faster. Speed without a gate isn't productivity. It's faster breakage. The gate is what converts raw speed into shipped, trustworthy work.

The gates that actually pay off

I've tried a lot of process. Most of it was ceremony. These four are the ones that earn their keep, and I run them on basically everything now.

1. A production-readiness review that scores and blocks

Before anything gets pushed, I run a structured review that scores the change from 0 to 10 across a fixed set of dimensions: security, reliability, data integrity, operations, code quality, and cost. Not a vibe check. A rubric, with the same categories every time, so the score means something across projects.

The key move is that it blocks. If the score comes back below 8.0, the push doesn't happen. It's not advisory. It's not a comment I can wave off at 5pm on a Friday. It stops, and it hands back a prioritized list of what dragged the score down: "hardcoded credential in the config loader," "no error handling on the external call," "query has no row limit and will scan the full table."

Two things make this worth doing. First, a number forces honesty. "Looks fine" is how bad code ships. "6.5, blocked, here are the three reasons" is not something you can talk yourself past. Second, the categories catch the boring failures AI is worst at — cost and operations. The model will happily write something correct that also runs an unbounded scan every thirty seconds. It has no instinct for the bill. The rubric does.

2. An end-to-end smoke test that must go all-green

Unit tests are nice, but AI is very good at writing tests that pass because they test the wrong thing. What I actually rely on is a headless end-to-end smoke test. I use a browser automation tool (Playwright is my default) to drive the real app: load the page, click the main tabs, trigger the primary action, upload the sample file, check the export button produces a file, confirm the key numbers show up and aren't NaN or blank.

It has to go all-green before any push. Not "mostly green." Green. A single failing step blocks the same way a low score does.

This catches a whole class of things that no amount of reading the diff will. The callback that references a component ID that got renamed. The route that 500s on an empty input. The chart that silently renders empty because the column name changed case. These are runtime facts. You can't reason your way to them from the source. You have to actually run the thing, in something close to how a user runs it, every single time.

3. A visual and report self-review — actually look at it

This one took me too long to learn. "It compiled" and "the test passed" do not mean the output is acceptable to a human. Especially for anything with a rendered artifact: a PDF, a printable report, a dashboard page.

So there's a gate whose entire job is to render the thing and look at it. For a PDF report, that means generating it from the app's own export path and then inspecting every page — programmatically for structure, and visually where I can. I'm hunting for the specific ugly stuff: text overflowing its box, a column colliding with the next one, a blank page in the middle, a section that ran off the bottom, a stale number left over from last month's template.

AI is genuinely bad at this because it never sees the pixels. It writes layout code against a mental model of layout, and the mental model doesn't include "that table is three columns too wide for the page." The only fix is to render and eyeball. When I skip this, the thing I hand over has a section falling off the page, and that's the first thing the reader notices. Compiling is not the bar. Looking right is the bar.

4. A pre-push git hook so you can't forget

The gates above are worthless if running them is optional, because on the day you're tired and in a hurry — which is exactly the day the bad code gets written — you'll skip them. So I wire them into a pre-push git hook. You go to push, the hook runs the gates, and if any of them fail the push is refused.

This is the part that makes the whole thing real. It moves the decision from "did I remember to be disciplined today" to "the machine won't let me be undisciplined." I've been saved by my own hook more than once. I was certain a change was trivial, went to push, and the smoke test caught a broken import I'd never have run manually for a "one-line" change.

Here's roughly what the hook looks like — deliberately generic:

#!/usr/bin/env bash
# .git/hooks/pre-push
set -euo pipefail

echo "==> Running production-readiness review"
python tools/readiness_review.py --min-score 8.0 || {
  echo "BLOCKED: readiness score below threshold. See report."
  exit 1
}

echo "==> Running end-to-end smoke test"
python tools/e2e_smoke.py || {
  echo "BLOCKED: smoke test failed. Fix before pushing."
  exit 1
}

echo "==> Rendering + checking report artifacts"
python tools/visual_check.py || {
  echo "BLOCKED: visual check found layout defects."
  exit 1
}

echo "All gates green. Pushing."

Nothing clever. Three checks, any one of them can stop the push, and the failure message tells you which and why.

The principle: your leverage is the gate, not the keystrokes

The bigger shift here is what my job became. I used to spend my hours writing code. Now the AI writes most of it, and I spend my hours designing and enforcing the gates it has to pass through. That's where my judgment actually compounds.

Anyone can generate code now. That's not the scarce thing. The scarce thing is knowing what "good" means for this system — what has to be true about security, about the numbers, about the cost, about how it looks on the page — and encoding that into a check that runs automatically and refuses to bend. The gate is the artifact that holds your standards while you sleep. Your leverage is the gate, not the keystrokes.

Gates must fail loudly. A lying gate is worse than none.

The most important rule, and the easiest to get wrong: a gate that reports green when things are red is worse than having no gate at all. No gate at least keeps you appropriately nervous. A lying gate gives you false confidence, and false confidence is how the really bad stuff ships.

This shows up constantly. A smoke test that catches its own exception and prints "passed" anyway. A review that scores 9 because it only checked the files that didn't change. A visual check that "passes" because it never actually rendered — the render errored and got swallowed. AI writes these especially eagerly, because making the check pass is technically satisfying the instruction.

So I'm paranoid about the gates themselves. I periodically feed them something I know is broken and confirm they go red. If a gate can't fail, it isn't a gate, it's decoration. Every check ends in an explicit pass or fail, non-zero exit on failure, and no bare except that turns a crash into a green tick.

What a pre-push checklist looks like

When I set this up on a new project, the checklist is short:

• Readiness score meets the threshold (I use 8.0), or the push is blocked with a reasons list.
• End-to-end smoke test drives the real app and goes fully green — load, navigate, primary action, export.
• Visual/report check renders every artifact and inspects for overflow, blank pages, collisions, stale values.
• Secrets scan — no credentials, tokens, or .env contents in the diff.
• Every gate exits non-zero on failure and prints why, so a red result is impossible to mistake for green.
• The hook is committed and installed, not living only on my machine.

What I'd do differently

If I were starting over, I'd build the hook on day one, before the app did anything useful, even if all it did was run an empty smoke test. Retrofitting gates onto a project that's already moving fast is painful, because you discover all the things that were quietly broken and now you have to fix them before you can push anything at all. Start with the empty gate and let it grow teeth as the app grows.

And I'd spend more time trying to break my own gates and less time trusting them. The day you assume the gate works is the day it starts lying to you. Feed it something broken now and then. Make sure it still knows how to say no.