How I run a gated build pipeline for AI-generated production apps

I build a lot of internal software with an AI coding agent. Dashboards, data-entry tools, scheduled jobs, report generators. The kind of thing a company used to hand to a contractor for six weeks and $40k, and now one person can stand up in an afternoon.

The afternoon part is real. What nobody tells you is that the afternoon build is usually wrong in ways you won't notice until it's in front of someone who matters. The chart looks great. The number is off by 14%. The auth works on your machine and fails for every other user. The PDF export has a blank page three. The app runs the most expensive model on every keystroke and you find out when the bill arrives.

So the interesting problem stopped being "can AI write the app." It obviously can. The problem became: how do I make AI-written software trustworthy without personally reading every line? My answer is a build pipeline with hard gates. The AI does the typing. The gates decide whether the work is allowed to move forward. The human — me — designs and enforces the gates.

This is the whole method. If you take one thing from this post: your leverage isn't the code the AI writes, it's the gate the code has to pass.

The shape of the pipeline

Five stages, two of them are human approval gates, one is a hard automated gate:

Discover data  →  [DATA GATE]  →  Mockup  →  [LOOK GATE]  →  Build  →  Test  →  [PROD GATE]  →  Deploy
   (agent)         (human)       (agent)     (human)        (agent)  (agent)   (automated)     (ship)

Each stage has a single owner and a single question it answers. You don't advance until the question is answered yes. That's it. The discipline is in refusing to skip.

Every expensive failure I've had came from skipping a gate because I was in a hurry. Every time I ran the full pipeline, the thing shipped correct the first time. The pipeline is slower per attempt and dramatically faster per working result, because you're not doing the build ten times.

Let me walk each stage.

Stage 1 — Discover the data (and prove it)

This is the stage everyone skips and it's the one that costs the most.

An AI agent will happily build you a beautiful dashboard on top of the wrong table. It has no way to know that the obvious-looking transactions table is actually a trading ledger that double-counts every movement, and that the real source is a different feed three joins away. It will pick the obvious table, write clean code against it, and hand you something that looks completely finished and is completely wrong.

So before any code gets written, the agent's job is to produce a data contract:

• Inventory the candidate tables/views for the goal.
• Profile each one: grain (what's one row?), freshness, row counts, null rates.
• Map the join keys and flag fan-out risk (the join that turns 1,000 rows into 40,000 and inflates every total).
• Actively hunt for a better source than the obvious one.
• Land on exact, validated queries and a column→meaning map.

The single most important artifact here is a reproducible anchor. Pick a number you already know is true from some trusted report — say, "last month's total volume was exactly 1,240,000 units" — and don't trust the query until it reproduces that number to the penny. If it comes back 1,410,000, you have a fan-out bug or a wrong filter, and you just caught it before writing a single line of app code instead of in a meeting with an executive.

-- Anchor check: does the "obvious" source reproduce the known-true monthly total?
SELECT COUNT(*)             AS raw_rows,
       COUNT(DISTINCT txn_id) AS unique_rows,     -- if these differ, you have dupes
       SUM(quantity)        AS total_qty          -- must equal the trusted 1,240,000
FROM   candidate_source
WHERE  period = '2026-06';
-- raw_rows 40,120 but unique_rows 12,004  ->  fan-out. Dedup by txn_id before trusting anything.

The gate: I read the data contract and confirm the anchor ties. Data wrong → stop. Nothing downstream can fix a wrong source; it just makes the wrongness prettier.

Stage 2 — Mockup, before any app code

The second most expensive loop is the "build it, then redo it ten times because it doesn't look right" loop. AI is great at producing a layout. It's not great at reading your mind about the layout.

So the next artifact is a standalone, high-fidelity mockup — a static HTML page plus a short design-system spec (colors, type, spacing, component patterns). No live data, no callbacks, no backend. Just the look and the layout, rendered well enough that I can point at it and say "that, but move the KPIs above the table."

The gate: I look at the mockup and approve the look before any real code exists. Changing a mockup is cheap. Changing a built app with wired-up callbacks is not. This gate alone probably saves me more total time than any other.

Stage 3 — Build

Now, and only now, the agent writes the actual application. Because the data is proven and the look is approved, this stage is mostly mechanical: wire the validated queries to the approved layout. The agent isn't guessing about sources or design anymore — the two things it's worst at have already been decided by a human.

A few standards get enforced here regardless of what the app does, because they're the difference between "runs on my laptop" and "runs in production":

• A real health endpoint the platform can hit to know the app is alive.
• An auth model that's correct for the environment — queries running as the logged-in user when that's what you need, versus a service account for housekeeping, decided deliberately, not by accident.
• The server object exposed the way the production WSGI server expects it.
• Config and secrets out of the code. No credentials in the repo, ever.

These are boring and they're exactly the things a fast afternoon build forgets.

Stage 4 — Test

The agent writes and runs the tests, and this is where "AI is fast and confident" becomes a liability you have to actively manage. Fast and confident means it will produce plausible-but-wrong code at volume. Speed without a test is just faster breakage.

Two kinds of tests matter here:

1. Calculation / anchor regression. If the app computes something, prove it still reproduces the anchor number from Stage 1, and prove a fix to one number didn't silently move another. This is how you catch the "off by 14%" before anyone else does.
2. End-to-end smoke. Drive the actual running app with a headless browser. Does it load, do the tabs switch, does the export produce a file, does nothing throw? Green means green.

And if the app produces a document — a PDF, a report — render it and actually look at it. "It compiled" is not "it's correct." I've shipped PDFs where the code ran perfectly and page three was blank, or a column ran off the edge of the page. Now the pipeline renders the artifact and inspects every page before it's allowed to be called done.

Stage 5 — The production gate (the one that's automated and mean)

This is the gate that makes the whole thing work, because it doesn't rely on my discipline. It's a structured production-readiness review that scores the code 0–10 across six dimensions:

• Security — secrets, injection, authz, exposure.
• Reliability — error handling, timeouts, what happens when the data source is down.
• Data integrity — does it still tie to the anchor, are edge cases handled.
• Operations — health checks, logging, can you tell when it breaks.
• Code quality — is the next person going to be able to maintain this.
• Cost — is it going to run the biggest model on every request and surprise me.

Below an 8.0, the gate blocks the push and emits a prioritized, machine-actionable remediation list. The fix skills work that list, then it re-scores. It doesn't ask my permission to be strict.

I wire this into a pre-push git hook so it is physically impossible to forget. You can't push if the gate is red. That's the point — a gate you can skip when you're tired is not a gate.

# .git/hooks/pre-push  (conceptual)
run_production_review || { echo "BLOCKED: score below 8.0 — see remediation plan"; exit 1; }
run_e2e_smoke        || { echo "BLOCKED: smoke tests not green"; exit 1; }
echo "gates green — push allowed"

One rule that sounds obvious and isn't: a gate that lies is worse than no gate. If a smoke test marks red as green, it's not a safety net, it's a blindfold that feels like a safety net. Gates have to fail loudly and honestly. When a test fails, I want the ugly output in my face, not a green checkmark that papers over it.

Deploy discipline

Shipping follows the same "prove it in a safe place first" logic. Build on a dev branch, deploy to a dev environment that auto-updates, verify it there against real conditions, then open a pull request into main. main is branch-protected and requires a review to merge. Nobody — including me, including the AI — pushes straight to production. The protection isn't about distrust, it's about making the safe path the only path.

Why this is the FDE skill, not the coding skill

If you're a forward-deployed engineer, or hiring one, here's the thing worth noticing: none of the above is about writing code faster. The AI already writes code fast. The value I'm adding is judgment encoded as gates — knowing that data correctness is the real risk, that look-approval belongs before the build, that a production score below 8 shouldn't ship, that a lying test is a liability. That judgment is what turns "AI can generate an app" into "this specific app is safe to put in front of a customer on Monday."

Anyone can prompt an AI into a demo. The job is turning demos into things that don't break at 2am. The gates are how.

What I'd do differently / the takeaway

Early on I treated the gates as bureaucracy and skipped them when I was confident. I was wrong every time confidence and correctness diverged, which is exactly the case gates exist for. The lesson: build the gates first, before you're in a hurry, because the moment you need them is the moment you'll be tempted to skip them.

If you want to start Monday, you don't need my exact tooling. You need five questions you refuse to advance past:

1. Is the data provably correct? (Anchor number ties.)
2. Do we agree on the look? (Mockup approved.)
3. Does it actually run end to end? (Smoke green.)
4. Would I be embarrassed if a stranger read this code / saw this output? (Score ≥ 8, PDF eyeballed.)
5. Can I ship without touching production directly? (dev → PR → main.)

Wire even two of those into a hook you can't bypass and your AI-built software stops being a fast way to make confident mistakes and starts being a fast way to ship correct work.