Prove the data before you write a line of app code

The most expensive failures I've shipped with AI-generated apps weren't bugs in the code. The code was fine. The charts rendered, the filters worked, the export button produced a clean PDF. The failure was that the number on the screen was wrong, and nobody caught it until an executive did.

That's the trap I want to talk about: AI will happily build a beautiful dashboard on top of the wrong table, and it will do it fast. You'll get a polished result in an afternoon. You just won't find out it's wrong until the person you least want to disappoint spots a total that's off by a factor you can't explain in the meeting.

The fix is boring and it works. Before you write a line of app code, you prove the data. You write a short data contract, you validate your queries against a number you already know is true, and only then do you let the build start. This post is how I run that gate.

Why data failures are the expensive ones

A UI bug is cheap. Someone clicks the thing, it breaks, you fix it in ten minutes, nobody's trust is damaged. A data bug is different. It ships silently, it looks authoritative, and it erodes trust in the whole tool the moment it's discovered. After that, every number you show gets a raised eyebrow, and you spend the next month re-earning credibility you didn't need to lose.

Here's the part that makes AI especially dangerous here. When you ask a coding assistant to "build me a dashboard of monthly volume by customer," it does not stop to ask whether the table it found is the right table. It finds a table with a volume column and a customer column and a date, and it builds. It's confident. The output looks correct. And "looks correct" is exactly the failure mode, because a wrong total and a right total render identically.

The assistant is genuinely useful for the mechanical parts of data work. It's terrible at the judgment part. Knowing which of four plausible tables is the one your finance team actually closes the books on is not something you can pattern-match from column names. That's the human's job, and the data contract is where the human does it.

The data contract, step by step

A data contract is a short document you produce before building. Not a schema, not documentation for its own sake. It's the record of you proving to yourself that you know where the numbers come from. Mine has five parts.

1. Inventory the candidate tables. For any real metric there is never one obvious source. There's the transaction feed, the ledger, a reporting rollup somebody built two years ago, and a "master" attributes table. List them all. Ask the assistant to help you find them — it's fast at scanning a catalog and surfacing everything with a matching column name. Then treat that list as candidates, not answers.

2. Profile each candidate. For every table, get the boring facts: grain (what does one row mean?), row count, date range and freshness, and null rates on the columns you care about. This is where AI earns its keep. It'll write you a dozen profiling queries in the time it takes to describe them. A table that claims to have daily data but hasn't had a new row in three months is disqualified before you build anything on it.

3. Map join keys and fan-out risk. Write down the key you'll join on and, crucially, whether that join is one-to-one or one-to-many. This is where totals go to die (more on that below). If a customer has many contracts and you join customer-to-contract and then sum a customer-level amount, you just multiplied your total by the average number of contracts per customer. Nobody meant to. It happens constantly.

4. Actively hunt for a better source than the obvious one. The obvious table is obvious because it's named after the thing you want. That doesn't make it right. I make it a rule to look for at least one alternative and write down why I rejected it or switched to it. The best source is often a reporting view a finance person already reconciles, not the raw feed the assistant found first.

5. Anchor it to a known-true number. This is the whole ballgame, so it gets its own section.

The anchor number

Pick a number you already know is correct, from outside the system you're building. Last month's total. A figure from a report someone already signed off on. A count you can verify by hand. Then make your query reproduce that number to the penny before you trust anything downstream.

Say you know, from a report finance already closed, that last month's total was exactly 1,240,000 units. Your query should return 1,240,000. Not 1,238,905. Not 1,510,000. Exactly 1,240,000.

If it's off by a rounding hair, you have a filter or a type problem. If it's off by 20%, you're missing or double-counting a category. If it's off by 4x, you almost certainly have a fan-out join or you're on a ledger table when you wanted transactions. Every one of those gaps is a real thing you need to understand now, in a query, not later, in a meeting.

The discipline here is that "close" doesn't count. A number that's 98% right is more dangerous than one that's obviously broken, because it survives the smell test and ships. Reproduce the anchor exactly, or keep digging until you understand every unit of the difference.

The traps that actually get you

A handful of data traps show up over and over. Once you've been burned by each you start smelling them early.

Fan-out from a bad join. Covered above. The tell is a total that's larger than it should be by a suspiciously clean multiple. Always check whether your join is one-to-many before you sum across it.

Duplicate rows. The same business event lands in the table more than once, sometimes from an upstream retry, sometimes because the "grain" isn't what you assumed. Raw SUM counts all of them. You need to dedup by a genuine business key, not by "all columns are equal," because two legitimately-different rows can look identical after you drop the columns you're ignoring.

Ledger vs. transaction tables. This one's brutal because both tables are correct — they just answer different questions. A ledger or accounting table can carry many booking lines per real-world event and produce totals many times larger than the transaction feed. If you point a volume dashboard at the ledger, your numbers explode and you won't know why until you realize you're summing accounting entries, not events.

IDs that don't bridge between systems. The customer ID in one system is not the customer ID in another, even when they're both called customer_id. Older records especially tend to have drifted-apart conventions, and the bridge table that's supposed to connect them is incomplete. Test the join coverage — what percentage of rows actually match — before you rely on it. A join that silently drops 15% of rows is a wrong total waiting to happen.

Imputed values masquerading as measured ones. Some "volume" figures aren't metered at all — they're derived, backed into from a dollar amount divided by an assumed rate, or filled from an estimate. They sit in the same column as the real measurements and look identical. If your anchor won't tie out and you've ruled out joins, ask whether some of these rows are computed rather than observed. That's a caveat your users need to know, not something to bury.

A tiny dedup-and-anchor check

Here's the shape of the query I run to prove an anchor while guarding against duplicates. Nothing fancy — dedup to the true business grain first, then sum, then compare to the number I already trust.

-- One row per real event, newest wins, keyed on the true business key
with deduped as (
  select *,
         row_number() over (
           partition by event_id            -- the business key, not "every column"
           order by loaded_at desc           -- keep the latest load of each event
         ) as rn
  from raw_events
  where event_date >= '2026-06-01'
    and event_date <  '2026-07-01'
    and category <> 'excluded_type'          -- match the report's scope exactly
)
select sum(units) as total_units             -- expect EXACTLY 1,240,000
from deduped
where rn = 1;

If total_units comes back as 1,240,000, the source, the scope filter, and the grain all agree with a number a human already signed off on. Now I trust it. If it doesn't, I don't build — I find out why first.

The deliverable

The data contract itself is short. One page is fine. Mine is a Markdown file that lives next to the app and contains:

• The validated queries — the exact SQL, not a paraphrase, so anyone can rerun it.
• A column-to-meaning map — what each field actually represents in business terms, because a column named amt could be six different things.
• Join keys and their cardinality — one-to-one or one-to-many, spelled out.
• Caveats — the imputed rows, the sites that don't bridge, the category you excluded and why.
• The anchor number — the known-true value and the query that reproduces it exactly.

That document is the thing that makes the build safe. When the assistant writes the app, it builds on proven queries instead of guessing at a table. When someone questions a number six months later, you have the receipt. And when you hand the app to the next person, the hard-won knowledge about where the bodies are buried travels with it instead of living in your head.

What I'd do differently, and what I do now

I learned this the way everyone does — by shipping a confident, wrong number and having to explain it. The lesson wasn't "check your work harder." It was that data correctness is a gate, not a step you do while building. If it's a step, it gets skipped under deadline pressure. If it's a gate, nothing gets built until the anchor ties out.

So now the rule is simple: no app code until the data contract exists and the anchor reproduces to the penny. AI makes the profiling and the query-writing fast enough that the gate costs an hour, maybe two. That hour is the cheapest insurance you'll ever buy against the one failure that actually damages trust. Let the assistant build fast — after you've proven what it's building on.