Trust Is a Provenance Field

During QA on our restaurant-analytics product, the agent told a test persona that a location had served 5,121 guests. The database, queried directly for the same locations and the same dates, held 1,320. That is 3,801 phantom guests, an inflation of 288%. The answer read confidently. The chart rendered cleanly. Every human who had watched this agent in demos had nodded along at numbers exactly like it.

The thing that caught it was not a person. It was an assertion harness that takes every metric the agent claims, re-derives the same metric straight from the database, and diffs the two. A variance of 3 to 5% logs a warning. Above 10% is a failure. The guest count failed by a margin no reasonable person would call a rounding issue, and it failed for all three test personas across both QA rounds.

Here is the uncomfortable part I want on the record: the inflation was in the tool layer, likely a missing DISTINCT plus voided transactions leaking into the aggregate. It had been sitting there through weeks of internal demos, and nobody saw it, including me. Guest counts feel plausible across a wide range. Revenue you sanity-check against your own memory of the business. Guests you just believe. That gap between what looks right and what is right cost us the staging deployment, and the remediation plan the QA report laid out ran four days before we could even rerun the gate.

The QA run was structured as three operator personas (cost-conscious, revenue-focused, detail-oriented) across two iterations, producing 118 assertions spanning eight of the semantic layer's nine domains. The results: 88 of 118 passed (74.6%), 10 warned (8.5%), and 20 failed (16.9%). Every one of the 20 failures was a data issue, meaning a mismatch between what the tool claimed and what the database verified. Not a formatting bug. Not a tone problem. Wrong numbers.

The order-count bug is my favorite of the batch because it points the opposite direction from the guest bug. The tool reported 221 orders when the database held 835, dropping 614 orders, or 73.5% of reality. One tool inflates, another deflates, and both pass a casual read. The period-ambiguity warnings were subtler: single-period claims matched the database within 1 to 2%, but when a persona asked for a multi-period comparison, the agent sometimes answered with only the most recent period, producing a 46.9% gap against the full requested range.

In fairness to the system, the pass analysis matters as much as the failure list, because it tells you where the architecture is sound. Core revenue metrics, net revenue, and discounts landed within 1% of the database. Labor metrics, employee counts and shift hours, within 0.5%. Product metrics within 2%, operations metrics within 3%, payment-type distribution within 1%. The failures were concentrated in two aggregation paths and one prompt-level ambiguity, which is a very different diagnosis from "the agent makes things up." Nothing here was hallucination. The tool layer ran specific queries wrong, deterministically, every time. That distinction decides whether you fix prompts or fix SQL, and misdiagnosing it wastes weeks.

The trajectory across iterations was real but modest: 39 of 55 assertions passed in iteration 2 (71%), 49 of 63 in iteration 3 (78%). The deployment gate in the report was blunt: do not deploy until order and guest variance drop below 2% and an iteration-4 run clears 85%. I wrote that gate, and the system I wanted to ship failed it. Good. That is the gate doing its job.

The QA harness catches wrong numbers before release. It does nothing for the operator staring at a number in production, deciding whether to act on it. For that, the answer itself has to carry evidence. So every insight the platform generates is a structured object with four tiers of disclosure: the headline summary always visible, breakdowns with charts one click down, the action plan below that, and at the bottom, a raw-data modal with the actual data table and the SQL debug view. Tier 4 exists for exactly one reason. When an operator, or our own support team, does not believe a number, the path from the number to the query that produced it is one click, not a Jira ticket.

Attached to the insight is a provenance record, a frozen Pydantic model that rejects unknown fields and cannot be mutated after creation: the exact date range queried, the skill that generated the analysis, and the location IDs it ran against. Frozen matters. Provenance you can edit after the fact is marketing.

Underneath that sits a query-capture layer. Every SQL statement the semantic layer executes gets recorded with its full text, bound parameters, a SHA-256 hash of the sorted result set, the row count, execution time in milliseconds, the domain, the tool that triggered it, and a timestamp. The store holds up to 10,000 captured queries with a 60-minute TTL.

captured = CapturedQuery(
    query_id=str(uuid4()),
    sql=sql,                      # full statement, not a summary
    params=params,                # bound parameters
    result_hash=result_hash,      # SHA-256 of the result set
    row_count=row_count,
    execution_time_ms=execution_time_ms,
    source_table=source_table,
    domain=domain,                # which semantic domain ran this
    tool_name=tool_name,          # which agent tool triggered it
    captured_at=datetime.now(timezone.utc),
)

The hash is the part people underestimate. There is a verification endpoint that takes a captured query ID, re-executes the same SQL against the live database, and compares the new result hash to the original. If the hashes match, the number the agent showed is still the number the database produces. If they diverge, you know the data moved underneath the answer, and you know when, because the capture carries its timestamp. A second endpoint runs golden-dataset checks the same way. Verification stops being an argument and becomes a button.

Insights that combine several queries get a multi-source capture instead of a single record. Each contributing source is stored with its own SQL, its own record count, its own hash, and up to 100 sample rows, alongside a plain statement of the aggregation logic that combined them. When a skill blends revenue and labor to compute sales per labor hour, the receipt shows both queries and the join between them, not just the blended output. Composite metrics are where analytics products usually hide, because no single query can be pointed at. We removed that hiding place on purpose.

The last field on the insight is an observability trace ID. From any number on screen there is a walkable path: insight, to provenance record, to captured queries, to the full agent trace with every tool call and token count. My team's operating rule is written into the repo instructions: check the trace before touching agent code, because guessing without evidence is how you fix the wrong layer. The same discipline that debugs the system is what lets us stand behind it. Those are not two systems. Receipts for the user and forensics for the engineer come out of the same plumbing, which is most of the argument for building the plumbing once and early.

None of this works if the agent can freehand SQL or read from a stale replica. All agent reads go through a semantic layer that fronts nine domains: revenue, labor, guest, product, operations, payments, tax compliance, forecasting, and server performance. The agent asks for a metric in a domain; the layer owns the SQL. That is the single choke point that makes query capture complete, because there is no second path to the data that bypasses the recorder.

Caching is tiered by how settled the data is, and the tiers are deliberate. Historical data more than a day old caches for 24 hours because it is immutable. Yesterday caches for one hour because late adjustments still land. Anything including today caches for 300 seconds. An operator looking at this morning's revenue is never more than five minutes behind the source of truth, and the source of truth is the live operational database, not a copy we sync when convenient. I have watched competitors demo dashboards that were quietly eight hours stale. Their numbers were internally consistent and operationally useless.

Board math, because this is where provenance stops being a philosophy and becomes margin. Restaurant operators staff against guests per labor hour. If the guest count is 3.9x reality (5,121 claimed against 1,320 actual), every ratio downstream of it is fiction, and the schedule built on that fiction burns real payroll. The period-ambiguity bug is worse in the other direction: an operator planning against a claimed $84,800 when the true figure for the requested range was $159,808 is making decisions with 46.9% of their revenue picture missing. The cost of a wrong number is never the number. It is the payroll decision, the order quantity, the price change built on top of it.

Against that, the cost of the trust stack: an assertion harness that runs 118 checks, a provenance field on every insight, a query recorder, and two verification endpoints. It is boring engineering. It found 20 wrong numbers in a system I would have sworn was ready, and it blocked a deployment that would have put those numbers in front of people running businesses on thin margins. The harness paid for itself before it finished its second iteration.

The harness and the provenance stack are the same bet placed at two timescales. The harness catches the wrong number before release; provenance catches it after, when a real operator pulls the receipt on a number that smells off. Between them sits the class of bug neither can catch alone: the query that is wrong in a way the database agrees with, like the period-scope ambiguity, where the SQL was correct for a question the user did not ask. That one needed the personas. Three synthetic operators with different obsessions found what one generic test account never would have.

Here is the position, stated so you can disagree with it: an AI analytics product that cannot show the SQL behind an answer should not be deployed to operators. Not because operators read SQL. Most never open Tier 4. The point is that the receipt exists, that support can pull it in one click, and that the vendor knows every number it displays can be audited down to the query and re-executed against live data by anyone with access. That knowledge changes how a team builds. You write different code when your output testifies against you.

The common objection is that provenance UI is clutter and users just want the answer. I agree they want the answer, which is why the receipts live four tiers down and the headline stays clean. But I have also watched what a 288% inflated number looks like when it is dressed in a confident sentence and a clean chart. Users do not trust AI answers because you ask them to. They trust answers that carry their own receipts, and they only need to pull the receipt once, after the first time the number saves them or burns them. If your product cannot survive that pull, you do not have a trust problem. You have a correctness problem, and the trust problem is on its way.