bfxos.ai · field notes

What a real AI ROI dashboard tells your CFO.

Four things a legitimate ROI dashboard has to show — and the ways most "AI success" dashboards quietly avoid showing them.

The BfxOS Team 6 min read Field Notes · No. 5

Most AI dashboards a CFO sees were built by the team being measured.

That's a reasonable place to start being skeptical from. A dashboard that only shows metrics chosen by the team it's evaluating has an obvious incentive problem, whether or not anyone intends it. Trust gets built by what a dashboard is willing to show, not just what it shows well. Four things separate a dashboard a CFO can actually rely on from one that just looks like it.

I.

Price it the way finance already does

Full loaded cost — compute, review time, rework — not just the model API line item. A dashboard that reports "$400 in API spend this month" while omitting the twelve reviewer-hours spent fixing what the agent got wrong isn't lying, exactly. It's reporting a number that flatters the initiative by construction.

The test
Ask what's excluded from the cost figure before asking what's included. If review time and rework aren't in there, the number is incomplete, not wrong — but incomplete in one consistent direction.
II.

Show trend, not a snapshot

One month of good numbers proves a pilot worked once. A CFO needs to know whether automation is compounding — is cost-per-outcome actually declining as the team tunes the process, or was month one a demo dressed up as a result? Trend over at least a full quarter is the minimum bar for a number that belongs in a board deck.

III.

Break down by process, not by model

Model choice is an engineering decision. ROI is an operating decision, and it should be organized the way the business is organized — by process, by team, by business unit — not by which model happened to run it.

A CFO doesn't care that one model cost two cents more per call. They care whether claims processing got cheaper this quarter.
IV.

Reconcile against a baseline that doesn't move

The moving-goalposts problem: if the "before" number gets quietly revised every quarter to make the "after" number look better, the ROI figure stops being a measurement and becomes a negotiation. A real dashboard locks the baseline once, at the start, and holds it there — even when that's uncomfortable for whoever's being measured against it.

The test
Ask when the baseline was last recalculated, and why. "We updated it to reflect the new process" is the answer that should worry you — a baseline that adapts to the thing it's measuring can't measure it.
V.

Where this leaves you

None of this is about making AI look better on a dashboard. It's about building the one dashboard a CFO would build themselves, if they had access to the same data engineering does — which, if it's built right, they now do.

See the dashboard, not a slide about it.

Bring a real workflow to a demo — the same rollup your CFO would see, on day one.

Field Notes — the series
05What a real AI ROI dashboard should tell your CFOpublished