In production is not in use

In August 2012 I was marketing director at Wheely, the chauffeur app, and I posted a number from a Wheely survey on my blog. The survey ran for about a month among Londoners and found that only 1.5% had tried ordering a taxi through a mobile app, even though these apps always sat at the top of innovation rankings. About 74% still booked a taxi by phone, the way their parents did. The apps had plenty of fans. Not many people used them yet.

The same gap now sits on AI slides, under a new label: "in production".

Before: the status line

Here is how many AI lines look in a 2026 plan. A tool name, a cost, a status. The status says live, deployed or in production, and the room moves on. A tool count goes on the summary page as proof of progress.

The market is full of that status. G2's 2026 AI Agents Insights Report, published on 14 October and based on 1,019 B2B decision-makers surveyed in July, says AI agent adoption among those companies rose from 69% to 86% in a year. 66% of companies now run agents in limited or full production.

The same report shows what the status line hides. 31% of respondents say employees spend at least six hours a week overseeing agent work. 45% say oversight and rework eat more than a quarter of the productivity gains they expected. 46% report higher agent operating costs than expected. G2 calls this the Distrust Tax. It rarely makes the slide.

And the money keeps coming. Demand Gen Report's 2026 B2BMX Insights Benchmark, also out on 14 October, says AI-powered marketing tools replaced ABM as the No. 1 area for increased budget. The 2027 plan you are writing now will probably carry more AI lines than the 2026 one, each with a status next to it.

"In production" is the 2026 version of topping the innovation rankings. It is true and sounds like progress, and it says almost nothing about use.

After: three lines per tool

For every AI tool in the 2027 marketing plan, I would ask for three numbers before the line is approved.

1. Weekly users: how many people used it in each of the last four weeks, by name or by role. Not seats bought, not logins in the first month.
2. Hours spent checking it: how many hours a week people spend reviewing, correcting and redoing its output. If nobody tracks this, ask the two heaviest users to log it for two weeks.
3. The removal test: one workflow where switching the tool off would change cost or headcount. "The team would be sad" does not count. "We would need a second freelance writer" or "the weekly pipeline report would take a day again" does.

No line passes without the third. The first two tell you whether a tool is used and what it costs to trust. The third tells you whether it changed how the work gets done.

The three-line AI budget test

Three numbers before an AI line is approved.

Operating test from the EK Note, not survey data.

Pick one AI line in your plan. Tick each number you can show for it.

1. 1 Weekly usersPeople who used it in each of the last four weeks. Not seats bought.
2. 2 Hours checkingHours a week spent reviewing, correcting and redoing its output.
3. 3 Removal testOne workflow where switching it off changes cost or headcount.

0 of 3 Tick each number you can show for this line.

No line passes without the third.

G2 gives a hint of how hard the third answer is. 35% of respondents say they redeployed employees into higher-value roles. Only 14% report reducing headcount. Redeployment can be real value, but only if someone can name the workflow it came from.

What changes in the meeting

The before meeting reads a status column and approves a total. The after meeting reads three numbers per tool and argues about the third one. Some tools lose their line. Some get more budget, because the removal test is strong and nobody had written it down.

There is a brake here. Low usage is not a failure on its own. A tool used by two people that turns a full reporting day into an hour can be the best AI line in the plan. It passes on the third test and should be funded. The opposite case is the expensive one: forty seats, daily logins, six hours a week of checking and no workflow that would change if it disappeared.

Confirmed and plausible

Confirmed: the G2 figures above, read on G2's press release on 26 October 2026, and the B2BMX budget ranking, read on Demand Gen Report's preview the same day. G2 sells to software vendors, its sample covers B2B decision-makers in all functions, not marketing only, and the answers are self-reported. The B2BMX preview gives the ranking but no percentage.

Plausible, not proven: that the three-line test cuts AI spend or raises returns. I found no public data that measures it. The test is cheap. Run it on the current stack before the 2027 plan locks, and see how many lines can name a workflow.

In 2012 the rankings were right that taxi apps were innovative. They just measured something other than riders. "In production" has the same problem, and the 2027 budget is where the gap between live and used gets paid for.