The AEO checklist is not the lever

Somewhere this quarter a marketing team will show an "AI visibility score" that went up four points. The slide will credit the FAQ blocks, the schema markup and the faster pages shipped last month. Nobody in the room will ask whether the score would have moved anyway if someone had captured the same answers again the next morning.

I think both the checklist and the single score are weaker than they look. Two pieces of research from the past few weeks show why.

What actually predicts a citation

In September, Ben Moore and Liam Dunne of Discovered Labs, an AEO benchmarking firm, posted a paper to arXiv on about two million citations from ChatGPT, Claude, Google AI Overviews and Gemini. They collected them over roughly six months and joined them to about 10,000 crawled pages from 19 B2B SaaS companies. They tested more than 60 page features, and an effect only counted if it survived nine separate statistical checks.

One feature dominated: how closely the words on a page match the words in the prompts buyers ask. The authors call it prompt-content alignment. One standard deviation more alignment went with roughly 30% more citations at the geometric mean, and the effect held in every robustness test, including a hold-out month. It was stronger on a brand's own pages than on competitors' pages, which is where a marketing team actually controls the copy.

The checklist did much worse. FAQ blocks, structured data and Core Web Vitals looked helpful in pooled data. Once the authors compared pages within the same domain, those effects collapsed to about zero or reversed. Their explanation is Simpson's paradox. Big incumbent brands collect most of the citations and differ from challengers on most page-level signals, so pooled data gives the brand's credit to the checklist. Domain authority was the other heavy predictor, about six times the strongest page feature after alignment, a figure the authors call an upper bound.

Why one score hides the answer

The second source is the OMNIFAMOUS AI Visibility Report, published 6 September. It ran the same 7,949 prompts through ChatGPT, Perplexity, Gemini, Google AI Mode and AI Overviews three times between late July and early September. Of the domains each engine cited in July, the share still cited in September ranged from 14.8% on AI Mode to 64.6% on Perplexity. Even two captures of the same prompt on the same day disagreed: ChatGPT, Gemini and AI Mode failed to keep roughly half of their cited domains.

Their sharpest point is about denominators. The same 8,429 brand changes on ChatGPT come out as 9.76% or 38.25%, depending on what you divide by. They also split apart outcomes a blended score merges. A mention, a recommendation and a citation of your own site are different events, and an answer can contain any one of them without the others.

So a score that moved four points could reflect the engines changing, the capture changing, or plain noise. Either way, it does not tell you which page to fix.

Some cautions before this goes into a deck. Both sources come from companies that sell AEO measurement. Both are observational, and the paper's authors say plainly they cannot tell whether engines reward alignment or whether aligned pages are better in ways they did not measure. Both cover US English. Neither measured Microsoft Copilot: the paper left out Perplexity and Bing, and OMNIFAMOUS dropped Copilot for incomplete coverage. Microsoft Advertising is one of the platforms I work with, so that gap matters to me. In Cyprus, Malta, Kazakhstan and Azerbaijan, buyers often research in more than one language, and I have no comparable data for any of them. Treat the mechanism as a hypothesis to test there.

Where the checklist still earns its place

There is a brake on this. If you are the incumbent with a strong domain, the checklist is cheap upkeep, and the paper found on-page signals mattered more on bottom-of-funnel pages; pricing pages had the largest positive effect of any page type. Domain authority is also not fixed. The authors describe it as the accumulated result of years of content and coverage. A challenger cannot buy it with schema, but it can build it, and alignment is the page-level route in.

How to tell which world you are in: if your pages already get cited for the prompts that matter and you lead the category, keep the hygiene. If you are a challenger and you are absent, schema will not close the gap.

What I would change this week

Three changes look cheap enough to start now. (1) Build a registry of 20–30 prompts from sales-call notes, support tickets and on-site search, the way OMNIFAMOUS built its set, and tag each one with the decision behind it: fit, comparison, pricing or switching. (2) Capture each prompt twice on the same day, and report mention, recommendation and own-site citation as three numbers, each with its denominator. (3) Rewrite one high-value page a week so it answers one buyer question in the buyer's words, near the top of the page, with the conditions (plan, region, date) next to the answer. The paper found engines cite paragraphs from the top third of a page.

If your AEO report cannot name the buyer question you lost and the page that should have won it, it is measuring the checklist. I would rather measure the question.