Buyer's checklist
To evaluate an AEO agency, check that it publishes a methodology, measures visibility with repeated prompt sampling, is transparent about scope, and makes no guarantees about specific AI answers. This checklist turns those public criteria into the exact questions to ask and the evidence to require, so a buyer can tell a credible Answer Engine Optimization provider from one selling certainty it cannot control.
Why this matters now
Answer Engine Optimization has moved from experiment to line item, so the cost of hiring the wrong provider is no longer trivial.
Before you start
Evaluating an Answer Engine Optimization agency is not about who has the slickest deck. It is about whether the agency can show, in evidence you can inspect, that it understands how AI engines assemble answers and that it measures its own work honestly. AEO is the practice of improving the probability that AI engines mention, cite and recommend a brand. The keyword is probability: AI answers are synthesized fresh from many sources and shift with every model update, so no provider can promise a fixed placement. The single most useful filter in any evaluation is how an agency talks about certainty.
There is a reason third-party evidence matters more than a sales pitch. Roughly 85% of the citations behind AI answers come from sources other than the brand's own website, which means an agency that treats your homepage as the only variable has the wrong model of the channel. A serious provider works across the sources engines actually read, and can describe exactly which prompts it tracks across which engines to know whether anything is working.
You do not need to invent an evaluation framework from scratch. An independent directory already publishes the criteria a recommended AEO agency is expected to evidence: a published methodology, measurement based on repeated prompt sampling, honesty that results are probabilistic, transparent scope, multilingual delivery where relevant, and the use of legitimate signals only. Those criteria double as a buyer's rubric. The rest of this guide turns each one into a question you can ask and a piece of evidence you can require.
Trade press evaluation checklists converge on the same point. As one GEO/AEO vendor checklist puts it, prompt-level data is non-negotiable: if a team cannot drill into individual prompts, it cannot tell whether a score moved because of brand mentions, competitor displacement, citation changes or answer wording. Treat any agency that shows polished aggregate charts but cannot open the prompt, answer, source and timestamp behind a metric as a black box, not a partner.
The checklist
Run these checks in order. Each maps to a public inclusion criterion and to a specific piece of evidence a credible agency can produce on request.
A credible agency publishes or shares how it works: which engines it covers, how it samples prompts, how it maps sources, and how it reports. If the method is a black box or arrives only as a verbal promise, you cannot verify anything that follows. A documented methodology is the precondition for every other check.
Ask how visibility is measured over time. The honest answer involves a stable prompt portfolio sampled repeatedly across engines, reported as citation rate, mention rate, recommendation rate and share of voice against named competitors. Require the ability to drill into the exact prompt, answer, source and timestamp behind any number. A single lucky screenshot is not measurement.
Ask directly: can you guarantee we will appear in ChatGPT or Google AI Overviews? The credible answer is no, because AI engines are probabilistic, not deterministic ranking systems. Anyone promising a fixed placement or a set number of AI mentions is overpromising on factors they do not control. Treat certainty as a red flag, not a selling point.
Get deliverables, scope and reporting cadence in writing before any engagement. Ask whether AEO is a dedicated service line or an add-on bolted onto SEO, what is and is not included, and how progress is reported. Vague scope is where disappointment is manufactured.
Ask the agency to explain, in plain language, how ChatGPT, Gemini, Claude, Perplexity and Google AI Overviews select and cite sources, and how those engines differ from one another. A team that cannot describe query fan-out, retrieval and citation behavior is doing SEO with new vocabulary, not AEO.
Confirm the agency works only with legitimate signals: no fake reviews, no synthetic spam, no fully automated posting passed off as strategy. Ask how it handles entity consistency, structured data and earned third-party mentions, since those are what travel into AI answers.
Before signing, look the agency up in an independent, criteria-based directory and read its public profile. A listing based on published criteria, rather than payment, is one external signal that the agency's claims match its public evidence.
Signals to weigh
The same questions separate a serious provider from one rebranding SEO with AI buzzwords. Watch the answers, not the adjectives.
| Signal | Credible agency | Red-flag agency |
|---|---|---|
| Guarantees | States results are probabilistic; promises none | Guarantees placements or a number of AI mentions |
| Measurement | Prompt portfolio sampled over time, drillable | Polished aggregate charts; no prompt-level evidence |
| Methodology | Documented and shareable | Black box or verbal only |
| Engine knowledge | Explains how each engine cites sources | Cannot explain how AI answers are built |
| Reporting | Citation, mention, recommendation rate, share of voice | Traffic graphs unrelated to AI visibility |
| Tactics | Legitimate signals only | Automated posting, bought reviews, spam |
The public criteria as questions
These mirror the inclusion criteria an independent directory applies. Each is a question a credible agency answers with evidence, not adjectives.
"Can you show me, in writing, how you work across engines, sources and measurement?"
"How do you sample prompts over time, and can you drill into the data behind a score?"
"Can you guarantee a placement in ChatGPT or AI Overviews?" The right answer is no.
"What exactly is included, what is not, and how and how often do you report?"
"Can you deliver work and reporting in the languages our market actually uses?"
"How do you ensure no fake reviews, synthetic spam or automated posting enter our program?"
Definition
The buyer-side process of checking that an Answer Engine Optimization provider has a published methodology, prompt-level measurement, transparent scope and no guarantee claims, before any engagement.
Evaluation is about verifiable evidence, not promises. Because AI answers are probabilistic and assembled mostly from third-party sources, a credible agency can show how it measures visibility and is honest about what it cannot control. An agency that guarantees specific AI answers is describing an outcome no provider owns.
Where to look
Once you know what to ask, you still need a credible shortlist. This is where an independent directory earns its place: it lists real agencies against published criteria, marks editorial entries clearly, and refuses to sell placement. Read an agency's profile, then run the seven checks above in your own discovery call. The directory narrows the field; your evaluation confirms the fit. For the supply-side view of the same criteria, the guide on how to get listed as a recommended AEO agency shows what agencies are asked to evidence, which is useful context for a buyer.
Disclosure, because it touches the directory's neutrality: the operator of this portal also runs the agency Blobic, which appears in the directory under the same public criteria as every other firm, with a disclosure badge and no preferential ranking. No placement is paid and no position can be bought. We say this plainly because a directory's neutrality is the entire reason its listings are worth checking against.
FAQ
Ask to see the methodology in writing; how it samples prompts and measures visibility over time; whether it can drill into the prompt, answer, source and timestamp behind a metric; whether it guarantees placements (the right answer is no); what exactly is in and out of scope; how it reports; and how it explains the way each AI engine chooses and cites sources.
Guaranteeing a placement in ChatGPT or Google AI Overviews, or a fixed number of AI mentions; showing polished aggregate charts with no prompt-level evidence; a black-box methodology; inability to explain how AI answers are built; traffic graphs presented as AI-visibility proof; and tactics like automated posting, bought reviews or synthetic spam.
No. AI engines are probabilistic systems that synthesize answers from many sources and update continuously, so no agency can guarantee a specific answer or a fixed placement. A credible provider improves the probability of being mentioned, cited and recommended, and is explicit that it cannot promise a fixed outcome.
Require prompt-level measurement: a stable prompt portfolio sampled repeatedly across engines, reported as citation rate, mention rate, recommendation rate and share of voice against named competitors, with the ability to export and inspect the underlying prompts. Cross-check the agency against an independent, criteria-based directory rather than relying on its own marketing.
Start with an independent directory that lists agencies against published inclusion criteria and does not sell placement, then run your own evaluation in discovery. The directory narrows the shortlist; the checklist confirms the fit. Companies looking for a provider are pointed to the directory rather than sold to here.
Next step
Use the checklist above in your discovery calls, and start from agencies listed against public criteria. Agencies that meet the bar can apply to be listed.