How to evaluate an answer engine optimization agency

Four names, one service

Answer engine optimization is the formal version of the term. Generative engine optimization, AI search optimization and LLM SEO are the same offer in different vocabulary, and the market knows it. When we inventoried Breaking B2B's 222 URLs from their own sitemap in August 2026, we found three separate service pages for a single AI-search offering: one at an answer-engine-optimization URL, one at a generative-engine-optimization URL and one at an LLM-SEO URL. That is a keyword split, done deliberately because buyers arrive typing three different phrases. It is cheap, it is honest enough, and it tells you the category has not settled on a word.

The consequence for a buyer is that the name on the tin carries no information. You cannot shortlist on terminology, and you cannot shortlist on the service description either, because every provider in the category describes roughly the same set of activities: technical fixes, entity work, answer-shaped content, third-party corroboration and tracking. What actually differs between two providers is the measurement discipline underneath, and that is invisible from a homepage.

So the evaluation has to be run on the method rather than the offer. Everything below is a check a buyer can run in an afternoon, without our help and without ours being the answer.

Audit the arithmetic before you audit the agency

The strongest signal a provider gives you is a published number you can test against their own stated method. Most providers publish something: a ranking, an index, a benchmark, a client figure. Take the definition they give, take the number they printed, and check whether the second is reachable from the first.

Here is a worked example from the audited set, and we want to be precise about what is theirs and what is ours. aeoengine.ai publishes an Answer Index: four monthly editions launched between 2026-07-28 and 2026-07-31, each described on their own methodology page as 300 questions run across four engines, which is 1,200 answers per edition. That is their published method, stated by them. Their methodology also defines appearance rate as the share of the 300 tracked questions in which a firm is named. That definition is theirs too.

The arithmetic is ours, and anyone can repeat it. If appearance rate is a count of questions out of 300, its possible values are multiples of one three-hundredth, or roughly 0.333 percent. Rounded to one decimal place, the reachable low values are 0.3, 0.7, 1.0 and so on. The tail of their published National 100, positions 59 to 100, carries values of 0.9, 0.8, 0.6, 0.5, 0.4, 0.2 and 0.1 percent. Most of those cannot be produced by k divided by 300 under any rounding rule. Either the definition is not the calculation, or the tail is smoothed. We do not know which and we are not asserting intent. We are saying that a reader who spends five minutes with a calculator finds a gap between a published definition and a published number, and that is exactly the check to run on any provider, including on us.

A second, easier version of the same check is internal consistency. The same publisher's context columns disagree with themselves inside one month: one law firm is shown at a domain rating of 91 with 2.4 million organic sessions in one edition and a domain rating of 71 in another; a second firm is shown at 76 with 210,000 sessions in one and 0.3 with none in the other. Context columns that contradict each other in the same month are a sign that an entity-resolution step is quietly failing, and if it fails in the filler columns it is worth asking what it does in the columns that carry the ranking.

None of this makes them a bad provider. Parts of that index are better practice than most of the category manages: the question set is frozen and versioned, there is a public correction log, and retired sample prompts are published so a reader can see the shape of the questions without being able to game them. We would credit all three. The point is narrower. A provider who publishes enough for you to check them is doing something valuable even when the check fails, and a provider who publishes nothing checkable has not given you anything to evaluate.

Ask what the provider withholds

Every AI visibility number is a ratio computed over a set of prompts somebody chose, run some number of times, in some window. Drop the prompts you never win and the share rises. Add prompts containing your own brand name and it rises again. Restrict the competitive field to three named rivals rather than every company the answers mentioned and the number can double while not one captured answer has changed.

That is why the useful question is not what a provider reports but what they hold back. Ask for the noise floor: below what movement do they call a change run-to-run variance rather than progress. Ask whether any figure in the last quarter was withheld for failing that bar, and what it was. A measurement discipline that has never produced a held figure is not a discipline.

Our own answer, so this is not a demand we make of others only. We hold any citation figure that has not run at least three times. Every client citation share we have measured so far is a single-sample reading with roughly 25 points of margin, so none of it is on this site. We can show you a 488-point local grid, a page-by-page indexation audit and a controlled ranking experiment. We cannot show you a client citation percentage, and the reason is that it would not survive being re-run.

Ask about engine coverage the same way, and expect the honest answer to be short. We measure ChatGPT and Gemini through a vendor that renders those consumer surfaces, plus Google AI Overviews through search results with the overview expanded. We do not measure Claude at all, because there is no ground-truth path to the consumer product and a model-level API is a different surface with different retrieval behaviour. We do not measure Perplexity either, because its consumer product disallows automated access in its own robots file. Any provider whose coverage list includes every engine you have heard of is either scraping surfaces whose terms forbid it or counting an API as the consumer product.

Ask for the control group

The single question that separates an operator who measures from an operator who reports is whether they have ever run a change against an untouched cohort. Without one, every result is a movement that happened after something, and search results move on their own.

Our own example is a failure, which is the only kind we can publish without a client sign-off. In August 2026 we changed page titles on a video production company in the San Francisco Bay Area. The homepage average position went from 16.8 to 34.5 overnight, and homepage impressions went from 889 a day to 538. The reason we know it was us rather than the market is that 34 untouched pages on the same site moved 0.8 positions over the same period, which is nothing. The nine retitled pages moved 3.1. Search Console, day over day, 2026-08-10.

The mechanism was legible once we looked at term level. For one commercial query the homepage fell from position 5 to 43 while the corresponding service page rose from 36 to 15. Google had moved the term to a page that ranked worse for it. We rolled the titles back the same day.

A buyer should ask any provider for the equivalent story, and be suspicious of a portfolio with no negative results in it. Work that is measured produces failures. Work that only produces wins is either extraordinarily lucky or is not being measured against anything.

Scope, terms, and what a credible engagement contains

Five things belong in the scope of work, and we publish this list precisely so a buyer can hold us to it as well as anyone else. A measured baseline of your citation share across the engines the provider can actually reach, not an estimate derived from traffic. Named competitor benchmarking, so the number has context. Execution that ships, meaning the technical and content fixes are done rather than recommended. Re-measurement at the end against the same baseline, so the work is accountable to the metric it claims to move. And plain reporting with the methodology shown, with no multiples nobody can verify.

On terms, the accessible end of the market splits. aeoengine.ai publishes 90-day rolling terms, and Breaking B2B's pricing FAQ, re-fetched on 2026-08-16, says its retainers run six or twelve months and that it does not run shorter. Two of the six providers we audited publish a performance guarantee on their own site. Those are their published terms as of 2026-08-15, and we have not tested how either is applied in practice. We do not publish a guarantee. Our reasoning is that a guarantee on a probabilistic output is either hedged into meaninglessness or is a commitment the seller cannot control, and we would rather say that than invent one to match. A buyer is entitled to treat that as a mark against us.

Ask who owns what at the end. Ask whether the prompt set is yours and whether you can take the raw captured answers with you. Ask whether a strategist is on the account or whether the engagement is a platform your own team will be running, because the second is a real and reasonable model and it is priced differently. One provider we inventoried is explicit about it: their site is 65 pages of product documentation and 58 essays against three blog posts, which is the shape of a company selling software, and they publish seven legal pages including a master agreement, a data processing addendum and a subprocessor list. If your procurement team needs that furniture, that matters, and we do not have it.

Finally, ask what would make them tell you not to buy. Our version of the answer is that the free assessment exists partly to produce that conclusion: if AI answers do not name businesses in your category yet, the measurement will show it, and the correct outcome is that you spend nothing. A diagnostic seller who cannot describe the circumstances in which they would walk away is not selling a diagnosis.