The short answer
Benchmark AI visibility by choosing a legitimate peer set, freezing the same buyer questions and market context for every brand, and comparing observable answer outcomes across the same providers and collection windows. Report mention coverage, first-choice coverage, citations, claim accuracy, and provider splits separately. Keep the raw counts and answers attached. The result describes the tested sample; it is not a universal share of AI search or customer demand.
Editorial method · evidence boundaries
This guide proposes a review method. Worked examples and pilot sizes are illustrations unless a dated study is explicitly identified. Provider documentation supports only the behavior and product surface it describes; it does not prove that an AIBL workflow improves recommendations or business outcomes.
Historical AIBL records have limitations: completed jobs can contain truncated answers, and native citation metadata was not retained uniformly. Verify answer usability and source availability before scoring or comparing results. Missing saved metadata is not proof that a provider supplied no sources.
What businesses notice
Common signs of the problem
- Leadership asks whether the brand is ahead of competitors, but the dashboard shows only an isolated score.
- A competitor appears often, yet nobody knows whether it was recommended first or merely named.
- The comparison set mixes direct rivals with brands that serve different markets, audiences, or price tiers.
- A report calls a handful of prompts “AI market share” without showing questions, providers, failures, or raw answers.
01
Define the benchmark before collecting answers
A competitive benchmark is a declared test, not a leaderboard assembled after seeing the results. Write a one-page measurement contract that names the decision, eligible brands, buyer questions, providers, markets, dates, repeats, and evaluation rules.
Choose the decision the benchmark supports
Decide whether the team needs to understand category discovery, shortlist inclusion, first-choice preference, source presence, or factual positioning. One benchmark can retain all of these observations, but its primary decision determines the prompts and headline view.
Create an eligibility rule for competitors
Include organizations a buyer could reasonably choose for the tested need in the stated market. Record why each brand belongs. Separate direct peers, substitutes, marketplaces, publishers, and irrelevant entities so a famous but ineligible name does not distort the comparison.
Freeze context for every brand
Use the same substantive question, provider, location or market, language, time window, and visible settings. Do not add favorable details for the tracked brand or test competitors with easier wording. If a brand is named in a prompt, label that cell as aided rather than unaided discovery.
Bound the conclusion
NIST distinguishes performance on a fixed benchmark from performance generalized to a broader population. State that the result applies to the declared question-provider sample unless the design and uncertainty analysis justify a wider inference.
Read NIST AI 800-3 ↗02
Build buyer questions that expose competitive choice
The prompt set should represent materially different buyer decisions. Rephrasing one flattering question ten times adds volume without adding much coverage.
Start with unaided discovery
Ask for options that solve a category, problem, role, or use-case need without naming any candidate. These cells test whether a brand enters the consideration set at all. Exclude prompts where the brand is not legitimately eligible.
Add fit and comparison questions
Test concrete constraints that could change the shortlist: company size, workflow, integration, location, budget band, or service model. Use only constraints that reflect a real buying decision, and keep each materially different intent as its own labeled group.
Keep branded verification separate
Questions such as “Does Acme support SSO?” measure description and claim accuracy after the brand is named. They do not measure unaided competitive visibility. Preserve them in a verification stratum instead of blending them into discovery coverage.
Use real language and a stable core
Seed questions from sales calls, support conversations, customer research, site search, and recurring objections. OpenAI evaluation guidance recommends task-specific objectives, representative cases, explicit metrics, logging, and continuous evaluation; those principles help keep the portfolio tied to the decision.
Review evaluation best practices ↗03
Preserve one evidence row per observation
A defensible benchmark can be reconstructed from its underlying answers. Store the provider output before applying labels, totals, or competitive interpretations.
Record the collection context
Keep question ID and version, exact prompt, buyer-intent group, provider and visible product surface, market or location, timestamp, repeat number, completion state, retry reason, and the full answer. A screenshot without this context is not a durable benchmark row.
Classify the brand role
For every eligible brand, distinguish absent, passing mention, considered option, qualified recommendation, first choice, and caution or rejection. A citation title, caveat, or excluded option should not be counted as a recommendation.
Capture sources and claims separately
Record citations or source links when returned, then check whether they support the nearby statement. OpenAI warns that search results and citations can be incomplete, outdated, or incorrect, so source presence is evidence to inspect rather than proof that the answer is accurate.
Review ChatGPT Search source guidance ↗Keep failures out of brand performance
Report planned attempts, completed eligible answers, failures, refusals, and exclusions. A timeout or unusable response is a collection failure, not an omission by every brand. Freeze the completed-answer denominator before calculating coverage.
04
Calculate a layered competitor benchmark
No single percentage captures competitive visibility. Start with simple rates whose numerators and denominators can be traced to answer rows, then show the breakdowns that make differences actionable.
Mention coverage
For each brand, divide completed eligible answers that mention it by all completed answers in which it was eligible. Report the raw count beside the percentage. This measures presence in the tested sample, not recommendation quality.
Recommendation and first-choice coverage
Calculate the share of completed eligible answers where the brand is a genuine recommended option, then the share where it is the clear first choice. If the answer gives an unordered list, record first choice as not applicable instead of converting list order into rank.
Share of observed brand mentions
Divide one brand’s mentions by all eligible brand mentions across the same completed cells. Label this precisely: one answer can mention several brands, so the denominator is observed mentions—not searches, customers, revenue, demand, or market share.
Evidence and accuracy layers
Show owned-source citation coverage, third-party source coverage, material-claim accuracy, and unresolved claims beside visibility. A frequently mentioned competitor may still be described incorrectly, while a less visible brand may have stronger source support.
05
Work through a 24-cell example
This fictional example demonstrates the method; it is not customer evidence or a claim about any provider. Suppose a team freezes six buyer questions, runs them in ChatGPT and Gemini, and repeats each once. That creates 24 planned question-provider-repeat cells.
Publish collection coverage first
If 23 answers complete and one fails, collection coverage is 23/24, or 96% when rounded. Brand performance uses the 23 completed eligible answers. The failed cell remains visible and is not scored as an omission.
Compare presence and preference
Suppose the tracked brand appears in 12 of 23 completed answers and is first choice in 4; Competitor A appears in 16 and is first choice in 8; Competitor B appears in 10 and is first choice in 3. Report 12/23 and 4/23, not just 52% and 17%. The gap is larger at first choice than at presence.
Locate the segment creating the gap
Break the 23 cells into category discovery, use-case fit, and comparison or verification groups. If the tracked brand is present in five of seven use-case answers but only one of eight discovery answers, the benchmark identifies an unaided-discovery gap that a blended total would hide.
Inspect the evidence behind the leader
Open the eight answers where Competitor A is first. Check whether the recommendation is consistent across providers and repeats, which rationale appears, whether citations support it, and whether material facts are accurate. The benchmark reveals where to investigate; it does not prove why the provider chose the brand.
06
Read competitive gaps without inventing causes
A benchmark shows what the configured systems returned under declared conditions. It can prioritize research, but it cannot expose a provider’s private ranking logic or prove that one content change caused an answer.
Separate broad gaps from provider-specific gaps
Show the same metrics by provider before presenting any rollup. A brand that leads in ChatGPT and disappears in Gemini needs a different investigation from one that is consistently absent across both.
See the provider disagreement protocol →Trace repeated rationale to public evidence
Compare recurring claims with official product or service pages, maintained structured facts, and relevant third-party evidence. Mark supported, contradicted, outdated, and unresolved rather than treating repeated wording as truth.
Review material claims before repeating them
A competitive report can become sales or advertising material. The FTC says objective express and implied advertising claims need a reasonable basis before dissemination. Verify competitor and tracked-brand claims, use clear qualifications, and involve the appropriate reviewer for regulated or high-consequence statements.
Read the FTC advertising FAQ ↗Turn gaps into testable hypotheses
Write a bounded hypothesis such as “our category page does not clearly support this use case,” then identify evidence that would confirm or weaken it. Improve the public record only where the claim is true and useful; do not manufacture parity with a competitor your offer cannot match.
07
Protect the benchmark when it changes
A trend is useful only when readers can distinguish competitive movement from prompt edits, provider changes, collection failures, and evaluation drift.
Version the measurement contract
Keep question wording, peer eligibility, provider coverage, market context, evaluation rubric, and formulas under version control. Mark a series break when a material field changes rather than joining unlike samples into one line.
Use a stable core and exploratory edge
Repeat a stable set for comparable history while testing new buyer needs in a separately labeled exploratory set. Promote a question into the core only when its business relevance and interpretation are documented.
Require repeatable movement
Treat one changed answer as a review signal. Check adjacent questions, providers, repeats, citations, and collection health before calling it a competitive gain or loss. AI systems can produce incorrect or misleading output, so important conclusions should be verified against reliable sources.
Review ChatGPT accuracy guidance ↗Publish limitations beside the chart
State the tested brands, question groups, providers, markets, dates, repeats, completion rate, formula, and known exclusions where readers see the conclusion. Do not bury the scope after a headline that implies universal market leadership.
08
Start with a two-provider competitive evidence cell
The free AI visibility snapshot is a practical starting point, not a complete competitor benchmark. It runs one buyer-style question across ChatGPT and Gemini and preserves the answer evidence needed to decide whether a larger study is worthwhile.
Choose one neutral buyer question
Use a real category, problem, use-case, or local question where the tracked brand and named peer set are legitimately eligible. Do not name the tracked brand in an unaided discovery test or write the prompt to force a preferred conclusion.
Run a free AI visibility snapshot →Inspect both complete answers
Record which brands appear, which are genuine options, whether a first choice exists, the rationale, sources, and material factual issues. Note provider disagreement rather than averaging the answers into one score.
Decide whether the result changes a decision
Expand into a benchmark only if the evidence affects a real choice: positioning work, source review, market prioritization, client reporting, competitor research, or monitoring. A surprising screenshot without an owner or decision does not justify a large program.
Scale with the same contract
Add questions, providers, markets, and repeats deliberately while preserving the row-level evidence and eligibility rules. AI Brand Lens can organize the continuing observations, but the benchmark remains trustworthy only when its scope and denominators stay inspectable.
Review the AIBL measurement method →Continue the research
Related AI visibility guides
Primary sources
Primary documentation used for this guide
AI products, search behavior, and platform policies change. Check these maintained first-party sources before making technical decisions.
- Expanding the AI Evaluation Toolbox with Statistical Models ↗
National Institute of Standards and Technology
NIST AI 800-3 distinguishes performance conditioned on a fixed benchmark from estimates intended to generalize to a broader item population and emphasizes explicit assumptions and uncertainty. This guide applies that distinction conservatively to competitive answer samples.
- Evaluation best practices ↗
OpenAI Platform Documentation
First-party guidance on task-specific objectives, representative datasets, explicit metrics, logging, comparison, continuous evaluation, and human calibration. It informs the benchmark workflow but does not validate any brand metric.
- Searching the web with ChatGPT ↗
OpenAI Help Center
First-party explanation of ChatGPT Search citations, source panels, query rewriting, location context, and the need to verify incomplete, outdated, or incorrect results and citations.
- Does ChatGPT tell the truth? ↗
OpenAI Help Center
First-party guidance that generated answers can be incorrect or misleading and that important claims, quotations, data, and references should be checked against reliable sources.
- Advertising FAQs: A Guide for Small Business ↗
Federal Trade Commission
FTC guidance that objective express and implied advertising claims need a reasonable basis and that qualifications should be clear and conspicuous. This guide is not legal advice.
Measure your visibility
Turn the questions in this guide into an evidence-backed baseline.
Get AI Visibility ScoreTalk to us
Have a visibility question?
Tell us what your team is trying to measure or improve.
Contact AI Brand Lens →Keep learning
Explore every guide.
Browse practical answers about AI visibility, competitors, and measurement.
View all guides →