Prompt research · Record keeping

Which ten fields belong in your prompt log?

Every metric you can compute later is a function of the columns you wrote down at collection time. The schema is the experiment; the dashboard is only a view of it.

Topic in Prompt research12 min readUpdated 30 Aug 202611 sources, 10 primary and 1 vendor-described

Back to the roadmap Next: Retiring a prompt

On this page

The short answer

Log ten fields on every run: prompt ID and version, prompt class, engine, surface or mode, model name if exposed, whether web search was enabled, timestamp, locale and country, whether the answer text named your brand, and the full list of cited URLs including everybody else's. Write a row for the runs where nothing happened too, because a presence rate is only as good as the denominator underneath it. The 2026 critical survey's minimum checklist for a GEO study asks for seven configuration values on every observation, product, mode, model, date, locale, account and search state, and for denominators and null outcomes to be retained.¹

Key takeaways

The configuration fields feel like nothing at collection time, and are the only columns that can later tell you whether the engine changed or you did.
Mention and citation are two columns, never one: a link with no brand name is a content problem, a brand name with no link is a model-priors problem.
Keep failed collections behind a status field: dropping 25 timeouts out of 200 runs turns a 30.0% presence rate into 34.3%.

The schema is the experiment

Why does the log schema decide what you can conclude?

Because no analysis recovers a column that was never written down, and the columns teams skip are always the same ones. The 2026 critical survey of this field publishes a minimum checklist for a GEO study, and it is unusually specific: record product, mode, model, date, locale, account and search state; run closely spaced repetitions across multiple time windows; retain denominators and null outcomes; and keep the pipeline stages separate rather than read a one-off score as a stable rank. Its standing recommendation is to measure retrieval, citation and fidelity separately.¹ Most of that is instruction about the log, not about the analysis.

Notice what the first line consists of. Those seven are configuration, not results, and none of them feels like data while you are collecting, which is exactly why they get dropped. Their value shows up months later, when a rate steps down by fifteen points and the only question that matters is whether the engine changed or you did. A log without those columns cannot answer it.

The schema is also what makes a panel version mean something. The prompt research stage treats the panel as a versioned instrument; the log is where that version is stamped onto each observation, so a version 2 number is never silently compared with a version 3 one.

The schema

Which ten fields belong in every row?

One row per run, written before anyone looks at the answer, carrying the ten fields below.

The third column is what a missing field costs you later.

Field	Example value	What breaks without it
1. Prompt ID and version	P-014 v2	A reworded prompt becomes a different question wearing the same name
2. Prompt class	objection	No class split, so the blended rate hides its own composition
3. Engine	engine A	Pooling averages surfaces that overlap at Jaccard 0.11–0.18⁷
4. Surface or mode	chat with web search	Two rows that look identical turn out to be different experiments
5. Model name, if exposed	as shown in the UI	A vendor's model swap gets attributed to your content change
6. Search enabled	yes / no / unknown	Retrieval-grounded answers get mixed with answers from model priors
7. Timestamp with timezone	2026-08-24 09:12 UTC	Batching goes undetected, and batched runs correlate
8. Locale and country	en-GB / UK	A panel that mixes locales cannot be compared with itself
9. Brand named in the text	true / false	No mention rate at all
10. Cited URLs, all of them	full list, not just yours	No citation rate, and no map of who owns the answer

One field the checklist implies without naming belongs beside those ten: the run status. Record whether the collection completed, was rate-limited, timed out or was refused, because a run that never happened and a run that happened without naming you are the same blank cell and opposite facts. Without it you have no denominator.

Three derived columns follow from the ten and should be computed rather than typed. Mention comes from field 9. Citation comes from field 10, by testing whether any cited URL is on your domain. Fidelity comes from neither: it is a human reading of the sentence about you, on a sample, scored as accurate, incomplete, misleading or wrong. Three columns, three failure modes, three different fixes.¹

The denominator

Why must null outcomes stay in the denominator?

Because a presence rate is a fraction, and a fraction that quietly drops the runs where nothing happened is not a rate. The obvious version, keeping only the 60 runs out of 200 that named you and reporting 100%, nobody does on purpose.

The version that actually happens is about failed collections rather than absent brands. Suppose 25 of those 200 runs errored, and the pipeline drops them because a failed row has no answer text to parse. Presence is now 60 of 175, or 34.3%, instead of 60 of 200, or 30.0%. That 4.3-point gap is a data-handling decision, and more than half the width of the confidence band a 200-observation panel carries. Worse, failures are rarely random: long answers time out more often, some locales rate-limit harder, and one engine refuses questions another answers, so the bias has a direction you did not choose.

Three rules fix it permanently. Write a row for every attempted run, including the ones that failed, with the status field from the previous section. Decide the denominator rule once and put it in the report: “presence over completed runs, 175 of 200 attempted” is a full disclosure and costs nine words. And watch the completion rate itself as a quality metric, because a pipeline that silently degrades from 95% to 80% completion moves your headline number without anything happening in the world. It is the checklist line teams break most often, and the only one that corrupts every metric downstream of it at once.¹

Two columns

Why must mention and citation be separate columns?

Because they diverge widely, they diverge in opposite directions on different engines, and each points at different work by a different team. A mention is your brand name appearing in the prose; a citation is one of your URLs being attached, whether or not the prose names you. Collapsing them into one “we appeared” flag destroys the distinction that tells you what to do next.

The size of the gap is best documented in vendor work, which should be read as directional. A June 2026 study of 115 prompts producing 3,981 domain appearances across four platforms and fourteen countries reported 62% of appearances as links whose brand the answer text never named, and the pattern reversing between engines: one assistant roughly 84% mention against 21% citation, another roughly 21% mention against 87% citation.¹¹ The sample is small and vendor-published; treat the percentages as unverified and the reversal as the finding.

The two failures have different fixes. A link with no name means retrieval worked and identification failed, usually because the quoted passage was useful but never said who was speaking; a content team fixes that by writing sentences that name the subject. A name with no link means the model is drawing on what it already knows rather than what it just retrieved, which on-page work will not move quickly; the diagnostic is to rerun the prompt with search disabled. Neither column says anything about whether the description was true. A Tow Center audit of sixteen hundred queries across eight generative search tools found more than 60% of answers incorrect,³ an earlier Tow test of two hundred quotes drew partially or entirely incorrect responses on 153 occasions with uncertainty signalled only seven times,² independent work found 51.5% of generative-answer sentences fully supported by their citations,⁴ and a 2026 study classified 11.0% of 98,020 atomic claims as unsupported by the pages cited.⁵ That is what the fidelity column is for.

The eleventh column

Should you store the raw answer text as well?

Yes, because it is what keeps the history re-analysable. Mention is a name match against prose, and that matcher is wrong at the edges on day one: it misses possessives and misspellings, fires on a competitor sharing a word with your product, and cannot tell a recommendation from a warning. With the answer text stored you rerun the corrected matcher over the whole history. Without it the corrected series starts today, which is not a series.

The fidelity column has the same dependency: nobody scores it live at collection time, so the stored text is what lets a reviewer read last month's runs this month. Keep the citation payload raw too, because a list flattened to bare domains at write time can no longer say which page was cited.

One limit keeps this honest: a backlog nobody rereads is cost without benefit. Set a retention window deliberately and write it into the report, because it is the ceiling on how far back a recomputed series can reach.

Limits of the schema

Which fields can you not reliably capture?

Three of the ten are frequently unavailable, and the right response is to record the absence rather than guess. Model name is the worst: most consumer surfaces expose a marketing label rather than a build identifier, and it can stay constant across a change that moves your numbers. Search state is second: some surfaces decide per query whether to retrieve and never say so, so the honest entry is a proxy, whether the answer carried any citations at all, marked as one. Locale is third: what you requested and what the engine inferred can differ, so store both.

There is also a whole category answer-side logging cannot see. An engine that consults a source without citing it leaves no trace in your log, and the distinction is not academic: the 4,706-query audit in Findings of ACL 2026 measures the domains AI Overviews consults and reports 53% of them absent from the organic top 10, on data collected in September 2025, while a 55,393-query study collected 13 March to 21 April 2026 measures reference domains and reports 29.8% absent from the first page.⁸⁵ Different verbs, different denominators, different dates, and a gap wide enough that quoting either as a point estimate would be wrong. Your log only ever sees the citing half.

Your server log is the complement: it records fetches rather than answers, so it catches the retrieval failures your panel cannot see. That belongs to the measuring AI visibility stage, along with which of these metrics hold up; panel size belongs to the companion article.

The honest limit of this article

This schema is a convention assembled from one survey's checklist plus operating experience, not a standard. Nobody has validated that these ten fields are sufficient, or any one necessary, and no shared format exists that would let two teams compare their logs directly. The fidelity column is the weakest part: no published work reports how consistently two readers score the same answer for this task, so treat a fidelity rate as an internal tripwire rather than a number to publish.

Where a product fits, and where it does not

A spreadsheet with these columns is a complete implementation, worth building first even if you later buy something: the schema is where the thinking is, the collection is only labour. What tooling changes is whether the boring columns get filled at a thousand runs a week. Status, timestamp, locale and the full cited-URL list are the fields a human under time pressure stops recording, and the ones that make a denominator defensible. Bavior runs a fixed prompt set across five engines on a schedule and records which sources each answer cited, keeping the null runs in the record. It cannot expose a model identifier an engine does not publish, cannot tell you whether the sentence about your product is true, and does not decide your denominator rule for you. The free AI visibility check and the free GEO audit run without a paid plan; paid plans are from $99/mo billed monthly, or $79.17/mo billed annually (as of 29 Aug 2026).

Sources, all checked 29 Aug 2026

“Optimizing Visibility in Generative Engines: A Critical Survey of Generative Engine Optimization (2023–2026)”, 15 Jul 2026, arXiv:2607.14035; Table 6 (minimum checklist for a GEO study) and §7.5: arxiv.org/abs/2607.14035
Tow Center for Digital Journalism, Columbia, “How ChatGPT Search (Mis)represents Publisher Content”, Nov 2024; two hundred quotes, 153 incorrect responses, uncertainty signalled seven times: cjr.org
Jaźwińska & Chandrasekar, “AI Search Has a Citation Problem”, Tow Center, 6 Mar 2025; sixteen hundred queries, eight generative search tools; “incorrect answers to more than 60 percent of queries”: cjr.org
Liu et al., 2023; 51.5% of sentences in generative-search answers fully supported by their citations (reported in the critical survey at note 1)
Xu, Iqbal & Montgomery, 2026; 55,393 trending queries, 13 March to 21 April 2026; 11.0% of 98,020 atomic claims unsupported by the cited pages; “29.8% of AIO-cited domains do not appear in those first-page results” (preprint): arxiv.org/abs/2605.14021
Vykopal et al., 2026; credible-source shares of 71.4–86.3% depending on assistant and topic (reported in the critical survey at note 1)
Grossman et al., SIGIR 2026; 11,500 queries; “Jaccard similarities between 0.11 and 0.18” across Google organic, AI Overviews and Gemini: arxiv.org/abs/2604.27790
Kirsten et al., Findings of ACL 2026; 4,706-query audit, data collected September 2025; “on average 53% (27%) of domains that AIO consults are not contained in top-10 (top-100) Organic search results”: aclanthology.org/2026.findings-acl.526
Sielinski, “Quantifying Uncertainty in AI Visibility”, Mar 2026 (preprint): arxiv.org/abs/2603.08924
Google Search Central, “AI features and your website”, updated 10 Dec 2025 (surfaces, eligibility; first-party): developers.google.com/search/docs/appearance/ai-features
Vendor study, June 2026; 115 prompts, 3,981 domain appearances, four platforms, fourteen countries; 62% of appearances were links whose brand the answer text never named, and the mention-to-citation ratio reverses between assistants. Vendor-published; described rather than linked, per this curriculum's sourcing rule.

Continue reading

Learn GEO GEO roadmap Learn GEO Previous: Panel size Learn GEO Next: Retiring a prompt

FAQ

Frequently asked questions.

Do I really need to log the runs where I did not appear?

Yes, and they are the more important half of the log. A presence rate is a fraction whose denominator is every run you attempted, so an absence is data and a deleted absence is a corrupted metric. The failure that actually bites teams is subtler than deleting non-appearances: it is dropping the runs that errored. If 25 of 200 collections time out and the pipeline discards them, a 30.0% presence rate becomes 34.3% with no change in the world, and the shift is larger than half the confidence band the panel carries.

What is the difference between a mention and a citation?

A mention is your brand name appearing in the answer's prose; a citation is one of your URLs being attached to it, whether or not the prose names you. They diverge substantially and they mean different things. A link with no name means retrieval worked and identification failed, usually because the quoted passage was useful without saying who was speaking, which is a content fix. A name with no link means the model is drawing on what it already knows rather than what it just retrieved, which on-page work will not move quickly. Report both, per engine.

Should I log the URLs cited for competitors too?

Log every cited URL in the answer, not only yours, because that list is the most actionable artefact the panel produces. Your own presence tells you whether you are in the answer; the full list tells you who is, which converts a visibility number into a map of the pages that actually own your category's questions. On commercial questions much of that list will be third-party write-ups and community threads rather than any vendor's own pages, which is the finding that decides whether the fix belongs to your content team or to off-site work.

The engine will not tell me which model answered. What do I record?

Record what the interface displayed, mark it as a label rather than a build identifier, and treat any change in it as a hypothesis rather than a fact. Most consumer surfaces expose a marketing name that can stay constant across an underlying change, so the field is weak evidence in both directions: a stable label does not prove stability, and a changed label does not prove the change caused what you observed. The value of logging it anyway is that it gives you something to check first when a rate steps by more than your confidence band.

Bavior Editorial

The team that researches and maintains Bavior’s writing on Reddit marketing and AI search visibility. Every figure here is attributed to a named source with the date it was checked, and none of our links are affiliate links.

Found a number that looks wrong? Tell us and we will re-check it: support@bavior.com