AI Search Evidence Lab

Turn repeated AI-answer captures into evidence you can audit: exact denominators, citation persistence, topic-specific source patterns, prompt coverage, fact conflicts, URL failures, and bounded intervention results.

Built by Patrick Stox

Example data — fabricated observations that demonstrate repeated citations, a broken URL, a fact conflict, and a correction.

Bundled methodology and source registries last verified Jul 28, 2026.

Runs entirely in your browser — nothing you paste is uploaded, and data is saved locally only when you explicitly choose the browser-save action. Clear saved removes that browser copy without changing the text currently in the editor. Anonymous run-level outcome counters may be used for aggregate research; URLs, domains, IPs, and identifiers are never included, and no statistic is released below 100 runs.

Feedback
Report a bug

Found something broken in Ai Search Evidence Lab? Let us know what happened — this goes straight to a private triage queue, not a public list.

What will be sent 
 No tool inputs, uploads, pasted source, complete results, query parameters, or URL fragments are attached automatically. You can edit or remove the selected passage above. Browser and anti-abuse metadata is processed for spam prevention. 
  
 
  
 
 
  
 
 
   
 
     
 Local data 
Saved targets, named lists, and recent check summaries remain only in this browser.
 
   
 
      
 
What this adds to AI-search measurement
 
     
  • Observation provenance: exact prompt version, surface, model/checkpoint, retrieval mode, sample group, date, and extraction method.
    •  
  • Passage-level evidence: distinguish a known cited URL from a citation whose supporting passage was actually captured.
    •  
  • Repeatability: persistence and 95% Wilson intervals instead of a single-run visibility label.
    •  
  • Source influence: repeated citations grouped by normalized domain, with prompt and surface breadth shown separately.
    •  
  • Correction fixtures: human overrides remain attached to the original observation and classifier version.
    •  
  • Intervention discipline: before/after panels remain directional unless compatible control evidence exists.
    •  
 
Packet structure
 
The top-level object accepts observations, corrections, and interventions. Historical records should be append-only. Change a prompt by creating a new version; change a classifier or denominator by creating a new methodology version. Do not rewrite earlier captures.
 
Required observation fields
id, capturedAt, promptId, promptVersion, sampleGroup, sampleOrdinal, source, provider, surface, retrievalMode, evaluationState, extractorVersion, methodologyVersion, evidenceState
 
How to use it
 
     
  1. Capture repeated answers under the same prompt version, surface, retrieval mode, locale, and methodology.
    2.  
  2. Include citations and their passages when the source exposes them. Use a passage hash when retention rules prevent storing the text.
    3.  
  3. Paste or upload the normalized packet and review excluded observations before reading rates.
    4.  
  4. Investigate fact conflicts and broken URLs manually. A conflict shows disagreement, not which value is correct.
    5.  
  5. When evaluating an edit, preserve the deployment and recrawl dates and add a comparable control panel when possible.
    6.  
 
Limitations
 
The lab analyzes supplied evidence; it cannot see undisclosed retrieval, internal ranking, cached context, or every answer shown to every user. Citation frequency is not rank, authority, causal influence, traffic, or conversion. A verified crawler request proves that request, not later use in an answer. Human corrections are counted but never silently rewrite raw evidence.
 
Frequently asked questions
 
Does this query AI systems for me?
No. It analyzes observation packets you already captured or exported. Keeping acquisition separate prevents an API execution from being mislabeled as a consumer-product result.
What counts in the denominator?
Only evaluated observations with usable evidence. Refusals, provider failures, and not-evaluated records remain visible but do not become zeroes.
How many repeats do I need?
The ai-search-evidence-v1 contract requires at least three compatible observations before applying a stability label and prefers five. Even then, the result is a sample, not a universal rank.
How does the topic-specific source map work?
Add version-matched prompt records with topic labels. The report groups observed citation domains by those supplied labels and preserves citation appearances, distinct prompt breadth, surface breadth, and captured-passage counts. It does not infer topical authority.
Does a before-and-after increase prove my edit caused it?
No. An uncontrolled change is directional evidence. A compatible control panel makes the inference stronger, but the tool still avoids causal language.
Is pasted data uploaded or stored?
Analysis runs in your browser. Nothing is saved unless you explicitly choose Save in this browser; you can clear that local copy at any time.
 
 
Next stepQuotability & Entity-Preserving Rewriter — generate the corrected version.

People also explore
ToolAI Citation Readiness CheckerToolCitation Gap CheckerToolAI Content AuditArticleAI Search Measurement MethodologyArticleRetrieved vs. Mentioned vs. Cited in AIArticleAI Search Measurement and Reporting
 
 
 
Feature requests for Ai Search Evidence Lab
 

Upvote what you want most. New ideas can be submitted from the floating Feedback menu;
      requests appear here once approved, and the most-wanted rise to the top.

   
Loading…
  
  ➕ Request a feature   
New requests are reviewed before they appear here.