Crawl Export Comparator

Free, no signup. Compare two crawl exports without forcing unlike vendor issue taxonomies into fake equivalence. Shared page evidence is normalized; unmapped columns and incomplete crawl coverage remain visible.

Built by Patrick Stox

Runs entirely in your browser — nothing you paste is uploaded or stored. Both files stay in your browser. Files over 15 MiB and rows beyond the 100,000-row comparison cap are rejected or reported as truncated. Anonymous run-level outcome counters may be used for aggregate research; URLs, domains, IPs, and identifiers are never included, and no statistic is released below 100 runs.

Feedback
Report a bug

Found something broken in Crawl Export Comparator? Let us know what happened — this goes straight to a private triage queue, not a public list.

What will be sent 
 No tool inputs, uploads, pasted source, complete results, query parameters, or URL fragments are attached automatically. You can edit or remove the selected passage above. Browser and anti-abuse metadata is processed for spam prevention. 
  
 
  
 
 
  
 
 
   
 
 
Sample report Static output-shape example
 
5 changed URLs · 2 added · 2 removed · 1 field-changed
/products/:number — 1 added, 1 removed; verify whether product IDs changed or the crawl scopes differed.
https://example.com/ — meta description and word count changed.
 
This illustrates the report structure. It is not a live observation or a claim that every difference is a defect.
 
  
 
How to use it
 
  1. Export page-level CSV or TSV files from the before and after crawls.
  2. Use comparable crawl scope, robots, rendering, authentication, and limits.
  3. Load both files and review the detected vendor, mapped fields, ignored columns, invalid URLs, duplicates, and caps.
  4. Triage inventory changes by path template, then inspect field changes per URL.
  5. Export the normalized change list and verify important regressions in the source crawlers or on the live pages.
 
 
     
 Local data 
Saved targets, named lists, and recent check summaries remain only in this browser.
 
   
 
     
 
How it works
 
The browser parses quoted CSV or TSV, detects known vendor headers, normalizes public HTTP(S) URLs, and keeps the last row when an export contains duplicate URLs. Only fields present in both exports are compared. Numeric evidence is compared numerically; canonicals and content types receive bounded normalization. The path-template rollup replaces only numeric IDs, UUIDs, and long hexadecimal segments.
 
What the results mean
 
  • Added — the normalized URL appears only in the after export.
  • Removed — it appears only in the before export.
  • Changed — at least one mutually mapped evidence field differs.
  • Ignored column — the source data is preserved in the original file but is not given cross-vendor meaning here.
 
Limitations
 
Different crawl configurations are the largest confounder. This tool does not reproduce rendering modes, authentication, robots behavior, custom extraction, vendor scoring, crawl scheduling, or site completeness. It does not infer that two differently named vendor checks mean the same thing. Template grouping is path-shape grouping, not a learned CMS template classification. The report is triage evidence, not proof of a regression.
 
Frequently asked questions
Which crawl exports can I compare?
The tool recognizes common page-export headers from Screaming Frog, Sitebulb, Scout Site Audit Free, and generic CSV or TSV files. It needs a URL column and compares only fields recognized in both files.
Are vendor issue names treated as equivalent?
No. Vendor-specific issue taxonomies can have different definitions and thresholds. The comparator normalizes shared page evidence such as status, indexability, title, canonical, H1, word count, depth, and inlinks. Issue labels are compared as literal sorted text only when both files provide them.
Does an added or removed URL prove a crawlability change?
No. A URL can appear or disappear because of crawl scope, limits, authentication, robots rules, settings, or a real site change. Confirm comparable crawl configurations before treating inventory differences as regressions.
Are the exports uploaded?
No. Parsing, normalization, comparison, filtering, and CSV export happen in your browser. The selected files are not sent to this site.
How are path templates created?
The grouping replaces numeric IDs, UUIDs, and long hexadecimal tokens in path segments. It is a deterministic triage aid, not a CMS-aware template detector.
 
 
Next stepXML Sitemap Generator — generate the corrected version.

Explore more
ToolHTTP Status & Redirect CheckerToolCMS & Tech DetectorToolHTTP Header CheckerArticleHow to Audit Hreflang at ScaleArticleEcommerce SEO AuditArticleHTTPS for SEO
 
 
 
Feature requests for Crawl Export Comparator
 

Upvote what you want most. New ideas can be submitted from the floating Feedback menu;
      requests appear here once approved, and the most-wanted rise to the top.

   
Loading…
  
  ➕ Request a feature   
New requests are reviewed before they appear here.