Sprachen

Crawl Export Comparator

Free, no signup. Compare two crawl exports without forcing unlike vendor issue taxonomies into fake equivalence. Shared page evidence is normalized; unmapped columns and incomplete crawl coverage remain visible.

Built by Patrick Stox

Runs entirely in your browser — nothing you paste is uploaded or stored. Both files stay in your browser. Files over 15 MiB and rows beyond the 100,000-row comparison cap are rejected or reported as truncated. Anonymous run-level outcome counters may be used for aggregate research; URLs, domains, IPs, and identifiers are never included, and no statistic is released below 100 runs.

Feedback
Report a bug

Found something broken in Crawl Export Comparator? Let us know what happened — this goes straight to a private triage queue, not a public list.

What will be sent 
 No tool inputs, uploads, pasted source, complete results, query parameters, or URL fragments are attached automatically. You can edit or remove the selected passage above. Browser and anti-abuse metadata is processed for spam prevention. 
  
 
  
 
 
  
 
 
   
 
 
Sample report Static output-shape example
 
5 changed URLs · 2 added · 2 removed · 1 field-changed
/products/:number — 1 added, 1 removed; verify whether product IDs changed or the crawl scopes differed.
https://example.com/ — meta description and word count changed.
 
This illustrates the report structure. It is not a live observation or a claim that every difference is a defect.
 
  
 
How to use it
 
  1. Export page-level CSV or TSV files from the before and after crawls.
  2. Use comparable crawl scope, robots, rendering, authentication, and limits.
  3. Load both files and review the detected vendor, mapped fields, ignored columns, invalid URLs, duplicates, and caps.
  4. Triage inventory changes by path template, then inspect field changes per URL.
  5. Export the normalized change list and verify important regressions in the source crawlers or on the live pages.
 
 
     
 Local data 
Saved targets, named lists, and recent check summaries remain only in this browser.
 
   
 
     
 
How it works
 
The browser parses quoted CSV or TSV, detects known vendor headers, normalizes public HTTP(S) URLs, and keeps the last row when an export contains duplicate URLs. Only fields present in both exports are compared. Numeric evidence is compared numerically; canonicals and content types receive bounded normalization. The path-template rollup replaces only numeric IDs, UUIDs, and long hexadecimal segments.
 
What the results mean
 
  • Added — the normalized URL appears only in the after export.
  • Removed — it appears only in the before export.
  • Changed — at least one mutually mapped evidence field differs.
  • Ignored column — the source data is preserved in the original file but is not given cross-vendor meaning here.
 
Limitations
 
Different crawl configurations are the largest confounder. This tool does not reproduce rendering modes, authentication, robots behavior, custom extraction, vendor scoring, crawl scheduling, or site completeness. It does not infer that two differently named vendor checks mean the same thing. Template grouping is path-shape grouping, not a learned CMS template classification. The report is triage evidence, not proof of a regression.
 
Frequently asked questions
Which crawl exports can I compare?
The tool recognizes common page-export headers from Screaming Frog, Sitebulb, Scout Site Audit Free, and generic CSV or TSV files. It needs a URL column and compares only fields recognized in both files.
Are vendor issue names treated as equivalent?
No. Vendor-specific issue taxonomies can have different definitions and thresholds. The comparator normalizes shared page evidence such as status, indexability, title, canonical, H1, word count, depth, and inlinks. Issue labels are compared as literal sorted text only when both files provide them.
Does an added or removed URL prove a crawlability change?
No. A URL can appear or disappear because of crawl scope, limits, authentication, robots rules, settings, or a real site change. Confirm comparable crawl configurations before treating inventory differences as regressions.
Are the exports uploaded?
No. Parsing, normalization, comparison, filtering, and CSV export happen in your browser. The selected files are not sent to this site.
How are path templates created?
The grouping replaces numeric IDs, UUIDs, and long hexadecimal tokens in path segments. It is a deterministic triage aid, not a CMS-aware template detector.
 
 
Next stepXML Sitemap Generator — generate the corrected version.

People also explore
ToolHTTP Status & Redirect CheckerToolCMS & Tech DetectorToolHTTP Header CheckerArticleHow to Audit Hreflang at ScaleArticleEcommerce SEO AuditArticleHTTPS for SEO
 
 
 
Feature requests for Crawl Export Comparator
 

Upvote what you want most. New ideas can be submitted from the floating Feedback menu;
      requests appear here once approved, and the most-wanted rise to the top.

   
Loading…
  
  ➕ Request a feature   
New requests are reviewed before they appear here.
 
     
  
  
 
 
 
 
     
    
Über das Tool
Normalisiere zwei Screaming-Frog-, Sitebulb-, Scout-Site-Audit- oder generische CSV-/TSV-Crawl-Exporte lokal und prüfe hinzugefügte, entfernte und feldgeänderte URLs. Unkartierte Spalten, ungültige URLs, Duplikate und unvollständige Abdeckung bleiben sichtbar; Pfadvorlagen dienen nur als deterministische Triagehilfe.
Ein Unterschied ist keine bewiesene Regression. Nutze vergleichbare Crawl-Konfigurationen und verifiziere wichtige Befunde im Quell-Crawler oder auf der Live-Seite.
Funktionen
  • Erkennung gängiger Vendor-Header und URL-Normalisierung
  • Vergleich gemeinsamer Seitenfelder wie Status, Indexierbarkeit, Title, Canonical, H1, Wortzahl, Tiefe und Inlinks
  • Gruppierung hinzugefügter, entfernter und geänderter URLs nach Pfadvorlage
  • Browserlokale Verarbeitung und CSV-Export der normalisierten Änderungsliste
Funktionsweise
Exportiere Seiten-CSV oder TSV vor und nach dem Crawl mit vergleichbarem Scope, Robots, Rendering, Authentifizierung und Limits. Lade beide Dateien, prüfe Vendor-Erkennung, gemappte Felder, ignorierte Spalten, ungültige URLs, Duplikate und Caps. Triage die Inventaränderungen nach Pfadvorlage, inspiziere Feldänderungen pro URL und exportiere anschließend die normalisierte Änderungsliste.
Einschränkungen
  • Unterschiedliche Rendering-, Authentifizierungs-, Robots-, Extraktions-, Scoring- und Crawl-Einstellungen werden nicht reproduziert.
  • Vendor-spezifische Issue-Taxonomien werden nicht als äquivalent angenommen; Issue-Labels werden nur als sortierter Text verglichen, wenn beide Dateien sie liefern.
  • Ein hinzugefügter oder entfernter URL beweist keine Crawlability-Änderung; Konfiguration, Limits, Auth, Robots oder echte Site-Änderungen können Ursachen sein.
Häufig gestellte Fragen
Welche Crawl-Exporte kann ich vergleichen?
Der Comparator erkennt häufige Seitenexporte von Screaming Frog, Sitebulb, Scout Site Audit Free und generische CSV-/TSV-Dateien. Eine URL-Spalte ist erforderlich; verglichen werden nur Felder, die beide Dateien erkennen.
Werden Vendor-Issue-Namen als gleich behandelt?
Nein. Unterschiedliche Taxonomien können andere Definitionen und Schwellen haben. Gemeinsame Seitenevidenz wird normalisiert, Issue-Labels nur als wörtlich sortierter Text verglichen, wenn beide Exporte sie enthalten.
Wie entstehen Pfadvorlagen?
Numerische IDs, UUIDs und lange Hex-Tokens in Pfadsegmenten werden ersetzt. Das ist eine deterministische Triagehilfe, kein CMS-bewusster oder gelernter Template-Detektor.
Beweist eine hinzugefügte oder entfernte URL eine Crawlability-Änderung?
Nein. Scope, Limits, Authentifizierung, Robots, Einstellungen oder echte Site-Änderungen können die Differenz erklären. Bestätige vergleichbare Konfigurationen, bevor du eine Regression annimmst.