言語

Crawl Export Comparator

Free, no signup. Compare two crawl exports without forcing unlike vendor issue taxonomies into fake equivalence. Shared page evidence is normalized; unmapped columns and incomplete crawl coverage remain visible.

Built by Patrick Stox

Runs entirely in your browser — nothing you paste is uploaded or stored. Both files stay in your browser. Files over 15 MiB and rows beyond the 100,000-row comparison cap are rejected or reported as truncated. Anonymous run-level outcome counters may be used for aggregate research; URLs, domains, IPs, and identifiers are never included, and no statistic is released below 100 runs.

Feedback
Report a bug

Found something broken in Crawl Export Comparator? Let us know what happened — this goes straight to a private triage queue, not a public list.

What will be sent 
 No tool inputs, uploads, pasted source, complete results, query parameters, or URL fragments are attached automatically. You can edit or remove the selected passage above. Browser and anti-abuse metadata is processed for spam prevention. 
  
 
  
 
 
  
 
 
   
 
 
Sample report Static output-shape example
 
5 changed URLs · 2 added · 2 removed · 1 field-changed
/products/:number — 1 added, 1 removed; verify whether product IDs changed or the crawl scopes differed.
https://example.com/ — meta description and word count changed.
 
This illustrates the report structure. It is not a live observation or a claim that every difference is a defect.
 
  
 
How to use it
 
  1. Export page-level CSV or TSV files from the before and after crawls.
  2. Use comparable crawl scope, robots, rendering, authentication, and limits.
  3. Load both files and review the detected vendor, mapped fields, ignored columns, invalid URLs, duplicates, and caps.
  4. Triage inventory changes by path template, then inspect field changes per URL.
  5. Export the normalized change list and verify important regressions in the source crawlers or on the live pages.
 
 
     
 Local data 
Saved targets, named lists, and recent check summaries remain only in this browser.
 
   
 
     
 
How it works
 
The browser parses quoted CSV or TSV, detects known vendor headers, normalizes public HTTP(S) URLs, and keeps the last row when an export contains duplicate URLs. Only fields present in both exports are compared. Numeric evidence is compared numerically; canonicals and content types receive bounded normalization. The path-template rollup replaces only numeric IDs, UUIDs, and long hexadecimal segments.
 
What the results mean
 
  • Added — the normalized URL appears only in the after export.
  • Removed — it appears only in the before export.
  • Changed — at least one mutually mapped evidence field differs.
  • Ignored column — the source data is preserved in the original file but is not given cross-vendor meaning here.
 
Limitations
 
Different crawl configurations are the largest confounder. This tool does not reproduce rendering modes, authentication, robots behavior, custom extraction, vendor scoring, crawl scheduling, or site completeness. It does not infer that two differently named vendor checks mean the same thing. Template grouping is path-shape grouping, not a learned CMS template classification. The report is triage evidence, not proof of a regression.
 
Frequently asked questions
Which crawl exports can I compare?
The tool recognizes common page-export headers from Screaming Frog, Sitebulb, Scout Site Audit Free, and generic CSV or TSV files. It needs a URL column and compares only fields recognized in both files.
Are vendor issue names treated as equivalent?
No. Vendor-specific issue taxonomies can have different definitions and thresholds. The comparator normalizes shared page evidence such as status, indexability, title, canonical, H1, word count, depth, and inlinks. Issue labels are compared as literal sorted text only when both files provide them.
Does an added or removed URL prove a crawlability change?
No. A URL can appear or disappear because of crawl scope, limits, authentication, robots rules, settings, or a real site change. Confirm comparable crawl configurations before treating inventory differences as regressions.
Are the exports uploaded?
No. Parsing, normalization, comparison, filtering, and CSV export happen in your browser. The selected files are not sent to this site.
How are path templates created?
The grouping replaces numeric IDs, UUIDs, and long hexadecimal tokens in path segments. It is a deterministic triage aid, not a CMS-aware template detector.
 
 
Next stepXML Sitemap Generator — generate the corrected version.

People also explore
ToolHTTP Status & Redirect CheckerToolCMS & Tech DetectorToolHTTP Header CheckerArticleHow to Audit Hreflang at ScaleArticleEcommerce SEO AuditArticleHTTPS for SEO
 
 
 
Feature requests for Crawl Export Comparator
 

Upvote what you want most. New ideas can be submitted from the floating Feedback menu;
      requests appear here once approved, and the most-wanted rise to the top.

   
Loading…
  
  ➕ Request a feature   
New requests are reviewed before they appear here.
 
     
  
  
 
 
 
 
     
    
ツールについて
Screaming Frog、Sitebulb、Scout Site Audit Free、または汎用CSV／TSVの2つのクロールエクスポートを、URL、フィールド、パステンプレートでローカル比較します。
同じクロール範囲、robots、レンダリング、認証、上限のファイルを読み込み、URLごとの追加・削除・変更と証拠を確認します。
機能
  • ベンダー形式を判定するCSV／TSVパーサー
  • 共通フィールドだけを正規化して比較
  • URLとパステンプレート別の変更一覧
  • ブラウザー内のエクスポートとプライバシー保護
仕組み
引用付きCSVまたはTSVをブラウザーで解析し、既知のベンダーヘッダーを判定してHTTP(S) URLを正規化します。重複URLは最後の行を保持し、両方に存在するフィールドだけを比較します。数値、canonical、content typeを制限付きで正規化し、パスの数値ID、UUID、長い16進文字列だけをテンプレートに置き換えます。
制限事項
  • 異なるクロール設定、認証、robots、レンダリング、抽出項目、スケジュール、サイト完全性は再現しません。ベンダーの異なる問題名を同一の意味とは推測せず、パステンプレートはCMS分類ではありません。レポートは回帰の証明ではなくトリアージ用の証拠です。
よくある質問
どのクロールエクスポートを比較できますか？
Screaming Frog、Sitebulb、Scout Site Audit Free、または汎用CSV／TSVを比較できます。URL列が必要で、両方で認識されたフィールドだけを使います。
エクスポートはアップロードされますか？
いいえ。解析、正規化、比較、CSV出力はブラウザー内で行われ、選択したファイルはこのサイトへ送信されません。
追加または削除されたURLはクロール可能性の変化を証明しますか？
いいえ。範囲、上限、認証、robots、設定、実際のサイト変更で差が生じます。比較可能な設定を確認してから回帰として扱ってください。