Crawl Depth
Crawl depth means two things — the click depth of a page (clicks from the homepage) and a crawler's depth-limit setting. Why depth comes from internal links not URL slashes, whether the "3-click rule" is real, and how to measure and reduce it.
Crawl depth is used two ways: a page's click depth (how many clicks from the homepage to reach it) and a crawler's depth-limit setting (how many levels deep a crawl goes before it stops). Depth is defined by internal links, not URL folders — Mueller: 'we don't count slashes in the URLs.' Pages close to the homepage are discovered faster, crawled more often, and read as more important. The '3-click rule' is folklore: Bing names three clicks, Google never has, and UX research finds the count alone doesn't predict success. Reduce depth with internal links, hub pages, and tidy navigation — but don't flatten everything, or Google reads it all as equally important.
TL;DR — Crawl depthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops. usually means click depth: how many clicks it takes to reach a page from your homepage. The homepage is 0, a page linked straight from it is 1, and so on. Pages close to the homepage get foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore. faster and crawled more often, so keep the pages you care about a few clicks from home. The famous “everything within 3 clicks” rule is a guideline, not a Google law.
What crawl depth means
When most people say “crawl depth,” they mean click depth: the number of clicks it takes to get from your homepage to a given page by following links.
- Your homepage is depth 0.
- A page linked directly from the homepage, your main menu, or your footer is depth 1.
- A page you can only reach by clicking through to that page first is depth 2, and so on.
So crawl depth is really a measure of how buried a page is. A page nobody links to at all is an “orphan” — effectively infinitely deep, because there’s no path to it. Evidence for this claim Google recommends crawlable anchor elements with href attributes and says every important page should be linked from another page. Scope: Google link discovery; it does not establish a universal click-depth threshold. Confidence: high · Verified: Google: Link best practices
Why it matters
Search engines find pages by following links. The closer a page is to your homepage, the sooner a crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. stumbles onto it and the more often it comes back to check for changes. Bury something ten clicks deep and it may get found slowly, crawled rarely, or — if nothing links to it — never found at all.
There’s a second reason. Google treats distance from the homepage as a hint about importance. As Google’s John Mueller put it, what matters is “how easy it is to actually find the content.” If a page is one click from home, that signals it’s a page you consider important.
The thing people get wrong
Crawl depth is about links, not your URL. A page at
yoursite.com/a/b/c/d/product/ that’s linked from your homepage is one click
deep. A page with a short, tidy URL that’s buried six clicks down is six deep.
Mueller said it directly: Google doesn’t “count slashes in the URLs.” Don’t
restructure your folders to fix depth — fix where you link. Evidence for this claim Google's URL guidance addresses logical, descriptive URL structure but does not equate folder count with click depth. Scope: Google URL structure guidance. Confidence: high · Verified: Google: URL structure best practices
And the “3-click rule” — the idea that everything should be reachable within three clicks — is folklore, not a Google rule. It’s a sensible aim, and Bing does suggest keeping important pages within about three clicks, but Google has never named a number, and usability research has found the click count alone doesn’t decide whether people succeed.
Want the tool-by-tool detail, the flat-vs-pyramid argument, and the crawl-budget link? Switch to the Advanced tab.
TL;DR — “Crawl depthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops.” conflates two things: a page’s click depth (clicks from the homepage following internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them.) and a crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. depth-limit setting (how many levels a crawl traverses before stopping). Depth is defined by links, not URL folders — Mueller: “From our point of view we don’t count slashes in the URLs.” Shallow pages are discovered, crawled, and weighted as more important; deep/orphan pagesAn orphan page is a page on your site that no other page links to internally. Because crawlers discover pages by following links, an orphan page is effectively invisible to search engines unless it's in an XML sitemap or linked from an external site. aren’t. The “3-click rule” is folklore — Bing names three, Google names none, NN/g foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore. the count alone doesn’t predict success. Aim for a pyramid: important pages shallow, but don’t flatten everything (Mueller: then “we think, oh all of these are equally important”). Reduce depth with internal links, hubs, breadcrumbsBreadcrumbs are a secondary navigation trail (Home > Category > Page) that shows where a page sits in a site's hierarchy. They create internal links that pass PageRank, and when marked up with BreadcrumbList structured data they can drive the path Google shows in desktop search results., and cleaner nav; watch redirectsA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. and JS-only links that inflate effective depth.
Two meanings, defined separately
The term is used two ways, and most articles quietly switch between them. Pin them down first:
-
Click depth (page depth). The number of clicks to reach a page from the homepage — or from the crawl’s start URL — following internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them.. Homepage is depth 0; a page linked from the homepage/nav/footer is depth 1. This is a property of an individual page describing how buried it is. It’s what Screaming Frog’s “Crawl Depth” column and Ahrefs Site Audit’s “Depth” report.
-
Crawl depth as a crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. setting. A configurable limit on how many levels deep a crawler follows links before it stops. In Ahrefs Site Audit this is the “maximum click depth” — set it to 4 and pages that can’t be reached within four clicks won’t be crawled at all.
The two interlock: a page’s click depth determines whether a depth-limited crawl ever reaches it, and a deeper page is generally discovered less reliably and crawled less often.
Precisely, a page’s click depth is the length of the shortest path from your crawl’s chosen start URL to that page, through links the crawler counts as crawlable. That makes depth relative to whatever you seed the crawl with — start from the homepage and you get homepage-relative depth; add sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. URLs, a URL list, or other seeds, and the numbers stop being comparable to a homepage-only crawl. State what you seeded from before you compare depth numbers.
Depth is about internal links, not URL folders
This is the most common confusion, so it’s worth stating bluntly. Depth comes from the link path, not the URL string. Mueller, in a 2018 Webmaster Central hangout:
“From our point of view we don’t count slashes in the URLs.”
A page at /a/b/c/d/page/ that’s linked from the homepage is depth 1. A page at
/page/ that’s six clicks down the link graph is depth 6. Google’s own URL-structure
guidance asks you to keep URLs logical and readable — but it never equates folder
depth with click depth. So don’t flatten your folder structure to “reduce depth”;
change where pages are linked from. Evidence for this claim Google's URL guidance addresses logical, descriptive URL structure but does not equate folder count with click depth. Scope: Google URL structure guidance. Confidence: high · Verified: Google: URL structure best practices
A corollary: Google only follows links it can see. Per Search Central, “Google can
only crawl your link if it’s an <a> HTML element with an href attribute.” A
“link” that’s really an onclick handler or a <button> with no href may not be
followed — so a page you think is one click deep via a JavaScript menu can be far
deeper (or undiscoverable) in practice. The same goes for template and device
variants: a desktop mega-menu and a collapsed mobile nav don’t always expose the
same links, so a depth audit should say which rendered version — and which
device — it measured. In my How Search Works talk I make the
same point about Google’s link parsing missing JS-driven navigation; it’s a real way
to accidentally inflate effective depth. Evidence for this claim Google recommends crawlable anchor elements with href attributes and says every important page should be linked from another page. Scope: Google link discovery; it does not establish a universal click-depth threshold. Confidence: high · Verified: Google: Link best practices
Why crawl depth matters for SEO
Five reasons, roughly in order of how concrete they are:
- Discovery. Crawlers find pages by extracting links. A deep page is found later; an orphan (no internal links) may never be found. Google’s link guidance is blunt about the fix: “Every page you care about should have a link from at least one other page on your site.”
- Crawl frequencyCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial.. Pages closer to the homepage tend to be crawled more often. Depth feeds into how a crawler prioritizes, which ties into crawl budgetThe number of URLs an engine will crawl in a timeframe. — though for most sites that budget isn’t a constraint to begin with.
- Importance signal. Google infers importance from proximity to the homepage. Mueller: “What does matter for us a little bit is how easy it is to actually find the content.” If something is one click from home, that’s a vote for its importance.
- Link equity. PageRankPageRank is Google's original recursive link-graph algorithm: a page's score depends on the scores of the pages linking to it, and in the published model each page's score is split across its outbound links (the simplified version: links are weighted votes). Google says it's evolved since launch but still part of its core ranking systems. flows through internal links. Shallow, well-linked pages accumulate more of it; pages stranded at the bottom of the graph get less.
- User experience. People also find things faster when they aren’t ten clicks deep. The SEO and UX cases point the same direction here.
A useful framing from my own Beginner’s Guide to Technical SEO: “Internal links are links from one page on your site to another page on your site. They help your pages be found and also help the pages rank better.” Crawl depth is just the architectural expression of that.
Is the “3-click rule” real?
Short version: no — not as a Google rule.
The “everything within three clicks” idea is a UX rule of thumb (it traces back to the early 2000s), and it’s intuitive enough that it became SEO gospel. But:
- Google has never named a number. There’s no Google statement of a click-count threshold. What Google says is about reachability and importance, not “three.”
- Bing does name three. Bing’s webmaster guidance recommends a broad-to-specific structure so important pages are reachable, in their phrasing, “ideally within three clicks from the homepage.” So if you’ve heard “three clicks” attached to a search engine, it’s Bing, not Google.
- UX research doesn’t back the rule either. Nielsen Norman Group tested it and concluded “user dropoff does not increase when the task involves more than 3 clicks, nor does satisfaction decrease.” The number of clicks alone doesn’t predict whether people succeed; clarity of the path matters more.
So the honest framing: keep important pages shallow because it helps discovery, crawl frequencyCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial., and importance signals — not because three is a magic threshold. Treat a three- or four-click threshold as a triage heuristic — a fast way to flag important pages that might be under-linked — not a pass/fail test to apply uniformly across every URL on the site.
A note on a stat you’ll see floating around. A claim that “94% of top pages are within 3 clicks” (and that anything past five clicks is an orphan) gets repeated on a lot of SEO blogs. I couldn’t find a primary source for it, and it appears to be a garbled echo of an unrelated Ahrefs figure. I’m leaving it out rather than repeat an unsourced number.
Flat vs. deep vs. pyramid
It’s tempting to “fix” depth by flattening everything — linking thousands of pages from the homepage so nothing is far away. Don’t. Mueller’s warning about overly flat architectures: “Whereas if it’s very flat, then we think, oh all of these are equally important.” If everything is one click from home, you’ve thrown away the hierarchy Google uses to understand what’s important.
The other extreme is just as bad — burying content so deep that, in Mueller’s words, “you don’t want it to be such that it’s like you have to click through a million times to actually get to the actual content.”
His recommendation is the middle path: “So from my point of view, I think for a lot of sites it makes sense to have more of a pyramid structure.” Homepage at the top, broad category/hub pages a click down, specific pages below those. Important pages sit a few clicks from home; the hierarchy still communicates relative importance.
How to check crawl depth
Every serious crawler reports depth; the gotchas are in how they compute it.
- Screaming Frog SEO Spider. The “Crawl Depth” column is clicks from the start URL (depth 0). It flags pages with high crawl depth as an issue. Watch two things: the start URL is depth 0 (crawl from the homepage if you want homepage-relative depth), and a redirect counts as a hop — a page reached via a 301 reads one level deeper than the same page linked directly.
- Ahrefs Site Audit. Reports each page’s Depth, and the maximum click depth setting controls how deep the crawl itself goes. If you cap it, you’ve also capped the depth numbers you’ll see — a page beyond the cap shows as uncrawled, not as “very deep.”
- Semrush / Sitebulb. Both expose a crawl-depth metric and depth-based reports for the same purpose.
A useful audit doesn’t stop at “what’s the deepest page.” Bucket every page you care about by depth and by template, so a handful of intentionally deep pages don’t hide a widespread architecture problem — or a single outlier doesn’t make the whole site look worse than it is.
The general gotchas: the start URL changes every number; redirect chainsA → B → C instead of A → C. Each hop loses link equity and adds latency. add hops; JS-only links can hide the real path; the render mode and device you crawl with (JS-rendered vs. raw HTML, desktop vs. mobile nav) can expose a different link graph entirely; and the crawler’s own depth-limit setting can artificially cap reported depth — pages beyond that cap show up as uncrawled, not as proven to sit at any particular depth. Two tools can disagree on the same site purely because of these settings.
For the ground truth on what bots actually reached — versus what a simulated crawl predicts — server-side log file analysisLog file analysis is reading a web server's raw access logs to see exactly which URLs search engine crawlers actually requested, when, how often, and what status code they got. Unlike crawl tools or Search Console, logs are the unsampled, ground-truth record of what really happened. is the complement to a depth audit.
How to reduce crawl depth
In rough priority order:
- Add internal links from shallow, authoritative pages. A contextual link from a high-traffic, low-depth page is the most direct way to pull a deep page up.
- Use hub / category pages. A well-organized hub linking to its cluster keeps every member of that cluster a click or two from the hub (and the hub a click from home).
- Tidy navigation, footer, and breadcrumbsBreadcrumbs are a secondary navigation trail (Home > Category > Page) that shows where a page sits in a site's hierarchy. They create internal links that pass PageRank, and when marked up with BreadcrumbList structured data they can drive the path Google shows in desktop search results.. These are site-wide links; what you put in them defines the shallow layer. Breadcrumbs also create consistent upward link paths.
- Fix orphan pagesAn orphan page is a page on your site that no other page links to internally. Because crawlers discover pages by following links, an orphan page is effectively invisible to search engines unless it's in an XML sitemap or linked from an external site.. Give every page you care about at least one internal link.
- Handle paginationPagination splits a large set of content — product listings, blog archives, search results — across multiple sequentially numbered URLs. For SEO, each paginated page should be crawlable, indexable, and self-canonical; Google no longer uses rel=prev/next, but Bing still does. and faceted navigationFaceted navigation (faceted search, product filtering) lets visitors refine a list of products or content by attribute — price, color, size, brand, rating. The SEO problem: each filter combination can spawn a distinct crawlable URL, turning a small catalog into millions of near-duplicate pages that waste crawl budget and dilute ranking signals.. Deep paginated archives and facet combinations are the classic way products end up eight clicks deep. Favor “view all” or hub pages over forcing crawlers through endless “next” links, and don’t hide the real path behind load-more JavaScript.
- Fix redirect chains. Each hop in a chain adds a level; collapsing chains to a single 301 removes the extra depth.
- Add an HTML sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. for very large sites as a supplementary shallow path.
And the counterweight: don’t over-flatten. The goal is important pages within a few clicks, not everything one click from home. As crawl efficiency goes, you raise discovery far more by removing waste and linking smartly than by trying to make the whole site one level deep.
If you’re testing a depth change — a new hub, a nav restructure — measure it properly. Crawl before and after with the same configuration (same seed, render mode, and depth cap) so the numbers are actually comparable, decide up front which pages and templates you expect to move, and give it a defined window. Then look at crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., and rankings as separate outcomes: a shallower path doesn’t guarantee any of them moves, and a traffic change in that window isn’t proof the depth change caused it.
When crawl depth matters most
Depth is a bigger deal on:
- Large sites and e-commerce with deep faceted catalogs, where products can sink many clicks deep.
- News and archive sites where pagination pushes older content down.
- New sites, where discovery is the bottleneck and you can’t afford pages that aren’t reachable.
These are the same conditions where crawl budgetThe number of URLs an engine will crawl in a timeframe. starts to matter — the two are related but distinct: depth is about a single page’s distance and discoverability, while crawl budget is the broader rate-and-demand envelope. As I cover in my crawl budget guide, most sites never need to worry about budget at all; depth, though, is worth a quick audit on almost any site.
AI summary
A condensed take on the Advanced version:
- Two meanings. (1) A page’s click depth — clicks from the homepage following internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. (homepage = 0). (2) A crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. depth-limit setting — how many levels a crawl traverses before stopping. Don’t conflate them.
- Links, not slashes. Depth comes from the internal linkAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. path, not URL folders.
Mueller: “we don’t count slashes in the URLs.” JS-only “links” can inflate
effective depth because Google follows
<a href>links. - Why it matters: discovery (orphans don’t get foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore.), crawl frequencyCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial., importance signal (proximity to home), link-equity flow, and UX.
- The “3-click rule” is folklore. Google names no number; Bing suggests ~three clicks; NN/g found “user dropoff does not increase when the task involves more than 3 clicks.” Treat any threshold as a triage heuristic, not a pass/fail test. Avoid the unsourced “94% within 3 clicks” stat.
- Depth is seed- and render-relative. It’s the shortest path from your crawl’s chosen start URL, through links the crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. counts as crawlable — different seeds, render modes, or devices (desktop vs. mobile nav) can produce different numbers for the same site. Audit the distribution by depth and template, not just the maximum.
- Aim for a pyramid, not a pancake. Mueller: a very flat site reads as “all of these are equally important,” which dilutes hierarchy.
- Reduce depth with internal links from shallow pages, hubs, breadcrumbsBreadcrumbs are a secondary navigation trail (Home > Category > Page) that shows where a page sits in a site's hierarchy. They create internal links that pass PageRank, and when marked up with BreadcrumbList structured data they can drive the path Google shows in desktop search results., cleaner nav, fixing orphans/paginationPagination splits a large set of content — product listings, blog archives, search results — across multiple sequentially numbered URLs. For SEO, each paginated page should be crawlable, indexable, and self-canonical; Google no longer uses rel=prev/next, but Bing still does./facets, and collapsing redirect chainsA → B → C instead of A → C. Each hop loses link equity and adds latency.. When testing a change, crawl before/after with the same configuration and judge crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., and rankings as separate outcomes — don’t infer causality from traffic movement alone.
- Measure with Screaming Frog (Crawl DepthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops.), Ahrefs Site Audit (Depth + max click depth), Semrush/Sitebulb — mind start URL, redirectsA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't.-as-hops, and depth-limit settings.
- Matters most for large/e-commerce/news/new sites; related to but distinct from crawl budgetThe number of URLs an engine will crawl in a timeframe..
Official documentation
Primary-source documentation relevant to crawl depthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops. and internal-link discovery.
- Crawlable links / SEO link best practices — links as a discovery signal, the “link every page you care about” rule, and the
<a href>requirement. - URL structure best practices — keep URLs logical and readable; note it does not equate folder depth with click depth.
- Crawling and Indexing — the hub for sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing., internal linkingAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them., and crawl controls.
- Optimize your crawl budget — where depth feeds into crawl prioritization (and who actually needs to care).
Bing / Microsoft
- Bing Webmaster Guidelines — the broad-to-specific structure guidance and the “within ~three clicks” recommendation (the one search engine that names a number).
Quotes from the source
On-the-record statements relevant to crawl depthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops.. Each link jumps to the quoted passage on the source page.
John Mueller, Google — depth is links, not URLs (2018)
- “From our point of view we don’t count slashes in the URLs.” — the canonical “URL depth doesn’t matter” quote. Jump to quote
- “What does matter for us a little bit is how easy it is to actually find the content.” — depth as findability. Jump to quote
- “if it’s one click from the home page to one of these stores then that tells us that these stores are probably pretty relevant, and that probably we should be giving them a little bit of weight in the search results as well.” — homepage-proximity as an importance signal. Jump to quote
John Mueller, Google — flat vs. pyramid (2021)
- “Whereas if it’s very flat, then we think, oh all of these are equally important” — why over-flattening backfires. Jump to quote
- “you don’t want it to be such that it’s like you have to click through a million times to actually get to the actual content” — and why over-deepening is just as bad.
- “So from my point of view, I think for a lot of sites it makes sense to have more of a pyramid structure.” Jump to quote
Google Search Central docs — links and discovery
- “Google uses links as a signal when determining the relevancy of pages and to find new pages to crawl.” Jump to quote
- “Every page you care about should have a link from at least one other page on your site.” — the closest thing Google has to an orphan/depth rule. Jump to quote
Nielsen Norman Group — the “3-click rule” is false (2019)
- “user dropoff does not increase when the task involves more than 3 clicks, nor does satisfaction decrease” — Page Laubheimer, NN/g. Jump to quote
Reduce-your-crawl-depth checklist
A pass to pull important pages closer to the homepage:
- Crawl from your homepage as the start URL so depth is homepage-relative.
- Identify pages deeper than ~4 clicks that you actually care about (Screaming Frog Crawl DepthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops. / Ahrefs Site Audit Depth).
- Add contextual internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. to those pages from shallow, authoritative pages — not just from other deep pages.
- Confirm every important page has at least one internal linkAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. (no orphans).
- Use hub / category pages so each cluster sits a click or two from a hub.
- Review main nav, footer, and breadcrumbsBreadcrumbs are a secondary navigation trail (Home > Category > Page) that shows where a page sits in a site's hierarchy. They create internal links that pass PageRank, and when marked up with BreadcrumbList structured data they can drive the path Google shows in desktop search results. — these define your shallow layer.
- Check that important links are real
<a href>links, not JSonclick/<button>navigation a crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. may not follow. - Tame paginationPagination splits a large set of content — product listings, blog archives, search results — across multiple sequentially numbered URLs. For SEO, each paginated page should be crawlable, indexable, and self-canonical; Google no longer uses rel=prev/next, but Bing still does. and faceted nav — favor “view all”/hubs over forcing crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. through endless “next” links; avoid burying products many clicks deep.
- Collapse redirect chainsA → B → C instead of A → C. Each hop loses link equity and adds latency. (each hop adds a level of depth).
- Don’t over-flatten — keep a sensible pyramid, not a homepage linking to everything.
The mental models
1. Two depths, never conflate them. Click depth is a property of a page (clicks from home). Crawl depthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops. setting is a property of your crawl (how far it traverses). When a number looks wrong, ask which one you’re actually looking at.
2. Depth is a link-graph distance, not a URL string.
Picture your site as nodes connected by <a href> links. Depth is the shortest path
from the homepage to a node. Folders and slashes don’t appear in this picture at all
— Mueller: “we don’t count slashes.”
3. The pyramid, not the pancake. Homepage → hubs/categories → specific pages. Important pages stay shallow; the hierarchy still tells Google what’s important. A pancake (everything one click from home) destroys that signal: “all of these are equally important.”
4. The “3 clicks” honesty test. Three clicks is a Bing guideline and a UX rule of thumb — not a Google threshold and not empirically magic. Use “important pages reachable in a few clicks” as the goal; drop “exactly three” as a rule.
5. Depth → discovery → crawl frequencyCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial.. A shallow page is foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore. sooner and re-crawled more often; a deep/orphan pageAn orphan page is a page on your site that no other page links to internally. Because crawlers discover pages by following links, an orphan page is effectively invisible to search engines unless it's in an XML sitemap or linked from an external site. may be found late or never. Reach is the first-order effect of depth; this is the bridge to crawl budgetThe number of URLs an engine will crawl in a timeframe. on large sites.
Patrick's relevant free tools
- robots.txt Tester — Test pages against bots with a matcher ported from Google's open-source robots.txt parser — a blocked/allowed matrix with the exact winning rule per cell, file lint, sitemap-conflict detection, a diff mode for proposed changes, and a separate live robots.txt fetch for each entered origin.
- XML Sitemap Validator — Paste, upload, or fetch a sitemap by URL — errors, warnings, and a health score with line numbers. Pasted and uploaded sitemaps are validated entirely in your browser.
- XML Sitemap Generator — Generate an XML sitemap from a capped, robots-respecting same-site crawl. Noindex, off-canonical, failed, and uncertain URLs remain visibly separate; lastmod dates are emitted only when the page provides evidence.
Tools for measuring crawl depth
- Screaming Frog SEO SpiderA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. — the Crawl DepthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops. column reports clicks from the start URL (depth 0), and it flags pages with high crawl depth. Remember a redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. counts as a hop.
- Ahrefs Site Audit — each page’s Depth, plus a maximum click depth setting that controls how far the crawl itself goes (and therefore the depths you can see). Structure Explorer visualizes the architecture by depth.
- Semrush Site Audit / Sitebulb — both expose a crawl-depth metric and depth-based reports for finding buried pages.
- Server log file analysisLog file analysis is reading a web server's raw access logs to see exactly which URLs search engine crawlers actually requested, when, how often, and what status code they got. Unlike crawl tools or Search Console, logs are the unsampled, ground-truth record of what really happened. — the ground-truth complement: what bots actually reached, versus what a simulated crawl predicts. (See log file analysisLog file analysis is reading a web server's raw access logs to see exactly which URLs search engine crawlers actually requested, when, how often, and what status code they got. Unlike crawl tools or Search Console, logs are the unsampled, ground-truth record of what really happened..)
- Ahrefs Webmaster ToolsThe free, first-party consoles search engines give site owners — Google Search Console and Bing Webmaster Tools — to see how their pages are crawled, indexed, and ranked, and to fix problems. The legacy name is 'webmaster tools.' — a free Site Audit (with Depth) for sites you verify.
Whichever tool you use, note the seed URL, render mode (JS-rendered vs. raw HTML), and device (desktop vs. mobile nav) it crawled with — each can change the reported numbers — and look at the distribution of important pages by depth and template, not just the single deepest page.
Counting URL folders instead of link clicks
/a/b/c/page/ can be one click from the homepage, while /page/ can be buried behind
six links. Measure the shortest crawlable link path from a defined start URL. Do not
rewrite URLs just to make their folder shape look shallow.
Treating three clicks as a universal Google rule
Three clicks can be a useful prioritization prompt, but Google does not publish it as a hard limit. Investigate whether important pages are harder to discover and less prominent than comparable pages instead of failing every URL at depth four.
Flattening the site into a mega-menu
Linking every page from every page can reduce measured depth while destroying hierarchy and usability. A useful architecture keeps important paths short through hubs, categories, and contextual links without making every page look equally important.
Measuring from the wrong start URL
A crawl launched from an XML sitemapAn XML sitemap is a UTF-8 file listing the canonical URLs on your site (with optional lastmod) so search engines can discover and prioritize them. It's a discovery and diagnostic aid, not a guarantee of indexing — and Google ignores its priority and changefreq tags. or a complete URL list does not reproduce click depth from the homepage. Seed discovery from the intended root and keep sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.-only discovery separate in the report.
Fixing depth with redirects
A redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. changes the route a request takes, not the internal-link distance at which the destination is discovered. Add a useful link from a shallower page and remove unnecessary redirect hops from that path.
A page’s depth differs between crawlers
Symptom: two tools report different depths for the same canonical URLHow search engines pick one canonical URL among duplicates and consolidate signals onto it.. Likely cause: different start URLs, crawl limits, redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. counting, JavaScript renderingTurning HTML, CSS, and JavaScript into the final visual page and DOM., or URL normalization. Fix: align the seed, user agentA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target., render mode, canonical rules, and maximum depth, then compare the shortest path each crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. recorded.
Important pages appear only through the sitemap
Symptom: the URL is submitted and indexable but absent from a homepage-seeded link
crawl. Likely cause: it is orphaned or its only links require unsupported client
interaction. Fix: add a normal <a href> path from a relevant hub or related page
and verify it in raw or rendered HTML as appropriate.
A new hub did not lower depth
Symptom: the hub links to buried pages, yet the next crawl reports no improvement. Likely cause: the hub itself is deep, the links are paginated out of the crawl, or the crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. cannot discover them. Fix: inspect the recorded path to the hub and the rendered link elements, then connect the hub from a shallower, stable parent.
Depth jumps after a redesign
Symptom: whole templates move one or more clicks farther from home. Likely cause: navigation, paginationPagination splits a large set of content — product listings, blog archives, search results — across multiple sequentially numbered URLs. For SEO, each paginated page should be crawlable, indexable, and self-canonical; Google no longer uses rel=prev/next, but Bing still does., related modules, or category links changed globally. Fix: diff the old and new internal-link graphs by template and restore the lost paths before patching individual URLs.
Crawl-depth audit cheat sheet
| Finding | Confirm with | Typical response |
|---|---|---|
| Important URL has high shortest-path depth | CrawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. path report plus business-priority list | Add a relevant path from a hub, category, or strong contextual page |
| URL is sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.-only | Homepage-seeded crawl compared with sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. inventory | Add at least one crawlable internal linkAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. |
| URL folder count looks deep but click depth is low | Shortest link path | Leave the URL alone unless another migration reason exists |
| Depth rose for one template | Before/after graph grouped by template | Restore or replace the shared link module |
| CrawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. disagree on depth | Seed, render mode, limits, redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. and canonical settings | Normalize crawl configuration before changing the site |
| Many pages are depth one | Homepage/menu link count and navigation usability | Rebuild hierarchy; do not optimize the number in isolation |
| A redirect sits in the shortest path | Path report with response status per hop | Update the source link to the final canonical URLHow search engines pick one canonical URL among duplicates and consolidate signals onto it. |
| Deep pages receive little crawler activity | Depth buckets joined to server logsLog file analysis is reading a web server's raw access logs to see exactly which URLs search engine crawlers actually requested, when, how often, and what status code they got. Unlike crawl tools or Search Console, logs are the unsampled, ground-truth record of what really happened. | Prioritize valuable pages and investigate capacity or demand separately |
Metrics for crawl-depth health
| Metric | What it tells you | How to pull it | Benchmark or realistic range | Cadence |
|---|---|---|---|---|
| Depth distribution by template | Where the site’s reachable pages sit in the internal-link graph | Homepage-seeded crawl; group canonical URLsHow search engines pick one canonical URL among duplicates and consolidate signals onto it. by shortest path and template | Use the current distribution as baseline; compare important templates with peers rather than enforcing one universal depth | Monthly and after navigation releases |
| Priority-page depth | Whether business-critical pages are as prominent as intended | Join a priority URL list to the crawl’s shortest-depth field | Set targets by page role and site size; document exceptions | Weekly during remediation, then monthly |
| SitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.-only and orphan count | Which known URLs lack a crawlable internal path | Compare sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing., analytics, or indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. exports with the homepage-seeded crawl | The target for indexable priority pages is zero; archives may need explicit classification | Monthly |
| Depth change after release | Whether navigation changes shortened or lengthened paths | Diff URL-level depth between normalized before/after crawls | Investigate material movement by template; do not celebrate a lower average if hierarchy degraded | Every architecture release |
| CrawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. requests by depth bucket | Whether deeper groups receive different real bot attention | Join server-log requests to the latest crawl-depth buckets | Compare with the site’s own history; bot demand and content change frequency affect the pattern | Monthly on large sites |
Test yourself: Crawl Depth
Resources worth your time
My related writing
- The Beginner’s Guide to Technical SEO — where internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. and architecture fit in the bigger picture.
- When Should You Worry About Crawl Budget? — depth’s bigger sibling: capacity, demand, and who actually needs to care.
- Enterprise SEO — depth and crawl efficiency on very large sites.
My speaking
- How Search Works (SlideShare) — my walkthrough of crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., renderingTurning HTML, CSS, and JavaScript into the final visual page and DOM., indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., and ranking, including how Google’s link parsing focuses on
<a href>links and can miss JavaScript navigation. (My standing disclaimer applies: this is my understanding of these systems, not a guarantee it’s 100% complete or accurate.)
From others
- The 3-Click Rule for Navigation Is False — Page Laubheimer, Nielsen Norman Group. The UX research debunking the rule.
- Google: Click Depth Matters More for SEO than URL Structure — Search Engine Journal’s write-up of the Mueller “we don’t count slashes” session.
- John Mueller Recommends Pyramid Site Structure — Search Engine Journal’s coverage of Mueller’s 2021 flat-vs-pyramid statements.
- Google Recommends Pyramid Navigation Structures — Search Engine Roundtable’s take on the same Mueller guidance.
- Pages With High Crawl Depth — Screaming Frog’s reference on the issue.
- How to Audit Crawl Depth & Improve Crawl Efficiency — Sitebulb’s practical guide to measuring and fixing crawl depthCrawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops..
- Crawl Depth: How To Audit Crawl Depth With Screaming Frog — GoFish Digital’s step-by-step audit walkthrough.
Crawl depth
Crawl depth usually means click depth — how many clicks it takes to reach a page from the homepage by following internal links. It can also mean a crawler setting that limits how many levels deep a crawl goes before it stops.
Related: Crawling, Crawl Budget
Crawl depth
“Crawl depth” gets used two different ways, and the two are easy to conflate.
The common meaning is click depth (also called page depth): the number of clicks it takes to reach a page from the homepage — or from wherever a crawl started — by following internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them.. The homepage is depth 0, a page linked directly from the homepage, main nav, or footer is depth 1, and so on. This is a property of an individual page describing how buried it is in your architecture, and it’s what Screaming Frog’s “Crawl Depth” column and Ahrefs Site Audit’s “Depth” report.
The other meaning is crawl depth as a crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. setting: a configurable limit on how many levels deep a crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. will follow links before it stops. Set a tool’s maximum click depth to 4 and any page that can’t be reached within four clicks simply won’t be crawled.
The two are linked. A page’s click depth determines whether a depth-limited crawl reaches it at all, and pages that sit deeper tend to be discovered less reliably and crawled less often. Importantly, depth is defined by internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them., not URL folders — John Mueller has said Google doesn’t count the slashes in a URL. A page at /a/b/c/d/page/ that’s linked from the homepage is one click deep; a page at /page/ buried six clicks down is six deep.
Related: Crawling, Crawl Budget
Build-time retrieval analysis plus live signals for this exact article. The automatic chunk report includes a deterministic readiness score and is ready without a model download.
Search Console
sampleGA4 traffic (28d)
sampleCloudflare traffic (7d)
sampledCrUX field data (28d, phone)
sampleGoogle NLP entities
localChangelog
Updated Jul 18, 2026.
Editorial summary and recorded change details.Summary
Tightened the definition of crawl depth to a seed-relative shortest path and added measurement-methodology caveats (render mode/device, depth distribution vs. maximum, and how to test a depth change without over-claiming causality).
Change details
-
Defined click depth precisely as the shortest path from a stated crawl seed, and noted that seed, render mode, and device (desktop vs. mobile nav) can each change the reported number.
-
Added guidance to audit the distribution of important pages by depth and template rather than only the site's maximum depth.
-
Reframed the three/four-click threshold explicitly as a triage heuristic, not a pass/fail rule.
-
Added a methodology note for testing depth changes: like-for-like before/after crawls, a defined page/template scope, and treating crawling, indexing, and rankings as separate outcomes rather than inferring causality from traffic movement.
Full comparison unavailable — no prior snapshot was archived for this revision.