URL Case Sensitivity

TL;DR — /Apple and /apple are distinct URL strings, even when a server returns the same page for both. Hostnames are case-insensitive, while path and query handling depends on the server and application. Mixed variants can create duplicate crawl paths and mismatched robots.txt rules, so choose a casing convention and enforce it consistently.

Are URLs case-sensitive?

Yes — the part after your domain is. Google says it plainly in its own documentation: “Be aware that URLs are case sensitive.” So these two addresses are treated as two separate URLs:

https://example.com/Apple
https://example.com/apple

Even if both load the exact same page, a search engine sees two different URLs with their own content. That’s the whole issue in one sentence.

The one exception is your domain name (the hostname). Example.com and example.com are the same site — capitals in the domain never matter. It’s only the path, the filename, and the query stringThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. — everything after the domain — where case matters.

Why this causes problems

Two things go wrong:

1. Duplicate contentThe same or very similar primary content reachable at more than one URL. There's no general duplicate content penalty — the real costs are possible signal dilution, the wrong URL getting chosen, and less-efficient crawling.. If people (or your own site) link to both /Apple and /apple, you’ve got the same page living at two addresses. Google may sort that out on its own — but it also may not, and now your rankings and links are split across two versions instead of pooled into one.
2. Broken robots.txt rules. If your robots.txt says Disallow: /Private/ (capital P), it does not block /private/ (lowercase p). The rule only matches the exact casing you wrote.

How to fix it

• Pick lowercase. It’s the near-universal convention and what most website platforms already use.
• Make your server enforce it — set up a rule that redirectsA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. any uppercase URL to its lowercase version. (The Advanced and Scripts tabs have the actual code for Apache, Nginx, and IIS.)
• Keep your own links lowercase — in your menus, your content, and your sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing., always link to the lowercase version.

The thing most people get wrong

“Google is smart enough that this doesn’t matter.” It’s usually smart enough to merge identical case-variant pages — but “usually” isn’t “always,” and John Mueller has said outright that “hope should not be a part of an SEO strategy.” Don’t leave it up to Google to guess.

Want the why behind all this — how your server’s operating system decides whether /Apple works or 404s, the full robots.txt trap, and copy-paste server fixes? Switch to the Advanced tab.

TL;DR — Hostnames are case-insensitive; path and query comparison may be case-sensitive and ultimately depends on server and application routing. Filesystem defaults often influence behavior, but do not define URL semantics by themselves. Google can crawl differently cased paths as distinct URLs; it can canonicalize them together when content matches, but “usually… not always ideal” is not a strategy. robots.txt values are case-sensitive too, so a Disallow for one casing silently misses the others — a distinct access problem, not just a duplicate-content one. Fix priority: prevent variants at the source, then a server-level 301 to lowercase (Apache RewriteMap tolower, Nginx map, IIS URL Rewrite), with rel=canonical as a fallback when you can’t touch the server.

What is (and isn’t) case-sensitive

Start with the exact boundary, because casual phrasing (“URLs are case-sensitive”) gets it wrong half the time. John Mueller drew the line cleanly: “URL path, filename, and query parametersThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. are case-sensitive, the hostname / domain name aren’t. Case-sensitivity matters for canonicalizationHow search engines pick one canonical URL among duplicates and consolidate signals onto it., so it’s a good idea to be consistent there.”

So:

EXAMPLE.com/page and example.com/page are the same URL. example.com/Page and example.com/page are not. This is the same class of “one page, many URLs” problem that trailing slashesA trailing slash is the forward slash (/) at the end of a URL — example.com/page/ versus example.com/page. Except at the bare root domain, the two versions are different URLs to search engines, so you pick one format and enforce it., www vs non-www, and query parameters create — the URL structure and canonicalizationHow search engines pick one canonical URL among duplicates and consolidate signals onto it. deep dives cover those variants, and I won’t re-derive them here; casing is one more axis in the same duplicate-URL grid.

That table is the URL-handling default engines like Google follow, per the spec — not a promise about what your specific origin server, CDN, or application will actually return. Treat it as where to start testing, not as a substitute for testing: request the path in both casings and read the real response (the Scripts tab has a copy-paste curl check) before you assume the rule applies to your stack.

One more edge case worth flagging rather than glossing over: Unicode slugs. Two visually identical characters — a Latin “a” and a Cyrillic “а,” for example — can be different code points that survive ASCII-style lowercasing completely untouched, because lowercasing operates on the characters you have, not the ones a reader perceives. If you generate slugs from non-English titles, decide your encoding and normalization policy up front rather than assuming a .toLowerCase() call catches everything international.

Why this happens: it’s your filesystem

Most write-ups state “URLs are case-sensitive” as received wisdom and move on. The useful part is why, because it explains the single most confusing symptom — the same URL working on one server and 404-ing on another.

Linux/Unix filesystems are case-sensitive. On the ext4, XFS, and similar filesystems that run the overwhelming majority of the web, Apple and apple are literally different directory entries — different inodes. When Apache or Nginx maps an incoming request path to a file or a route, it compares the casing byte-for-byte. Ask for /Apple when only /apple exists and you get a genuine 404.

Windows/NTFS is case-insensitive by default — but case-preserving. It stores Apple with its capital A intact (so it displays “correctly”), yet when it looks the name up, it ignores case. IIS, Microsoft’s web server, inherits that behavior: request /Apple or /APPLE or /apple and IIS serves the same file. It doesn’t fix the SEO problem — Google still sees distinct URLs — but the server itself will happily 200 all of them.

The consequence is the classic dev-to-prod bug: a developer on Windows/IIS types a path with the wrong case, it works fine locally because Windows doesn’t care, then the same request 404s the moment it hits a Linux/Apache/Nginx production server that does. Nothing in your CMSA content management system (CMS) is software that lets users create, manage, and publish digital content — like blog posts and pages — without writing raw code. WordPress, Drupal, and Joomla are the most common open-source CMS platforms. or SEO plugin caused it — the two machines’ operating systems just disagree about whether case matters. (Microsoft documents this behavior directly in its Windows case-sensitivity guidance.)

This is worth knowing during any platform migration: move a site between a case-insensitive host and a case-sensitive one and mixed-case URLs that “always worked” can suddenly start breaking or duplicating.

The filesystem is the bottom of the stack, not the whole stack. Everything above it — a reverse proxy, a CDN edge, a load balancer, or the application’s own router — gets a chance to normalize, rewrite, or short-circuit a request’s casing before it ever reaches the filesystem the origin server sits on. A Linux origin doesn’t guarantee case-sensitive behavior at the URL you actually request if something in front of it is already folding cases together, and an application router can just as easily impose its own case rules independent of the OS underneath it. “Linux + Apache/Nginx = case-sensitive” is the right default assumption, not a substitute for checking the actual response on your stack.

How Google actually treats case-different URLs

Google isn’t inventing case sensitivity — it’s following the URL spec that every compliant HTTP client follows. From the docs: “Like any other HTTP client following IETF STD 66, Google Search’s URL handling is case sensitive (for example, Google treats both /APPLE and /apple as distinct URLs with their own content).” That STD 66 groundingGrounding is anchoring an AI model's answer to source documents it retrieves at the moment you ask — not to the patterns frozen into its weights during training. Retrieval-Augmented Generation (RAG) is the most common way to do it. matters: this is standards behavior, not a Google quirk.

Google’s own advice is to normalize: “If upper and lower case text in a URL is treated the same by your web server, convert all text to the same case so it’s easier for Google to determine that URLs reference the same page.” Note the “if” — that’s aimed squarely at case-insensitive servers (IIS) that serve every casing, which is exactly where the duplicate risk lives.

Can Google merge case-duplicates on its own? Often, yes — through canonicalization, casing is one of the common duplicate-URL patterns it groups. Mueller: “If a website still shows the same content in these cases, search engines will try to figure it out on their own and usually that works out well. But it’s not always ideal.” And in a separate exchange about mismatched canonicals on uppercase URLs, he was sharper: “If it serves the same content, it’ll probably be seen as a duplicate and folded together, but ‘hope’ should not be a part of an SEO strategy.”

That’s the honest framing. This isn’t a five-alarm fire — but relying on Google to guess right is a choice you don’t have to make when the fix is a redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. rule.

There’s also a crawl-efficiency cost. Mueller: “Search engines will try to crawl all variations of the URL that they find. This can make it a bit slower for them to find other useful content on your website.” If your site is spraying case-variant URLs into the link graph, botsA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. waste fetches re-crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. the same content in different casings instead of discovering your real pages — the same crawl-waste dynamic the parameters and crawl-budget topics describe, just with casing as the multiplier.

How Bing treats case-different URLs

Bing behaves differently, and honesty about sourcing matters here: there is no dedicated Bing documentation page on URL case sensitivityURL case sensitivity is the rule that uppercase and lowercase letters in the path, filename, and query string are treated as different characters — so /Apple and /apple are two distinct URLs — while the hostname is not case-sensitive. comparable to Google’s STD 66 callout, so this is based on webmaster reports and Microsoft community threads, not a formal spec.

What those reports consistently describe: BingbotBingbot is Microsoft Bing's primary web crawler — the bot that discovers, fetches, and renders pages to build the Bing index. That index also powers Yahoo, DuckDuckGo, Ecosia, and Microsoft Copilot, so Bingbot's reach is far wider than Bing's own search-market share. tends to normalize toward lowercase — crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. and indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. the lowercase version of a URL it discovers, and Bing Webmaster ToolsMicrosoft's free portal for monitoring and improving how a site appears in Bing search — the peer to Google Search Console, plus IndexNow instant indexing, richer backlink data, and keyword volumes. Because Bing's index also feeds Microsoft Copilot, it doubles as a window into AI-search visibility. has been observed auto-lowercasing URLs submitted in sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.. Webmasters have raised both behaviors on the Microsoft Community Hub. So Bing appears to do more of the case-collapsing for you than Google’s strict distinct-URL treatment does.

Either way, the fix is the same. Standardizing on lowercase satisfies Google’s distinct-URL model and aligns with whatever Bing is normalizing toward — you don’t optimize for one engine’s behavior at the expense of the other.

The robots.txt case trap

This deserves its own section because it’s a different kind of failure from duplicate contentThe same or very similar primary content reachable at more than one URL. There's no general duplicate content penalty — the real costs are possible signal dilution, the wrong URL getting chosen, and less-efficient crawling.. Duplicate content splits signals; a broken robots.txt rule is an access-control failure — a path you meant to keep bots out of gets crawled anyway.

The rule, straight from Google’s robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. spec: the directive name is case-insensitive, but its value is case-sensitive. “The field name (disallow) is case-insensitive, but its value is case-sensitive.” And: “The path value must start with / to designate the root and the value is case-sensitive.” Google’s own example spells it out — a rule for /fish “Matches any path that starts with /fish. Note that the matching is case-sensitive.”

So Disallow, disallow, and DISALLOW all work identically — the directive name casing is irrelevant. But the path value is matched exactly. Mueller confirmed the practical implication directly: “The robots.txt file also uses exact URLs, so if you have entries there which refer to one version of a URL they would not apply to other versions.”

Two concrete ways this bites — the second is the more dangerous:

Example 1 — the classic. Your robots.txt has:

User-agent: *
Disallow: /Private/

That blocks /Private/. It does nothing for /private/. If anything on your site links to the lowercase version, it’s fully crawlable despite your rule.

Example 2 — the post-migration mismatch. You replatform, and the new CMS generates cart and admin paths in lowercase. Your old robots.txt, copied over verbatim, still says:

Disallow: /Checkout/
Disallow: /Admin/

The live URLs are now /checkout/ and /admin/. The disallows silently miss them, and thin, no-value (or sensitive) paths get crawled and can end up indexed. This is exactly the scenario migration teams walk into, because robots.txt almost never gets audited against the actual casing of the new URLs.

(To be clear, this rarely causes catastrophic problems in practice — Mueller has said case mismatches in robots.txt are rare to see cause real issues. But “rare and silent” is precisely the kind of bug that sits unnoticed for months, so it’s worth a five-minute check.)

The fix for the trap is the same discipline as everywhere else: write your robots.txt rules in the casing your URLs actually use, and enforce a single casing so there’s only one version to block.

Fixing case-duplicate URLs

Standard technical-SEO remediation hierarchy, most to least preferred:

1. Prevent variants at the source

The best fix is never generating mixed-case URLs in the first place — this is a CMS/URL-generation-layer concern, and it varies by platform (verify current behavior on your own stack rather than trusting a blanket claim):

• Shopify lowercases product and collection handles automatically, so native URLs are largely safe — but app-generated or legacy-imported URLs can still introduce case variants.
• WordPress permalinks are lowercase by convention, but custom post types, tags, and manually entered slugs can introduce mixed case; Yoast and RankMath don’t natively force-lowercase URLs.
• Enterprise, custom, and Java/.NET-backed platforms are the most exposed — casing often falls out of the routing framework and hosting OS with no deliberate policy, especially on Windows/IIS stacks where nothing forces the issue locally.

2. Server-level lowercase redirect (the definitive fix)

A 301 to the canonical lowercase URL is the hard fix — it consolidates signals and sends users/bots to one address. The implementation differs by server (full copy-paste config is in the Scripts tab):

• Apache has a native lowercasing tool — RewriteMap with int:tolower — so you can convert an entire path to lowercase in a rewrite rule.
• Nginx has no built-in string-lowercasing function, so it’s genuinely more involved than Apache’s one-liner: you either use the njs/Lua module to lowercase the URI, or build an explicit map lookup that catches uppercase characters. Don’t let anyone tell you it’s a like-for-like map one-liner — it isn’t.
• IIS uses the URL Rewrite module with a rule condition and the {ToLower:...} function. Since IIS serves every casing by default, this is where enforcement matters most.

Don’t ship a blanket sitewide rule blind. Lowercase is the right convention for new paths, but a mixed-case-to-lowercase redirect is not automatically safe on an existing site — inventory your actual case variants first (crawl export, access logs, sitemap, internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them.), and specifically check for paths where the case is legitimate and load-bearing: uploaded filenames, generated tokens, anything a CMS or app produced with intentional mixed case. Confirm each route actually returns equivalent content in both casings before you fold it into the rule, roll the redirect out in a bounded way rather than flipping every path at once, and watch error rates and the GSC Page indexing reportThe Google Search Console report (formerly Index Coverage) showing how many of your URLs are indexed vs. not indexed, and grouping the not-indexed ones by reason. afterward — there’s no guarantee a lowercase cleanup recovers lost rankings or crawl coverage, only that it stops the signal-splitting going forward.

3. rel=canonical as a fallback

When you can’t touch server config — shared hosting, a platform that won’t let you add rewrite rules, or a case where legitimate case-variant URLs must stay live — a rel=canonical pointing every variant at the lowercase version is the safety net. It’s a hint, not a directive, and weaker than a 301 — but far better than nothing. Mueller’s recommendation: “Using internal linkingLinks between pages on the same site. to link to a consistent version makes your preference clear. Adding a link rel=“canonical” element also helps to confirm that.”

That said, canonical only works once you’ve confirmed the variants actually serve duplicate content. If a case variant returns genuinely different content, or an error, a rel=canonical element can’t merge it — a hint can’t consolidate two different resources into one. Verify the response before you point a canonical at it, not after.

4. Keep internal links and sitemaps consistent

Whatever casing you choose, your own links have to agree with it. Every internal link, navigation entry, and sitemap URL should use the canonical (lowercase) version. A perfect redirect rule undermined by menus that still link to /Products/ just makes bots follow redirect hops they didn’t need to.

How to find case-duplicate URLs on your site

Three complementary methods — GSC shows what Google actually did; crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. let you catch variants before they become an indexing problem:

• Google Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. → Page indexing report. Look for “Duplicate without user-selected canonicalA Google Search Console Page Indexing status: Google found this page to be a duplicate, you didn't declare a canonical, so Google chose a different page as the canonical — and this URL isn't indexed.” and “Duplicate, Google chose different canonical than user” statuses, then eyeball the flagged URL pairs for case differences. GSC won’t label the cause as “casing” — it reports every duplicate cause the same way — so you have to spot the case-only pairs yourself.
• Ahrefs Site Audit. A crawl surfaces near-duplicate URLs; casing differences show up as separate crawled URLs with identical or near-identical content, worth cross-checking under the duplicate-content reporting.
• Screaming Frog. Export the full crawl, lowercase the URL column, and sort/filter to spot pairs that differ only in case. You can also test specific known paths in both casings to confirm server behavior — does /Apple return 200, 301, or 404? That single test tells you whether your server is case-insensitive (IIS-style 200), already enforcing lowercase (301), or case-sensitive with no variant (404).

Where this sits

Case sensitivity is one axis in the broader duplicate-URL problem this cluster keeps circling — it compounds with trailing slashes, www vs non-www, and query parameters, and it’s one of the roughly 40 signals that feed canonicalization. The URL structureURL structure is how the parts of a web address — scheme, domain, path, query string, and fragment — are organized and formatted. It mostly affects crawling, usability, and how engines understand a page, not rankings directly. hub, the trailing slashA trailing slash is the forward slash (/) at the end of a URL — example.com/page/ versus example.com/page. Except at the bare root domain, the two versions are different URLs to search engines, so you pick one format and enforce it. and URL parametersThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. deep dives, and the canonicalization article each cover their own axis; the discipline is identical across all of them — pick one clean version and enforce it with redirects, canonicals, consistent internal links, and matching sitemaps.

AI summary

A condensed take on the Advanced version:

• What’s case-sensitive: the path, filename, and query stringThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. — not the hostname. /Apple ≠ /apple; Example.com = example.com. Google follows IETF STD 66, so it treats casings as distinct URLs with their own content. Treat the rule as a default to verify, not a guarantee — request both casings and read the real response for your own stack. Non-ASCII slugs are a separate risk: visually identical Unicode characters can be different code points that survive lowercasing untouched, so international slugs need an explicit encoding/normalization policy.
• Why it happens — the filesystem: Linux/Unix (and Apache/Nginx on it) are case-sensitive — Apple and apple are different files. Windows/NTFS and IIS are case-insensitive by default (case-preserving). So the same request can 200 on IIS and 404 on Linux — the classic dev-to-prod / migration bug. But the filesystem is the bottom of the stack, not the whole stack: a proxy, CDN, load balancer, or application router in front of it can override that default before a request ever reaches the filesystem.
• Google can merge case-duplicates via canonicalizationHow search engines pick one canonical URL among duplicates and consolidate signals onto it., but “usually… not always ideal” — and “hope should not be a part of an SEO strategy.” There’s also a crawl-efficiency cost: botsA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. waste fetches crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. every casing.
• Bing appears to normalize toward lowercase (crawls/indexes the lowercase version, auto-lowercases sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. URLs) — per webmaster reports, not an official spec.
• The robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. trap (an access failure, not a duplicate one): the directive name is case-insensitive but the path value is case-sensitive. Disallow: /Private/ does not block /private/. Watch for post-migration mismatches (/Checkout/ rule vs. lowercase live paths).
• Fix priority: (1) prevent variants at the CMSA content management system (CMS) is software that lets users create, manage, and publish digital content — like blog posts and pages — without writing raw code. WordPress, Drupal, and Joomla are the most common open-source CMS platforms./source; (2) server-level 301 to lowercase — Apache RewriteMap int:tolower, Nginx via njs/Lua or a map table (no native lowercasing — and make sure the redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. preserves the query string with $is_args$args, since the njs function only lowercases the path), IIS URL Rewrite {ToLower:...}; (3) rel=canonical as a fallback, only once you’ve confirmed the variant actually serves duplicate contentThe same or very similar primary content reachable at more than one URL. There's no general duplicate content penalty — the real costs are possible signal dilution, the wrong URL getting chosen, and less-efficient crawling.; (4) keep internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. and sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. consistent. Before a sitewide redirect, inventory legitimately case-sensitive paths (uploaded files, tokens) and roll it out bounded — there’s no ranking or crawl recovery guarantee, only that it stops future signal-splitting.
• Detect it: GSC Page indexing reportThe Google Search Console report (formerly Index Coverage) showing how many of your URLs are indexed vs. not indexed, and grouping the not-indexed ones by reason. (duplicate statuses → eyeball case pairs), Ahrefs Site Audit, Screaming Frog (lowercase-and-sort the export; test /Apple for 200/301/404).
• Compounds with trailing slashA trailing slash is the forward slash (/) at the end of a URL — example.com/page/ versus example.com/page. Except at the bare root domain, the two versions are different URLs to search engines, so you pick one format and enforce it., www/non-www, and parameters — same fix: pick one version, enforce it.

Official documentation

Primary-source documentation on case handling.

Google

• URL structure best practices — “Be aware that URLs are case sensitive”; the IETF STD 66 /APPLE vs /apple framing; the convert-to-same-case recommendation.
• How Google interprets the robots.txt specification — the directive name is case-insensitive but the path value is case-sensitive; the /fish case-sensitive-matching example.
• Canonicalization — casing listed among the common duplicate-URL scenarios (alongside HTTP/HTTPSHTTPS is the encrypted version of HTTP — it uses TLS to authenticate the server and protect data in transit between a browser and a website. Google announced it as a lightweight ranking signal in 2014 and today conditionally prefers HTTPS pages as canonical; Chrome marks plain HTTP pages 'Not Secure.', www, trailing slashA trailing slash is the forward slash (/) at the end of a URL — example.com/page/ versus example.com/page. Except at the bare root domain, the two versions are different URLs to search engines, so you pick one format and enforce it., and parameters).
• Page indexing report — Search Console Help — the “Duplicate without user-selected canonicalA Google Search Console Page Indexing status: Google found this page to be a duplicate, you didn't declare a canonical, so Google chose a different page as the canonical — and this URL isn't indexed.” and “Duplicate, Google chose different canonicalA Google Search Console Page Indexing status: you declared a canonical for this URL, but Google overrode your choice, picked a different page as the canonical, and indexed that one instead. than user” statuses that surface case-duplicates.

Bing / Microsoft

• Bing changing URLs from upper to lower case — Microsoft Community Hub — webmaster reports of Bing’s lowercase normalization (community evidence, not an official spec page).
• Adjust case sensitivity — Microsoft Learn — Windows/NTFS case-insensitive-but-case-preserving behavior, the mechanism behind the IIS-vs-Linux difference.

Quotes from the source

On-the-record statements from Google’s documentation and John Mueller. Where the source is documentation, the link deep-links to the passage.

Google Search Central docs — URLs are case-sensitive

• “Be aware that URLs are case sensitive.” Jump to quote
• “Like any other HTTP client following IETF STD 66, Google Search’s URL handling is case sensitive (for example, Google treats both /APPLE and /apple as distinct URLs with their own content).” Jump to quote
• “If upper and lower case text in a URL is treated the same by your web server, convert all text to the same case so it’s easier for Google to determine that URLs reference the same page.” Jump to quote

Google Search Central docs — robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. is case-sensitive

• “The field name (disallow) is case-insensitive, but its value is case-sensitive.” Jump to quote
• “The path value must start with / to designate the root and the value is case-sensitive.” Jump to quote
• “Matches any path that starts with /fish. Note that the matching is case-sensitive.” Jump to quote

John Mueller, Google — what is (and isn’t) case-sensitive

• “URL path, filename, and query parametersThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. are case-sensitive, the hostname / domain name aren’t. Case-sensitivity matters for canonicalizationHow search engines pick one canonical URL among duplicates and consolidate signals onto it., so it’s a good idea to be consistent there.” Coverage (Search Engine Journal)

John Mueller, Google — don’t rely on Google figuring it out

• “If it serves the same content, it’ll probably be seen as a duplicate and folded together, but ‘hope’ should not be a part of an SEO strategy.” Coverage (Search Engine Journal)
• “If a website still shows the same content in these cases, search engines will try to figure it out on their own and usually that works out well. But it’s not always ideal.” Coverage (Search Engine Journal)

John Mueller, Google — robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. and crawl efficiency

• “The robots.txt file also uses exact URLs, so if you have entries there which refer to one version of a URL they would not apply to other versions.” Coverage (Search Engine Journal)
• “Search engines will try to crawl all variations of the URL that they find. This can make it a bit slower for them to find other useful content on your website.” Coverage (Search Engine Journal)
• “Using internal linkingAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. to link to a consistent version makes your preference clear. Adding a link rel=“canonical” element also helps to confirm that.” Coverage (Search Engine Journal)

URL case-sensitivity checklist

A quick pass to catch and close case-duplicate risk:

• Decide on a single casing convention — lowercase unless you have a hard reason not to.
• Test a known path in both casings (/Apple and /apple): does the uppercase version 301 to lowercase, or 200/404? Anything but a 301 is a gap.
• Know your host OS/server: Linux + Apache/Nginx = case-sensitive (mixed-case links can 404); Windows + IIS = case-insensitive (every casing 200s → duplicate risk).
• Before deploying a blanket lowercase redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't., inventoried existing paths where case is legitimate (uploaded filenames, tokensA token is the smallest unit of text (or image/audio/video) an LLM processes — roughly 4 characters, or about ¾ of an English word. A context window is the maximum number of tokens (input plus output) a model can hold at once, like its short-term memory., generated slugs) and confirmed they’re excluded or tested for equivalence — not just assumed safe.
• Server-level lowercase redirect rule is in place (Apache RewriteMap, Nginx map/njs, or IIS URL Rewrite) and, if Nginx, redirects preserve the original query stringThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. ($is_args$args) rather than dropping it.
• Every robots.txt Disallow/Allow value matches the actual casing of your live URLs — no /Private/ rules guarding /private/ paths.
• Internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them., navigation, and menus all use the canonical (lowercase) casing.
• XML sitemapAn XML sitemap is a UTF-8 file listing the canonical URLs on your site (with optional lastmod) so search engines can discover and prioritize them. It's a discovery and diagnostic aid, not a guarantee of indexing — and Google ignores its priority and changefreq tags. lists only the canonical-cased URLs.
• rel=canonical points every reachable case-variant at the lowercase version (as a backstop, especially where you can’t add a redirect).
• GSCA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. Page indexing reportThe Google Search Console report (formerly Index Coverage) showing how many of your URLs are indexed vs. not indexed, and grouping the not-indexed ones by reason. checked for duplicate statuses; flagged pairs eyeballed for case-only differences.
• Post-migration: robots.txt, canonicals, and internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. re-audited against the new platform’s URL casing.

URL case sensitivity — cheat sheet

What’s case-sensitive

Server behavior by platform

These are OS/server defaults, not guarantees — a reverse proxy, CDN, or application router in front of the origin can override them. Test the real response for your stack.

The robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. trap

Directive name casing (Disallow / disallow) doesn’t matter — the path value casing does.

Fix priority

1. Prevent variants at the CMSA content management system (CMS) is software that lets users create, manage, and publish digital content — like blog posts and pages — without writing raw code. WordPress, Drupal, and Joomla are the most common open-source CMS platforms./source.
2. Server-level 301 → lowercase (definitive).
3. rel=canonical → lowercase (fallback, it’s a hint).
4. Consistent internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. + sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing..

One-line diagnosis: request /Apple — a 301 to /apple means you’re covered; a 200 (IIS) or 404 (Linux with no variant) means you’re not.

Enforce lowercase URLs — server config

Copy-paste-ready starting points for each server. Test against your own stack before shipping — mod_rewrite availability, module installs, and rule ordering vary by host, and an over-broad rule can loop or lowercase things you didn’t mean to (like case-sensitive query values).

Apache (.htaccess / vhost) — the native tolower approach

Apache has a built-in lowercasing map. Define it once in your main server config (RewriteMap can’t live in .htaccess), then use it in a rule:

# In httpd.conf / vhost config (NOT .htaccess):
RewriteMap lc int:tolower

# In .htaccess or vhost:
RewriteEngine On
# Only act when the path actually contains an uppercase letter
RewriteCond %{REQUEST_URI} [A-Z]
# Redirect the whole path to its lowercase form
RewriteRule (.*) ${lc:$1} [R=301,L]

int:tolower is Apache’s internal function that lowercases the captured path. The RewriteCond %{REQUEST_URI} [A-Z] guard means the rule only fires when there’s an uppercase letter to fix, avoiding a redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. on already-lowercase URLs.

Nginx — no native lowercasing (be honest about this)

Nginx’s core map directive cannot lowercase a string on its own, so there’s no Apache-style one-liner. Two real options:

Option A — njs (or Lua), the clean way. With the njs module, lowercase the URI in JavaScript and redirect:

# Load the njs module and a small JS file that lowercases the URI path
js_import lower from conf.d/lower.js;
js_set $lower_uri lower.uri;

server {
    # ...
    # Test $uri (path only), NOT $request_uri (path + query) — the njs
    # function below only lowercases the path, so the query string is
    # intentionally left untouched. Testing $request_uri would fire on an
    # uppercase-containing query even when the path is already lowercase.
    if ($uri ~ [A-Z]) {
        # $is_args$args re-attaches the original query string. Without it,
        # the redirect drops any query string entirely, since $lower_uri
        # (built from r.uri, which excludes the query) never contains one.
        return 301 $scheme://$host$lower_uri$is_args$args;
    }
}

// conf.d/lower.js
function uri(r) { return r.uri.toLowerCase(); }
export default { uri };

Two details that are easy to get wrong here and will silently break the fix: matching on $request_uri instead of $uri makes the rule fire on an uppercase query stringThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. even when the path is already lowercase, and forgetting $is_args$args on the redirect target drops the entire query string — tokensA token is the smallest unit of text (or image/audio/video) an LLM processes — roughly 4 characters, or about ¾ of an English word. A context window is the maximum number of tokens (input plus output) a model can hold at once, like its short-term memory., filter parameters, everything — on every single redirect, since the njs function only ever sees and lowercases the path. Test a path that has a mixed-case query string attached before you ship this.

Option B — an explicit map lookup table, if you can’t add a module. You build a map that only catches the uppercase-containing URIs you care about — verbose and partial, which is exactly why Option A is preferred:

map $request_uri $has_upper {
    default        0;
    "~[A-Z]"       1;   # flag URIs containing any uppercase letter
}
# You still need njs/Lua or per-path rewrites to do the actual lowercasing —
# map alone flags, it doesn't transform. Don't ship this as a complete fix.

The takeaway: on Nginx, plan for the njs/Lua route. Anyone claiming a plain map one-liner lowercases URLs is skipping the part where map can’t transform strings.

IIS (Windows) — URL Rewrite module

Because IIS serves every casing by default, this is where enforcement matters most. The URL Rewrite module has a {ToLower:...} function:

<!-- web.config -->
<configuration>
  <system.webServer>
    <rewrite>
      <rules>
        <rule name="Lowercase URLs" stopProcessing="true">
          <match url="[A-Z]" ignoreCase="false" />
          <conditions>
            <add input="{URL}" pattern="[A-Z]" ignoreCase="false" />
          </conditions>
          <action type="Redirect" url="{ToLower:{R:0}}" redirectType="Permanent" />
        </rule>
      </rules>
    </rewrite>
  </system.webServer>
</configuration>

ignoreCase="false" on the match is what lets the rule see uppercase letters at all; {ToLower:{R:0}} rewrites the matched path to lowercase; redirectType="Permanent" issues the 301.

Verify it worked

After deploying, confirm the redirect with a single request (macOS/Linux):

# Should show: HTTP/… 301 and a Location header pointing to the lowercase path
curl -sI https://example.com/Apple | grep -Ei "^HTTP|^location"

Windows (PowerShell):

# -MaximumRedirection 0 stops the redirect so you can read the 301 + Location
try { Invoke-WebRequest "https://example.com/Apple" -MaximumRedirection 0 } catch {
  $_.Exception.Response.StatusCode.value__
  $_.Exception.Response.Headers["Location"]
}

A 301 to the lowercase URL means enforcement is live. A 200 means the server is still serving the uppercase version; a 404 means that casing has no page (and you likely have case-sensitive hosting with no redirect layer).

Case-sensitivity anti-patterns

The mistakes that turn a trivial config detail into a real duplicate-content or crawl problem:

1. Assuming Google will always merge case-duplicates. It often does — but “usually” isn’t “always,” and Mueller’s own “hope should not be a part of an SEO strategy” is the counter. Leaving it to canonicalizationHow search engines pick one canonical URL among duplicates and consolidate signals onto it. splits signals when Google guesses wrong.

2. Writing robots.txt rules in a different casing than your live URLs. Disallow: /Private/ guarding a /private/ path is a silent no-op. The directive name’s casing is irrelevant; the path value’s casing is exact. This is an access-control bug, not a ranking one.

3. Copying robots.txt verbatim after a migration. The new platform generates /checkout/ and /admin/; the old robots.txt still disallows /Checkout/ and /Admin/. Everything you meant to block is now crawlable. Re-audit robots.txt against the new URL casing every migration.

4. Fixing the redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. but not the internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them.. A perfect lowercase-enforcement rule undermined by menus and content still linking to /Products/ just makes botsA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. (and users) take an extra redirect hop every time. The redirect is the backstop; consistent links are the actual fix.

5. Treating a rel=canonical as equivalent to a 301. Canonical is a hint; a 301 is a directive. Use canonical only when you genuinely can’t add a redirect — don’t reach for it first because it’s easier.

6. Believing “modern platform, so it’s handled.” Shopify lowercases native handles, sure — but imported, legacy, and app-generated URLs, plus anything on a Windows/IIS stack, still carry real risk. “Buy a modern CMSA content management system (CMS) is software that lets users create, manage, and publish digital content — like blog posts and pages — without writing raw code. WordPress, Drupal, and Joomla are the most common open-source CMS platforms. and forget about it” is not a policy.

7. Thinking lowercase is a ranking preference. Google has never said lowercase ranks better. The issue is duplicate-content consolidation and crawl efficiency — not a case-based ranking signal. Lowercase is a convention (and what most CMSs default to), not a ranking factor. Same class of myth as “shorter URLs rank better.”

8. Over-broad rewrite rules that lowercase query values. Blindly lowercasing the entire request URI can mangle case-sensitive query parameter values (tokensA token is the smallest unit of text (or image/audio/video) an LLM processes — roughly 4 characters, or about ¾ of an English word. A context window is the maximum number of tokens (input plus output) a model can hold at once, like its short-term memory., base64, signatures). Scope your redirect to the path, and leave query stringsThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. alone unless you know they’re safe to fold.

Common URL-case failures

Mixed-case and lowercase URLs both return 200

Symptom: /Products/Blue-Shoe and /products/blue-shoe load the same page. Likely cause: The application or host resolves paths case-insensitively without a normalization redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't.. Fix: Choose the established format, normally lowercase, redirect alternates in one hop, and align internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them., canonicals, and sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing..

A URL works on staging but returns 404 in production

Symptom: A mixed-case path succeeds on one environment and fails on another. Likely cause: The hosts use different case-sensitivity rules, commonly across Windows and Linux-based systems. Fix: Correct the source link to the exact route case and add deployment tests that request the production path verbatim.

A robots rule does not block the expected path

Symptom: A crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. can fetch /private/ even though robots.txt disallows /Private/. Likely cause: Robots path matching is case-sensitive. Fix: Match the exact URL case or normalize paths before relying on a single rule, then retest the real bot and URL combination.

Review mixed-case URL pairs

Paste a crawl export with URL, status, final URL, canonical, inlinks, sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. source, and organic signals into this prompt:

Group URLs that differ only by path or query-string case. For each group, identify
the strongest candidate preferred URL using only the supplied evidence. Flag
conflicting redirects, canonicals, internal links, sitemap entries, and robots rules.
Return an implementation table with preferred URL, alternate URL, redirect action,
source links to update, and unresolved evidence. Do not assume lowercase wins when
the data shows an established mixed-case canonical.

Tools for case normalization

• Redirect Chain Mapper — compare case variants and see whether they resolve directly to one preferred form or accumulate extra hops.
• Canonicalization Checker — identify conflicts between the response, canonical element, and final destination for a variant URL.
• robots.txt Tester — test the exact uppercase and lowercase paths against the relevant botA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. because path matching is case-sensitive.
• A full-site crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. — use case-insensitive grouping to find variants, then use inlink reports to correct the templates and content that keep generating them.

Verify the preferred-case redirect

Test to run: Request the preferred URL and several case variants with the Redirect Chain Mapper. Expected result: The preferred URL returns a successful response and every alternate permanently redirectsA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. to it in one hop. Failure interpretation: The rule is incomplete, ordered incorrectly, or incompatible with the application route. Monitoring window: Immediate after deployment. Rollback trigger: Valid routes begin to loop, redirect to the wrong page, or return errors.

Verify emitted URLs and crawl controls

Test to run: Crawl internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. and sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing., then test representative paths in the robots.txt Tester. Expected result: Templates, canonicals, and sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. emit only the preferred case, and robots rules behave the same for the exact URLs the site exposes. Failure interpretation: A source system still generates variants or a robots rule has the wrong case. Monitoring window: After a complete post-release crawl. Rollback trigger: Important preferred URLs become blocked or a large new variant set appears.

Test yourself: URL Case Sensitivity

Five quick questions on how case sensitivity actually works and how to fix it. Pick an answer for each, then check.

Resources worth your time

My related writing

• Google Uses ~40 Canonicalization Signals — my breakdown of how Google picks a representative URL; it lists case-variant URLs (/page/ vs /Page/) as one of the duplicate-URL example patterns, which is exactly the problem this article fixes.
• Trailing Slash: To Use or Not to Use? — a sibling canonicalizationHow search engines pick one canonical URL among duplicates and consolidate signals onto it. axis: same “pick one version and enforce it” discipline, applied to the slash instead of casing.
• URL Parameters: A Complete Guide for SEOs — the other big duplicate-URL multiplier, and why passive parameters waste crawl budgetThe number of URLs an engine will crawl in a timeframe. the same way stray casings do.
• Redirects for SEO: A Simple (But Complete) Guide — the 301 mechanics behind the server-level lowercase-enforcement fix.
• The Beginner’s Guide to Technical SEO — where URL hygiene sits in the bigger picture.

My speaking

• Troubleshooting Technical SEO Problems (SlideShare, Raleigh SEO Meetup) — duplicate-URL and canonicalizationHow search engines pick one canonical URL among duplicates and consolidate signals onto it. issues in context. (Standing disclaimer applies: this is my understanding of these systems, not an official spec.)

From around the industry

• URL structure best practices (Google Search Central) — the “URLs are case sensitive” / IETF STD 66 source.
• How Google interprets the robots.txt specification (Google Search Central) — the “value is case-sensitive” robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. rule.
• Google: URLs Are Case Sensitive (Search Engine Journal) — the anchor source for Mueller’s case-sensitivity, robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere., crawl-slowdown, and canonical-recommendation quotes.
• Google’s Advice On Canonicals: They’re Case Sensitive (Search Engine Journal) — the “hope should not be a part of an SEO strategy” exchange.
• URL Case Sensitivity and SEO (Practical Ecommerce) — the closest direct competitor; good on the ecommerce 404 symptom.
• Adjust case sensitivity (Microsoft Learn) — the Windows/NTFS case-insensitive-but-case-preserving behavior behind the IIS-vs-Linux difference.
• Bing changing URLs from upper to lower case (Microsoft Community Hub) — webmaster reports of Bing’s lowercase normalization (community evidence, not a formal spec).

URL Case Sensitivity

URL case sensitivity is the rule that uppercase and lowercase letters in the path, filename, and query string are treated as different characters — so /Apple and /apple are two distinct URLs — while the hostname is not case-sensitive.

Related: URL Structure, Canonicalization, Trailing Slash, 301 redirect

URL Case Sensitivity

URL case sensitivity refers to whether a web server and search engine treat uppercase and lowercase letters in a URL as different characters. They do — for everything after the hostname. example.com/Apple and example.com/apple are two separate, distinct URLs, even when they render identical content. This follows the URL standard (RFC 3986 / IETF STD 66), and Google’s crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. follows that standard exactly.

The one part that is never case-sensitive is the hostname. DNS doesn’t distinguish Example.com from example.com, so the domain resolves the same regardless of casing. Only the path, filename, and query stringThe `?key=value` data tacked onto the end of a URL after a question mark — used for tracking, sessions, filtering, sorting, and search — and one of the biggest sources of duplicate URLs and wasted crawling in SEO. carry case-sensitivity risk.

The practical problem is duplicate contentThe same or very similar primary content reachable at more than one URL. There's no general duplicate content penalty — the real costs are possible signal dilution, the wrong URL getting chosen, and less-efficient crawling.: when the same page is reachable under multiple casings, you can split signals across near-identical URLs. There’s also a sharper trap in robots.txt, where the directive value is case-sensitive — a Disallow: /Private/ rule does not block /private/ at all. The standard fix is to pick one case (lowercase is the convention), enforce it with a server-level redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't., and fall back to rel=canonical where a hard redirect isn’t feasible.

Related: URL Structure, Canonicalization, Trailing Slash, 301 redirect

Changelog