403 Forbidden

What an HTTP 403 error is, how Google treats it (blocked — similar to a noindex), common causes (access controls, bot blocking, misconfigured permissions), and how to fix 403s for SEO.

First published: Jun 27, 2026 · Last updated: Jul 17, 2026 · Advanced
demand #2 in HTTP Errors#2 in HTTP Status Codes#5 in Technical SEO#9 on the site
AIAuthored byAI v2
1 evidence signal on this page

A 403 Forbidden means the server understood the request but refused it — access was denied, and that refusal doesn't have to be about credentials at all (the server can even send a 404 instead to hide that a forbidden resource exists). It's not a 404 ("nothing here") and not a 401 ("authenticate first"); a 403 is an active "you can't have this." For SEO, a 403 that stays on a page you want public keeps it out of Google's index — a similar end result to a noindex, but a different mechanism, since Google can't read content from a 4xx response at all. Because Googlebot never sends credentials, a 403 served to it is worth investigating rather than assuming intentional — common causes include a CDN/WAF bot filter, a server IP/user-agent block, a security plugin, or an .htaccess/permissions error, though there's no reliable data on which is most frequent. Never use 403 to throttle crawling (that's 429/503, for a short window only). And check intent first: some 403s (staging, admin, gated content) are correct and don't need fixing. Note the flip: a 403 on robots.txt itself is treated permissively, while a 403 on a page is a hard block.

TL;DR — A 403 means the server understood the request and refused it on an access basis — distinct from 404 (gone) and 401 (authenticate). For indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. the outcome mirrors a noindex: Google won’t index a 403 URL and removes one that was indexed, though the mechanism (a server/CDN/WAF block) is nothing like a meta tag. Because GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. never sends credentials, a 403 to GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. is nearly always a misconfiguration — most often CDN/WAF botA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. filtering (Cloudflare Bot Fight Mode leads the pack), server IP/UA blocks, security plugins, or .htaccess/permissions. 403 has no effect on crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor. — never use it to throttle (that’s 429429 Too Many Requests is an HTTP client-error status code that means a client sent too many requests in a given time window — a mechanism called rate limiting. Unlike other 4xx codes, Google treats 429 as a server-overload signal and slows crawling instead of removing content./503503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime.). And the flip case matters: a 403 on robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. itself is treated permissively, while a 403 on a page is a hard block.

403 vs 401 vs 404 — get the mental model right first

These three get conflated constantly, and the distinction drives the whole diagnosis. The base definition comes from the HTTP spec itself, RFC 9110 §15.5.4: the server understood the request and refuses to fulfill it. A couple of nuances in that same section matter more than people give them credit for:

  • Refusal doesn’t have to be about credentials. RFC 9110 allows a 403 for reasons unrelated to authentication — it doesn’t universally prove the requester is “known” or that credentials were even involved. Don’t read every 403 as an authentication story.
  • A 403 doesn’t have to admit the resource exists. The spec explicitly permits an origin server that wants to hide whether a forbidden resource exists at all to respond with 404 instead. So the reverse is also true: a 404 doesn’t always mean “nothing was ever here” — sometimes it means “there’s something here I don’t want you to know about.”
  • 401 vs. 403 isn’t just “weaker vs. stronger.” A 401 is specifically an authentication challenge — the spec requires it to come with a WWW-Authenticate header telling the client how to authenticate. A 403 carries no such requirement; it’s a broader refusal that doesn’t promise re-authenticating (with the same or different credentials) will change anything. MDN’s plain-English version: a 403 is “similar to 401, except that … authenticating or re-authenticating makes no difference. The request failure is tied to application logic, such as insufficient permissions.” Evidence for this claim A 403 response means the server understood the request but refuses to fulfill it, and authenticating does not necessarily make a difference. Scope: RFC 9110 defines the protocol semantics; it does not identify the application, firewall, or policy responsible for a specific response. Confidence: high · Verified: IETF: RFC 9110 §15.5.4 — 403 Forbidden
  • 404 Not Found404 Not Found is the HTTP client-error status code a server returns when it can't find the requested URL — RFC 9110 defines it as no current representation, or unwillingness to disclose one. A \"hard 404\" actually returns the 404 status; a \"soft 404\" returns a success code (like 200) for a page that's really gone. 404s are normal and expected: the fact that some URLs 404 doesn't affect your site's other, successful pages, and Google de-indexes 404'd URLs over time (probably retrying for some period, less and less often)., for contrast — the ordinary case is “there’s nothing here,” though see the concealment point above.

That difference is part of why a 403 to Googlebot is worth a second look rather than business-as-usual. A 401 to a bot on a members-only page is arguably working — the area needs a login. A 403 to a bot on a page that’s supposed to be public means some rule decided the requester was unwelcome — though, as below, it’s still worth confirming that page was actually meant to be public before you call it a bug.

How Google treats a 403 (outcome resembles noindex; the mechanism doesn’t)

Google lumps 403 in with the rest of the 4xx family. From the Search Central docs: “Google doesn’t index URLs that return a 4xx status code, and URLs that are already indexed and return a 4xx status code are removed from the index.” And: “All 4xx errors, except 429, are treated the same: Google crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. inform the next processing system that the content doesn’t exist.” Google also notes that a known URL’s crawl frequency gradually decreases the longer it keeps returning a 4xx — that’s a per-URL effect, separate from your site’s overall crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor. (more on that distinction below). Evidence for this claim Google does not index 4xx URLs and removes already-indexed 4xx URLs over time. Scope: Google's crawler documentation supports the indexing outcome for persistent 403 responses, not the article's analogy to noindex. Confidence: high · Verified: Google: How HTTP status codes affect Google's crawlers

So the outcome resembles a noindex: the page won’t enter the index, and if it’s already there, it falls out over time. But the mechanism is genuinely different, not just cosmetically different: a noindex tag has to be fetched and parsed out of the page’s HTML to take effect, while a 403 stops Google from reading any content at all — there’s no page for Google to process. Two different paths that happen to converge on the same non-appearance-in-Search result. This is why in my HTTP Status Codes guide I describe 403 simply as “the client is known but doesn’t have access rights,” and note that 4xxs cause pages to drop from the index — though “known” there is Google’s report shorthand, not a claim that every 403 involves a credentialed requester.

Why a 403 to Googlebot is worth investigating (not always a mistake)

This section leans on Google’s own Page IndexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. help doc: “HTTP 403 means that the user agentA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target. provided credentials, but was not granted access. However, Googlebot never provides credentials, so your server is returning this error incorrectly. The page will not be indexed.” Read that in context, though — it’s report-specific guidance about a page Search ConsoleGoogle's free tool for monitoring crawling, indexing, and search performance. assumes you want indexed, not a universal claim that every 403 to Googlebot is a bug. Googlebot doesn’t authenticate, so a 403 framed strictly as “your credentials aren’t good enough” doesn’t quite apply to it — but plenty of 403s aren’t about credentials at all, and a server can legitimately decide Googlebot (or anyone else) doesn’t get a resource, full stop.

So before you chase a fix: is this a page you actually want public and indexed? If it’s staging, an admin area, a paywall, or anything gated, a 403 to Googlebot is correct and there’s nothing to fix (more in “When a 403 is actually fine,” below). If it is a page meant to be public, then yes, you’re most likely looking at a rule that fired on the wrong requester — a WAF that flagged the crawler as a bot to block, an IP-range block that swept up Google’s ranges, a security plugin’s default paranoia — but I don’t have reliable data on how often each of those is the actual cause, so treat the list below as candidates to check, not a diagnosis. Google’s guidance for a page that should be public: admit non-signed-in users, or explicitly allow Googlebot without authentication (after verifying its identity — more on that below).

403 has no effect on crawl rate — do not use it to throttle

People sometimes reach for 403 (or 404) to get Googlebot to back off a struggling server. Don’t. Google is explicit: “Don’t use 401 and 403 status codes for limiting the crawl rate. The 4xx status codes, except 429, have no effect on crawl rate.” Worth being precise about scope here: Google separately says an individual known URL’s crawl frequency gradually decreases the longer it keeps 4xx-ing — but that’s a narrowing of interest in that one URL, not a site-wide crawl-rate throttle. If you need the latter, 403 doesn’t do it.

Gary Illyes wrote an entire post about this ( Don’t 404 my yum): “All 4xx HTTP status codesAn HTTP status code is the three-digit number a server returns with every response to tell a browser or crawler what happened to its request — success, redirect, client error, or server error. For SEO the code matters as much as the content: it tells Google and Bing whether to index a page, follow a redirect, retry later, or drop the URL from the index. (again, except 429) will cause your content to be removed from Google Search.” The correct emergency lever is a 500, 503, or 429 returned for a short window (hours, not days) — and even that isn’t a free pass: Google warns that persistent 5xx responses over multiple days risk pages being dropped from the index too, so scope and duration matter, not just which status code you picked. Barry Schwartz’s write-up of an earlier incident captured the stakes bluntly: sites have “lost load[s] of their pages from our index because they were serving them with a 403 status code instead of a 503.” A 503 is understood as temporary; a 403 gets you deindexed.

The robots.txt gotcha: a 403 on robots.txt is permissive, not restrictive

Here’s a distinction nearly every competing article misses, and it flips the intuition. A 403 on a page is a hard block. But a 403 on your robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. file itself is treated the opposite way. From Google’s robots.txt spec: “Google’s crawlers treat all 4xx errors, except 429, as if a valid robots.txt file didn’t exist. This means that Google assumes that there are no crawl restrictions.”

So if your firewall starts 403-ing robots.txt, Google concludes you have no rules and crawls freely — including paths you meant to disallow. Illyes’ colorful version: if you had a rule disallowing your “dirty laundry,” now Googlebot knows about it too. Don’t conflate “my robots.txt returned a 403” (Google now ignores your rules) with “my pages returned a 403” (those URLs get deindexed). They have opposite effects, and misdiagnosing which one you’re looking at sends you fixing the wrong thing.

How Bing treats 403

Honestly, Bing’s public documentation on 403 specifically is thinner than Google’s, so I’ll scope this narrowly rather than pad it. BingbotBingbot is Microsoft Bing's primary web crawler — the bot that discovers, fetches, and renders pages to build the Bing index. That index also powers Yahoo, DuckDuckGo, Ecosia, and Microsoft Copilot, so Bingbot's reach is far wider than Bing's own search-market share. is denied by the same mechanisms Googlebot is — robots.txt rules, server-level IP/user-agent blocks, and WAF/firewall rules. Bing Webmaster ToolsMicrosoft's free portal for monitoring and improving how a site appears in Bing search — the peer to Google Search Console, plus IndexNow instant indexing, richer backlink data, and keyword volumes. Because Bing's index also feeds Microsoft Copilot, it doubles as a window into AI-search visibility. surfaces crawl errors under its crawl-error alerts. The practical takeaway is the same on both engines: allow the verified crawler through your security layer, and verify the bot by its published IP ranges and reverse DNS rather than by trusting a user-agent string. If anything, treat Bing as a reminder that “fix Googlebot” isn’t automatically “fix all bots” — check both engines’ tools after a WAF change.

Common causes (not ranked — I don’t have prevalence data across sites)

CDN/WAF bot protection — the most underserved cause in existing SEO content. This is where I’d start checking in 2026, even though I can’t tell you how often it’s the actual culprit versus the others below. Cloudflare’s Bot Fight Mode and Super Bot Fight Mode, WAF Managed Rules, and custom firewall rules routinely 403 Googlebot and Bingbot as collateral damage. The tell: the block is at the edge, so your origin server and CMSA content management system (CMS) is software that lets users create, manage, and publish digital content — like blog posts and pages — without writing raw code. WordPress, Drupal, and Joomla are the most common open-source CMS platforms. look totally clean while GSC still reports 403s. Check Security Events in your CDN dashboard for the crawler being challenged or blocked.

2. Server / hosting-level IP or user-agent blocks. Some hosts block by user-agent or rate-limit by default, and IP-range blocks meant for abusive traffic can catch crawler ranges.

3. robots.txt / .htaccess misconfiguration. A stray Deny from or a broken rewrite rule can forbid a whole directory. (And remember the robots.txt gotcha above.)

4. Security plugins. Wordfence, iThemes Security, and similar tools ship with aggressive bot-blocking defaults that can catch legitimate crawlers.

5. Login walls / authenticated content. Anything gated behind auth 403s a bot by design — Googlebot never logs in. This one is often intentional (see the last section).

6. File / directory permission errors. The classic server-admin cause. On WordPress, Rank Math documents sane values — directories 755/750, files 644/640, wp-config.php 400/440 — and “regenerate .htaccess” via the Permalinks settings as a common fix.

7. Malware / compromised site injecting bad access rules, and 8. geoGenerative Engine Optimization — visibility inside AI answer engines.-blocking that inadvertently catches a crawler’s IP range.

Diagnosing a 403 — isolate which layer is issuing it

Most guides jump straight to “disable your plugins.” The real skill is finding which layer is refusing the request — and a bare 403 status code doesn’t tell you that by itself; you need response headers, logs, or a security-event entry that actually points at the CDN, WAF, application, host, permissions, geography, or cache before you name a culprit. Don’t skip straight to “it’s probably the WAF” without evidence. Here’s how the fix differs by layer:

  1. GSC Page Indexing → “Blocked due to access forbidden (403)” to see affected URLs, then URL Inspection → Test Live URL for the current live response.
  2. Reproduce it with curl, swapping user-agents, to confirm the status the server actually returns: 
    # As a generic client
curl -I https://example.com/page/
# Spoofing Googlebot's UA (tests UA-based rules)
curl -I -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://example.com/page/
    If a normal browser UA gets 200 but the Googlebot UA gets 403, you’ve found a user-agent rule.
  3. Check robots.txt’s own status (is it 403-ing? different problem — see above).
  4. Review CDN/WAF Security Events for the crawler being challenged or blocked.
  5. Verify the crawler is really Googlebot via reverse + forward DNS, not the UA string (which is trivially spoofed).
  6. Isolate by disabling in stages — one WAF rule or plugin at a time — until the 403 clears.

Fixing it — allowlist bots the right way

The tempting fix is to allowlist the Googlebot user-agent string. Don’t stop there — UA strings are trivially spoofed, so allowlisting by UA alone is a security hole that lets any scraper claiming to be Googlebot straight in. Verify properly:

  • Confirm bots by reverse DNS + forward DNS or against Google’s/Bing’s published IP ranges.
  • Most CDNs/WAFs offer a “verified bots” category that does this validation for you — prefer that over a raw UA allow rule.
  • Fix the specific rule (a WAF managed rule, one firewall rule, one plugin setting) rather than disabling security wholesale.
  • Then Validate Fix in GSC’s Page Indexing reportThe Google Search Console report (formerly Index Coverage) showing how many of your URLs are indexed vs. not indexed, and grouping the not-indexed ones by reason. and, if it’s urgent, request re-indexing via URL Inspection.

When a 403 is actually fine — don’t “fix” these

Not every 403 is a bug. A 403 is correct and intentional for staging sites, admin areas, private member sections, and paywalled/gated content you never wanted indexed. In an Ahrefs or Screaming Frog audit, a 403 on those is a non-issue — the fix is only needed when a page that’s supposed to be public and indexable is getting blocked by accident. Don’t reflexively “resolve” every 403 an audit throws; confirm the page is one you actually want in the index first.

For the broader family — how 4xx vs 5xx differ, and where 403 sits among them — see my HTTP Status Codes & Their SEO Impact guide, and the sibling 401 Unauthorized401 Unauthorized is a client-error HTTP status code meaning the request lacks valid authentication credentials — the server wants you to log in before it serves the page. For SEO it means the content is gated: Googlebot never sends credentials, so a 401'd page can't be seen and won't be indexed. and 404 Not Found404 Not Found is the HTTP client-error status code a server returns when it can't find the requested URL — RFC 9110 defines it as no current representation, or unwillingness to disclose one. A \"hard 404\" actually returns the 404 status; a \"soft 404\" returns a success code (like 200) for a page that's really gone. 404s are normal and expected: the fact that some URLs 404 doesn't affect your site's other, successful pages, and Google de-indexes 404'd URLs over time (probably retrying for some period, less and less often). deep dives in this cluster.

TIP Reproduce the forbidden response outside your signed-in browser

A 403 can come from the origin, CDN, WAF, geo rule, or bot policy. The status confirms denial; it does not identify the responsible layer by itself.

Use my HTTP Status Checker with representative user agents to confirm which public URL and redirect hop returns 403. HTTP Status Checker Free

  1. Test without cookies from outside the trusted network.
  2. Compare ordinary browser and crawler behavior and inspect the redirect chain.
  3. Correct the responsible access rule, then rerun the same URL and user-agent combinations.

Try it live

This is a real endpoint on this site — not a simulation. Hit it from the button, open it in a new tab, or curl -i it from your terminal, and the server answers with the actual status code this article is about.

Open in new tab ↗

Add an expert note

Pin an expert quote

New person? Create their unclaimed profile at /admin/experts/ → Pin a quote first.