HTTP Error Codes

TL;DR — HTTP error codes are the “something went wrong” responses your server sends instead of a normal “200 OK.” They come in two families: 4xx (the page is missing or blocked — a client problem) and 5xx (the server itself is struggling). Google treats them very differently: a 4xx just drops the page from search; a 5xx makes Google slow down crawling your whole site until the server recovers. And despite the panic they cause, 404 errors are not a penalty — you only need to fix the ones that matter.

What HTTP error codes are

HTTP status codes are standardized response metadata grouped into client-error 4xx and server-error 5xx classes. Evidence for this claim Primary standard or official documentation supporting the adjacent article claim. Scope: Protocol semantics and Search behavior are kept separate; no indexing, ranking, or migration-timing guarantee is inferred. Confidence: high · Verified: 2026-07-14 RFC 9110: Status codes Google documents distinct crawling and indexing handling for those response families. Evidence for this claim Primary standard or official documentation supporting the adjacent article claim. Scope: Protocol semantics and Search behavior are kept separate; no indexing, ranking, or migration-timing guarantee is inferred. Confidence: high · Verified: 2026-07-14 Google: HTTP and network errors

Every time a browser or a search engine bot requests a page, your server answers with a three-digit status code. 200 means “here’s the page, all good.” The error codes are the ones in the 400s and 500s — they mean the request didn’t succeed.

There are two families, and the difference between them is the single most important thing on this page:

• 4xx — client errors. The problem is with the request: the page is gone (404), it’s been permanently removed (410), or you’re not allowed in (401, 403). To a search engine, a 4xx basically says “there’s no content here.”
• 5xx — server errors. The problem is on your end: the server crashed (500), a system it depends on failed (502, 504), or it’s overloaded or down for maintenance (503). To a search engine, a 5xx says “the server itself is broken right now.”

How Google reacts to each family

This is where the two families split:

• A 4xx page just gets dropped from search. Google decides the content doesn’t exist and removes the URL from its index over time. Importantly, a bunch of 404s does not slow down how Google crawls the rest of your site.
• A 5xx makes Google crawl your whole site more slowly. When the server is erroring, Google eases off to avoid making things worse. Your already-indexed pages stay in the index at first — Google only starts dropping them if the errors keep going. Once your server is healthy again, crawling speeds back up on its own.

The thing most people get wrong

A pile of 404 errors is not a penalty. Google’s John Mueller has said flat out that 404s are “not a quality signal, it’s not a SEO signal.” Broken and missing pages are a normal part of the web. So when you open Search Console and see thousands of “Not found” errors, don’t panic — you only need to fix the ones that actually matter: pages people still link to, pages in your sitemap, pages that were getting traffic. The genuinely dead ones can just stay 404.

The one pattern that does have a real, documented downside is a server that keeps throwing 5xx errors — that’s the case where Google throttles crawling and eventually drops pages. Even then it’s reversible: fix the server and it recovers.

A quick tour of the codes

Each of these has its own deep-dive page — here’s the one-line version:

• 401 / 403 — the page is blocked (you need to log in, or you’re forbidden).
• 404 — page not found. 410 — page gone for good.
• 429 — “too many requests” (rate limiting). Looks like a client error but Google treats it like a server error.
• 451 — blocked for legal reasons (a takedown, a country-level block).
• 500 / 502 / 503 / 504 — the server-error family (crash, bad gateway, unavailable, gateway timeout).
• Soft 404 — the sneaky one: a page that says “not found” but returns a 200 code, so Google keeps checking it.

Want the mechanics — exactly how Google’s crawl rate reacts, when pages get deindexed, how to take a site down the right way, and how to monitor all of this in Search Console? Switch to the Advanced tab.

TL;DR — HTTP error codes are the 4xx (client) and 5xx (server) status classes. Google draws a hard behavioral line between them: 4xx (except 429) means “the content doesn’t exist” — the URL is dropped from the index, with “no effect on crawl rate”; 5xx (and 429) means “the server is failing” — Google throttles crawling proportionately, preserves indexed URLs at first, and only deindexes if the errors persist, then ramps crawl rate back up gradually after recovery. 429 is numerically a 4xx but Google calls it “a server error.” Content from any error response is ignored. 404s are not a quality signal (Mueller) — triage by links/traffic, don’t fix everything, and remember an isolated error’s real impact depends on the URL (robots.txt has special error handling that an ordinary page doesn’t). Soft 404s (a 200 that reads as “not found”) are flagged by Google as wasting crawl budget — mainly a large-site concern, not a guaranteed effect on every site. For planned downtime use 503 + Retry-After, kept to “a few days at most,” and never 503 the robots.txt. Monitor the family through GSC’s Page Indexing report. This hub maps and routes to every individual code.

The one distinction that runs the whole topic

The protocol code describes the HTTP outcome; a Search Console label describes how Google classified an observed fetch. Evidence for this claim Primary standard or official documentation supporting the adjacent article claim. Scope: Protocol semantics and Search behavior are kept separate; no indexing, ranking, or migration-timing guarantee is inferred. Confidence: high · Verified: 2026-07-14 RFC 9110: Status codes Do not infer a single root cause or exact removal time from the family alone. Evidence for this claim Primary standard or official documentation supporting the adjacent article claim. Scope: Protocol semantics and Search behavior are kept separate; no indexing, ranking, or migration-timing guarantee is inferred. Confidence: high · Verified: 2026-07-14 Google: HTTP and network errors

Four layers get flattened into “it’s a 404” whenever people talk about errors, and keeping them apart is what makes the rest of this page make sense:

1. Protocol semantics — what the status code means per the HTTP spec (a 404 means “not found,” full stop).
2. The observed fetch — what Googlebot actually got back on a specific request at a specific time, which can differ from what a browser sees.
3. Search processing — how Google’s indexing pipeline classifies and acts on that observed fetch (drops the URL, throttles the crawl, ignores the body).
4. Root cause — the actual reason on your server (a bad deploy, an overloaded database, a WAF rule), which the status code alone never tells you.

Knowing the family (4xx vs 5xx) tells you which of the first three layers you’re looking at. It never substitutes for layer 4 — you still have to go find out why.

Google’s own docs frame the split cleanly, and it’s worth internalizing before anything else: 4xx errors say “the content doesn’t exist”; 5xx errors say “the server itself is failing.” Those are two completely different problems, and Google’s indexing pipeline responds to them in two completely different ways.

A useful anchor first: even a success code isn’t a promise. Google says plainly that “for Google Search, an HTTP 2xx (success) status code doesn’t guarantee indexing.” Error codes are actually the more deterministic half of the picture — they tell Google, unambiguously, either “gone” or “broken.”

How Google treats 4xx errors

For client errors, Google’s behavior is uniform and blunt: “Google doesn’t use the content from URLs that return 4xx status codes,” and “Google crawlers inform the next processing system that the content doesn’t exist.” Even if your 403 or 404 page renders a wall of real text, none of it gets indexed — the body of an error response is ignored.

Two consequences follow:

• Previously indexed URLs get dropped. Once a page reliably returns a 4xx, Google removes it from the index over time.
• There’s no crawl-rate penalty. This is the myth-buster: “The 4xx status codes, except 429, have no effect on crawl rate.” A mountain of 404s does not slow Google’s crawling of the rest of your site.

That last point kills a bad instinct: don’t try to throttle Googlebot with a 401 or 403. Google explicitly warns “don’t use 401 and 403 status codes for limiting the crawl rate.” An auth wall doesn’t slow the crawl — it just makes the content invisible.

How Google treats 5xx errors (and 429)

Server errors trigger the “the server is struggling” branch, and here Google is deliberately protective of your site:

• Crawl rate throttles first, proportionate to volume. “5xx and 429 server errors prompt Google’s crawlers to temporarily slow down with crawling,” and “the decrease in crawl rate is proportionate to the number of individual URLs that are returning a server error.” A handful of 500s barely registers; a site-wide outage cuts crawling hard.
• Indexed URLs are preserved… until the errors persist. “Already indexed URLs are preserved in the index, but eventually dropped,” and Google “removes from the index URLs that persistently return a server error.” A short blip doesn’t deindex you; a sustained failure does.
• Content from 5xx is ignored too. “Any content Google receives from URLs that return a 5xx status code is ignored.”
• Recovery is automatic but gradual. “Once the server starts responding with a 2xx status code, Google gradually increases the crawl rate for the site.” Google is fast to back off and cautious to ramp back up — there’s no manual “unlock,” you just fix the root cause and wait.

Why 429 lives in the server-error bucket

This is the single most-missed nuance in most guides. 429 Too Many Requests is numerically a 4xx, but Google treats it as a server signal: “Google’s crawlers treat the 429 status code as a signal that the server is overloaded, and it’s considered a server error.” So a WAF or rate limiter that starts firing 429s at Googlebot will throttle your crawl the same way a wave of 500s would — not drop individual pages the way a 404 does. When I break the codes down in my HTTP status codes guide at Ahrefs, I put 429 with the server errors for exactly this reason: it’s “a form of rate-limiting to protect the server,” and it makes Google slow down.

The error-code family map

Here’s the fast triage view of the whole family. Each code below is its own deep dive nested under this hub (they’re in the sidebar too):

Blocked / access errors

• 401 Unauthorized — the client hasn’t identified or verified itself when needed. Blocked to Googlebot behind an auth request.
• 403 Forbidden — the client is known but doesn’t have access rights.

Not-found errors

• 404 Not Found — the requested resource isn’t found.
• 404 vs 410 — the practical difference between “not found” and “gone” (it’s smaller than people think).
• 410 Gone — like a 404, but it also says the resource won’t be back. Drops pages slightly faster.

The dual-identity code

• 429 Too Many Requests — rate limiting; numerically 4xx but Google treats it as a server error for crawl-rate purposes.

Legal

• 451 Unavailable For Legal Reasons — blocked for a legal reason: country-level blocks, DMCA takedowns.

Server errors (5xx)

• 500 Internal Server Error — the server hit an issue it can’t handle.
• 502 Bad Gateway — a bad response from an upstream server.
• 503 Service Unavailable — the server is overloaded or down for maintenance (the correct code for planned downtime).
• 504 Gateway Timeout — no timely response from an upstream server.

The trap case

• Soft 404 — a page that returns 200 OK but reads as “not found.” The worst of both worlds — more on this below.

Broken redirects are error-adjacent but categorized separately (Google surfaces them as “Redirect error” in Search Console, distinct from a normal, working “Page with redirect”). By default Google’s crawlers “follow up to 10 redirect hops” before giving up; a chain that’s too long, loops, or contains a bad URL turns into that Redirect-error status.

Does an HTTP error hurt your SEO?

Lead with the verdict: an isolated error on an ordinary page is almost never a problem, and 404s specifically are not a ranking or quality signal. Mueller has been explicit and repeated about this. The reflex — “I have 50,000 404s, my site must be penalized” — is the number-one myth to defuse. Errors are a normal part of the web; having pages 404 or 410 is the technically correct way to handle URLs that don’t exist.

“Isolated” doesn’t mean “always harmless,” though — the real impact depends on what’s returning the error. An ordinary page 404ing is a non-event; a special resource is different. Google gives robots.txt its own error-handling rules distinct from regular URLs, so a server error on robots.txt can affect crawling in a way an ordinary page’s 404 never would. Judge an error by what URL it’s on and what depends on it, not by the status code alone.

The one pattern with a real, documented mechanism is persistent, mass 5xx: crawl-rate throttle → eventual deindexing. And even that is proportionate to how many URLs are erroring and generally reversible once the server recovers — Google describes the ramp-back-up as gradual, not instant, so treat the exact timing, retry cadence, and recovery speed as evidence-dependent (what Google’s docs describe happening) rather than a fixed guarantee. The distinction Mueller draws is the one to remember: crawl rate reacts to server errors (429/500/503/ timeouts), not to 404s.

Crawl waste vs deindexing — two different harms

It helps to separate the two ways errors can cost you:

• Deindexing — persistent 4xx (page dropped as “gone”) or persistent 5xx (pages dropped after sustained server failure). This is about pages leaving the index.
• Crawl-budget waste — mostly a large-site concern. Google’s crawl-budget guide notes “if the site slows down or responds with server errors, the limit goes down and Google crawls less.” But the real budget-waster is the soft 404: “soft 404 pages will continue to be crawled, and waste your budget.” Because a soft 404 looks alive (it returns 200), Google keeps re-checking a page that isn’t really there. That’s why the soft 404 is the “worst of both” case — it doesn’t cleanly drop like a true 404, it lingers and burns fetches.

The soft-404 trap has two common causes, both worth naming: a custom “page not found” template that returns 200 instead of a real 404, and a blanket redirect of every dead URL to the homepage — Google recognizes that pattern as a soft 404 too. A friendly 404 page is good for UX and fully recommended — as long as it still returns the actual HTTP 404 status code.

Doing planned downtime the right way

When you intentionally take a site (or a section) offline, the correct code is 503 Service Unavailable with a Retry-After header — never a 4xx. Google’s “Pause your online business” guidance is specific:

• “If you need to urgently disable the site for 1-2 days, then return an informational error page with a 503 HTTP response status code.”
• “This is an extreme measure that should only be taken for a very short period of time (a few days at most),” because “completely closing a site even for just a few weeks can have negative consequences on Google’s indexing of your site.”
• “Don’t block the website by returning 403, 404, 410 HTTP status codes” during downtime — a 4xx says “permanently gone,” which is exactly the wrong signal for a temporary outage.
• The trap almost everyone misses: “Don’t return a 503 HTTP response status code for the robots.txt file because this blocks all crawling.” 503 your pages, not your robots.txt.

How to monitor the whole family in Search Console

Day to day, you’ll meet these errors in Search Console’s Page Indexing report, where each maps to a distinct status:

• Not found (404) — “this page returned a 404 error when requested.”
• Server error (5xx) — “your server returned a 500-level error when the page was requested.”
• Blocked due to unauthorized request (401) — “the page was blocked to Googlebot by a request for authorization.”
• Blocked due to access forbidden (403) — a 403 where credentials were provided but access wasn’t granted.
• Blocked due to other 4xx issue — a 4xx not covered by another status; use URL Inspection to debug.
• Soft 404 — a “user-friendly ‘not found’ message but not a 404 HTTP response code.”
• Redirect error — chain too long, a loop, an over-long URL, or a bad URL in the chain.

Each status points at a different root cause and fix path. Once you’ve resolved one, use Validate Fix to prompt a recrawl — but set realistic expectations on timing; recrawl isn’t instant. And remember Google’s own framing: “it’s fine for a URL not to be indexed for the right reasons — for example… a 404 for a page that you’ve removed and have no replacement for.” Not every error is a to-do.

Search Console alone isn’t enough to act on — it’s one vantage point, sampled and delayed. When you log an error for triage, record at minimum: the URL, when you observed it, the vantage point (Search Console vs. a live check vs. server/CDN logs), the user agent the request came in on, the HTTP method, the final status code and response path (including any redirect chain), whether it’s a one-off or recurring, and a post-fix verification step once you’ve redeployed. Triangulating GSC against a live status check and your own logs is what turns a stale report row into a confirmed, fixable problem.

How to fix and prioritize errors

• Triage 404s by value. Fix the ones with inbound links, internal links, sitemap presence, or lingering traffic — 301-redirect those to a relevant page to recover the link equity. Let genuinely dead URLs 404 or 410. As I put it in my Ahrefs guide, the practical fix for most of these is that “you just need to 301 redirect each of these pages to a relevant page” — but only where a relevant target exists. Don’t blanket-redirect everything to the homepage (that’s a soft 404).
• 410 vs 404 is marginal. A 410 drops a page slightly faster than a 404; the practical SEO difference is minor. Use 410 when you want to be explicit that something is gone for good, but don’t expect it to be dramatically better.
• Root-cause 5xx. The fixes are on the server side: capacity and timeouts for 500s, upstream/CDN health for 502/504, and WAF or rate-limiting rules misfiring on Googlebot for 403/429. Confirm the real bot with a reverse/forward DNS check before you go rate-limiting it.
• Common root causes worth naming: app and database errors and overloaded hosts (5xx), upstream/CDN failures (502/504), rate limiting or bot-blocking WAFs (403/429), broken migrations and stale internal links (404), and misconfigured “friendly error pages” (soft 404).

Bing: a similar pattern, not independently verified code-for-code

Bing’s public statements point in the same direction as Google’s approach — 400-range codes are treated as missing or forbidden, and 500-range codes signal server trouble that works against crawl efficiency — but I haven’t independently verified full, current, code-by-code parity against Bing’s own documentation, so treat this as directional rather than a confirmed one-to-one match. Fabrice Canel frames Bing’s goal as a “crawl efficiency north star … to crawl a URL only when the content has been added … updated,” and persistent errors work directly against that — Bing spends crawl footprint on URLs that aren’t yielding fresh, indexable content. Bing also recommends a 503 with Retry-After for planned downtime rather than serving error pages as 200. You’ll find Bing’s crawl-error surfaces in Bing Webmaster Tools (URL Inspection, Crawl Control, Site Scan).

Where to go next

This page is the conceptual hub for the error-code family. It sits inside the broader HTTP Status Codes cluster (the full 1xx–5xx picture, plus redirects and success codes); this sub-hub is the map for the error half of that. Each code below is its own deep dive:

Blocked / access

• 401 Unauthorized — what triggers the “Blocked due to unauthorized request” status, and why auth walls don’t throttle Googlebot.
• 403 Forbidden — credentials-provided-but-denied, and the WAF/bot-blocking patterns that cause false 403s to Googlebot.

Not found

• 404 Not Found — how Google handles missing pages, why it’s not a penalty, and which 404s to actually fix.
• 404 vs 410 — the real, small difference, and when to reach for each.
• 410 Gone — the “permanently gone” signal and its slightly faster drop.

Rate limiting

• 429 Too Many Requests — the 4xx that behaves like a 5xx, and how to keep rate limits from throttling your crawl.

Legal

• 451 Unavailable For Legal Reasons — takedowns, country blocks, and how legal removals show up.

Server errors

• 500 Internal Server Error — the generic server failure and how to root-cause it.
• 502 Bad Gateway — upstream/proxy failures.
• 503 Service Unavailable — the correct code for maintenance and planned downtime (with the robots.txt trap).
• 504 Gateway Timeout — upstream timeouts.

The trap case

• Soft 404 — the 200-that-reads-as-gone, why it wastes crawl budget, and how to turn it into a real 404.

Broken redirects are handled separately as the Redirect error status — related but categorized on its own in Search Console.

AI summary

A condensed take on the Advanced version:

• Two families, two behaviors. 4xx = “content doesn’t exist”; 5xx = “the server is failing.” Google’s indexing pipeline responds to each very differently.
• 4xx (except 429): content is ignored, the URL drops from the index, and there’s “no effect on crawl rate.” Don’t use 401/403 to throttle Googlebot — Google explicitly says not to.
• 5xx (and 429): Google throttles crawling first (proportionate to how many URLs error), preserves indexed URLs at first, and deindexes only if errors persist. Recovery is automatic but the crawl rate ramps back up gradually.
• 429 is the odd one out: numerically 4xx, but Google treats it as “a server error” for crawl-rate purposes — it behaves like a 5xx.
• 404s are not a quality/ranking signal (Mueller). Don’t panic over an error count; triage by links, sitemap presence, and traffic — 301 the ones that matter, let dead pages 404/410.
• An isolated error’s impact depends on the URL, not just the status code — an ordinary page 404ing is a non-event, but special resources like robots.txt have their own error handling and can affect crawling differently.
• Soft 404 is the trap: a 200 that reads as “not found,” which Google flags as wasting crawl budget because it keeps re-checking a page that looks alive — mainly a large-site concern, not a guaranteed effect on every site.
• 410 vs 404 is marginal — 410 drops slightly faster; the practical difference is minor.
• Planned downtime: use 503 + Retry-After, “a few days at most,” never 403/404/410, and never 503 your robots.txt (it blocks all crawling).
• Monitor via GSC’s Page Indexing report — seven statuses (404, 5xx, 401, 403, other 4xx, soft 404, redirect error) — plus your own live checks and server logs, and use Validate Fix.
• Bing’s public statements point the same direction as Google’s — but full code-by-code parity isn’t independently verified; Bing also recommends 503 + Retry-After for downtime.

Official documentation

Primary-source documentation from the search engines.

Google

• HTTP status codes, network errors, and DNS errors — the definitive doc on how Google handles 4xx, 5xx, 429, redirects, and network/DNS errors.
• Page Indexing report (Search Console Help) — every error status you’ll see day to day (404, 5xx, 401, 403, other 4xx, soft 404, redirect error) and Validate Fix.
• Optimize your crawl budget — how server errors reduce the crawl limit and why soft 404s waste budget.
• Pause your online business in Google Search — the 503 + Retry-After playbook for planned downtime, and what not to do.
• Soft 404 errors — what a soft 404 is and how to fix it.
• Make your 404 pages more useful — older but still-standing guidance: a friendly 404 page is fine as long as it returns a real 404 code.

Bing / Microsoft

• Crawl error alerts (Bing Webmaster Tools Help) — where Bing surfaces crawl errors.
• bingbot Series: Maximizing Crawl Efficiency — Bing’s “crawl efficiency north star,” which persistent errors work against.

Quotes from the source

On-the-record statements from Google and Bing. Each link is a deep link that jumps to the quoted passage on the source page.

Google — the 4xx vs 5xx split

• “For Google Search, an HTTP 2xx (success) status code doesn’t guarantee indexing.” — Google Search Central docs. Jump to quote
• “Google doesn’t use the content from URLs that return 4xx status codes.” Jump to quote
• “The 4xx status codes, except 429, have no effect on crawl rate.” Jump to quote

Google — 5xx and 429

• “5xx and 429 server errors prompt Google’s crawlers to temporarily slow down with crawling.” Jump to quote
• “already indexed URLs are preserved in the index, but eventually dropped.” Jump to quote
• “Any content Google receives from URLs that return a 5xx status code is ignored.” Jump to quote
• “Google’s crawlers treat the 429 status code as a signal that the server is overloaded, and it’s considered a server error.” Jump to quote

Google — crawl budget and soft 404

• “If the site slows down or responds with server errors, the limit goes down and Google crawls less.” Jump to quote
• “Soft 404 pages will continue to be crawled, and waste your budget.” Jump to quote

Google — planned downtime

• “If you need to urgently disable the site for 1-2 days, then return an informational error page with a 503 HTTP response status code.” Jump to quote
• “Don’t return a 503 HTTP response status code for the robots.txt file because this blocks all crawling.” Jump to quote

Google — it’s fine not to be indexed

• “It’s fine for a URL not to be indexed for the right reasons — for example, an expected robots.txt rule on your site, a noindex tag on the page, a duplicate URL, or a 404 for a page that you’ve removed and have no replacement for.” — Search Console Help. Jump to quote

John Mueller, Google — 404s are not a quality signal

• “404s are not a quality signal, it’s not a SEO signal.” Read the coverage
• On crawl-rate reactions: “I’d only expect the crawl rate to react that quickly if they were returning 429 / 500 / 503 / timeouts… (404s are generally fine & once discovered, Googlebot will retry them anyway)… Once things settle down on the server, the crawl rate will return to normal automatically.” Read the coverage

Fabrice Canel, Microsoft Bing

• “Our crawl efficiency north star is to crawl a URL only when the content has been added (URL not crawled before), updated (fresh on-page context or useful outbound links).” Jump to quote

Which error do I deal with first?

When a batch of errors shows up, work the triage in this order — server health before missing pages, because that’s where the real, site-wide damage lives.

1. Are you seeing 5xx or 429 (Server error, or a spike of timeouts)?

• Yes → Treat it as urgent. This is the branch that throttles crawling site-wide and can deindex pages if it persists. Root-cause it now: server capacity/timeouts (500), upstream/CDN health (502/504), or a WAF / rate limiter misfiring on Googlebot (403/429). Recovery is automatic once you return 2xx, but crawl rate ramps back up gradually.
• No → Move on; 4xx has no crawl-rate effect.

2. Is it a soft 404 (a 200 that reads as “not found”)?

• Yes → Fix the response code. Either return a real 404/410, or if the page moved, 301 it to a relevant page. Stop blanket-redirecting dead URLs to the homepage. Soft 404s waste crawl budget because Google keeps re-checking them.
• No → Continue.

3. Is it a 404 / 410 (Not found / Gone)?

• Does the URL have inbound links, internal links, sitemap presence, or lingering traffic?
  • Yes → 301-redirect it to a relevant page to recover the equity.
  • No → Leave it. A genuinely dead page should just 404/410 — it’s not a penalty, and you don’t need to fix it.

4. Is it a 401 / 403 (Blocked)?

• Should this page be public? Yes → Fix the auth/permission config or the WAF rule that’s blocking Googlebot. No → It’s fine to be blocked — but don’t expect the 401/403 to throttle Googlebot; it only makes the content invisible.

5. Is it a 451 (Legal)?

• That’s a legal takedown or geo-block, not a technical bug — handle it through the legal/compliance process, not the SEO one.

6. Is it a Redirect error?

• Shorten the chain (Google follows up to ~10 hops), break the loop, or fix the bad/over-long URL in the chain so it resolves to a single final 200.

Deindexing or crawl waste — which harm am I actually facing?

• Pages leaving the index? → Look for persistent 4xx (dropped as “gone”) or persistent 5xx (dropped after sustained failure). Isolated errors don’t do this.
• Crawl budget being burned (large sites)? → Look for soft 404s and mass errors — the pages that keep getting re-crawled because they look alive.

Error-code → behavior → fix cheat sheet

How Google treats each code

Fast facts

• 4xx (except 429): content ignored, URL dropped, zero crawl-rate effect.
• 5xx + 429: throttle first, deindex only if persistent, recovery is gradual.
• 404s are not a quality/ranking signal — triage by links/traffic.
• Soft 404 = the trap: 200 that reads as gone → wastes crawl budget.
• Planned downtime = 503 + Retry-After, “a few days at most,” never 403/404/410 — and never 503 the robots.txt (blocks all crawling).
• Redirects: Google follows up to ~10 hops; over-long/looping chains = “Redirect error” in GSC.

GSC Page Indexing statuses to watch Not found (404) · Server error (5xx) · Blocked due to unauthorized request (401) · Blocked due to access forbidden (403) · Blocked due to other 4xx issue · Soft 404 · Redirect error — then Validate Fix once resolved.

Resources worth your time

My related writing

• HTTP Status Codes & Their SEO Impact — my full code-by-code breakdown: 4xx, 5xx, 429, and the practical fixes, across the whole status-code range.
• 11 Types Of Redirects & Their SEO Impact — the redirect side, which is a leading cause of both redirect errors and soft 404s.
• The Beginner’s Guide to Technical SEO — where status codes fit in the bigger picture.

My speaking

• How Search Works (SlideShare) — my walkthrough of crawling, rendering, indexing, and serving, including how status codes gate the pipeline. (My standing disclaimer applies: “This is my understanding of systems… not going to be 100% complete or accurate.”)

From around the industry

• Googlebot Crawl Slump? Mueller Points To Server Errors (Search Engine Journal) — the write-up anchoring the “crawl rate reacts to 429/500/503/timeouts, not 404s” distinction, in Mueller’s own words.
• The ultimate guide to HTTP status codes for SEO (Search Engine Land) — a broad reference across every status class.
• HTTP Status Codes Explained (Semrush) — implementation-focused overview of the code families.
• An SEO Guide to HTTP Status Codes (Search Engine Journal) — a code-by-code reference with SEO context.
• r/TechSEO — the community for crawl-error and indexing debugging.

The two-harm model: disappearance vs. crawl throttling

Do not prioritize an HTTP error from the first digit alone. Ask which of two harms it creates.

Harm 1: the URL says the content does not exist

Most 4xx responses tell Google the requested content is unavailable. Google does not index the returned body, and a persistently affected URL can leave the index. The scope is usually the URL itself; ordinary 4xx responses do not tell Google that the whole server is unhealthy. The exception is 429, which Google handles like a server availability problem.

Decision question: Is this URL meant to exist, receive traffic, or preserve links? If yes, fix the erroneous status or send users to a genuinely equivalent destination. If no, a 404 or 410 may be the correct outcome rather than an SEO issue.

Harm 2: the server says it cannot reliably respond

5xx responses and 429 signal availability or capacity trouble. Google can reduce crawling to avoid adding load; only sustained failures threaten indexed URLs. The harm can therefore spread beyond one failing URL into slower discovery and refresh across the site.

Decision question: Is the failure persistent or broad enough to change crawler behavior? If yes, treat it as an infrastructure incident and restore reliable responses. Do not “fix” it inside Search Console.

Priority overlay: value and intent

After choosing the harm, rank affected URLs by whether they should be live, have internal or external links, appear in a sitemap, receive search traffic, or represent a critical template. An error count without this context is not a priority list.

Evidence/action matrix: what to record per error

The two-harm model tells you which problem you’re facing. This matrix is what to actually record for each affected URL before you decide what to do with it — skipping any of these columns is how a triage list turns into a guess:

Prompt: turn a GSC error export into a triage queue

Export the affected URL list from Search Console and enrich it where possible with status code, sitemap presence, organic clicks/impressions, internal links, external links, and intended URL state. Paste only non-sensitive data.

You are triaging an HTTP-error URL export for technical SEO. Use only the supplied
columns. Do not assume that every 404 is a problem or that every 5xx is permanent.

For each URL:
1. Classify the response as 4xx disappearance risk, 5xx/429 crawl-throttling risk,
   soft-404 mismatch, or insufficient evidence.
2. Decide whether the URL appears intended to exist, based only on sitemap presence,
   links, traffic, template, and the supplied intended-state field.
3. Assign priority: incident-now, high, medium, low, or expected/no fix.
4. Explain the priority using the exact supplied signals.
5. Recommend the next verification, not a guessed root cause.

Prioritize broad or persistent 5xx/429 incidents first, then intended URLs with
traffic or links, then sitemap/internal-link hygiene. Keep intentional 404/410 URLs
out of the fix queue. Return a table plus a short list of missing data that would
change the ranking.

DATA:
[PASTE SANITIZED EXPORT HERE]

Verify the output against live status checks and the site’s intended URL map before assigning work.

Tools for error triage

• Bulk HTTP Status Code Checker — paste the exported URLs to collect live status codes, redirect chains, destinations, and latency, then filter and export the result. This converts a stale report row into current HTTP evidence.
• Website Down Checker — check whether a suspected 5xx incident is externally reproducible before changing application or crawler settings.
• Google Search Console Page Indexing report — identify Google’s reason groups and affected examples; use URL Inspection for a representative live check.
• Server or CDN access logs — measure crawler-facing error rates and distinguish a single URL problem from a site-wide availability pattern.

Search Console reports what Google observed; it does not replace a live request or the server evidence needed to find the cause.

Intended URLs returning errors

Metric: Count and share of URLs that are supposed to be live but return 4xx, 5xx, or 429, segmented by response family and template.

What it tells you: Separates real availability/indexing defects from intentional 404/410 cleanup.

How to pull it: Join a current crawl or Bulk HTTP Status Code Checker export with your sitemap, internal-link graph, and intended URL inventory.

Benchmark / realistic range: The target is zero unintended errors on URLs that must be live. Expected retired URLs should be tracked separately, not forced into a made-up site-wide error-rate target.

Cadence: Daily for critical templates and during releases; weekly for the broader inventory.

Crawler request error rate

Metric: Search-engine requests returning 4xx, 429, or 5xx as a share of verified crawler requests, split by status, hostname, and path group.

What it tells you: Shows whether crawler waste is concentrated in retired URLs or whether server failures may be suppressing crawl demand.

How to pull it: Aggregate verified crawler requests from CDN or server access logs; do not trust the user-agent string alone when verification is available.

Benchmark / realistic range: Establish the normal mix for your site. Intentional 404s may be legitimate; unexplained 5xx/429 responses on intended URLs should not be treated as normal baseline noise.

Cadence: Monitor continuously for 5xx/429 alerts; review segmented trends weekly.

Search Console error inventory trend

Metric: Affected URL counts by Page Indexing reason, paired with representative URL state and report update date.

What it tells you: Whether Google’s observed inventory is shrinking after fixes or expanding into additional templates.

How to pull it: Export the Page Indexing reason tables and retain dated snapshots; spot-check examples with URL Inspection and live requests.

Benchmark / realistic range: Compare with the site’s intended state and its own prior snapshots. Search Console is sampled and delayed, so do not invent a real-time pass threshold.

Cadence: Weekly and after Google has had time to recrawl a deployed fix; use logs and live checks for immediate incident monitoring.

Test yourself: HTTP Error Codes

Five quick questions on how Google handles 4xx and 5xx errors. Pick an answer for each, then check.

HTTP Error Code

The 4xx (client error) and 5xx (server error) HTTP status codes a server returns instead of a successful 2xx — and how Google treats each class very differently.

Related: 404 Not Found, 410 Gone, 429 Too Many Requests, 500 Internal Server Error, 503 Service Unavailable, Soft 404, HTTP Status Code

HTTP Error Code

An HTTP error code is any response in the 4xx (client error) or 5xx (server error) class that a server returns instead of a successful 2xx response. In SEO terms they matter because Googlebot’s reaction to an error code decides whether a URL gets indexed, stays indexed, or gets dropped — and, for 5xx and 429, how fast Google keeps crawling the whole site.

The fundamental split: 4xx means “the content doesn’t exist” and 5xx means “the server itself is failing.” Google treats those two situations very differently. 4xx codes (except 429) tell Google the content is gone, so the URL drops from the index — but they have no effect on crawl rate. 5xx codes (and 429) tell Google the server is struggling, so Google throttles crawling site-wide first and only drops the pages if the errors persist.

429 (Too Many Requests) is the odd one out: numerically it’s a 4xx, but Google explicitly treats it as a server error for crawl-rate purposes, so it behaves like a 5xx. A soft 404 is the trap case — a page that returns 200 OK but reads as “not found,” which wastes crawl budget instead of cleanly dropping the URL.

You monitor the whole family through Google Search Console’s Page Indexing report, where each error surfaces as its own status: Not found (404), Server error (5xx), Blocked due to unauthorized request (401), Blocked due to access forbidden (403), Blocked due to other 4xx issue, Soft 404, and Redirect error.

Related: 404 Not Found, 410 Gone, 429 Too Many Requests, 500 Internal Server Error, 503 Service Unavailable, Soft 404, HTTP Status Code

Changelog