504 Gateway Timeout

TL;DR — A 504 Gateway TimeoutA 504 Gateway Timeout is an HTTP 5xx server error a gateway or proxy (a CDN, load balancer, or reverse proxy) returns when it doesn't get a timely response from the upstream server behind it. Unlike a 502 (a bad/garbled response) or a 503 (server explicitly unavailable), a 504 means no response arrived in time. For SEO, isolated 504s are retried and tolerated, but sustained 504s and timeouts make Googlebot slow its crawl and eventually drop pages from the index. means something in front of your website — a CDN, load balancer, or proxy — waited for your server to answer and gave up because the answer took too long. It’s a timeout, not a broken response. A stray 504 here and there is fine; the problem is when they keep happening, because search engines will crawl you less and, eventually, drop pages.

What a 504 actually means

When you load a page, the request often doesn’t hit your web server directly. It usually passes through a gateway first — a CDN, a load balancer, or a reverse proxy. That gateway forwards the request to the server behind it (the upstream or origin), waits for a response, and passes it back to you.

A 504 Gateway Timeout is what the gateway returns when it waited for that upstream response and never got one in time. The origin might be slowly grinding through a database query, waiting on a third-party API, or just overloaded — but from the gateway’s point of view, the clock ran out. Evidence for this claim A 504 response means a gateway or proxy did not receive a timely response from an upstream server. Scope: RFC 9110 defines the timeout semantics; it does not determine why the upstream response was delayed. Confidence: high · Verified: 2026-07-14 IETF: RFC 9110 §15.6.5 — 504 Gateway Timeout

The key word is timeout. Nothing was necessarily “broken.” The response simply didn’t arrive fast enough.

How it’s different from its 5xx siblings

People mix these up constantly:

• 502 Bad GatewayA 502 Bad Gateway is an HTTP server error a gateway or reverse proxy (a CDN, load balancer, or proxy like Nginx) returns when it gets an invalid or no response from the upstream origin server it was trying to reach. It signals a communication failure *between* servers, not necessarily that the origin itself is down. For SEO, Google treats 502 exactly like 500 and 503: crawling slows, and persistent errors eventually get pages dropped from the index. — the gateway did get a response from the origin, but it was invalid or garbled.
• 503 Service Unavailable503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime. — the server explicitly said “I’m unavailable right now” (often on purpose, like a maintenance window).
• 504 Gateway Timeout — the gateway didn’t get a timely response from the upstream. That’s a narrower claim than “nothing came back” — the upstream may still be working on it — it just didn’t answer inside the waiting window.

So a 504 is almost always a performance symptom: something upstream is too slow.

Does a 504 hurt my SEO?

Not directly, and it’s not a penalty. Google isn’t judging your content — it literally can’t retrieve the page. But there’s a real, indirect cost:

• If GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. keeps hitting slow responses and timeouts, it backs off and crawls your site less so it doesn’t make the overload worse.
• A single 504 during a traffic spike gets retried and mostly ignored.
• 504s that keep happening over days are the ones that get pages dropped from the indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. — because Google can’t reliably fetch them. Evidence for this claim Google reduces crawling when server errors or slow responses indicate a site is having trouble responding. Scope: Google's documentation supports automatic crawl throttling for server trouble; it does not attribute a specific 504 to any particular bottleneck. Confidence: high · Verified: 2026-07-14 Google: Troubleshoot crawling errors

What to do about it

• Reload once or twice — a one-off 504 may just be a blip.
• Check your server and error logs to see what was slow (a database query, an external API, an overloaded app process).
• Don’t just crank up the timeout value and call it done — that hides the slow response instead of fixing it (more on why in the Advanced tab).
• Keep your server fast: cachingCaching stores a copy of a page or resource — in a browser, a CDN edge node, or a search crawler's own cache — so it can be served again without regenerating or re-downloading it. It isn't a direct ranking factor, but it feeds page speed and crawl efficiency., faster queries, and enough capacity are the real prevention.

Want the full technical version — where a 504 originates in the request chain, how GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer.’s crawl throttling works, and why TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. is the metric to watch? Switch to the Advanced tab.

TL;DR — A 504 is a gateway/proxy telling you the upstream server didn’t respond within the timeout window — a timeout, distinct from a 502 (bad response) or a 503503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime. (explicit unavailability). It’s an accessibilityWeb accessibility means designing sites so people with disabilities can use them, per the W3C's WCAG guidelines. It overlaps with SEO in specific, checkable ways — alt text, heading structure, descriptive link text, captions, and page speed all serve both audiences — but Google has said accessibility itself is not a ranking factor, and most WCAG success criteria (keyboard focus order, ARIA live regions, form labels) have no SEO effect at all. problem, not a penalty. Timeouts sit in the same bucket as 429429 Too Many Requests is an HTTP client-error status code that means a client sent too many requests in a given time window — a mechanism called rate limiting. Unlike other 4xx codes, Google treats 429 as a server-overload signal and slows crawling instead of removing content./500/503: GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. backs off when it sees them, and sustained 504s risk deindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist.. The durable fix is server response time (TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good.), not a longer timeout value — raising the timeout usually masks a slow upstream and can make things worse under load.

Where a 504 originates in the request chain

A modern request path looks roughly like: browser → CDN/edge → load balancer → reverse proxy (e.g. Nginx) → app server (PHP-FPM, Node, etc.) → database / third-party APIs. A 504 is generated by whichever component was waiting on the one behind it when its timeout expired. That’s the first diagnostic question: Evidence for this claim A 504 response means a gateway or proxy did not receive a timely response from an upstream server. Scope: RFC 9110 defines the timeout semantics; it does not determine why the upstream response was delayed. Confidence: high · Verified: 2026-07-14 IETF: RFC 9110 §15.6.5 — 504 Gateway Timeout

• CDN/edge timing out on origin → the fix is origin performance, or raising the CDN’s upstream timeout (carefully — see below).
• Load balancer timing out on the app server → check app-server health and autoscaling.
• Reverse proxy timing out on the app process → look at Nginx’s proxy_read_timeout / fastcgi_read_timeout and the slow query or process that’s actually behind it.

Getting the layer right matters, because “fix the timeout at the edge” and “fix the slow database query at the origin” are completely different jobs.

What causes 504s

The status code itself doesn’t prove a cause — it only tells you a gateway timed out waiting on an upstream. These are the usual suspects worth checking, not facts the 504 already established; confirm with logs and traces before you act on one:

• Slow database queries or upstream API calls. A single unindexed query or a laggy third-party dependency can push response time past the timeout.
• Server/app overload and resource exhaustion. Under enough concurrent load, requests queue, worker processes fill up, and responses stop arriving in time.
• Misconfigured timeout values across Nginx, Apache, the load balancer, or the CDN — often mismatched between layers so one gives up before another.
• Traffic spikes, botA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. floods, or DDoS that overwhelm capacity temporarily.

504s are often intermittent and load-dependent

This is what makes them nasty. Unlike a hard outage, a 504 frequently appears only under load — which means an uptime monitor pinging during a quiet window can show 100% green while GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer., crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. in heavier bursts, is quietly collecting timeouts. In Google Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results.’s URL Inspection you may see this surface as a “Hostload exceeded” condition rather than a clean, constant error. If your monitoring says everything’s fine but Crawl Stats disagrees, load-dependent 504s are a prime suspect.

How Googlebot (and Bingbot) handle timeouts

Google doesn’t treat a 504 as a content-quality judgment — it’s an availability signal, and the response is automatic throttling. Its crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. docs are explicit: “Googlebot will scale back its crawling if it detects that your servers are having trouble responding to crawl requests.” The large-site crawl-budget guide says the same thing in budget terms: “If the site slows down or responds with server errors, the limit goes down and Google crawls less.” Evidence for this claim Google reduces crawling when server errors or slow responses indicate a site is having trouble responding. Scope: Google's documentation supports automatic crawl throttling for server trouble; it does not attribute a specific 504 to any particular bottleneck. Confidence: high · Verified: 2026-07-14 Google: Troubleshoot crawling errors

The freshest framing tying this straight to slow/timing-out responses comes from Google’s March 2026 Inside Googlebot explainer, where Gary Illyes is quoted saying: “If your server is struggling to serve bytes, our crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. will automatically back off to avoid overloading your infrastructure, which will drop your crawl frequencyCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial..” (As reported by Search Engine Land’s coverage — worth confirming against the original recording or transcript if you’re citing it elsewhere.) A 504 is that struggle made visible.

One nuance worth understanding: Google doesn’t always see a literal “504” the way your browser does. If the timeout happens before Googlebot gets any status line at all, it’s logged as a timeout/network error in Crawl Stats rather than a clean 5XX. But when an intermediary proxy or CDN times out and generates its own 504 response, Googlebot receives a standard 5XX server error. Either way the effect is identical: throttled crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor., and, if sustained, removal from the index.

Bing documents timeouts as their own crawl-error category, separate from server errors — BingbotBingbot is Microsoft Bing's primary web crawler — the bot that discovers, fetches, and renders pages to build the Bing index. That index also powers Yahoo, DuckDuckGo, Ecosia, and Microsoft Copilot, so Bingbot's reach is far wider than Bing's own search-market share. stops trying to access pages when responses are too slow — and its standing recommendation is to check server response time, keep server software updated, optimize slow resources, and read server logs for systemic patterns rather than one-off blips. Bing hasn’t published the same level of throttle-and-recover detail Google has, so treat the two as broadly comparable in intent rather than confirmed to behave identically.

The throttle-then-recover feedback loop

The reassuring part: this loop is automatic and self-correcting. Google throttles down when it sees errors and timeouts, then ramps crawling back up gradually once responses are healthy again. There’s no manual “unthrottle” button you need to press once the underlying issue is fixed — as John Mueller noted, “Once things settle down on the server, the crawl rate will return to normal automatically.” He also flagged the asymmetry: throttling down happens fast to resolve an immediate problem; ramping back up is done cautiously.

Duration is what converts a blip into an indexing problem

Short, occasional 504s — a few during a spike, resolved quickly — get retried and are largely tolerated. What risks deindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist. is a sustained pattern of timeouts over an extended window. Google’s documented “come back later” mechanics — where you intentionally return 503 or 429 during overload — describe roughly a two-day horizon before URLs start dropping, but that specific number is documented for 503/429 by name, not for 504. Google’s broader 5xx guidance says crawl rate drops proportionally to how many URLs are erroring and recovers gradually once responses are healthy, without naming a fixed horizon for 504 specifically. Extending the two-day figure to 504 is a reasonable inference, not a documented fact — treat it as: brief 504s are survivable, and a sustained pattern over days is the range where you should worry, not a guaranteed trigger date.

Why this matters more on large and ecommerce sites

If you have a small site whose pages get crawled the same day they publish, you’ll almost never notice this. But on large or fast-changing sites — big ecommerce catalogs, news, marketplaces — crawl budgetThe number of URLs an engine will crawl in a timeframe. is already the constraint, and a wave of 504s during peak load can starve crawling of pages that genuinely need to be recrawled. Timeouts and crawl budget are the same conversation at scale.

Server response time and TTFB are a strong preventive lever — for origin-side causes

Here’s a point most 504 articles miss: waiting until errors appear and then log-grepping is reactive. Monitoring server response time continuously, so you see the drift before it tips into a timeout, is proactive. A server that’s slow but not yet timing out today becomes a 504-generating server tomorrow under slightly more load or a slightly slower dependency. Watching TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. (time to first byte) as an ongoing early-warning signal — not just a post-mortem metric — is how you catch that drift.

The caveat: TTFB monitoring addresses origin-side slowness. It won’t catch a CDN timing out on a healthy origin, a load balancer misconfigured to give up too early, or a network path problem between hops — those need the emitting hop identified first (the “Where a 504 originates” section above, or the decision tree below). TTFB is a real, durable fix for the common case of a genuinely slow upstream; it’s not a universal one. Once you’ve confirmed the origin is the bottleneck, optimize with caching (page, object, and CDN), faster database queries and proper indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., autoscaling for load, and sensible CDN upstream tuning.

Why “just raise the timeout” is the wrong instinct

Bumping proxy_read_timeout or the CDN’s upstream timeout can stop the 504 from showing — but it doesn’t make the upstream response any faster. Worse, a longer timeout window under load means requests pile up and tie up worker processes and connections for longer, which can make an overload spiral worse, not better. Raising the timeout is occasionally the right call (for a genuinely long-running, well-understood operation), but as a reflex it masks the real problem.

When downtime is planned or you’re deliberately shedding load, the right tool isn’t letting pages 504 — it’s returning a 503 (with a Retry-After header) so you send search engines a clean, intentional “come back later” instead of a messy timeout.

Diagnosing a 504 in practice

Identify the emitting hop before you guess at a cause — that’s the difference between fixing the actual problem and fixing a symptom:

• Reproduce it and identify which hop answered. Load it in a browser; hit it with curl and watch time_starttransfer for TTFB; run it through a crawler like Ahrefs Site Audit or Screaming Frog to see whether it’s site-wide or isolated. If you can, test direct-to-origin (bypassing the CDN/proxy) to see whether the origin itself completes.
• Read the logs. Server and proxy logs tell you which layer timed out and, ideally, on what — the slow query, the stuck upstream, the exhausted worker pool. Correlate by timestamp and request ID across layers rather than assuming.
• Check Search ConsoleGoogle's free tool for monitoring crawling, indexing, and search performance.. Crawl Stats surfaces response-code spikes and average response time; the Page Indexing reportThe Google Search Console report (formerly Index Coverage) showing how many of your URLs are indexed vs. not indexed, and grouping the not-indexed ones by reason. and URL Inspection show whether Google is hitting timeouts (including “Hostload exceeded”).

Reconciling tools matters: a browser 504, an Ahrefs Site Audit 5xx, and a GSC timeout can all describe the same underlying slow origin seen from different vantage points. Don’t assume three tools mean three problems.

Related codes

A 504 lives in a small family. A 500 is a generic server error with no more specific code; a 502 is a bad/garbled upstream response; a 503 is an explicit, often intentional “unavailable.” Knowing which one you’re actually returning — and returning the right one on purpose during planned downtime — is half the battle.

AI summary

A condensed take on the Advanced version:

• A 504 = a timeout, not a broken response. A gateway/proxy (CDN, load balancer, reverse proxy) waited on the upstream server and didn’t get a timely response — not necessarily “no response ever.” Distinct from 502 (bad response) and 503503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime. (explicit unavailability). The status code alone doesn’t prove which hop or which cause; identify the emitting hop before you fix anything.
• It’s not a penalty. It’s an accessibilityWeb accessibility means designing sites so people with disabilities can use them, per the W3C's WCAG guidelines. It overlaps with SEO in specific, checkable ways — alt text, heading structure, descriptive link text, captions, and page speed all serve both audiences — but Google has said accessibility itself is not a ranking factor, and most WCAG success criteria (keyboard focus order, ARIA live regions, form labels) have no SEO effect at all. problem — Google can’t retrieve the page, so ranking/indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. loss is a downstream consequence, not punishment.
• Timeouts trigger crawl throttling. Google: “GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. will scale back its crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. if it detects that your servers are having trouble responding.” Gary Illyes is quoted (via Search Engine Land’s March 2026 coverage) describing struggling servers causing fetchers to “automatically back off … which will drop your crawl frequencyCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial..”
• Duration matters, but the “2 days” figure isn’t Google’s documented number for 504. That horizon is documented for intentionally returned 503/429429 Too Many Requests is an HTTP client-error status code that means a client sent too many requests in a given time window — a mechanism called rate limiting. Unlike other 4xx codes, Google treats 429 as a server-overload signal and slows crawling instead of removing content. during overload. Google’s broader 5xx guidance says crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor. drops proportionally and recovers gradually, without naming a fixed timeline for 504 specifically — isolated 504s are retried and tolerated; a sustained pattern over an extended window is the range that risks deindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist..
• The loop is self-correcting. Once responses recover, crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor. returns automatically — no manual unthrottle needed.
• Often intermittent/load-dependent — invisible to uptime monitors pinging quiet windows; may show as “Hostload exceeded” in URL Inspection.
• Server response time (TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good.) is a strong preventive lever for origin-side causes, monitored as an early-warning signal — but it’s not universal: a 504 can also originate at the CDN, load balancer, or proxy layer, so confirm which hop is timing out first. Raising the timeout value isn’t a fix either way — it masks the slow upstream and can worsen overload.
• When downtime is planned, return a 503 with Retry-After, not a 504.

Official documentation

Primary-source documentation on timeouts, server errors, and how crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. respond.

Definition

• MDN — 504 Gateway Timeout — the authoritative definition and how it differs from a 502.

Google

• Troubleshoot crawling errors — how GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. scales back on server trouble, and when to return 503503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime./429429 Too Many Requests is an HTTP client-error status code that means a client sent too many requests in a given time window — a mechanism called rate limiting. Unlike other 4xx codes, Google treats 429 as a server-overload signal and slows crawling instead of removing content. for overload.
• Optimize your crawl budget — how slow responses and server errors lower the crawl limit.
• Crawl Stats report — the “Server error (5XX)” and “Page timeout” categories, and how GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. throttles to avoid overload.

Bing / Microsoft

• Bing Webmaster Tools — crawl error alerts — how Bing flags server errors and timeouts as separate crawl-error categories.

Quotes from the source

On-the-record statements. Each link is a deep link that jumps to the quoted passage on the source page.

MDN — the definition

• “The HTTP 504 Gateway Timeout server error response status code indicates that the server, while acting as a gateway or proxy, did not get a response in time from the upstream server in order to complete the request. This is similar to a 502 Bad Gateway, except that in a 504 status, the proxy or gateway did not receive any HTTP response from the origin within a certain time.” Jump to quote

Google — crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. and timeouts

• “GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. will scale back its crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. if it detects that your servers are having trouble responding to crawl requests.” — Google Search Central, Troubleshoot crawling errors. Jump to quote
• “If the site slows down or responds with server errors, the limit goes down and Google crawls less.” — Google Search Central, Optimize your crawl budgetThe number of URLs an engine will crawl in a timeframe.. Jump to quote

Gary Illyes, Google

• “If your server is struggling to serve bytes, our crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. will automatically back off to avoid overloading your infrastructure, which will drop your crawl frequencyCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial..” Jump to quote

John Mueller, Google (on Reddit, reported by Search Engine Journal, Aug 2025)

• “I’d only expect the crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor. to react that quickly if they were returning 429429 Too Many Requests is an HTTP client-error status code that means a client sent too many requests in a given time window — a mechanism called rate limiting. Unlike other 4xx codes, Google treats 429 as a server-overload signal and slows crawling instead of removing content. / 500 / 503503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime. / timeouts.” Jump to quote Search Engine Journal reproduces Mueller’s Reddit response and its surrounding context.

Server response-time / TTFB checklist

The goal is to keep responses fast enough that a 504 never fires — and to catch the drift before it does. Work top to bottom:

• Baseline TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. with curl -w "%{time_starttransfer}" (or a synthetic monitor) and record what “normal” looks like per template.
• Monitor TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. continuously, not just after an incident — alert on upward drift, not only on hard failures.
• Load-test at realistic (and peak) concurrency — 504s are usually load-dependent, so a quiet-window check won’t surface them.
• Profile the slowest upstream work — unindexed database queries, N+1 queries, and blocking third-party API calls are the usual culprits.
• Cache aggressively — page cache, object cache, and CDN edge cache — so most requests never touch the slow path.
• Check timeout values are consistent across layers (CDN, load balancer, Nginx proxy_read_timeout / fastcgi_read_timeout, app) so one layer isn’t giving up before another.
• Confirm autoscaling / capacity headroom for traffic and crawl spikes.
• Review GSC Crawl StatsA Google Search Console report (under Settings) that shows how Google has crawled your site over the last 90 days — total requests, download size, and average response time, broken down by response code, file type, Googlebot type, and purpose. It's only available for root-level properties (a Domain property or a URL-prefix property verified at the site's root). for timeout and 5XX spikes and rising average response time.
• Check server + proxy logs to identify which layer is timing out and on what.
• Verify uptime monitoring covers peak load, not just off-hours pings.
• Use 503503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime. + Retry-After for planned downtime instead of letting pages 504.

504 myths and mistakes to avoid

The traps that come up most often — several of these are widely-repeated myths worth correcting:

• “A 504 is a Google penalty.” No. It’s an accessibilityWeb accessibility means designing sites so people with disabilities can use them, per the W3C's WCAG guidelines. It overlaps with SEO in specific, checkable ways — alt text, heading structure, descriptive link text, captions, and page speed all serve both audiences — but Google has said accessibility itself is not a ranking factor, and most WCAG success criteria (keyboard focus order, ARIA live regions, form labels) have no SEO effect at all./crawlabilityCrawlability is how well search engine crawlers can discover, access, and fetch a site's pages. A crawlability issue is any technical condition — blocked access, broken links, server failures, or bloated URL inventory — that stops pages from reaching the index. problem, not an algorithmic action. Google isn’t judging your content — it can’t retrieve the page. Any ranking loss is a downstream consequence of inaccessibility, not punishment.
• “Any 504 will get my page deindexed immediately.” No. Short, occasional 504s are retried and tolerated. It’s sustained, frequent 504s over an extended window that risk deindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist..
• “504 and 503503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime. are basically the same — use them interchangeably.” No. A 503 can be a deliberate, controlled signal (Google explicitly supports 503 for planned downtime); a 504 is a timeout, almost always unplanned and symptomatic of a slow upstream. During planned downtime, return a 503 with Retry-After — don’t let pages 504.
• “It’s GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. being too aggressive.” Usually backwards. A server that reliably 504s under GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer.’s crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor. will also degrade for real users under similar load. The timeout is exposing a genuine capacity/performance problem (“Hostload exceeded” scenarios exist, but they’re the exception, not the rule).
• “Fixing it just means raising the timeout value.” Raising proxy_read_timeout masks a slow upstream instead of fixing it — and a longer timeout under load ties up worker processes for longer, which can make an overload worse. Fix the slow response, don’t extend the patience for it.
• “Uptime monitoring says we’re green, so we don’t have a 504 problem.” 504s are frequently load-dependent. A monitor pinging during a quiet window can miss the timeouts Googlebot collects during heavier crawl bursts. Trust Crawl StatsA Google Search Console report (under Settings) that shows how Google has crawled your site over the last 90 days — total requests, download size, and average response time, broken down by response code, file type, Googlebot type, and purpose. It's only available for root-level properties (a Domain property or a URL-prefix property verified at the site's root). and logs over off-hours pings.

Which layer is timing out?

Prompt: classify a batch of 504 incidents

Paste sanitized rows with URL, timestamp, response headers, TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. or total time, public-edge result, direct-origin result, application trace, database timing, and resource signals. Do not paste credentials, cookies, private addresses, or user data.

You are triaging HTTP 504 Gateway Timeout incidents. Classify each row into one of:
slow application or query, upstream dependency timeout, resource exhaustion,
gateway/origin timeout mismatch, CDN or network path, intermittent with insufficient
evidence, or not actually a 504.

For each row:
- quote the supplied evidence behind the classification;
- identify the missing observation that would most change the conclusion;
- separate response latency from the gateway's configured timeout;
- do not recommend increasing a timeout unless the evidence shows healthy work that
  legitimately needs longer;
- group rows sharing timestamps, routes, upstreams, or resource spikes.

Return a table with URL, likely cause, confidence, evidence, next diagnostic, and
incident group. Then list the first three engineering checks for the highest-impact
group. Do not invent thresholds or monitoring data.

DATA:
[PASTE SANITIZED INCIDENT DATA HERE]

Treat the result as a hypothesis queue. Confirm it in gateway, application, database, and infrastructure telemetry.

Measure status and time to first byte

Run this in a macOS/Linux shell. It reports the final response code and TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. for a single request without printing the body.

curl -sS -o /dev/null \
  -w 'status=%{http_code} ttfb=%{time_starttransfer}s total=%{time_total}s\n' \
  https://www.example.com/slow-path

Repeat it a few times to see whether the timeout is consistent or load-dependent:

for i in {1..5}; do
  curl -sS -o /dev/null \
    -w 'status=%{http_code} ttfb=%{time_starttransfer}s total=%{time_total}s\n' \
    https://www.example.com/slow-path
done

PowerShell equivalent:

1..5 | ForEach-Object {
  $watch = [System.Diagnostics.Stopwatch]::StartNew()
  try {
    $response = Invoke-WebRequest -Uri 'https://www.example.com/slow-path' -SkipHttpErrorCheck
    $watch.Stop()
    [pscustomobject]@{ Attempt = $_; Status = $response.StatusCode; TotalMs = $watch.ElapsedMilliseconds }
  } catch {
    $watch.Stop()
    [pscustomobject]@{ Attempt = $_; Status = 'request-failed'; TotalMs = $watch.ElapsedMilliseconds }
  }
}

The PowerShell version measures total request time, not TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good.. Use the shell command or your application telemetry when you need the first-byte split.

Tools for reproducing and scoping 504s

• Website Down Checker — test reachability from an external vantage point and capture response timing, redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't., and bounded DNS evidence. Use it to establish whether the incident is publicly reproducible.
• Bulk HTTP Status Code Checker — check a representative set of routes, compare latency, and export the failing subset. This helps separate one slow endpoint from a site-wide upstream problem.
• CDN/load-balancer logs — locate the gateway that emitted the 504 and correlate its request ID and timeout with the upstream attempt.
• Application tracing and database slow-query logs — show where the request spent its time after it reached the origin.
• Infrastructure monitoring — compare the incident window with CPU, memory, worker-pool, connection, and dependency saturation rather than guessing from the status code.

TTFB by route and percentile

Metric: Time to first byteTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. for representative routes, segmented by useful percentiles rather than only an average.

What it tells you: Rising tail latency is an early warning that requests are approaching a gateway timeout even before they start returning 504.

How to pull it: Use real-user/server monitoring or gateway logs; use the curl script in the Scripts tab for spot checks, not as a substitute for production telemetry.

Benchmark / realistic range: Establish a baseline per route and infrastructure path. The meaningful alert is a sustained change from that baseline or movement toward your actual configured gateway timeout, not a universal SEO number.

Cadence: Monitor continuously; review route-level trends weekly and during every 504 incident.

504 response rate

Metric: Requests returning 504 divided by all requests, segmented by route, gateway, origin, and crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index./user-agentA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target. class where available.

What it tells you: Whether timeouts are isolated, concentrated in one path, or broad enough to affect crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. and users.

How to pull it: Aggregate CDN, load-balancer, or server access logsLog file analysis is reading a web server's raw access logs to see exactly which URLs search engine crawlers actually requested, when, how often, and what status code they got. Unlike crawl tools or Search Console, logs are the unsampled, ground-truth record of what really happened. by status code and request dimensions.

Benchmark / realistic range: The healthy target is no unexplained 504s. For alert sensitivity, use your own normal incident-free baseline because traffic mix and retry behavior differ by stack.

Cadence: Alert continuously; review daily during recovery and in the regular weekly reliability report.

Upstream completion versus gateway timeout

Metric: The distribution of upstream completion times compared with each hop’s configured timeout.

What it tells you: Whether slow work is genuinely approaching the limit or a shorter gateway timeout is cutting off otherwise healthy upstream responses.

How to pull it: Join gateway timing fields with application traces using a request or trace ID.

Benchmark / realistic range: Keep normal completion comfortably inside the configured limit, with room for expected variance. Define that room from observed production distributions; do not invent a generic percentage.

Cadence: Review after configuration or dependency changes and whenever tail latency or 504 rate rises.

Test yourself: 504 Gateway Timeout

Five quick questions on what a 504 means and how it affects crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor.. Pick an answer for each, then check.

Resources worth your time

My related writing

• The Beginner’s Guide to Technical SEO — where server health and crawlabilityCrawlability is how well search engine crawlers can discover, access, and fetch a site's pages. A crawlability issue is any technical condition — blocked access, broken links, server failures, or bloated URL inventory — that stops pages from reaching the index. fit in the bigger picture.
• Meet the New Web Crawlers: AI Bots Are Closing in on Search Engine Bots — who’s actually hitting your server, and why load matters.

My speaking

• How Search Works (SlideShare) — my walkthrough of crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., renderingTurning HTML, CSS, and JavaScript into the final visual page and DOM., indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., and ranking, including how server responses govern crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor.. (My standing disclaimer applies: “This is my understanding of systems… not going to be 100% complete or accurate.”)

From around the industry

• MDN — 504 Gateway Timeout — the canonical definition and the 502 distinction.
• Google — Troubleshoot crawling errors — how GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. scales back on server trouble.
• Google — Managing crawl budget for large sites — how slow responses lower the crawl limit.
• Google explains how crawling works in 2026 (Search Engine Land) — the Illyes quote tying struggling servers to reduced crawl frequencyCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial..
• Googlebot Crawl Slump? Mueller Points To Server Errors (Search Engine Journal) — Mueller grouping timeouts with 429429 Too Many Requests is an HTTP client-error status code that means a client sent too many requests in a given time window — a mechanism called rate limiting. Unlike other 4xx codes, Google treats 429 as a server-overload signal and slows crawling instead of removing content./500/503 as the errors that drop crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor. fast.
• How to Fix the 504 Gateway Timeout Error (Kinsta) — a solid host-side technical walkthrough of the request chain and log diagnostics.

Videos

• Google Search Central (YouTube) — the How Google Search WorksSearch works in three stages — crawling, indexing, and serving (ranking). A page has to clear each one to appear in results: getting crawled doesn't mean you're indexed, and getting indexed doesn't mean you rank. series and Martin Splitt’s crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. explainers, useful for seeing how server responses and crawl rateCrawl rate is how fast a search engine crawler fetches pages from your site — the number of simultaneous requests it makes and the delay between them. Google sets it automatically based on your server's health; it's the supply side of crawl budget, not a ranking factor. interact. Channel

504 Gateway Timeout

A 504 Gateway Timeout is an HTTP 5xx server error a gateway or proxy (a CDN, load balancer, or reverse proxy) returns when it doesn't get a timely response from the upstream server behind it. Unlike a 502 (a bad/garbled response) or a 503 (server explicitly unavailable), a 504 means no response arrived in time. For SEO, isolated 504s are retried and tolerated, but sustained 504s and timeouts make Googlebot slow its crawl and eventually drop pages from the index.

Related: HTTP Status Code, Crawling, Crawl Budget

504 Gateway Timeout

A 504 Gateway Timeout is an HTTP 5xx server error returned when a server acting as a gateway or proxy — a CDN, load balancer, or reverse proxy — didn’t get a response in time from the upstream server it needed to complete the request. As MDN puts it, it’s “similar to a 502 Bad Gateway, except that in a 504 status, the proxy or gateway did not receive any HTTP response from the origin within a certain time.”

The word to notice is timeout. A 504 doesn’t mean the origin sent something broken (that’s a 502) or that the server explicitly said it was unavailable (that’s a 503503 Service Unavailable is the HTTP status code a server returns when it's temporarily unable to handle a request — usually because of overload or planned maintenance. It tells crawlers 'come back later' instead of 'this page is gone,' which is why Google recommends it (paired with a Retry-After header) for short, planned downtime.) — it means the response didn’t arrive before the gateway gave up waiting. That points diagnosis at slow upstream work: a slow database query, a lagging third-party API or microservice, an overloaded app server, or a timeout value set too low for the work behind it.

For SEO, the risk is graduated, not instant. Google retries isolated 504s and largely ignores them. But timeouts sit in the same class of error as 429429 Too Many Requests is an HTTP client-error status code that means a client sent too many requests in a given time window — a mechanism called rate limiting. Unlike other 4xx codes, Google treats 429 as a server-overload signal and slows crawling instead of removing content., 500, and 503: when GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. sees your server struggling to respond, it automatically backs off and crawls less, and if the pattern is sustained the affected URLs eventually drop from the indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.. A strong preventive lever for origin-side causes is server response timeTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. — keeping TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. low so a slow-but-not-yet-timing-out server never tips over into a 504 under GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer.’s heavier crawl bursts — though a 504 can also originate at the CDN, load balancer, or proxy layer, so confirm which hop is timing out before assuming TTFB is the fix.

Related: HTTP Status Code, Crawling, Crawl Budget

Changelog