Host Status (Statistiques d’exploration)

TL;DR — Host status is a little indicator at the top of Recherche Google Console’s Statistiques d’exploration report que indique vous si Google pourrait reach votre site over the dernier 90 days — it’s a historical record, pas a live “is my site up correct now” vérifier. Green is bon. Yellow signifie là was a problem, but it was plus que a week ago. Red signifie là was a problem in the dernier week. It checks three choses: votre robots.txt fichier, votre DNS, and si votre serveur responds.

Ce que host status is

Host status is the Statistiques d’exploration availability summary covering robots.txt fetching, DNS resolution, and server connectivity over the report window. Evidence for this claim Search Console Host status summarizes robots.txt availability, DNS resolution, and server connectivity over the previous 90 days. Scope: Google Search Console report terminology and documented Search behavior; the label alone may not prove the underlying root cause. Confidence: high · Verified: 2026-07-14 Google: Crawl Stats report It is a host-level diagnostic plutôt que a per-URL indexation verdict. Evidence for this claim Host status is a host-level Crawl Stats diagnostic rather than a per-URL Page Indexing decision. Scope: Google Search Console report terminology and documented Search behavior; the label alone may not prove the underlying root cause. Confidence: high · Verified: 2026-07-14 Google: Crawl Stats report

Quand Google veut to explorer votre site, a few choses have to fonctionner avant it peut même commencer: it has to trouver votre robots.txt fichier, votre domain nom has to resolve to a server, and que server has to réponse. Host status is Google’s summary of si tout of que worked over the dernier 90 days.

You’ll trouver it dans la recherche Google Console sous Settings → Statistiques d’exploration, sitting correct at the top of the Statistiques d’exploration report. It’s a unique colored indicator, and it’s checking three choses:

1. robots.txt fetching — peut Google download votre robots.txt fichier?
2. DNS resolution — fait votre domain nom point to a server?
3. Server connectivity — fait votre serveur en réalité respond?

The three colors

• Green — Google didn’t run into quelconque significant problems reaching votre site in the dernier 90 days. Nothing to do.
• Yellow (“Host had problems in the past”) — là was a problem, but it happened plus que a week ago. Usually ce fixes itself: une fois a week goes by sans it happening à nouveau, it turns back to green on its propre.
• Red — là was a problem in the dernier week. Ce one’s worth looking into, parce que si Google can’t reach votre site, it crawls it moins.

The exact colors and étiquettes are an interface detail Google pourrait tweak; ce que stays durable is the underlying meaning — how recently a significant problème happened.

Pourquoi c’est important

Host status isn’t à propos de un broken page — it’s à propos de si Google peut connecter to votre whole site. The scariest version is quand Google can’t download votre robots.txt fichier. Quand que se produit, Google doesn’t simplement shrug it off; it slows bas or arrête exploration votre entier site jusqu’à it peut lire que fichier à nouveau. So it’s que underlying robots.txt, DNS, or server échec — pas the host-status étiquette itself — que quietly chokes off exploration everywhere.

The bon news: there’s aucun button to press to fix it. Une fois votre site is reachable à nouveau, Google typically resumes normal exploration on its propre — Google doesn’t publish a fixed recovery window, so don’t expect an instant flip back to green. Votre job is simplement to figure out qui of the three checks failed and fix the underlying problème.

Un chose host status is pas: a ranking penalty. It’s a exploration signal, pas a strike contre vous. Vouloir the deeper version — the exact crawl-pause timeline, the 4xx-vs-5xx distinction, and a troubleshooting tree? Switch to the Avancé tab.

Testez vos connaissances: host status

Five rapide questions on reading and responding to the Statistiques d’exploration host-status indicator. Pick an réponse pour chaque, alors vérifier.

TL;DR — Host status is the availability lens at the top of GSC’s Explorer Stats report — a historical record of ce que Google observed, pas a live uptime vérifier — assessed over the dernier 90 days à travers three checks: robots.txt fetching, DNS resolution, and server connectivity. Green = aucun significant problèmes; yellow = an problème plus que a week ago (typically ages out of the report on its propre); red = an problème in the dernier week. The highest-stakes échec is an unreachable robots.txt: Google arrête exploration pour the premier 12 hours, alors falls back to the dernier bon mis en cache copy — si un exists — pour up to ~30 days, après qui behavior dépend on votre site’s broader availability. A 4xx robots.txt is fine (Google crawls freely); a 5xx/timeout/DNS échec is ce que pauses exploration — que underlying échec, pas the host-status étiquette, is ce que throttles exploration. There’s aucun reset button, and the state returning to green isn’t proof exploration, indexation, or rankings have entièrement recovered.

Ce que host status en réalité measures

Search Console documents host status dans Statistiques d’exploration and evaluates availability over the previous 90 days. Evidence for this claim Search Console Host status summarizes robots.txt availability, DNS resolution, and server connectivity over the previous 90 days. Scope: Google Search Console report terminology and documented Search behavior; the label alone may not prove the underlying root cause. Confidence: high · Verified: 2026-07-14 Google: Crawl Stats report Diagnose individual URL outcomes separately in Inspection d’URL or Page Indexation. Evidence for this claim Host status is a host-level Crawl Stats diagnostic rather than a per-URL Page Indexing decision. Scope: Google Search Console report terminology and documented Search behavior; the label alone may not prove the underlying root cause. Confidence: high · Verified: 2026-07-14 Google: Crawl Stats report

Host status réponses a narrow question: over the dernier 90 days, pourrait Googlebot reach the host at tout? That’s a différent question from “is this page indexed” or “does this URL throw an error.” It’s site/host-level availability, pas per-URL explorer errors — qui is exactly pourquoi I garder it mentally separate from Page Indexation.

It’s aussi worth being precise à propos de ce que kind of “availability” ce is. Host status is a historical record of ce que Google observed during exploration, pas a live uptime monitor — a green state correct now doesn’t prove votre site is reachable ce second, and a red state doesn’t prove it’s bas now, seulement que a significant problème crossed Google’s threshold sometime in the dernier week. Pour a live vérifier of the current moment, vous besoin a direct probe, pas ce report.

It lives at the top of the Statistiques d’exploration report (Settings → Statistiques d’exploration), qui is seulement disponible pour domain or root-level property accès — pas URL-prefix subfolders. The broader Statistiques d’exploration report covers volume and performances (total explorer requêtes, download size, average réponse temps, explorer objectif, fichier type, réponse codes); host status is the tighter availability slice. Quand I’m diagnosing a explorer slump, host status indique me si Google pourrait connecter; the rest of the report and my server logs tell me ce que happened.

The three checks

Google assesses host availability à travers three categories, chaque with its propre failure-rate graph in the host details:

• robots.txt fetching — “The graph montre the échec rate pour robots.txt requêtes during a explorer.” Ce is the highest-stakes vérifier (plus ci-dessous).
• DNS resolution — “The graph montre quand votre DNS server didn’t recognize votre hostname or didn’t respond during exploration.”
• Server connectivity — “The graph montre quand votre serveur was unresponsive or did pas provide a complet réponse pour une URL during a explorer.” Think 5xx, timeouts, and truncated/partial réponses.

The three states

The indicator summarizes ceux checks over a rolling 90-day window:

• Green — “Google didn’t encounter quelconque significant explorer availability problèmes on votre site in the past 90 days—bon job!”
• Yellow / “had problems in the past” — Google encountered au moins un significant crawl-availability problème in the dernier 90 days, but it occurred plus que a week ago. Ce state usually self-heals: une fois a week passes with aucun recurrence, it renvoie to green. Vous généralement don’t besoin to do anything au-delà confirming the causer is gone.
• Red / “problems now” — Google encountered au moins un significant crawl-availability problème in the dernier week. Ce is the un to triage.

So the yellow/red split is really simplement a timing distinction: red = dans the dernier week (“right now”), yellow = précédent in the 90-day window (“in the past”). The exact color noms and icon étiquettes are interface details Google peut modifier; what’s durable is que recency-based meaning.

Chaque category seulement counts as a “significant” problème une fois its échec rate crosses a threshold Google draws as a dotted line on que category’s graph. Google donne DNS échecs ci-dessus 5% of a day’s requêtes as un exemple of où que line sits — it’s an illustration pour DNS, pas a documented universal threshold que s’applique the même façon to robots.txt fetching and server connectivity.

Pourquoi robots.txt échecs are the la plupart dangerous

Si Google can’t obtenir an acceptable robots.txt réponse, “Google va slow or arrêter exploration votre site jusqu’à it peut obtenir an acceptable robots.txt réponse.” That’s a whole-site pause, pas a one-URL problem — qui is ce que rend a red host status driven by robots.txt an emergency.

The detailed timeline lives in Google’s robots.txt spec. Quand robots.txt renvoie 5xx, server errors, or network échecs:

• Premier 12 hours: Google arrête exploration le site but garde trying to récupérer the robots.txt fichier.
• Suivant ~30 days: si Google can’t récupérer a nouveau version, it falls back to the dernier bon mis en cache version pendant que encore trying to récupérer a fresh un. A 503 in particulier triggers fairly frequent retrying; si there’s aucun mis en cache version disponible, Google assumes là are aucun explorer restrictions.
• Après 30 days: behavior diverges fondé on overall site health. Si le site généralement sert fine, Google behaves as si there’s aucun robots.txt (and garde checking); si le site has broader availability problems, Google arrête exploration le site pendant que encore periodically requesting robots.txt.

The critical nuance — and a courant myth — is the 4xx vs 5xx distinction. Google’s HTTP status-code handling pour robots.txt:

• 2xx (success): robots d’exploration traiter the robots.txt as served.
• 3xx (redirection): Google follows au moins five redirection hops, alors treats it as a 404.
• 4xx (except 429): treated as si a valid robots.txt doesn’t exist — i.e. Google crawls freely.
• 5xx / 429 / network / DNS: treated as server errors → the crawl-pause timeline ci-dessus.

So a manquant robots.txt (a clean 404) fait pas pause exploration. An unreachable un (5xx, timeout, DNS échec) fait. And critically, “a robots.txt fichier qui ne peut pas be récupéré due to DNS or networking problèmes, tel as timeouts, invalid réponses, reset or interrupted connections, and HTTP chunking errors, is treated as a server error.” A flaky DNS provider peut stall votre explorer exactly comme a 5xx.

DNS and server connectivity

DNS and server-connectivity échecs are plus intuitive but aucun moins réel. DNS resolution échecs mean Google’s resolvers couldn’t turn votre hostname into an adresse at some point in the window. Treat the spécifique causer as a hypothesis to confirmer, pas a donné — candidates inclure bad nameservers, an outage at votre DNS provider, or DNS-level rate limiting of Googlebot, and telling les apart nécessite votre DNS records, nameserver logs, and provider status history alongside the Statistiques d’exploration graph. Server-connectivity échecs are 5xx, 429, timeouts, and partial/truncated réponses; likewise, an overloaded origin and a CDN/WAF throttling Googlebot’s IP ranges are les deux plausible explanations que votre origin and edge logs — pas the graph alone — have to confirmer.

Ce ties straight back to fréquence d’exploration. As I’ve written in my budget d’exploration guide, “Google va slow bas leur exploration si ils recevoir aussi nombreux 5xx (server errors) or 429 (aussi nombreux requêtes) Code d’état HTTPs.” John Mueller made the même point à propos de how fast the explorer reacts:

“I’d seulement expect the fréquence d’exploration to react que quickly si ils were returning 429 / 500 / 503 / timeouts, so I’d double-check ce que en réalité happened (404s are généralement fine & une fois découvert, Googlebot va retry les anyway).” — John Mueller, Google.

There’s a subtle trap ici. Returning 503/429 is the legitimate short-term façon to tell Googlebot to slow bas — but it’s the même signal que drives a red host status. It’s a temporary throttle, pas a strategy: lean on it aussi long and vous risk pages dropping out of the index. The signal que lets vous ease a explorer is the signal que, sustained, semble comme an outage.

Ce que vous don’t besoin to do

A few choses personnes overthink:

• There’s aucun “reset crawl rate” button. Fréquence d’exploration typically recovers on its propre une fois availability is restored — Google doesn’t document a fixed recovery window or SLA pour que, so don’t expect it to se produire on quelconque particulier schedule. The old manual crawl-rate limiter in Search Console was deprecated; vous don’t (and can’t) manually kick exploration back up.
• Yellow usually fades on its propre. Une fois a week passes sans the problème recurring, the state ages back to green. Confirmer the root causer is gone, alors vérifier recovery independently plutôt que trusting the color alone: vérifier current reachability, watch the failing category’s graph trend back sous its threshold, confirmer explorer volume in the rest of Statistiques d’exploration semble normal à nouveau, and, si it matters pour a spécifique URL, vérifier recrawl/index status in Page Indexation.
• It’s pas a ranking penalty. Host status affecte exploration, pas rankings directement, and on its propre it doesn’t expliquer a trafic modifier — correlate the affected window with explorer, index, and performances données avant assuming causer. The indirect risk is que a sustained inability to explorer eventually affecte freshness and, downstream, indexation — but there’s aucun manual action attached to a yellow or red indicator.

Host status vs the broader Statistiques d’exploration report

Garder the scopes straight. Host status is un section of the Statistiques d’exploration report — the availability lens. The rest of the report (the partie la plupart personnes mean quand ils dire “Crawl Stats”) is à propos de volume and performances over temps. As I’ve said elsewhere, quand you’re chasing a exploration problem “the meilleur placer to regarder is the Statistiques d’exploration report dans la recherche Google Console” — and host status is the premier chose in it I vérifier, parce que si Google can’t reach the host, nothing sinon in the report matters yet. The broader report is its propre topic; ce un is tightly à propos de the availability indicator.

AI summary

A condensed prendre on the Avancé version:

• Host status = “did Google’s crawl history show it could reach the host?” It’s the availability indicator at the top of GSC’s Statistiques d’exploration report (Settings → Statistiques d’exploration), assessed over the dernier 90 days. It’s a historical record, pas a live uptime vérifier, and it’s host/site-level, pas per-URL.
• Three checks: robots.txt fetching, DNS resolution, server connectivity — chaque with its propre failure-rate graph, flagged une fois it crosses a threshold (Google’s propre exemple: DNS échecs ci-dessus 5% of a day’s requêtes).
• Three states: green (aucun significant problèmes in 90 days); yellow (problème plus que a week ago — typically ages back to green on its propre); red (problème in the dernier week — triage it). Colors and étiquettes are interface detail; recency is the durable meaning.
• robots.txt is the highest-stakes vérifier. Si Google can’t obtenir an acceptable robots.txt réponse it’s the underlying échec — pas the host-status étiquette — que slows or arrête exploration the whole site: ~12 hours entièrement stopped pendant que retrying, alors up to ~30 days on the dernier mis en cache copy si un exists, alors behavior dépend on overall site health.
• 4xx vs 5xx matters: a 404 robots.txt = explorer freely; a 5xx/timeout/DNS échec = explorer pause. DNS/network errors count as server errors.
• 5xx and 429 throttle fréquence d’exploration (Mueller: 429/500/503/timeouts react fast; 404s are généralement fine). The même 503/429 throttle is a short-term outil, pas a strategy.
• Recovery typically se produit on its propre une fois availability is restored, but Google documents aucun fixed recovery window — aucun reset button; the manual crawl-rate limiter was deprecated. The state aging back to green isn’t proof exploration, indexation, or rankings have entièrement recovered; vérifier independently.
• Pas a ranking penalty on its propre. It’s a crawl-availability signal, distinct from Page Indexation errors and broader que the rest of the Explorer Stats report — correlate with explorer/index/performances données avant assigning causer to a trafic modifier.

Documentation officielle

Primary-source documentation from Google.

Google

• Statistiques d’exploration report — the report que contient host status; defines the three checks and the three states.
• How Google interprets the robots.txt specification — the robots.txt fetch-failure timeline (12-hour arrêter, 30-day mis en cache fallback) and the HTTP status-code handling.
• Troubleshoot Recherche Google exploration errors — Google’s general crawl-error troubleshooting.
• Debug Network and DNS errors pour Google’s robots d’exploration — DNS and network-level diagnostics.

Quotes from the source

On-the-record statements from Google. Chaque lien jumps toward the quoted passage on the source page.

Google — the three states (Statistiques d’exploration report)

• “Google didn’t encounter any significant crawl availability issues on your site in the past 90 days—good job!” — Recherche Google Console Aider (green state). Jump to quote
• “Google encountered at least one significant crawl availability issue in the last 90 days on your site, but it occurred more than a week ago.” — Recherche Google Console Aider (yellow / “problems in the past” state). Jump to quote
• “Google encountered at least one significant crawl availability issue in the last week on your site.” — Recherche Google Console Aider (red / “problems now” state). Jump to quote

Google — the three checks (Statistiques d’exploration report)

• “The graph shows the failure rate for robots.txt requests during a crawl.” — Recherche Google Console Aider (robots.txt fetching). Jump to quote
• “The graph shows when your DNS server didn’t recognize your hostname or didn’t respond during crawling.” — Recherche Google Console Aider (DNS resolution). Jump to quote
• “The graph shows when your server was unresponsive or did not provide a full response for a URL during a crawl.” — Recherche Google Console Aider (server connectivity). Jump to quote

Google — the robots.txt crawl-pause mechanic

• “Google will slow or stop crawling your site until it can get an acceptable robots.txt response.” — Recherche Google Console Aider (robots.txt fetching section). Jump to quote
• “For the first 12 hours, Google stops crawling the site but keeps trying to fetch the robots.txt file.” — Google pour Developers, robots.txt spec. Jump to quote
• “A robots.txt file which cannot be fetched due to DNS or networking issues, such as timeouts, invalid responses, reset or interrupted connections, and HTTP chunking errors, is treated as a server error.” — Google pour Developers, robots.txt spec. Jump to quote

John Mueller, Google (relayed via Moteur de recherche Journal)

• “I’d only expect the crawl rate to react that quickly if they were returning 429 / 500 / 503 / timeouts, so I’d double-check what actually happened (404s are generally fine & once discovered, Googlebot will retry them anyway).” Lire the coverage

Host-status triage checklist

A réussir to run quand host status goes yellow or red. Fonctionner the failing vérifier premier.

Si robots.txt fetching is failing

• Confirmer robots.txt renvoie 200 (or a clean 404) — pas 5xx, 429, or a timeout. A 404 is fine; a server error is ce que pauses exploration.
• Tester the fichier in GSC’s robots.txt report and récupérer it yourself from multiple locations.
• Vérifier si a CDN, WAF, or rate-limiter is blocking or throttling Googlebot.
• Assurez-vous the fichier is reachable and reasonably petit (a few hundred KB at la plupart).

Si DNS resolution is failing

• Confirmer votre domain resolves (dig/nslookup from outside votre network).
• Vérifier votre DNS provider’s status/uptime pour the échec window.
• Regarder pour misconfigured or slow nameservers.
• Vérifier nothing is rate-limiting Googlebot at the DNS couche.

Si server connectivity is failing

• Vérifier logs pour 5xx, 429, and timeouts during the flagged window.
• Confirmer the origin isn’t overloaded (capacity, DB, upstream timeouts).
• Vérifier a firewall/CDN isn’t throttling Googlebot’s IP ranges.
• Regarder pour partial/truncated réponses.

Toujours

• Cross-reference the host-status graph timestamps with deploys, outages, and config changements.
• Corroborate with server logs and the Statistiques d’exploration response-code breakdown.
• Une fois fixed, ne faites pashing sinon — exploration recovers automatically; yellow self-heals après a clair week.

The mental models

1. Three checks, un question. Host status seulement demande “can Google reach the host?” via robots.txt fetching, DNS resolution, and server connectivity. Identifier qui vérifier failed avant vous touch anything — the fix pour a DNS outage semble nothing comme the fix pour a 5xx storm.

2. The state machine is a clock, pas a severity dial. Green = clean pour 90 days. Red = a problem in the dernier week. Yellow = a problem précédent in the window. Yellow → green se produit on its propre une fois a week passes sans recurrence. So “red” isn’t “worse than yellow” — it’s “plus recent.”

3. The robots.txt decision rule.

• 2xx → processed as served.
• 3xx → followed ~5 hops, alors treated as 404.
• 4xx (except 429) → treated as aucun robots.txt → explorer freely.
• 5xx / 429 / timeout / DNS échec → server error → explorer pause (12h arrêter → ~30d mis en cache fallback → dépend on site health).

A manquant robots.txt is safe. An unreachable un is the emergency.

4. Host status is upstream of everything sinon. Si Google can’t connecter to the host, aucun per-URL report indique vous anything utile yet. Clair host status premier, alors lire the rest of Statistiques d’exploration and Page Indexation.

5. The throttle is aussi the symptom. 503/429 is the legitimate façon to tell Googlebot to slow bas — and the exact signal que lights up a red host status. Utiliser it briefly and on objectif; sustained, it arrête looking comme a throttle and starts looking comme an outage (with indexation risk).

Host status — cheat sheet

The three states (90-day window)

The three checks

robots.txt status-code handling

robots.txt crawl-pause timeline (on server error)

• Premier 12 hours: arrête exploration, garde retrying the fichier.
• Suivant ~30 days: uses dernier bon mis en cache copy, encore retrying.
• Après 30 days: dépend on overall site health.

Fast facts

• Host status lives in Settings → Statistiques d’exploration (top of the report).
• Domain/root-property seulement — pas URL-prefix subfolders.
• Recovery is typically hands-off une fois availability is restored — aucun reset button, aucun documented fixed timeline (manual crawl-rate limiter deprecated).
• It’s host-level, pas per-URL — distinct from Page Indexation errors.
• It’s pas a ranking penalty.

Outils pour diagnosing host status

• Recherche Google Console — Statistiques d’exploration report — où host status lives (Settings → Statistiques d’exploration), plus the three per-check failure-rate graphs and the response-code breakdown.
• GSC robots.txt report — confirmer Google peut récupérer votre robots.txt and voir le code d’état it got.
• Server log fichier analysis — the ground truth pour ce que Googlebot en réalité reçu (5xx/429/timeouts) and quand. Outils: Screaming Frog Log Fichier Analyser, or pipe logs into BigQuery / a log platform.
• DNS diagnostics — dig / nslookup, plus votre DNS provider’s status page, to confirmer resolution during the flagged window.
• Uptime / status monitoring — to line up host-status dips with réel outages, deploys, and config changements.

Ce que devrait I do with ce host-status state?

Host-status mistakes que delay recovery

Treating a manquant robots.txt fichier comme an outage

Pourquoi it’s incorrect: Google treats la plupart 4xx réponses, notamment a clean 404, as si aucun robots.txt restrictions exist. 5xx, 429, DNS échecs, and timeouts are the réponses que trigger explorer pausing. Do à la place: tester the réel réponse code and availability avant creating a fichier simply to faire the 404 disappear.

Waiting pour a manual crawl-rate reset

Pourquoi it’s incorrect: là is aucun reset button, and the old crawl-rate contrôler n’est pas the recovery chemin. Do à la place: restore stable réponses and let Googlebot augmenter exploration automatically.

Reading red and yellow as severity grades

Pourquoi it’s incorrect: the colors mainly encode recency. Red signifie a significant problem occurred in the dernier week; yellow signifie it occurred précédent in the rolling 90-day window. Do à la place: utiliser the failing vérifier and timestamps to judge severity.

En utilisant 503 or 429 as a permanent crawl-control strategy

Pourquoi it’s incorrect: ceux codes peut be a legitimate short-term throttle, but ils are aussi server-error signals que reduce exploration and peut become an indexation risk quand sustained. Do à la place: utiliser les briefly during réel overload and fix the capacity or edge rule causing the pressure.

Diagnosing a host-level échec un URL at a temps

Pourquoi it’s incorrect: host status summarizes robots.txt, DNS, and connectivity échecs at the host level. Do à la place: correlate the failure-rate graph with server, CDN, WAF, DNS, and deployment logs avant chasing isolated page templates.

Rapide host-availability checks

Vérifier robots.txt and its chaîne de redirections

Run ce in a macOS/Linux shell. Replace the hostname; -L follows redirections and the output exposes si the final réponse is acceptable or a crawl-pausing server error.

curl -sS -L -o /dev/null -w 'final=%{url_effective} code=%{http_code} redirects=%{num_redirects} time=%{time_total}s\n' https://example.com/robots.txt

Run the equivalent in PowerShell to voir chaque hop and final status.

$r = Invoke-WebRequest -Uri 'https://example.com/robots.txt' -MaximumRedirection 5
[pscustomobject]@{ Status = [int]$r.StatusCode; FinalUrl = $r.BaseResponse.ResponseUri.AbsoluteUri }

Vérifier authoritative DNS from the command line

Run ces in a macOS/Linux shell. A manquant or intermittent réponse points to the DNS chemin plutôt que the origin application.

dig +short A example.com
dig +short AAAA example.com
dig +trace example.com

Run the PowerShell equivalent on Windows.

Resolve-DnsName example.com -Type A
Resolve-DnsName example.com -Type AAAA

Summarize crawl-pause réponse codes from an accès log

Run ce in a shell contre a standard combined accès log. Adjust the log field si votre format differs; the expression counts 429 and 5xx réponses pour Googlebot.

awk 'tolower($0) ~ /googlebot/ && ($9 == 429 || $9 ~ /^5[0-9][0-9]$/) { count[$9]++ } END { for (code in count) print code, count[code] }' access.log | sort

Ressources utiles

My connexe writing

• Quand Devrait Vous Worry À propos de Budget d’exploration? — où the 5xx/429 throttle, the Statistiques d’exploration report, and host availability fit into the bigger crawl-budget picture.
• Robots.txt and SEO: Everything Vous devez Know — the fichier whose unreachability drives the highest-stakes host-status échec.

From others

• Googlebot Explorer Slump? Mueller Points To Server Errors — Moteur de recherche Journal, Aug 2025; the Mueller quote on 429/500/503/timeouts.
• 5 Top Statistiques d’exploration Insights dans la recherche Google Console — Moteur de recherche Journal, on reading the broader report host status sits à l’intérieur.
• How Google interprets the robots.txt specification — Google pour Developers; the authoritative source pour the 12-hour crawl-stop and 30-day cached-fallback timeline.
• Debug Network and DNS errors pour Google’s robots d’exploration — Google pour Developers; DNS and network-level diagnostics directement relevant to the DNS resolution vérifier.
• Troubleshoot Recherche Google exploration errors — Google pour Developers; Google’s propre guide pour diagnosing le serveur-connectivity échecs que drive a red host status.
• r/TechSEO — the community pour explorer/availability debugging.

Journal des modifications