Verify a claimed search crawler before allowlisting it
Check the request IP against the operator's published ranges and, where supported, forward-confirmed reverse DNS instead of trusting a user-agent string.
Free, no signup. Anything can put "Googlebot" in a user-agent string — the IP address is what actually tells you who's crawling. Paste it in and get a verdict from two independent methods: the operator's published IP ranges and forward-confirmed reverse DNS (FCrDNS). If it's a fake, you'll see who the IP really belongs to.
Crawler lists updated 18 days ago.
Published range snapshot is stale. Refresh the operator range files before relying on an IP-range verdict, then rerun the check.
Example data — replace with your own, then press Verify
Checks run from our server; we fetch the URL you enter and don't keep the results. Submitted IPs are checked in memory, never stored. Anonymous aggregate verdict counts feed our research posts; domains are counted once via an anonymized hash, never stored in the clear. Anonymous run-level outcome counters may be used for aggregate research; URLs, domains, IPs, and identifiers are never included, and no statistic is released below 100 runs.
Paste up to 500 IPs or log lines. Published-range matching and user-agent claim extraction run entirely in your browser. Use Spot-check only for the few rows where you want live reverse-DNS confirmation.
You spot this line in your access log and want to know if it's really Google:
66.249.66.1 - - [08/Jul/2026:14:22:10 +0000] "GET /pricing HTTP/1.1" 200 "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
You paste 66.249.66.1 and choose Googlebot. The tool returns:
Saved targets, named lists, and recent check summaries remain only in this browser.
Uses the same inputs as the result above. No request is made from this panel.
MCP tool
Arguments
Next steps
Compare search, AI-training, and AI-search bots, their published ranges, and what can actually be verified.
Understand the major crawlers, their purpose, and the trade-offs before blocking them.
Use a URL list when you need to triage affected requests, not just one IP.
Copy-paste rules for the exact IP. Blocking a single IP is whack-a-mole — for persistent fake-crawler traffic, block by ASN or use your firewall's bot rules.
Not every operator makes verification possible. Some publish IP ranges and support reverse DNS, some publish ranges only, and some publish nothing at all — a user agent alone proves nothing, since any script can send any user-agent string.
| Bot | Operator | Type | Published IP list | Reverse DNS |
|---|---|---|---|---|
| Googlebot · docs | Search | ✅ 315 ranges | ✅ .googlebot.com, .google.com | |
| Google special crawlers · docs | Search | ✅ 270 ranges | ✅ .google.com | |
| Google user-triggered fetchers · docs | User-triggered | ✅ 1550 ranges | ✅ .gae.googleusercontent.com, .google.com, .googleusercontent.com | |
| Bingbot · docs | Microsoft | Search | ✅ 28 ranges | ✅ .search.msn.com |
| YandexBot · docs | Yandex | Search | ❌ none | ✅ .yandex.ru, .yandex.net, .yandex.com |
| Baiduspider · docs | Baidu | Search | ❌ none | ❌ |
| Yeti · docs | Naver | Search | ❌ none | ❌ |
| GPTBot · docs | OpenAI | AI training | ✅ 21 ranges | ❌ |
| OAI-SearchBot · docs | OpenAI | AI search | ✅ 35 ranges | ❌ |
| ChatGPT-User · docs | OpenAI | User-triggered | ✅ 258 ranges | ❌ |
| ClaudeBot · docs | Anthropic | AI training | ✅ 20 ranges | ❌ |
| PerplexityBot · docs | Perplexity | AI search | ✅ 8 ranges | ❌ |
| Perplexity-User · docs | Perplexity | User-triggered | ✅ 4 ranges | ❌ |
| Applebot · docs | Apple | Search | ✅ 12 ranges | ✅ .applebot.apple.com |
| Amazonbot · docs | Amazon | AI training | ❌ none | ✅ .crawl.amazonbot.amazon |
| DuckDuckBot · docs | DuckDuckGo | Search | ✅ 481 ranges | ❌ |
| Meta-ExternalAgent · docs | Meta | AI training | ❌ none | ❌ |
| Bytespider | ByteDance | AI training | ❌ none | ❌ |
crawl-66-249-66-1.googlebot.com) at a domain-label boundary, and a forward lookup of that hostname must answer the exact same IP. Both directions must agree — a PTR record alone is trivially fakeable.The verdict tiers are deliberately distinct: reverse-DNS confirmed is proof; a published-list match is strong but not confirmation; and unverifiable bots such as Bytespider and Meta publish no machine-readable range list — treat those user agents with suspicion. ClaudeBot now has an official published list, so it can return a list match even though Anthropic does not document reverse-DNS verification.
Need evidence across many requests? Use the Log File Analyzer to inspect crawl budget by bot and section, status-code waste, and spoofing signals. Log parsing stays in your browser.
The badge colour maps to one of five verdicts:
Verification runs on a small server endpoint (/api/verify-bot) because it needs live
DNS and a network-owner lookup your browser can't do. It never runs JavaScript against your site or
stores the IP — the address is checked in memory and discarded.
Verification is only as good as what an operator publishes. Crawlers that publish neither a machine-readable range list nor a reverse-DNS method (including Bytespider and Meta) can only ever come back Unverifiable. Bulk mode can match up to 500 IPs or access-log lines against the committed range snapshot in your browser, but live reverse-DNS confirmation still runs one selected row at a time through Spot-check. For deeper crawl-budget analysis across a whole log, use the Log File Analyzer. The verifier reflects the IP as it is right now; a spoofer that has since moved will read differently. And it verifies identity, not intent — a genuinely verified crawler can still be one you'd rather rate-limit.
Take the IP address from your server logs (not the user-agent stringA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target., which anything can fake) and check it two ways: match it against GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer.'s published IP ranges, and run a forward-confirmed reverse DNS lookup — the IP's PTR record should resolve to a hostname ending in googlebot.com or google.com, and a forward lookup of that hostname must return the same IP. This tool does both automatically when you paste the IP and pick Googlebot.
FCrDNS is a two-way DNS check. First a reverse lookup turns the IP into a hostname via its PTR record; then a forward lookup of that hostname must resolve back to the exact same IP. Both directions have to agree at an official domain boundary. A PTR record on its own proves nothing because the owner of an IP block can set it to any string, so the forward confirmation is what makes it trustworthy.
Some operators — Bytespider and Meta's crawler among them — do not publish IP ranges or support reverse DNS, so there is no authoritative range/DNS check for their claims. The tool returns "unverifiable" rather than guessing. Anthropic is no longer in that bucket: it publishes an official crawler IP list. A match is reported specifically as a published-list match, not as reverse-DNS confirmation, and a user-agent string alone remains spoofable.
You can, and the tool gives you copy-paste block rules for Cloudflare, nginx, and Apache when a verdict comes back spoofed. But blocking a single IP is whack-a-mole — spoofers rotate addresses. For persistent fake-crawler traffic it is more effective to block by the ASN (network owner) the tool names in the evidence, or to use your firewall's bot-management rules.
No. Submitted IPs are checked in memory against the published range lists and live DNS, and are not written to any store. The range lists themselves are refreshed weekly from the operators' official sources, so a verdict reflects ranges that are at most a few days old.
Upvote what you want most. New ideas can be submitted from the floating Feedback menu; requests appear here once approved, and the most-wanted rise to the top.
You won't be emailed about that request anymore.
Loading…
New requests are reviewed before they appear here.
Where this tool helps
Check the request IP against the operator's published ranges and, where supported, forward-confirmed reverse DNS instead of trusting a user-agent string.
Compare a claimed identity with the matched range, reverse-DNS evidence, forward confirmation, and actual network owner before deciding whether the request is genuine.
Paste up to 500 IPs or log lines, extract user-agent claims, match committed crawler ranges in the browser, and promote only selected rows to a live spot-check.
Use the spoofed verdict, evidence, result card, and generated Cloudflare, nginx, or Apache rule as a starting point for a least-privilege security review.
Confirm who operates the request first, then make a separate decision about rate limits, access, or blocking based on behavior and site policy.
Watch the full workflow
This beginner walkthrough shows how Bot Verifier checks whether a request claiming to be Googlebot or another crawler is genuine. We will cover the practical use cases, a complete verified example, the evidence behind the result, browser-local bulk matching, a claimed-bot mismatch, exports, limitations, and the safest next step.
Anything can send text that says Googlebot. Bot Verifier starts with the request’s network address instead. It compares that address with current operator-published ranges and, where the operator supports it, checks that the reverse hostname belongs to the official domain and resolves forward to the same address. That gives you evidence about identity before you allow, challenge, rate-limit, or block traffic.
Use the tool before allowlisting a claimed search crawler, when investigating a suspicious request, when triaging a batch of log lines, or when creating a firewall handoff for a confirmed mismatch. The last use case matters most: identity and policy are separate. A genuine crawler can still be too aggressive, while an unverifiable claim is not automatically proof of abuse.
Paste the address from your server or edge log into “IP address”. Under “Claims to be”, choose the crawler named by the request’s user-agent string. Leaving the selector on auto-detect can find a known published range, but a claimed crawler is required for mismatch detection. Submitted addresses are checked in memory and are not stored, while anonymous aggregate verdict counts support research.
The sample report uses a known Googlebot address. The green badge says “Verified — reverse-DNS confirmed”. Beneath it, the evidence shows a Google range, an official Googlebot hostname, and a forward lookup that returns the same address. Two independent checks agree, so this is the strongest identity result the tool can show.
The workflow is simple. Copy the request address from your logs. Select the claimed crawler. Choose “Verify”. Open “Evidence” and review the published range, reverse hostname, forward confirmation, range date, and network owner when available. If a claimed identity fails, investigate the request and review any generated firewall rule before changing production security.
Choose the example labelled “66.249.66.1 as Googlebot”. It fills the address and crawler claim without spending a request. For this walkthrough, the verification endpoint is fulfilled locally with a fixed response built from the refreshed range snapshot. Every external response is blocked, so the result is safe, repeatable, and clearly a demonstration rather than a live lookup.
Choose “Verify”. The status reports that the tool is checking published ranges and the two-way name lookup, then the result opens below the workflow. A live answer can change when operators rotate ranges or name records, so important access decisions should be rerun against current evidence. This fixed capture demonstrates exactly how to read the result.
The verdict is “Verified — reverse-DNS confirmed”. The message explains that the address is genuinely Googlebot because the reverse record resolves to an official Googlebot hostname and a forward lookup returns the exact original address. The result-card download preserves a compact handoff, while “What this result means” jumps to the definitions for every verdict tier.
Open “Evidence”. The matched range is 66.249.66.0 slash 27 from the operator-published list. The reverse hostname ends at the official googlebot dot com boundary. The forward lookup says “Yes”, the resolver state says “confirmed”, and the range result records an observed match. My favorite part is that the evidence stays separate from the headline, so a reviewer can reproduce the reasoning instead of trusting a green badge alone.
The tool keeps five outcomes separate. Reverse-DNS confirmed is the strongest proof. “In published ranges” is a strong list match but not the same proof. “Spoofed” means a claimed crawler failed the available identity checks. “Unverifiable” means the operator publishes too little evidence to decide. “Not a known crawler” is neutral ordinary traffic or an unknown bot, not a security verdict by itself.
For a quick batch, paste up to five hundred addresses or access-log lines. Published-range matching and user-agent claim extraction run entirely in your browser. That keeps the first pass fast and private. Use “Spot-check” only for selected rows that need the live reverse and forward name test, because the local range match alone cannot provide that confirmation.
Choose “Match pasted rows”. The tool parses three fixed rows locally. The first Googlebot claim matches a published Google range. The documentation-only address with the same claim becomes “claim mismatch”, and the plain documentation address has no range match. These are triage labels. They help you choose what to investigate; they are not all complete identity verdicts.
The status confirms that three rows were matched locally. Each row keeps the address, detected user-agent claim, range result, and a “Spot-check” action. “Download CSV” creates a spreadsheet-friendly handoff for the batch. Use the spot-check on a current public address when a range mismatch would affect access, and keep the original log line and timestamp with the review.
Now check the fixed address 1.1.1.1 with a Googlebot claim. The fixture returns “Spoofed” because the address matches no Google crawler range, its hostname is not an official Googlebot name, and the network owner is Cloudflare rather than Google. This demonstrates the mismatch interface. It is not a fresh accusation about a live request.
A confirmed mismatch opens exact-address examples for Cloudflare, nginx, and Apache. Do not paste one directly into production without checking the current request, rule syntax, scope, logs, and rollback path. Blocking one address is also temporary when an abusive client rotates networks. Use the generated rule as an engineering handoff, then prefer the smallest verified-bot or behavior-based control your edge supports.
The coverage table shows each crawler, its operator and purpose, whether it publishes a machine-readable range list, and whether it documents reverse-domain verification. In this visible section, Googlebot supports both methods. GPTBot has a published list but no documented reverse-name method. Yandex documents reverse-domain checks without a list, while Baiduspider and Yeti show neither method.
This feature summary highlights two independent identity checks instead of a user-agent match, five distinct verdicts including an honest “Unverifiable”, an evidence drawer with ranges, hostnames, forward confirmation, network owner, and fetch date, exact-address firewall examples, the supported-crawler coverage table, version four and version six addresses, in-memory handling, and weekly range refreshes.
Verification is only as complete as the evidence an operator publishes. Bytespider and Meta can remain “Unverifiable” when neither a machine-readable list nor reverse-domain method is available. Local bulk mode uses the committed range snapshot, while live name confirmation runs one selected row at a time. Addresses can move, and a verified identity does not decide whether you want that crawler’s behavior. Rerun important checks, then make the access decision independently.
Save the request address, claimed crawler, range-list date, reverse and forward name evidence, verdict, and final access decision together. Recheck important requests against current operator evidence. Most of all, do not let a copied user-agent string become an allowlist. Verify identity first, then apply the smallest policy that fits the crawler’s behavior and your site.