How to Remove Pages from Google's Index (Deindexing)

TL;DR — DeindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist. means getting a page out of Google’s search results. There’s no single “delete from Google” button — you pick a method based on your situation. And the big trap: blocking a page in robots.txt does not remove it from Google. To actually remove a page, you usually let Google crawl it and add a noindex tag.

What deindexing is

Deindexing is the opposite of getting foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore.: it’s making a URL stop showing up in Google’s search results. People need it for all kinds of reasons — an outdated page, a duplicate, a thin or embarrassing page, a staging URL that leaked, or something with sensitive information on it.

The catch is there’s no one button. The right tool depends on three questions:

1. Do you own the page? (If not, you have one specific tool; if you do, you have several.)
2. Do you want it gone temporarily or permanently?
3. Should the content still exist for real visitors, or should it be deleted?

The trap almost everyone falls into

The single most common mistake is thinking you can remove a page by blocking it in robots.txt. You can’t. Here’s why, in plain terms: robots.txt controls crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. (whether Google is allowed to read the page), not indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. (whether the URL shows up in results). Those are two different systems. Evidence for this claim Blocking crawling with robots.txt is not a reliable way to remove a URL from Google Search. Scope: Google Search; a blocked URL can still be indexed without content when Google discovers it elsewhere. Confidence: high · Verified: 2026-07-14 Google Search Central: Introduction to robots.txt

If you block a page in robots.txt, Google can still list the URL in search results — it just won’t have the content, because it never read the page. Worse: if you also add a noindex tag to that page, Google can never see it, because you blocked Google from reading the page in the first place. As I put it in my Ahrefs write-up, “Indexed, though blocked by robots.txt” Can Be More Than A Robots.txt Block: crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. and indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. are two different things.

The methods, in one breath

• noindex — a tag that tells search engines “don’t list this.” The page stays live for visitors but drops from results. Must stay crawlable so Google can see the tag.
• Delete it (404/410) — remove the content entirely and it drops from the index once Google recrawlsCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial..
• Password-protect it — lock it behind a login and no crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. can reach it.
• The Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. in Google Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. — fast, but only temporary (about six months). Emergency lever, not a fix. Evidence for this claim Search Console's Removals tool temporarily blocks a URL from Google Search and is not a permanent removal method. Scope: Google Search Console verified properties; permanent removal requires a separate lasting method. Confidence: high · Verified: 2026-07-14 Google Search Console Help: Removals tool and SafeSearch Reporting Tool
• Canonical tagA rel=\"canonical\" annotation — in the HTML <head> or an HTTP Link header — that tells search engines which URL is the preferred version of duplicate or near-duplicate content. — for duplicate pages, points Google at the version you want kept. This is consolidation, not really “removal.”

The quick rules of thumb

• Want a page gone but still usable for visitors? Allow crawling + noindex.
• Want it gone forever and it should stop existing? Delete it (return 404 or 410).
• Something leaked and you need it hidden today? Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. — then apply a permanent method during that six-month window.
• Don’t own the page? If it’s just stale/removed content, use Google’s Refresh Outdated Content tool. If it’s personal or sensitive information about you, Google currently points people to its separate “Remove your personal information from Google” troubleshooter instead — see Advanced for the split.
• Never block a page in robots.txt to try to remove it, and never combine robots.txt disallow with noindex.

Want the full decision framework, the exact Google quotes, image removal (where robots.txt actually is the right tool), and how long each method really takes? Switch to the Advanced tab.

TL;DR — DeindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist. has no single button — the method depends on ownership, temporary vs. permanent, and whether the content should survive. The accuracy spine: a robots.txt Disallow does not deindex a page (it blocks crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., not indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.; the URL can still be indexed via links, and Google can’t see a noindex on a page it can’t crawl), and combining robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. with noindex backfires. The GSCA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. is emergency triage only (~6 months) — pair it with a permanent method: noindex (keep live, hide from search), 404/410 (delete), password protection (block access), or a canonical (consolidate duplicates). Robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. is the sanctioned way to remove images. Non-owners split into two tools depending on the problem: stale/removed content uses Refresh Outdated Content; personal or sensitive information uses Google’s separate “Remove your personal information from Google” troubleshooter. Nothing but the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. is instant — every permanent method waits on a recrawlCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial..

What “deindexing” actually means (and what it doesn’t)

There’s no “delete from Google” button, and that’s the first thing to internalize. Deindexing is a situation-dependent task, and the situation is defined by three questions: do you own the page, do you want it gone temporarily or permanently, and should the content itself still exist for users. Get those three straight and the right method usually falls out on its own.

It’s also worth separating two tracks up front:

• Emergency track — you need something hidden fast (a leaked URL, exposed personal info). The Removals tool (if you own the property) or the Refresh Outdated Content tool (if you don’t) buys you time.
• Permanent track — the actual fix: noindex, delete (404/410), password- protect, or canonicalize. This is what has to be applied during or immediately after the emergency step, because the emergency tools all expire.

The core myth: robots.txt does not remove pages from Google

This is the load-bearing teaching point of the entire topic, so it gets the most space.

Crawling and indexing are different systems

robots.txt is a crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. directive. noindex is an indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. directive. The two are decoupled, and that decoupling is the whole story. Google can index a URL it never crawled (if the URL is discoverable through links), and it can never see a noindex rule on a URL it’s blocked from crawling. Evidence for this claim Blocking crawling with robots.txt is not a reliable way to remove a URL from Google Search. Scope: Google Search; a blocked URL can still be indexed without content when Google discovers it elsewhere. Confidence: high · Verified: 2026-07-14 Google Search Central: Introduction to robots.txt

Google’s own docs on noindex state it flatly: “If the page is blocked by a robots.txt file or the crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. can’t access the page, the crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. will never see the noindex rule, and the page can still appear in search results, for example if other pages link to it.” And the reason: “We have to crawl your page in order to see <meta> tags and HTTP headers.”

I’ve written this same point in my own words in the Indexed, though blocked by robots.txt piece: if you block a page from being crawled, Google may still index it because crawling and indexing are two different things — and unless Google can crawl a page, it won’t see the noindex meta tag and may still index it because it has links.

What Google’s removal docs say, verbatim

Google’s “Remove a page hosted on your site from Google” doc is unambiguous. Its guidance for permanent removal lists three sanctioned methods — remove/update the content, password-protect the page, or add a noindex tag — and then says directly: “Don’t use robots.txt as a way to block your page."

"Indexed, though blocked by robots.txt” — the Search Console symptom

If you’ve tried the robots.txt approach, you’ve probably seen the Search ConsoleGoogle's free tool for monitoring crawling, indexing, and search performance. status “Indexed, though blocked by robots.txtA Google Search Console Page Indexing status: the URL was excluded from indexing because your robots.txt disallows crawling it. Usually intentional and benign — robots.txt blocks crawling, not indexing..” That label describes exactly this phenomenon: a URL Google has indexed (usually because it found the URL via links) despite being disallowed from crawling it. In my Ahrefs write-up I break down four distinct causes of that status — a genuine robots.txt crawl block, an intermittent/caching block, a user-agent-specific block, and an IP-based block — so it’s worth diagnosing rather than assuming. (Confirm the exact current label wording in a live Search Console property before quoting it — Google occasionally adjusts the UI text.)

Why combining robots.txt disallow with noindex backfires

This is the single most common self-inflicted failure. If you want a page gone and you add a noindex tag and a robots.txt disallow, you’ve guaranteed the noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. never works. Google’s robots meta tag docs are explicit: “If a page is disallowed from crawling through the robots.txt file, then any information about indexing or serving rules will not be found and will therefore be ignored. If indexing or serving rules must be followed, the URLs containing those rules cannot be disallowed from crawling.”

The rule of thumb: if you want a page removed, leave it crawlable and add noindex. Only consider blocking it in robots.txt after it’s confirmed dropped — and even then, robots.txt won’t stop it resurfacing via link-based indexing.

A quick history lesson: the 2019 noindex-in-robots.txt deprecation

There used to be an unofficial noindex: directive you could put inside robots.txt. It was never an official standard, was inconsistently honored, and Google formally retired even its unofficial handling. From Google’s note on unsupported rules in robots.txt (Gary Illyes, July 2019): “we’re retiring all code that handles unsupported and unpublished rules (such as noindex) on September 1, 2019.” In my own Ahrefs coverage I’ve made the same point: while Google used to unofficially support noindex in robots.txt, it was never an official standard and they’ve now formally removed support. Gary was characteristically blunt about the directive on the way out, calling it something that “sticks out like a nun in a mosh pit.”

Emergency removal: the Search Console Removals tool

When something needs to be hidden now, the Removals tool is the fast lever. Google: “For quick removals, use the Removals tool to remove a page hosted on your site from Google’s search results within a day.” John Mueller has confirmed that for verified owners “it works very quickly.”

But it is emergency triage, not a fix, and the docs are blunt about its two limits:

• It’s temporary. “Requests made in the Removals tool last for about 6 months.” The Search Console Help version says the same: “A successful request lasts only about six months.” Evidence for this claim Search Console's Removals tool temporarily blocks a URL from Google Search and is not a permanent removal method. Scope: Google Search Console verified properties; permanent removal requires a separate lasting method. Confidence: high · Verified: 2026-07-14 Google Search Console Help: Removals tool and SafeSearch Reporting Tool
• It doesn’t stop crawling or truly remove anything. “Blocking a URL does not prevent Google from crawling your page, only from showing it in Search results.”

So without a permanent method applied in parallel, the page can return once the window expires. Use the Removals tool to buy time, then apply noindex, delete the page, or password-protect it during that window.

Bing’s equivalent

Bing has a parallel Block URLs tool, and this time the exact wording is confirmed. Bing Webmaster ToolsMicrosoft's free portal for monitoring and improving how a site appears in Bing search — the peer to Google Search Console, plus IndexNow instant indexing, richer backlink data, and keyword volumes. Because Bing's index also feeds Microsoft Copilot, it doubles as a window into AI-search visibility.’ own interface text for the feature: “This feature allows you to block a URL from appearing in the Bing search results. This block will remain in place for maximum of 90 days. If the URL still returns a 200 OKHTTP 200 OK is the standard 2xx success status code, meaning the server received, understood, and fulfilled the request and is returning the resource. It's the code every page you want indexed should return — but a 200 alone doesn't guarantee Google will index the page. code when visited by our crawler after 90 days, it will reappear in our search results.” Bing also warns you 8 days ahead of a block expiring so you can renew it, and there’s no cap on how many times you can re-block the same URL. In my own Ahrefs piece I noted the comparison in my own words: the temporary effect lasts about six months in Google, while Bing’s equivalent lasts about three months (90 days). Bing’s separate help article on permanent removal (delete + status code, or NOINDEX) is a client-rendered page I still couldn’t extract verbatim text from, so its IndexNowIndexNow is an open push protocol that lets you instantly tell participating search engines (Bing, Yandex, Naver, Seznam, and Yep) which URLs you've added, changed, or removed via a simple HTTP request — and one submission is shared across all of them. Google does not use it.-submission recommendation stays paraphrased below.

Permanent method #1 — noindex (meta tag / X-Robots-Tag)

noindex is the most effective way to remove a URL from the index when crawling is allowed. It tells search engines not to list the page while the page stays live for users and direct visitors. As I’ve put it: a noindex meta robots tag or X-Robots-Tag header response will tell search engines to remove a page from the index.

Two implementations:

• Meta tag in the HTML <head>: <meta name="robots" content="noindex">.
• X-Robots-Tag HTTP header — for non-HTML files like PDFs where there’s no <head>. Google: “Any rule that can be used in a robots meta tagThe robots meta tag is an HTML element in a page's head — <meta name=\"robots\" content=\"noindex\"> — that tells search engines how to index and serve that page. It's crawl-then-obey: a page blocked in robots.txt is never fetched, so the tag is never seen. can also be specified as an X-Robots-TagThe X-Robots-Tag is an HTTP response header that carries the same indexing and serving directives as the robots meta tag (noindex, nofollow, nosnippet, and the rest). Because it lives in the header rather than the HTML, it's how you control indexing for non-HTML files like PDFs, images, and videos..”

The one requirement that trips everyone up: the page must stay crawlable. If it’s blocked in robots.txt, Google never sees the tag. Mueller’s two-step model via Search Engine Journal is the clean mental frame: first make sure Google can access the page (if it’s blocked by robots.txtA Google Search Console Page Indexing status: the URL was excluded from indexing because your robots.txt disallows crawling it. Usually intentional and benign — robots.txt blocks crawling, not indexing., the noindex won’t be seen), then add the noindex tag.

Use it when: the page should stay live for users (or direct/logged-in access) but shouldn’t appear in search — thank-you pages, internal search results, filtered/faceted variants, thin archive pages.

Permanent method #2 — delete the content (404/410)

If the content genuinely shouldn’t exist anymore, delete it and return a 404 (Not Found) or 410 (Gone). Google: both status codes “mean that the page does not exist, which will drop such URLs from Google’s index once they’re crawled and processed.” In my own words: the page will be removed from the index shortly after it’s re-crawled.

404 vs 410: both work. 410 is a marginally more explicit, slightly faster “gone forever” signal, but the practical difference is small — see the dedicated 404-vs-410 comparison for the nuance. The catch with this method: you can’t keep the page live for users while returning a 410. If people still need the page, that’s a noindex job, not a delete job.

Use it when: the content is genuinely retired and no one needs it anymore.

Permanent method #3 — password protection / login walls

Locking a page behind authentication is the most airtight removal. Google: “Password-protect your page. Limiting access to your page enables the right users to view your page, while preventing GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. and other web crawlers from accessing it.” From the 2019 blog post: hiding a page behind a login will generally remove it from Google’s index (unless you use subscription/paywall markup).

Its two advantages: it blocks all crawlers across all search engines (not just Google), and it’s not link-defeatable the way a robots.txt block is. Its trade-off: it blocks all users without credentials too — so it’s for genuinely private content, not pages you still want the public to reach.

Use it when: the content is sensitive/private and only authenticated users should ever see it.

Canonical consolidation — related, but not removal

A canonical tagA rel=\"canonical\" annotation — in the HTML <head> or an HTTP Link header — that tells search engines which URL is the preferred version of duplicate or near-duplicate content. is not a removal tool, and conflating the two causes real damage. CanonicalizationHow search engines pick one canonical URL among duplicates and consolidate signals onto it. is consolidation: when you have genuine duplicate or near-duplicate URLs, rel="canonical" tells Google which one to treat as the representative version and folds ranking signals onto it. Both URLs stay live and crawlable.

Critically, Google specifically advises against using noindex to manage which duplicate is canonical within a site: “We don’t recommend using noindex to prevent selection of a canonical page within a single site, because it will completely block the page from Search.” In other words: if the problem is “two URLs for the same content, keep one,” use a canonical. If the problem is “this page shouldn’t be in search at all,” use noindex. Different problems, different tools.

Use it when: you have real duplicate URLs and want signals consolidated onto one — not as a substitute for noindex on a page you simply don’t want indexed.

Removing images specifically — the exception to the myth

Here’s the one place the robots.txt myth flips: for images, robots.txt genuinely is Google’s sanctioned removal method. Google’s image removal docs: “To prevent images from your site appearing in Google’s search results, add a robots.txt file to the root of the site that hosts the image.” The example:

User-agent: Googlebot-Image
Disallow: /images/dogs.jpg

Why does this work for images when it fails for pages? Because Google Images indexing depends primarily on crawling the image file itself, which robots.txt directly blocks — whereas ordinary web-page indexing can happen from link discovery alone, without crawling. That’s the resolution to what looks like a contradiction: the myth is specifically about pages, not images. Use the Googlebot-Image user-agent to remove an image from Google Images specifically, or the general Googlebot user-agent to remove it from all Google surfaces.

There’s also an X-Robots-Tag alternative for images: “Alternatively, you can remove images hosted on your site from Google’s search results by adding the noindex X-Robots-Tag to the HTTP response headers of the images you want to remove” — but in that case you must allow crawling the image URLs so Googlebot can extract the rule (same crawl-to-see-the-rule logic as pages). Google notes the two methods have the same effect, so implementing both doesn’t make sense — pick one.

If you don’t own the page — two different tools, for two different problems

Everything above assumes you control the page. If you don’t, Google splits this into two separate paths depending on what’s wrong — and picking the wrong one wastes the emergency-track time you don’t have.

The page’s content is stale or already gone, but Google’s snapshot hasn’t caught up. Use the Refresh Outdated Content tool. Google: “Use the Refresh Outdated Content tool to have Google update search results for pages or images that no longer exist, or pages that have deleted important (sensitive or critical) content.” The key constraint: it only works if the content is already gone or significantly changed on the live page. It can’t force removal of content a site owner still wants live — it just prompts Google to update its results to match the current reality of a page. It requires a login but not Search Console verification.

The problem is personal or sensitive information, and the page may still be live. Google’s own current guidance routes this case differently. The doc this article leans on for permanent-removal methods says, verbatim, in its “if you don’t own the page” callout: “If you don’t own the page, see Remove your personal information from Google instead.” That’s a separate troubleshooter built for personal-information removal requests — doxxing, exposed IDs or financial info, non-consensual explicit imagery, and similar categories — and it’s the path Google points to first, not Refresh Outdated Content, when what’s exposed is about you rather than merely outdated. (This is per the currently live version of Google’s removal-information page, last updated 2025-12-10 per its own footer — Google may keep adjusting exactly how it signposts these two paths, so re-check before treating this as permanent.)

If it’s genuinely just outdated/removed content and not a personal-information matter, Refresh Outdated Content remains the right non-owner tool.

Decision framework — which method for which situation

See the Decision Trees tab for an interactive walkthrough.

How long does deindexing actually take?

None of the permanent methods are instant — all of them wait on Google recrawling the URL to see the change (whether that’s the noindex tag or the 404/410 status). That’s precisely why the Removals tool exists: it’s the only genuinely fast lever (within a day, per Google’s docs), which is what makes it the emergency option despite being temporary. If a page you noindexed is still appearing, per Google it’s “probably because we haven’t crawled the page since you added the noindex rule.” Requesting indexing/recrawl in URL Inspection can nudge it along.

Where this fits

This page is the practical hub tying the indexing cluster together. For the directives themselves, the individual deep dives — noindex, meta-robots, x-robots-tag, robots-txt, the removals-tool, canonicalization, and the 404-vs-410 / 410-gone status codes — each go further. For the whole pipeline (discovery, crawling, renderingTurning HTML, CSS, and JavaScript into the final visual page and DOM., indexing, serving), see the How Search Works clusterSearch works in three stages — crawling, indexing, and serving (ranking). A page has to clear each one to appear in results: getting crawled doesn't mean you're indexed, and getting indexed doesn't mean you rank.. And when a robots.txt block did leave a page indexed, the Indexed, though blocked by robots.txt write-up is the direct troubleshooting companion.

AI summary

A condensed take on the Advanced version:

• No single “delete from Google” button. Method depends on three questions: do you own the page, temporary vs. permanent, and should the content still exist.
• The core myth: a robots.txt Disallow does not remove a page. Robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. controls crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., noindex controls indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. — they’re decoupled. A blocked URL can still be indexed via links, and Google can’t see a noindex on a page it can’t crawl.
• Never combine robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. disallow with noindex — the block guarantees the noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. is never seen. Leave the page crawlable and add noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed..
• The Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. is emergency triage, ~6 months — temporary, doesn’t stop crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor.. Pair it with a permanent method or the page returns. Bing’s Block URLs tool is ~90 days.
• Four permanent methods: noindex (keep live, hide from search; must stay crawlable), 404/410 (delete entirely; can’t keep live for users), password protection (blocks all crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index./engines, most airtight, blocks users too), canonical (consolidate real duplicates — not removal).
• Images are the exception: robots.txt (Googlebot-Image disallow) genuinely removes images, because image indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. depends on crawling the file directly.
• Don’t own the page? It depends on the problem. Stale or already-removed content → the Refresh Outdated Content tool (only works if the content is already gone/changed on the live page). Personal or sensitive information → Google’s separate “Remove your personal information from Google” troubleshooter, which is what Google’s own removal-information doc currently points non-owners to first.
• Nothing but the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. is instant — permanent methods all wait on a recrawlCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial..
• The 2019 history: Google formally retired the unofficial noindex: directive inside robots.txt effective September 1, 2019.

Official documentation

Primary-source documentation from the search engines.

Google

• Remove a page hosted on your site from Google — the three permanent methods, the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index.’s ~6-month limit, and “don’t use robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. to block your page.”
• Block Search Indexing with noindex — why the page must stay crawlable for noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. to work.
• Robots meta tag, data-nosnippet, and X-Robots-Tag specifications — why you can’t combine a robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. disallow with an indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. rule.
• A note on unsupported rules in robots.txt (Gary Illyes, July 2019) — the deprecation of the unofficial noindex: directive and the sanctioned alternatives.
• Removals and SafeSearch reports tool — Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. Help: the tool is temporary, ~6 months, and doesn’t stop crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor..
• Refresh Outdated Content tool — the public, non-owner tool for updating outdated results (only works if the content is already gone/changed on the live page).
• Remove your personal information from Google — the troubleshooter Google’s own removal-information doc currently points non-owners to when the issue is personal/sensitive information rather than merely outdated content.
• Remove your own site’s images from Google Search — the Googlebot-Image robots.txt method and the X-Robots-TagThe X-Robots-Tag is an HTTP response header that carries the same indexing and serving directives as the robots meta tag (noindex, nofollow, nosnippet, and the rest). Because it lives in the header rather than the HTML, it's how you control indexing for non-HTML files like PDFs, images, and videos. alternative for images.
• How to specify a canonical URL — why not to use noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. to manage canonical selectionHow search engines pick one canonical URL among duplicates and consolidate signals onto it. within a site.

Bing / Microsoft

• How To Permanently Remove a URL or Page from Bing or Copilot — Bing’s permanent removal guidance (delete + status code, or NOINDEX).
• Block URLs from Bing — Bing’s temporary Block URLs tool (the parallel to Google’s Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index.).

Quotes from the source

On-the-record statements from Google and its reps. Each link is a deep link that jumps to the quoted passage on the source page.

Google — the core myth (crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. vs. indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.)

• “If the page is blocked by a robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. file or the crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. can’t access the page, the crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. will never see the noindex rule, and the page can still appear in search results, for example if other pages link to it.” — Google Search Central docs. Jump to quote
• “Don’t use robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. as a way to block your page.” — Google Search Central docs. Jump to quote
• “If a page is disallowed from crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. through the robots.txt file, then any information about indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. or serving rules will not be foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore. and will therefore be ignored. If indexing or serving rules must be followed, the URLs containing those rules cannot be disallowed from crawling.” Jump to quote

Google — the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. (temporary)

• “For quick removals, use the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. to remove a page hosted on your site from Google’s search results within a day.” Jump to quote
• “Requests made in the Removals tool last for about 6 months.” Jump to quote
• “A successful request lasts only about six months.” — Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. Help. Jump to quote
• “Blocking a URL does not prevent Google from crawling your page, only from showing it in Search results.” — Search ConsoleGoogle's free tool for monitoring crawling, indexing, and search performance. Help. Jump to quote

Google — permanent methods

• “Password-protect your page. Limiting access to your page enables the right users to view your page, while preventing GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. and other web crawlers from accessing it.” Jump to quote
• 404 and 410: “Both status codes mean that the page does not exist, which will drop such URLs from Google’s index once they’re crawled and processed.” Jump to quote
• The 2019 deprecation: “we’re retiring all code that handles unsupported and unpublished rules (such as noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed.) on September 1, 2019.” — Gary Illyes, Google. Jump to quote

Google — canonical vs. noindex, and images

• “We don’t recommend using noindex to prevent selection of a canonical page within a single site, because it will completely block the page from Search.” Jump to quote
• “To prevent images from your site appearing in Google’s search results, add a robots.txt file to the root of the site that hosts the image.” Jump to quote

Google — the non-owner tools

• “Use the Refresh Outdated Content tool to have Google update search results for pages or images that no longer exist, or pages that have deleted important (sensitive or critical) content.” — Search Console Help. Jump to quote
• “If you don’t own the page, see Remove your personal information from Google instead.” — Google Search Central, “Remove a page hosted on your site from Google” (confirmed live, last updated 2025-12-10 UTC per the page’s own footer). Read the doc

Bing — Block URLs (temporary, ~90 days)

• “This feature allows you to block a URL from appearing in the Bing search results. This block will remain in place for maximum of 90 days. If the URL still returns a 200 OKHTTP 200 OK is the standard 2xx success status code, meaning the server received, understood, and fulfilled the request and is returning the resource. It's the code every page you want indexed should return — but a 200 alone doesn't guarantee Google will index the page. code when visited by our crawler after 90 days, it will reappear in our search results.” — Bing Webmaster ToolsMicrosoft's free portal for monitoring and improving how a site appears in Bing search — the peer to Google Search Console, plus IndexNow instant indexing, richer backlink data, and keyword volumes. Because Bing's index also feeds Microsoft Copilot, it doubles as a window into AI-search visibility. interface text. Bing Block URLs help

Google reps

• “Why would you need to unindex them? … (Also, to unindex, don’t block with robots.txt, use noindex instead.)” — John Mueller, Google, March 8, 2021. Read the coverage
• On the unofficial noindex: robots.txt directive: it “sticks out like a nun in a mosh pit.” — Gary Illyes, Google, April 2019. Jump to quote
• On the Removals tool for verified owners: “It works very quickly.” — John Mueller, Google. Jump to quote

Which removal method should I use?

Answer a couple of questions and land on the right method for your situation.

Deindexing mistakes to avoid

The recurring self-inflicted failures, each with why it’s wrong and what to do instead.

Blocking a page in robots.txt to remove it

• Why it’s wrong: robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. controls crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., not indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.. A blocked URL can still be indexed via links, and Google can’t read any content or noindex tag on a page it can’t crawl. You often end up with “Indexed, though blocked by robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere..”
• Do instead: leave the page crawlable and add noindex — or delete it (404/410).

Combining a robots.txt disallow with a noindex tag (“belt and suspenders”)

• Why it’s wrong: it’s actively counterproductive. The disallow prevents Google from ever crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. the page, so it never sees the noindex — Google’s docs say a disallowed page’s indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. rules “will not be foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore. and will therefore be ignored.” You’ve guaranteed the removal fails.
• Do instead: pick one. To remove, use noindex with crawling allowed; only consider robots.txt after the page has dropped (and know it still won’t stop link-based indexing).

Trying to noindex inside robots.txt itself

• Why it’s wrong: the unofficial noindex: directive was never a standard, and Google formally retired even its unofficial handling on September 1, 2019.
• Do instead: use a noindex meta tag or X-Robots-Tag header on the page.

Treating the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. as permanent

• Why it’s wrong: it’s explicitly temporary — “a successful request lasts only about six months” — and it doesn’t stop crawling. The page can return when the request expires.
• Do instead: treat it as emergency triage and apply a permanent method (noindex, 404/410, or password protection) during that window.

Using noindex to manage duplicate/canonical selectionHow search engines pick one canonical URL among duplicates and consolidate signals onto it. within your site

• Why it’s wrong: Google explicitly doesn’t recommend it — noindex fully blocks the page from Search instead of consolidating signals onto the version you’d keep.
• Do instead: use rel="canonical" for genuine duplicates; reserve noindex for pages you truly don’t want in search at all.

Assuming robots.txt never works for anything — including images

• Why it’s wrong: for images, robots.txt (via Googlebot-Image) is Google’s sanctioned removal method, because image indexing depends on crawling the file directly. The myth is about pages, not images.
• Do instead: use a Googlebot-Image disallow to remove images; use noindex (crawlable) for pages.

Expecting removal to be instant

• Why it’s wrong: every permanent method waits on Google recrawling the URL to see the change. Only the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. is fast.
• Do instead: if it’s urgent, use the Removals tool first; otherwise be patient and optionally request recrawlCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial. in URL InspectionA Google Search Console feature that reports how Google sees one specific URL on a property you own. By default it shows the last-indexed snapshot; a separate \"Test live URL\" mode fetches the current version..

SOP — permanently remove a page you own from Google

A repeatable procedure for taking a single URL out of Google’s indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. for good.

1. Decide the outcome. Should the content still exist for users? If yes → noindex. If it’s private → password protection. If it should stop existing → 404/410. If it’s a duplicate of a page you’d keep → rel="canonical" (that’s consolidation, not removal). Use the Decision Trees tab if unsure.
2. Confirm the page is crawlable. Check robots.txt — the URL must not be disallowed, or Google can’t see any tag you add. Remove any conflicting Disallow first.
3. Apply the permanent method:
   • noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed.: add <meta name="robots" content="noindex"> to the <head>, or set X-Robots-Tag: noindex in the HTTP header (for PDFs/non-HTML).
   • delete: remove the content and configure the server to return 404 or 410 for that URL.
   • password: put the page behind authentication.
4. (Optional) If it’s urgent, submit a Removals request in Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. (IndexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. → Removals → New request) to hide it within a day. Remember this is temporary (~6 months) — the permanent method from step 3 is what makes it stick.
5. Request a recrawlCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial.. In URL InspectionA Google Search Console feature that reports how Google sees one specific URL on a property you own. By default it shows the last-indexed snapshot; a separate \"Test live URL\" mode fetches the current version., run “Test live URL” then “Request indexing” so Google re-fetches the page and sees the change sooner.
6. Verify. After Google recrawls, confirm in URL Inspection that the page is no longer indexed (for noindex, it should report the page as excluded by the noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. tag; for 404/410, as not foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore.). If it still shows, per Google it’s usually just that the page hasn’t been recrawled yet.
7. Don’t undo yourself. Do not later add a robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. disallow to the same URL thinking it’s extra-safe — that would stop Google from re-confirming the noindex/404 on future crawls.

Deindexing checklist

Before and after you try to remove a page:

• Confirmed which method fits the situation (noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. / 404-410 / password / canonical / Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. / Refresh Outdated Content).
• The target URL is not blocked in robots.txt (required for noindex, 404/410, and canonical to be seen).
• You are not combining a robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. disallow with a noindex tag.
• For a page that should stay live: noindex applied via meta tag or X-Robots-Tag, crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. allowed.
• For a deleted page: server returns 404 or 410 (not a soft-404 with a 200 status).
• For private content: page is behind authentication (no paywall markup unless it’s genuinely subscription content).
• For duplicates: rel="canonical" used instead of noindex.
• For images: Googlebot-Image disallow in robots.txt (or X-Robots-TagThe X-Robots-Tag is an HTTP response header that carries the same indexing and serving directives as the robots meta tag (noindex, nofollow, nosnippet, and the rest). Because it lives in the header rather than the HTML, it's how you control indexing for non-HTML files like PDFs, images, and videos. with crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. allowed).
• If urgent: Removals request submitted and a permanent method applied in the same window.
• If you don’t own the page: confirmed which non-owner tool fits — Refresh Outdated Content (stale/already-removed content) vs. Google’s “Remove your personal information from Google” troubleshooter (personal/sensitive-info cases) — and, for Refresh Outdated Content, that the content is already gone/changed on the live page.
• RecrawlCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial. requested in URL InspectionA Google Search Console feature that reports how Google sees one specific URL on a property you own. By default it shows the last-indexed snapshot; a separate \"Test live URL\" mode fetches the current version.; removal verified after Google refetches.

Deindexing cheat sheet

What each method does

Fast facts

• robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. disallow does not deindex a page — it blocks crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., not indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed..
• Never combine a robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. disallow with noindex — the noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. is never seen.
• Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index.: Google ~6 months; Bing Block URLs ~90 days — both temporary.
• 404 vs 410: both work; 410 is marginally faster.
• robots.txt is the sanctioned way to remove images (Googlebot-Image).
• Don’t use noindex for canonical selection within a site — it fully blocks the page.
• Nothing but the Removals tool is instant — the rest wait on a recrawl.

The two-line mental model

1. Want it gone but usable? Allow crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. + noindex.
2. Want it gone forever? Delete it (404/410). Urgent? Removals tool first, then a permanent method.

Snippets for deindexing work

Add a noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. meta tag (HTML <head>)

<meta name="robots" content="noindex">

Set noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. via X-Robots-TagThe X-Robots-Tag is an HTTP response header that carries the same indexing and serving directives as the robots meta tag (noindex, nofollow, nosnippet, and the rest). Because it lives in the header rather than the HTML, it's how you control indexing for non-HTML files like PDFs, images, and videos. (Apache .htaccess) — for PDFs / non-HTML

<Files "private-report.pdf">
  Header set X-Robots-Tag "noindex"
</Files>

Set noindex via X-Robots-TagThe X-Robots-Tag is an HTTP response header that carries the same indexing and serving directives as the robots meta tag (noindex, nofollow, nosnippet, and the rest). Because it lives in the header rather than the HTML, it's how you control indexing for non-HTML files like PDFs, images, and videos. (Nginx)

location = /private-report.pdf {
  add_header X-Robots-Tag "noindex";
}

Return a 410 GoneA 410 Gone status code tells search engines a page was intentionally and permanently removed, prompting slightly faster de-indexing than a standard 404. for a retired URL (Apache)

Redirect 410 /old-retired-page/

Remove an image from Google (robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. — the sanctioned image method)

User-agent: Googlebot-Image
Disallow: /images/private-photo.jpg

Check a URL’s status code and indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. headers (shell)

# Confirm the server returns 404/410, or shows the X-Robots-Tag noindex header
curl -sI https://example.com/old-page/ | grep -iE 'HTTP/|x-robots-tag'

Verify a page isn’t accidentally blocked in robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. before you noindex (shell)

# Print the robots.txt so you can confirm the URL is NOT under a Disallow
curl -s https://example.com/robots.txt

Find noindex/blocking directives on a live page — Chrome DevTools Console

// Paste in the Console on the page you're auditing.
// Reports any robots meta tag and the value of the noindex directive.
const m = document.querySelector('meta[name="robots"], meta[name="googlebot"]');
console.log('robots meta:', m ? m.content : '(none)');
console.log('has noindex:', !!(m && /noindex/i.test(m.content)));

Bookmarklet — flag whether the current page is set to noindex

javascript:(function(){var m=document.querySelector('meta[name="robots"],meta[name="googlebot"]');var c=m?m.content:'';alert('robots meta: '+(c||'(none)')+'\nnoindex: '+(/noindex/i.test(c)?'YES':'no'));})();

Extract every <a href> pointing at a URL you’re about to remove — DevTools Console

// Find internal links to the page you're deindexing so you can update/remove them
// (a page with no inbound links is far less likely to linger via link-based indexing).
const target = '/old-page/';
[...document.querySelectorAll('a[href]')]
  .filter(a => a.getAttribute('href').includes(target))
  .forEach(a => console.log(a.getAttribute('href'), '←', a.textContent.trim()));

Remember: for noindex and 404/410 to take effect, the URL must stay crawlable — never pair any of these with a robots.txt disallow on the same URL.

Common deindexing problems

Real symptoms readers hit while trying to remove a page, with the likely cause and the fix.

”Indexed, though blocked by robots.txt” in Search Console

• Symptom: the Page Indexing reportThe Google Search Console report (formerly Index Coverage) showing how many of your URLs are indexed vs. not indexed, and grouping the not-indexed ones by reason. shows the URL under this status — Google has it indexedStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. but the content is grayed out / not previewable.
• Likely cause: the URL is disallowed in robots.txt, but Google foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore. it through inbound links and indexed the URL anyway (it just can’t read the page to see any noindex tag).
• Fix: decide what you actually want. To keep the page removed, remove the robots.txt disallow, leave the page crawlable, and add noindex (or delete it). Confirm the fix with the robots-txt-tester tool (/tools/robots-txt-tester) and re-check the status in Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. after Google recrawlsCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial..

Page still shows in search after you added noindex

• Symptom: you added a noindex meta tag or X-Robots-Tag days ago, but the URL still returns in a site: search or Search ConsoleGoogle's free tool for monitoring crawling, indexing, and search performance. still lists it as indexed.
• Likely cause: almost always one of two things — Google hasn’t recrawled the page yet, or the page is also blocked in robots.txt so Google can’t see the tag at all.
• Fix: check robots.txt isn’t disallowing the URL, then use URL InspectionA Google Search Console feature that reports how Google sees one specific URL on a property you own. By default it shows the last-indexed snapshot; a separate \"Test live URL\" mode fetches the current version. to “Test live URL” and confirm the tag is present in the live rendered response, then “Request indexing.” Per Google, a lingering indexed page after noindex is usually just “we haven’t crawled the page since you added the rule.”

A Removals-tool request expired and the page came back

• Symptom: a page you hid with the GSC Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. reappears in search a few months later.
• Likely cause: the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. is temporary by design (about six months) — if no permanent method was applied during that window, the URL is eligible to be shown again once the request expires.
• Fix: re-check whether a permanent method (noindex, 404/410, or password protection) was ever applied. If not, apply one now and submit a fresh Removals request to bridge the gap while it takes effect.

Deleted page (404/410) still appears in results

• Symptom: the page returns a genuine 404 or 410 when you check it directly, but it’s still listed in search results.
• Likely cause: Google hasn’t recrawled and processed the status code yet — removal only happens “once they’re crawled and processed,” not the moment the status code changes.
• Fix: confirm the status code with the HTTP Status Checker (/tools/http-status-checker), then request a recrawl via URL Inspection. If it’s urgent, use the Removals tool in the meantime.

An image still shows in Google Images after a robots.txt block

• Symptom: you added a Googlebot-Image disallow but the image is still appearing in Google Images results.
• Likely cause: either the disallow rule doesn’t match the actual image path, a syntax error elsewhere in robots.txt is interfering, or the image hasn’t been recrawled/dropped yet.
• Fix: verify the exact disallow path matches the image URL with the robots-txt-tester tool (/tools/robots-txt-tester), and give it time — image results update on Google’s normal recrawl schedule, not instantly.

Rankings dropped after noindex-ing a “duplicate” page

• Symptom: you noindexed what you thought was a duplicate page to “clean up” the index, and the site’s overall visibility for that content dropped instead of consolidating.
• Likely cause: noindex doesn’t consolidate signals onto another URL — it just removes the page from Search entirely. If the two pages weren’t truly redundant, or if you needed the ranking signals to fold onto the other version, noindex was the wrong tool.
• Fix: for genuine duplicates, use rel="canonical" pointing at the version you want to keep instead — check the current canonical relationship with the Canonical Checker (/tools/canonical-checker).

Tools for deindexing work

Verify the crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. and indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. signals directly rather than guessing.

• Robots.txt Tester — check whether a URL is disallowed before you rely on noindex, 404/410, or a canonical actually being seen by Google. This is the first check for almost every method above, since all of them require the page to stay crawlable.
• HTTP Status Checker — confirm a “deleted” URL is genuinely returning 404 or 410 (and not a soft-404 serving a 200), and confirm a page you noindexed is still returning 200 with the tag present.
• Canonical Checker — verify which URL Google is actually treating as canonical when you’re consolidating duplicates instead of removing them.
• Google Search Console CSV Analyzer — review Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results.’s Page IndexingThe Google Search Console report (formerly Index Coverage) showing how many of your URLs are indexed vs. not indexed, and grouping the not-indexed ones by reason. data, including statuses like “Indexed, though blocked by robots.txtA Google Search Console Page Indexing status: the URL was excluded from indexing because your robots.txt disallows crawling it. Usually intentional and benign — robots.txt blocks crawling, not indexing.,” without digging through the UI by hand.

Third-party tools worth using alongside these: Google Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. itself for the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index., URL InspectionA Google Search Console feature that reports how Google sees one specific URL on a property you own. By default it shows the last-indexed snapshot; a separate \"Test live URL\" mode fetches the current version., and the Page Indexing reportThe Google Search Console report (formerly Index Coverage) showing how many of your URLs are indexed vs. not indexed, and grouping the not-indexed ones by reason.; Bing WebmasterMicrosoft's free portal for monitoring and improving how a site appears in Bing search — the peer to Google Search Console, plus IndexNow instant indexing, richer backlink data, and keyword volumes. Because Bing's index also feeds Microsoft Copilot, it doubles as a window into AI-search visibility. Tools for the Block URLs tool; and Screaming Frog for auditing robots.txt, status codes, and canonical tagsA rel=\"canonical\" annotation — in the HTML <head> or an HTTP Link header — that tells search engines which URL is the preferred version of duplicate or near-duplicate content. across a whole site before a bulk removal push.

Validation tests — did the removal actually take effect?

Machine-verifiable checks for each permanent method, so you’re not just assuming Google saw the change.

Test: noindex tag is present and the page is crawlable

• Test to run: run the URL through the Robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. Tester (/tools/robots-txt-tester) to confirm it’s not disallowed, then run curl -sI https://example.com/page/ (or check the rendered <head>/response headers) for the noindex meta tag or X-Robots-Tag: noindex header.
• Expected result: the URL is allowed in robots.txt AND either the meta tag or header shows noindex.
• Failure interpretation: if robots.txt disallows the URL, the noindex will never be seen — fix the disallow first. If neither the tag nor header is present, the removal request has nothing to act on.
• Monitoring window: immediate for the tag/header check itself; 1–2 weeks for Google’s recrawlCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial. and the resulting drop from the indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed..
• Rollback trigger: if the page needs to come back, remove the noindex tag/header and request indexing again — there’s nothing else to undo.

Test: deleted page returns 404/410, not a soft-404

• Test to run: the HTTP Status Checker (/tools/http-status-checker) or curl -I https://example.com/old-page/.
• Expected result: a genuine 404 or 410 status code in the HTTP response, not a 200 that happens to display an error message (a soft 404A soft 404 is a URL that returns a success status code (usually 200 OK) even though the page is empty, missing, or shows a 'not found' message. It isn't a status code a server sends — it's a label search engines apply after comparing the response code against the rendered content, and they treat the page like a 404 for indexing.).
• Failure interpretation: a 200 status with error-page content means Google still treats the URL as a live, indexable page — the status code is what has to change, not just the visible content.
• Monitoring window: immediate for the status code; 1–4 weeks for the URL to actually drop out of Google’s index once recrawled and processed.
• Rollback trigger: if the page needs to return, restore the content and serve a 200 again, then request indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed..

Test: robots.txt is NOT blocking a page you want removed via noindex or deletion

• Test to run: Robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. Tester (/tools/robots-txt-tester) against the exact URL, or curl -s https://example.com/robots.txt and check the URL against the rules by hand.
• Expected result: the URL is allowed (no matching Disallow).
• Failure interpretation: if the URL is disallowed, any noindex tag or status code change on that URL will never be seen by Google — this is the single most common reason a removal “doesn’t work.”
• Monitoring window: immediate.
• Rollback trigger: n/a — this is a precondition check, not a change to roll back.

Test: canonical consolidation is being respected

• Test to run: the Canonical Checker (/tools/canonical-checker) on both the duplicate and the preferred URL, and Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results.’s URL Inspection toolA Google Search Console feature that reports how Google sees one specific URL on a property you own. By default it shows the last-indexed snapshot; a separate \"Test live URL\" mode fetches the current version.’s “Google-selected canonicalA Google Search Console Page Indexing status: you declared a canonical for this URL, but Google overrode your choice, picked a different page as the canonical, and indexed that one instead.” field.
• Expected result: Google’s selected canonical matches the URL you declared.
• Failure interpretation: if Google selects a different canonical than the one you specified, it’s using other signals (internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them., sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.) to override your declaration — check those signals rather than adding noindex to force the issue.
• Monitoring window: 2–4 weeks — canonical selection is not instant and can shift as Google re-evaluates signals.
• Rollback trigger: if consolidation causes a visibility drop for content that wasn’t truly duplicate, remove the canonical tagA rel=\"canonical\" annotation — in the HTML <head> or an HTTP Link header — that tells search engines which URL is the preferred version of duplicate or near-duplicate content. and let both URLs index independently.

Test: Removals-tool request is active and a permanent method is in place before it expires

• Test to run: check the request status in Search ConsoleGoogle's free tool for monitoring crawling, indexing, and search performance. (Indexing → Removals), and separately confirm the permanent method (noindex, 404/410, or password) is live using the relevant test above.
• Expected result: the Removals entry shows “Removed” and the permanent method’s own test also passes.
• Failure interpretation: if the Removals entry is active but the permanent method test fails, the page will reappear once the ~6-month window expires.
• Monitoring window: check again well before the 6-month mark (set a reminder at ~4 months) so there’s time to fix a failed permanent method before expiry.
• Rollback trigger: if removal was a mistake, cancel the Removals request in Search Console and remove the permanent method (undo noindex, restore the content, or remove the password wall).

Deindexing choices in practice

Keep the page for users, remove it from search

Before: an internal account page returns 200, is publicly reachable, and can appear in search. After: keep it crawlable and return noindex; do not block it in robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere., because crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. must fetch the page to see the directive.

Delete content with no replacement

Before: an expired page returns 200 even though its content is gone. After: return 404 or 410 and remove it from internal linksAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them. and sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.. Do not redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. it to an irrelevant homepage merely to avoid a 404.

Consolidate a true duplicate

Before: two equivalent URLs are indexable and compete as separate candidates. After: redirect the duplicate when users do not need it, or keep it crawlable with a canonical pointing to the preferred URL. This is consolidation, not a noindex job.

Hide sensitive material immediately and permanently

Before: a private document is public and indexedStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.. After: restrict access or remove it at the server, use the Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. for temporary search suppression, and keep the permanent access/removal control in place.

Test yourself: deindexing

Five questions on how to actually get a page out of Google. Pick an answer for each, then check.

Resources worth your time

My related writing

• How to Remove URLs From Google Search (5 Methods) — my own walkthrough of the five removal methods, including the Google-vs-Bing tool-duration comparison.
• “Indexed, though blocked by robots.txt” Can Be More Than A Robots.txt Block — why blocked pages still get indexedStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., and the four distinct causes of that Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. status.
• Robots.txt and SEO: Everything You Need to Know — what robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. does (crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor.) and doesn’t (indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.) do.
• The Beginner’s Guide to Technical SEO — where indexing controls fit in the bigger picture.

My speaking

• How Search Works (SlideShare) — my walkthrough of crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor., renderingTurning HTML, CSS, and JavaScript into the final visual page and DOM., indexing, and serving, where deindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist. controls live. (My standing disclaimer applies: “This is my understanding of systems… not going to be 100% complete or accurate.”)

Official

• Google — Remove a page hosted on your site from Google and Block Search Indexing with noindex.
• Google — A note on unsupported rules in robots.txt (Gary Illyes, 2019) — the noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed.-in-robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. deprecation.

From around the industry

• When Unindexing Pages From Google Search Use Noindex, Not Robots.txt (Search Engine Roundtable, Schwartz) — John Mueller’s 2021 tweet on using noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. over robots.txt to unindex.
• Google’s John Mueller Clarifies How To Remove Pages From Search (Search Engine Journal, Southern) — Mueller’s two-step “allow crawling, then noindex” model and “it works very quickly” on the Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index..
• Google Might Change How NoIndex In Robots.txt Works (Search Engine Roundtable) — the origin of the 2019 deprecation, with Gary Illyes’s “nun in a mosh pit” line.
• John Mueller’s Site No Longer In Google Search (Search Engine Roundtable) — the March 2024 cautionary tale of Mueller’s own site vanishing over a robots.txt parsing edge case.
• What is a Noindex Tag? (Ahrefs glossary) — a plain-language reference on the noindex directive and why canonical (not noindex) is the tool for duplicate contentThe same or very similar primary content reachable at more than one URL. There's no general duplicate content penalty — the real costs are possible signal dilution, the wrong URL getting chosen, and less-efficient crawling..
• r/TechSEO — the community for crawl/index and deindexing debugging.

Deindexing

Deindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist.

Related: Noindex, Removals tool

Deindexing

DeindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist. (also called “removing a page from Google” or “unindexing”) means getting a URL to stop appearing in search results. There is no single “delete from Google” button — the right method depends on three questions: do you own the page, do you want it gone temporarily or permanently, and should the content itself still exist for users?

The core toolbox:

• noindex — a meta tag or X-Robots-Tag HTTP header that tells search engines not to list a page. It’s the most effective way to remove a URL when crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. is allowed — the page must stay crawlable so Google can actually see the rule.
• 404 / 410 status codesA 410 Gone status code tells search engines a page was intentionally and permanently removed, prompting slightly faster de-indexing than a standard 404. — delete the content and the URL drops from the indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. once it’s recrawled (410 is a marginally more explicit “gone”).
• The Search ConsoleA free Google service that reports how a site performs in Google Search and surfaces problems with how Google crawls, indexes, and serves it. It's first-party data straight from Google — but you don't need it to appear in results. Removals toolA Google Search Console feature for verified owners to manage how their URLs appear in Search. A Temporary Removal hides a URL for about six months — it does not delete the page from the index. — fast (within a day) but only temporary, about six months; pair it with a permanent method.
• Password protection — blocks crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. access entirely; the most airtight option, and it works across every search engine, not just Google.
• Canonical consolidation (rel="canonical" or a redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't.) — for folding duplicate URLs into one indexed version. This is consolidation, not true removal.

The persistent myth is that a Disallow rule in robots.txt removes a page from the index. It doesn’t. Robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. is a crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. directive, not an indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. one — Google can still index a URL it was blocked from crawling if that URL is discoverable through links, and it will never see a noindex rule on a page it can’t crawl. Robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. is, however, the sanctioned method for removing images specifically (via a Googlebot-Image disallow), because image indexing depends on crawling the image file directly.

Related: Noindex, Removals tool

Changelog