How to Remove Pages from Google's Index (Deindexing)

TL;DR — Deindexing means getting a page out of Google’s search results. There’s no single “delete from Google” button — you pick a method based on your situation. And the big trap: blocking a page in robots.txt does not remove it from Google. To actually remove a page, you usually let Google crawl it and add a noindex tag.

What deindexing is

Deindexing is the opposite of getting found: it’s making a URL stop showing up in Google’s search results. People need it for all kinds of reasons — an outdated page, a duplicate, a thin or embarrassing page, a staging URL that leaked, or something with sensitive information on it.

The catch is there’s no one button. The right tool depends on three questions:

1. Do you own the page? (If not, you have one specific tool; if you do, you have several.)
2. Do you want it gone temporarily or permanently?
3. Should the content still exist for real visitors, or should it be deleted?

The trap almost everyone falls into

The single most common mistake is thinking you can remove a page by blocking it in robots.txt. You can’t. Here’s why, in plain terms: robots.txt controls crawling (whether Google is allowed to read the page), not indexing (whether the URL shows up in results). Those are two different systems. Evidence for this claim Blocking crawling with robots.txt is not a reliable way to remove a URL from Google Search. Scope: Google Search; a blocked URL can still be indexed without content when Google discovers it elsewhere. Confidence: high · Verified: 2026-07-14 Google Search Central: Introduction to robots.txt

If you block a page in robots.txt, Google can still list the URL in search results — it just won’t have the content, because it never read the page. Worse: if you also add a noindex tag to that page, Google can never see it, because you blocked Google from reading the page in the first place. As I put it in my Ahrefs write-up, “Indexed, though blocked by robots.txt” Can Be More Than A Robots.txt Block: crawling and indexing are two different things.

The methods, in one breath

• noindex — a tag that tells search engines “don’t list this.” The page stays live for visitors but drops from results. Must stay crawlable so Google can see the tag.
• Delete it (404/410) — remove the content entirely and it drops from the index once Google recrawls.
• Password-protect it — lock it behind a login and no crawler can reach it.
• The Removals tool in Google Search Console — fast, but only temporary (about six months). Emergency lever, not a fix. Evidence for this claim Search Console's Removals tool temporarily blocks a URL from Google Search and is not a permanent removal method. Scope: Google Search Console verified properties; permanent removal requires a separate lasting method. Confidence: high · Verified: 2026-07-14 Google Search Console Help: Removals tool and SafeSearch Reporting Tool
• Canonical tag — for duplicate pages, points Google at the version you want kept. This is consolidation, not really “removal.”

The quick rules of thumb

• Want a page gone but still usable for visitors? Allow crawling + noindex.
• Want it gone forever and it should stop existing? Delete it (return 404 or 410).
• Something leaked and you need it hidden today? Removals tool — then apply a permanent method during that six-month window.
• Don’t own the page? If it’s just stale/removed content, use Google’s Refresh Outdated Content tool. If it’s personal or sensitive information about you, Google currently points people to its separate “Remove your personal information from Google” troubleshooter instead — see Advanced for the split.
• Never block a page in robots.txt to try to remove it, and never combine robots.txt disallow with noindex.

Want the full decision framework, the exact Google quotes, image removal (where robots.txt actually is the right tool), and how long each method really takes? Switch to the Advanced tab.

TL;DR — Deindexing has no single button — the method depends on ownership, temporary vs. permanent, and whether the content should survive. The accuracy spine: a robots.txt Disallow does not deindex a page (it blocks crawling, not indexing; the URL can still be indexed via links, and Google can’t see a noindex on a page it can’t crawl), and combining robots.txt with noindex backfires. The GSC Removals tool is emergency triage only (~6 months) — pair it with a permanent method: noindex (keep live, hide from search), 404/410 (delete), password protection (block access), or a canonical (consolidate duplicates). Robots.txt is the sanctioned way to remove images. Non-owners split into two tools depending on the problem: stale/removed content uses Refresh Outdated Content; personal or sensitive information uses Google’s separate “Remove your personal information from Google” troubleshooter. Nothing but the Removals tool is instant — every permanent method waits on a recrawl.

What “deindexing” actually means (and what it doesn’t)

There’s no “delete from Google” button, and that’s the first thing to internalize. Deindexing is a situation-dependent task, and the situation is defined by three questions: do you own the page, do you want it gone temporarily or permanently, and should the content itself still exist for users. Get those three straight and the right method usually falls out on its own.

It’s also worth separating two tracks up front:

• Emergency track — you need something hidden fast (a leaked URL, exposed personal info). The Removals tool (if you own the property) or the Refresh Outdated Content tool (if you don’t) buys you time.
• Permanent track — the actual fix: noindex, delete (404/410), password- protect, or canonicalize. This is what has to be applied during or immediately after the emergency step, because the emergency tools all expire.

The core myth: robots.txt does not remove pages from Google

This is the load-bearing teaching point of the entire topic, so it gets the most space.

Crawling and indexing are different systems

robots.txt is a crawling directive. noindex is an indexing directive. The two are decoupled, and that decoupling is the whole story. Google can index a URL it never crawled (if the URL is discoverable through links), and it can never see a noindex rule on a URL it’s blocked from crawling. Evidence for this claim Blocking crawling with robots.txt is not a reliable way to remove a URL from Google Search. Scope: Google Search; a blocked URL can still be indexed without content when Google discovers it elsewhere. Confidence: high · Verified: 2026-07-14 Google Search Central: Introduction to robots.txt

Google’s own docs on noindex state it flatly: “If the page is blocked by a robots.txt file or the crawler can’t access the page, the crawler will never see the noindex rule, and the page can still appear in search results, for example if other pages link to it.” And the reason: “We have to crawl your page in order to see <meta> tags and HTTP headers.”

I’ve written this same point in my own words in the Indexed, though blocked by robots.txt piece: if you block a page from being crawled, Google may still index it because crawling and indexing are two different things — and unless Google can crawl a page, it won’t see the noindex meta tag and may still index it because it has links.

What Google’s removal docs say, verbatim

Google’s “Remove a page hosted on your site from Google” doc is unambiguous. Its guidance for permanent removal lists three sanctioned methods — remove/update the content, password-protect the page, or add a noindex tag — and then says directly: “Don’t use robots.txt as a way to block your page."

"Indexed, though blocked by robots.txt” — the Search Console symptom

If you’ve tried the robots.txt approach, you’ve probably seen the Search Console status “Indexed, though blocked by robots.txt.” That label describes exactly this phenomenon: a URL Google has indexed (usually because it found the URL via links) despite being disallowed from crawling it. In my Ahrefs write-up I break down four distinct causes of that status — a genuine robots.txt crawl block, an intermittent/caching block, a user-agent-specific block, and an IP-based block — so it’s worth diagnosing rather than assuming. (Confirm the exact current label wording in a live Search Console property before quoting it — Google occasionally adjusts the UI text.)

Why combining robots.txt disallow with noindex backfires

This is the single most common self-inflicted failure. If you want a page gone and you add a noindex tag and a robots.txt disallow, you’ve guaranteed the noindex never works. Google’s robots meta tag docs are explicit: “If a page is disallowed from crawling through the robots.txt file, then any information about indexing or serving rules will not be found and will therefore be ignored. If indexing or serving rules must be followed, the URLs containing those rules cannot be disallowed from crawling.”

The rule of thumb: if you want a page removed, leave it crawlable and add noindex. Only consider blocking it in robots.txt after it’s confirmed dropped — and even then, robots.txt won’t stop it resurfacing via link-based indexing.

A quick history lesson: the 2019 noindex-in-robots.txt deprecation

There used to be an unofficial noindex: directive you could put inside robots.txt. It was never an official standard, was inconsistently honored, and Google formally retired even its unofficial handling. From Google’s note on unsupported rules in robots.txt (Gary Illyes, July 2019): “we’re retiring all code that handles unsupported and unpublished rules (such as noindex) on September 1, 2019.” In my own Ahrefs coverage I’ve made the same point: while Google used to unofficially support noindex in robots.txt, it was never an official standard and they’ve now formally removed support. Gary was characteristically blunt about the directive on the way out, calling it something that “sticks out like a nun in a mosh pit.”

Emergency removal: the Search Console Removals tool

When something needs to be hidden now, the Removals tool is the fast lever. Google: “For quick removals, use the Removals tool to remove a page hosted on your site from Google’s search results within a day.” John Mueller has confirmed that for verified owners “it works very quickly.”

But it is emergency triage, not a fix, and the docs are blunt about its two limits:

• It’s temporary. “Requests made in the Removals tool last for about 6 months.” The Search Console Help version says the same: “A successful request lasts only about six months.” Evidence for this claim Search Console's Removals tool temporarily blocks a URL from Google Search and is not a permanent removal method. Scope: Google Search Console verified properties; permanent removal requires a separate lasting method. Confidence: high · Verified: 2026-07-14 Google Search Console Help: Removals tool and SafeSearch Reporting Tool
• It doesn’t stop crawling or truly remove anything. “Blocking a URL does not prevent Google from crawling your page, only from showing it in Search results.”

So without a permanent method applied in parallel, the page can return once the window expires. Use the Removals tool to buy time, then apply noindex, delete the page, or password-protect it during that window.

Bing’s equivalent

Bing has a parallel Block URLs tool, and this time the exact wording is confirmed. Bing Webmaster Tools’ own interface text for the feature: “This feature allows you to block a URL from appearing in the Bing search results. This block will remain in place for maximum of 90 days. If the URL still returns a 200 OK code when visited by our crawler after 90 days, it will reappear in our search results.” Bing also warns you 8 days ahead of a block expiring so you can renew it, and there’s no cap on how many times you can re-block the same URL. In my own Ahrefs piece I noted the comparison in my own words: the temporary effect lasts about six months in Google, while Bing’s equivalent lasts about three months (90 days). Bing’s separate help article on permanent removal (delete + status code, or NOINDEX) is a client-rendered page I still couldn’t extract verbatim text from, so its IndexNow-submission recommendation stays paraphrased below.

Permanent method #1 — noindex (meta tag / X-Robots-Tag)

noindex is the most effective way to remove a URL from the index when crawling is allowed. It tells search engines not to list the page while the page stays live for users and direct visitors. As I’ve put it: a noindex meta robots tag or X-Robots-Tag header response will tell search engines to remove a page from the index.

Two implementations:

• Meta tag in the HTML <head>: <meta name="robots" content="noindex">.
• X-Robots-Tag HTTP header — for non-HTML files like PDFs where there’s no <head>. Google: “Any rule that can be used in a robots meta tag can also be specified as an X-Robots-Tag.”

The one requirement that trips everyone up: the page must stay crawlable. If it’s blocked in robots.txt, Google never sees the tag. Mueller’s two-step model via Search Engine Journal is the clean mental frame: first make sure Google can access the page (if it’s blocked by robots.txt, the noindex won’t be seen), then add the noindex tag.

Use it when: the page should stay live for users (or direct/logged-in access) but shouldn’t appear in search — thank-you pages, internal search results, filtered/faceted variants, thin archive pages.

Permanent method #2 — delete the content (404/410)

If the content genuinely shouldn’t exist anymore, delete it and return a 404 (Not Found) or 410 (Gone). Google: both status codes “mean that the page does not exist, which will drop such URLs from Google’s index once they’re crawled and processed.” In my own words: the page will be removed from the index shortly after it’s re-crawled.

404 vs 410: both work. 410 is a marginally more explicit, slightly faster “gone forever” signal, but the practical difference is small — see the dedicated 404-vs-410 comparison for the nuance. The catch with this method: you can’t keep the page live for users while returning a 410. If people still need the page, that’s a noindex job, not a delete job.

Use it when: the content is genuinely retired and no one needs it anymore.

Permanent method #3 — password protection / login walls

Locking a page behind authentication is the most airtight removal. Google: “Password-protect your page. Limiting access to your page enables the right users to view your page, while preventing Googlebot and other web crawlers from accessing it.” From the 2019 blog post: hiding a page behind a login will generally remove it from Google’s index (unless you use subscription/paywall markup).

Its two advantages: it blocks all crawlers across all search engines (not just Google), and it’s not link-defeatable the way a robots.txt block is. Its trade-off: it blocks all users without credentials too — so it’s for genuinely private content, not pages you still want the public to reach.

Use it when: the content is sensitive/private and only authenticated users should ever see it.

Canonical consolidation — related, but not removal

A canonical tag is not a removal tool, and conflating the two causes real damage. Canonicalization is consolidation: when you have genuine duplicate or near-duplicate URLs, rel="canonical" tells Google which one to treat as the representative version and folds ranking signals onto it. Both URLs stay live and crawlable.

Critically, Google specifically advises against using noindex to manage which duplicate is canonical within a site: “We don’t recommend using noindex to prevent selection of a canonical page within a single site, because it will completely block the page from Search.” In other words: if the problem is “two URLs for the same content, keep one,” use a canonical. If the problem is “this page shouldn’t be in search at all,” use noindex. Different problems, different tools.

Use it when: you have real duplicate URLs and want signals consolidated onto one — not as a substitute for noindex on a page you simply don’t want indexed.

Removing images specifically — the exception to the myth

Here’s the one place the robots.txt myth flips: for images, robots.txt genuinely is Google’s sanctioned removal method. Google’s image removal docs: “To prevent images from your site appearing in Google’s search results, add a robots.txt file to the root of the site that hosts the image.” The example:

User-agent: Googlebot-Image
Disallow: /images/dogs.jpg

Why does this work for images when it fails for pages? Because Google Images indexing depends primarily on crawling the image file itself, which robots.txt directly blocks — whereas ordinary web-page indexing can happen from link discovery alone, without crawling. That’s the resolution to what looks like a contradiction: the myth is specifically about pages, not images. Use the Googlebot-Image user-agent to remove an image from Google Images specifically, or the general Googlebot user-agent to remove it from all Google surfaces.

There’s also an X-Robots-Tag alternative for images: “Alternatively, you can remove images hosted on your site from Google’s search results by adding the noindex X-Robots-Tag to the HTTP response headers of the images you want to remove” — but in that case you must allow crawling the image URLs so Googlebot can extract the rule (same crawl-to-see-the-rule logic as pages). Google notes the two methods have the same effect, so implementing both doesn’t make sense — pick one.

If you don’t own the page — two different tools, for two different problems

Everything above assumes you control the page. If you don’t, Google splits this into two separate paths depending on what’s wrong — and picking the wrong one wastes the emergency-track time you don’t have.

The page’s content is stale or already gone, but Google’s snapshot hasn’t caught up. Use the Refresh Outdated Content tool. Google: “Use the Refresh Outdated Content tool to have Google update search results for pages or images that no longer exist, or pages that have deleted important (sensitive or critical) content.” The key constraint: it only works if the content is already gone or significantly changed on the live page. It can’t force removal of content a site owner still wants live — it just prompts Google to update its results to match the current reality of a page. It requires a login but not Search Console verification.

The problem is personal or sensitive information, and the page may still be live. Google’s own current guidance routes this case differently. The doc this article leans on for permanent-removal methods says, verbatim, in its “if you don’t own the page” callout: “If you don’t own the page, see Remove your personal information from Google instead.” That’s a separate troubleshooter built for personal-information removal requests — doxxing, exposed IDs or financial info, non-consensual explicit imagery, and similar categories — and it’s the path Google points to first, not Refresh Outdated Content, when what’s exposed is about you rather than merely outdated. (This is per the currently live version of Google’s removal-information page, last updated 2025-12-10 per its own footer — Google may keep adjusting exactly how it signposts these two paths, so re-check before treating this as permanent.)

If it’s genuinely just outdated/removed content and not a personal-information matter, Refresh Outdated Content remains the right non-owner tool.

Decision framework — which method for which situation

See the Decision Trees tab for an interactive walkthrough.

How long does deindexing actually take?

None of the permanent methods are instant — all of them wait on Google recrawling the URL to see the change (whether that’s the noindex tag or the 404/410 status). That’s precisely why the Removals tool exists: it’s the only genuinely fast lever (within a day, per Google’s docs), which is what makes it the emergency option despite being temporary. If a page you noindexed is still appearing, per Google it’s “probably because we haven’t crawled the page since you added the noindex rule.” Requesting indexing/recrawl in URL Inspection can nudge it along.

Where this fits

This page is the practical hub tying the indexing cluster together. For the directives themselves, the individual deep dives — noindex, meta-robots, x-robots-tag, robots-txt, the removals-tool, canonicalization, and the 404-vs-410 / 410-gone status codes — each go further. For the whole pipeline (discovery, crawling, rendering, indexing, serving), see the How Search Works cluster. And when a robots.txt block did leave a page indexed, the Indexed, though blocked by robots.txt write-up is the direct troubleshooting companion.

AI summary

A condensed take on the Advanced version:

• No single “delete from Google” button. Method depends on three questions: do you own the page, temporary vs. permanent, and should the content still exist.
• The core myth: a robots.txt Disallow does not remove a page. Robots.txt controls crawling, noindex controls indexing — they’re decoupled. A blocked URL can still be indexed via links, and Google can’t see a noindex on a page it can’t crawl.
• Never combine robots.txt disallow with noindex — the block guarantees the noindex is never seen. Leave the page crawlable and add noindex.
• The Removals tool is emergency triage, ~6 months — temporary, doesn’t stop crawling. Pair it with a permanent method or the page returns. Bing’s Block URLs tool is ~90 days.
• Four permanent methods: noindex (keep live, hide from search; must stay crawlable), 404/410 (delete entirely; can’t keep live for users), password protection (blocks all crawlers/engines, most airtight, blocks users too), canonical (consolidate real duplicates — not removal).
• Images are the exception: robots.txt (Googlebot-Image disallow) genuinely removes images, because image indexing depends on crawling the file directly.
• Don’t own the page? It depends on the problem. Stale or already-removed content → the Refresh Outdated Content tool (only works if the content is already gone/changed on the live page). Personal or sensitive information → Google’s separate “Remove your personal information from Google” troubleshooter, which is what Google’s own removal-information doc currently points non-owners to first.
• Nothing but the Removals tool is instant — permanent methods all wait on a recrawl.
• The 2019 history: Google formally retired the unofficial noindex: directive inside robots.txt effective September 1, 2019.

Official documentation

Primary-source documentation from the search engines.

Google

• Remove a page hosted on your site from Google — the three permanent methods, the Removals tool’s ~6-month limit, and “don’t use robots.txt to block your page.”
• Block Search Indexing with noindex — why the page must stay crawlable for noindex to work.
• Robots meta tag, data-nosnippet, and X-Robots-Tag specifications — why you can’t combine a robots.txt disallow with an indexing rule.
• A note on unsupported rules in robots.txt (Gary Illyes, July 2019) — the deprecation of the unofficial noindex: directive and the sanctioned alternatives.
• Removals and SafeSearch reports tool — Search Console Help: the tool is temporary, ~6 months, and doesn’t stop crawling.
• Refresh Outdated Content tool — the public, non-owner tool for updating outdated results (only works if the content is already gone/changed on the live page).
• Remove your personal information from Google — the troubleshooter Google’s own removal-information doc currently points non-owners to when the issue is personal/sensitive information rather than merely outdated content.
• Remove your own site’s images from Google Search — the Googlebot-Image robots.txt method and the X-Robots-Tag alternative for images.
• How to specify a canonical URL — why not to use noindex to manage canonical selection within a site.

Bing / Microsoft

• How To Permanently Remove a URL or Page from Bing or Copilot — Bing’s permanent removal guidance (delete + status code, or NOINDEX).
• Block URLs from Bing — Bing’s temporary Block URLs tool (the parallel to Google’s Removals tool).

Quotes from the source

On-the-record statements from Google and its reps. Each link is a deep link that jumps to the quoted passage on the source page.

Google — the core myth (crawling vs. indexing)

• “If the page is blocked by a robots.txt file or the crawler can’t access the page, the crawler will never see the noindex rule, and the page can still appear in search results, for example if other pages link to it.” — Google Search Central docs. Jump to quote
• “Don’t use robots.txt as a way to block your page.” — Google Search Central docs. Jump to quote
• “If a page is disallowed from crawling through the robots.txt file, then any information about indexing or serving rules will not be found and will therefore be ignored. If indexing or serving rules must be followed, the URLs containing those rules cannot be disallowed from crawling.” Jump to quote

Google — the Removals tool (temporary)

• “For quick removals, use the Removals tool to remove a page hosted on your site from Google’s search results within a day.” Jump to quote
• “Requests made in the Removals tool last for about 6 months.” Jump to quote
• “A successful request lasts only about six months.” — Search Console Help. Jump to quote
• “Blocking a URL does not prevent Google from crawling your page, only from showing it in Search results.” — Search Console Help. Jump to quote

Google — permanent methods

• “Password-protect your page. Limiting access to your page enables the right users to view your page, while preventing Googlebot and other web crawlers from accessing it.” Jump to quote
• 404 and 410: “Both status codes mean that the page does not exist, which will drop such URLs from Google’s index once they’re crawled and processed.” Jump to quote
• The 2019 deprecation: “we’re retiring all code that handles unsupported and unpublished rules (such as noindex) on September 1, 2019.” — Gary Illyes, Google. Jump to quote

Google — canonical vs. noindex, and images

• “We don’t recommend using noindex to prevent selection of a canonical page within a single site, because it will completely block the page from Search.” Jump to quote
• “To prevent images from your site appearing in Google’s search results, add a robots.txt file to the root of the site that hosts the image.” Jump to quote

Google — the non-owner tools

• “Use the Refresh Outdated Content tool to have Google update search results for pages or images that no longer exist, or pages that have deleted important (sensitive or critical) content.” — Search Console Help. Jump to quote
• “If you don’t own the page, see Remove your personal information from Google instead.” — Google Search Central, “Remove a page hosted on your site from Google” (confirmed live, last updated 2025-12-10 UTC per the page’s own footer). Read the doc

Bing — Block URLs (temporary, ~90 days)

• “This feature allows you to block a URL from appearing in the Bing search results. This block will remain in place for maximum of 90 days. If the URL still returns a 200 OK code when visited by our crawler after 90 days, it will reappear in our search results.” — Bing Webmaster Tools interface text. Bing Block URLs help

Google reps

• “Why would you need to unindex them? … (Also, to unindex, don’t block with robots.txt, use noindex instead.)” — John Mueller, Google, March 8, 2021. Read the coverage
• On the unofficial noindex: robots.txt directive: it “sticks out like a nun in a mosh pit.” — Gary Illyes, Google, April 2019. Jump to quote
• On the Removals tool for verified owners: “It works very quickly.” — John Mueller, Google. Jump to quote

Which removal method should I use?

Answer a couple of questions and land on the right method for your situation.

Deindexing mistakes to avoid

The recurring self-inflicted failures, each with why it’s wrong and what to do instead.

Blocking a page in robots.txt to remove it

• Why it’s wrong: robots.txt controls crawling, not indexing. A blocked URL can still be indexed via links, and Google can’t read any content or noindex tag on a page it can’t crawl. You often end up with “Indexed, though blocked by robots.txt.”
• Do instead: leave the page crawlable and add noindex — or delete it (404/410).

Combining a robots.txt disallow with a noindex tag (“belt and suspenders”)

• Why it’s wrong: it’s actively counterproductive. The disallow prevents Google from ever crawling the page, so it never sees the noindex — Google’s docs say a disallowed page’s indexing rules “will not be found and will therefore be ignored.” You’ve guaranteed the removal fails.
• Do instead: pick one. To remove, use noindex with crawling allowed; only consider robots.txt after the page has dropped (and know it still won’t stop link-based indexing).

Trying to noindex inside robots.txt itself

• Why it’s wrong: the unofficial noindex: directive was never a standard, and Google formally retired even its unofficial handling on September 1, 2019.
• Do instead: use a noindex meta tag or X-Robots-Tag header on the page.

Treating the Removals tool as permanent

• Why it’s wrong: it’s explicitly temporary — “a successful request lasts only about six months” — and it doesn’t stop crawling. The page can return when the request expires.
• Do instead: treat it as emergency triage and apply a permanent method (noindex, 404/410, or password protection) during that window.

Using noindex to manage duplicate/canonical selection within your site

• Why it’s wrong: Google explicitly doesn’t recommend it — noindex fully blocks the page from Search instead of consolidating signals onto the version you’d keep.
• Do instead: use rel="canonical" for genuine duplicates; reserve noindex for pages you truly don’t want in search at all.

Assuming robots.txt never works for anything — including images

• Why it’s wrong: for images, robots.txt (via Googlebot-Image) is Google’s sanctioned removal method, because image indexing depends on crawling the file directly. The myth is about pages, not images.
• Do instead: use a Googlebot-Image disallow to remove images; use noindex (crawlable) for pages.

Expecting removal to be instant

• Why it’s wrong: every permanent method waits on Google recrawling the URL to see the change. Only the Removals tool is fast.
• Do instead: if it’s urgent, use the Removals tool first; otherwise be patient and optionally request recrawl in URL Inspection.

SOP — permanently remove a page you own from Google

A repeatable procedure for taking a single URL out of Google’s index for good.

1. Decide the outcome. Should the content still exist for users? If yes → noindex. If it’s private → password protection. If it should stop existing → 404/410. If it’s a duplicate of a page you’d keep → rel="canonical" (that’s consolidation, not removal). Use the Decision Trees tab if unsure.
2. Confirm the page is crawlable. Check robots.txt — the URL must not be disallowed, or Google can’t see any tag you add. Remove any conflicting Disallow first.
3. Apply the permanent method:
   • noindex: add <meta name="robots" content="noindex"> to the <head>, or set X-Robots-Tag: noindex in the HTTP header (for PDFs/non-HTML).
   • delete: remove the content and configure the server to return 404 or 410 for that URL.
   • password: put the page behind authentication.
4. (Optional) If it’s urgent, submit a Removals request in Search Console (Indexing → Removals → New request) to hide it within a day. Remember this is temporary (~6 months) — the permanent method from step 3 is what makes it stick.
5. Request a recrawl. In URL Inspection, run “Test live URL” then “Request indexing” so Google re-fetches the page and sees the change sooner.
6. Verify. After Google recrawls, confirm in URL Inspection that the page is no longer indexed (for noindex, it should report the page as excluded by the noindex tag; for 404/410, as not found). If it still shows, per Google it’s usually just that the page hasn’t been recrawled yet.
7. Don’t undo yourself. Do not later add a robots.txt disallow to the same URL thinking it’s extra-safe — that would stop Google from re-confirming the noindex/404 on future crawls.

Deindexing checklist

Before and after you try to remove a page:

• Confirmed which method fits the situation (noindex / 404-410 / password / canonical / Removals tool / Refresh Outdated Content).
• The target URL is not blocked in robots.txt (required for noindex, 404/410, and canonical to be seen).
• You are not combining a robots.txt disallow with a noindex tag.
• For a page that should stay live: noindex applied via meta tag or X-Robots-Tag, crawling allowed.
• For a deleted page: server returns 404 or 410 (not a soft-404 with a 200 status).
• For private content: page is behind authentication (no paywall markup unless it’s genuinely subscription content).
• For duplicates: rel="canonical" used instead of noindex.
• For images: Googlebot-Image disallow in robots.txt (or X-Robots-Tag with crawling allowed).
• If urgent: Removals request submitted and a permanent method applied in the same window.
• If you don’t own the page: confirmed which non-owner tool fits — Refresh Outdated Content (stale/already-removed content) vs. Google’s “Remove your personal information from Google” troubleshooter (personal/sensitive-info cases) — and, for Refresh Outdated Content, that the content is already gone/changed on the live page.
• Recrawl requested in URL Inspection; removal verified after Google refetches.

Deindexing cheat sheet

What each method does

Fast facts

• robots.txt disallow does not deindex a page — it blocks crawling, not indexing.
• Never combine a robots.txt disallow with noindex — the noindex is never seen.
• Removals tool: Google ~6 months; Bing Block URLs ~90 days — both temporary.
• 404 vs 410: both work; 410 is marginally faster.
• robots.txt is the sanctioned way to remove images (Googlebot-Image).
• Don’t use noindex for canonical selection within a site — it fully blocks the page.
• Nothing but the Removals tool is instant — the rest wait on a recrawl.

The two-line mental model

1. Want it gone but usable? Allow crawling + noindex.
2. Want it gone forever? Delete it (404/410). Urgent? Removals tool first, then a permanent method.

Snippets for deindexing work

Add a noindex meta tag (HTML <head>)

<meta name="robots" content="noindex">

Set noindex via X-Robots-Tag (Apache .htaccess) — for PDFs / non-HTML

<Files "private-report.pdf">
  Header set X-Robots-Tag "noindex"
</Files>

Set noindex via X-Robots-Tag (Nginx)

location = /private-report.pdf {
  add_header X-Robots-Tag "noindex";
}

Return a 410 Gone for a retired URL (Apache)

Redirect 410 /old-retired-page/

Remove an image from Google (robots.txt — the sanctioned image method)

User-agent: Googlebot-Image
Disallow: /images/private-photo.jpg

Check a URL’s status code and indexing headers (shell)

# Confirm the server returns 404/410, or shows the X-Robots-Tag noindex header
curl -sI https://example.com/old-page/ | grep -iE 'HTTP/|x-robots-tag'

Verify a page isn’t accidentally blocked in robots.txt before you noindex (shell)

# Print the robots.txt so you can confirm the URL is NOT under a Disallow
curl -s https://example.com/robots.txt

Find noindex/blocking directives on a live page — Chrome DevTools Console

// Paste in the Console on the page you're auditing.
// Reports any robots meta tag and the value of the noindex directive.
const m = document.querySelector('meta[name="robots"], meta[name="googlebot"]');
console.log('robots meta:', m ? m.content : '(none)');
console.log('has noindex:', !!(m && /noindex/i.test(m.content)));

Bookmarklet — flag whether the current page is set to noindex

javascript:(function(){var m=document.querySelector('meta[name="robots"],meta[name="googlebot"]');var c=m?m.content:'';alert('robots meta: '+(c||'(none)')+'\nnoindex: '+(/noindex/i.test(c)?'YES':'no'));})();

Extract every <a href> pointing at a URL you’re about to remove — DevTools Console

// Find internal links to the page you're deindexing so you can update/remove them
// (a page with no inbound links is far less likely to linger via link-based indexing).
const target = '/old-page/';
[...document.querySelectorAll('a[href]')]
  .filter(a => a.getAttribute('href').includes(target))
  .forEach(a => console.log(a.getAttribute('href'), '←', a.textContent.trim()));

Remember: for noindex and 404/410 to take effect, the URL must stay crawlable — never pair any of these with a robots.txt disallow on the same URL.

Common deindexing problems

Real symptoms readers hit while trying to remove a page, with the likely cause and the fix.

”Indexed, though blocked by robots.txt” in Search Console

• Symptom: the Page Indexing report shows the URL under this status — Google has it indexed but the content is grayed out / not previewable.
• Likely cause: the URL is disallowed in robots.txt, but Google found it through inbound links and indexed the URL anyway (it just can’t read the page to see any noindex tag).
• Fix: decide what you actually want. To keep the page removed, remove the robots.txt disallow, leave the page crawlable, and add noindex (or delete it). Confirm the fix with the robots-txt-tester tool (/tools/robots-txt-tester) and re-check the status in Search Console after Google recrawls.

Page still shows in search after you added noindex

• Symptom: you added a noindex meta tag or X-Robots-Tag days ago, but the URL still returns in a site: search or Search Console still lists it as indexed.
• Likely cause: almost always one of two things — Google hasn’t recrawled the page yet, or the page is also blocked in robots.txt so Google can’t see the tag at all.
• Fix: check robots.txt isn’t disallowing the URL, then use URL Inspection to “Test live URL” and confirm the tag is present in the live rendered response, then “Request indexing.” Per Google, a lingering indexed page after noindex is usually just “we haven’t crawled the page since you added the rule.”

A Removals-tool request expired and the page came back

• Symptom: a page you hid with the GSC Removals tool reappears in search a few months later.
• Likely cause: the Removals tool is temporary by design (about six months) — if no permanent method was applied during that window, the URL is eligible to be shown again once the request expires.
• Fix: re-check whether a permanent method (noindex, 404/410, or password protection) was ever applied. If not, apply one now and submit a fresh Removals request to bridge the gap while it takes effect.

Deleted page (404/410) still appears in results

• Symptom: the page returns a genuine 404 or 410 when you check it directly, but it’s still listed in search results.
• Likely cause: Google hasn’t recrawled and processed the status code yet — removal only happens “once they’re crawled and processed,” not the moment the status code changes.
• Fix: confirm the status code with the HTTP Status Checker (/tools/http-status-checker), then request a recrawl via URL Inspection. If it’s urgent, use the Removals tool in the meantime.

An image still shows in Google Images after a robots.txt block

• Symptom: you added a Googlebot-Image disallow but the image is still appearing in Google Images results.
• Likely cause: either the disallow rule doesn’t match the actual image path, a syntax error elsewhere in robots.txt is interfering, or the image hasn’t been recrawled/dropped yet.
• Fix: verify the exact disallow path matches the image URL with the robots-txt-tester tool (/tools/robots-txt-tester), and give it time — image results update on Google’s normal recrawl schedule, not instantly.

Rankings dropped after noindex-ing a “duplicate” page

• Symptom: you noindexed what you thought was a duplicate page to “clean up” the index, and the site’s overall visibility for that content dropped instead of consolidating.
• Likely cause: noindex doesn’t consolidate signals onto another URL — it just removes the page from Search entirely. If the two pages weren’t truly redundant, or if you needed the ranking signals to fold onto the other version, noindex was the wrong tool.
• Fix: for genuine duplicates, use rel="canonical" pointing at the version you want to keep instead — check the current canonical relationship with the Canonical Checker (/tools/canonical-checker).

Tools for deindexing work

Verify the crawling and indexing signals directly rather than guessing.

• Robots.txt Tester — check whether a URL is disallowed before you rely on noindex, 404/410, or a canonical actually being seen by Google. This is the first check for almost every method above, since all of them require the page to stay crawlable.
• HTTP Status Checker — confirm a “deleted” URL is genuinely returning 404 or 410 (and not a soft-404 serving a 200), and confirm a page you noindexed is still returning 200 with the tag present.
• Canonical Checker — verify which URL Google is actually treating as canonical when you’re consolidating duplicates instead of removing them.
• Google Search Console CSV Analyzer — review Search Console’s Page Indexing data, including statuses like “Indexed, though blocked by robots.txt,” without digging through the UI by hand.

Third-party tools worth using alongside these: Google Search Console itself for the Removals tool, URL Inspection, and the Page Indexing report; Bing Webmaster Tools for the Block URLs tool; and Screaming Frog for auditing robots.txt, status codes, and canonical tags across a whole site before a bulk removal push.

Validation tests — did the removal actually take effect?

Machine-verifiable checks for each permanent method, so you’re not just assuming Google saw the change.

Test: noindex tag is present and the page is crawlable

• Test to run: run the URL through the Robots.txt Tester (/tools/robots-txt-tester) to confirm it’s not disallowed, then run curl -sI https://example.com/page/ (or check the rendered <head>/response headers) for the noindex meta tag or X-Robots-Tag: noindex header.
• Expected result: the URL is allowed in robots.txt AND either the meta tag or header shows noindex.
• Failure interpretation: if robots.txt disallows the URL, the noindex will never be seen — fix the disallow first. If neither the tag nor header is present, the removal request has nothing to act on.
• Monitoring window: immediate for the tag/header check itself; 1–2 weeks for Google’s recrawl and the resulting drop from the index.
• Rollback trigger: if the page needs to come back, remove the noindex tag/header and request indexing again — there’s nothing else to undo.

Test: deleted page returns 404/410, not a soft-404

• Test to run: the HTTP Status Checker (/tools/http-status-checker) or curl -I https://example.com/old-page/.
• Expected result: a genuine 404 or 410 status code in the HTTP response, not a 200 that happens to display an error message (a soft 404).
• Failure interpretation: a 200 status with error-page content means Google still treats the URL as a live, indexable page — the status code is what has to change, not just the visible content.
• Monitoring window: immediate for the status code; 1–4 weeks for the URL to actually drop out of Google’s index once recrawled and processed.
• Rollback trigger: if the page needs to return, restore the content and serve a 200 again, then request indexing.

Test: robots.txt is NOT blocking a page you want removed via noindex or deletion

• Test to run: Robots.txt Tester (/tools/robots-txt-tester) against the exact URL, or curl -s https://example.com/robots.txt and check the URL against the rules by hand.
• Expected result: the URL is allowed (no matching Disallow).
• Failure interpretation: if the URL is disallowed, any noindex tag or status code change on that URL will never be seen by Google — this is the single most common reason a removal “doesn’t work.”
• Monitoring window: immediate.
• Rollback trigger: n/a — this is a precondition check, not a change to roll back.

Test: canonical consolidation is being respected

• Test to run: the Canonical Checker (/tools/canonical-checker) on both the duplicate and the preferred URL, and Search Console’s URL Inspection tool’s “Google-selected canonical” field.
• Expected result: Google’s selected canonical matches the URL you declared.
• Failure interpretation: if Google selects a different canonical than the one you specified, it’s using other signals (internal links, sitemaps) to override your declaration — check those signals rather than adding noindex to force the issue.
• Monitoring window: 2–4 weeks — canonical selection is not instant and can shift as Google re-evaluates signals.
• Rollback trigger: if consolidation causes a visibility drop for content that wasn’t truly duplicate, remove the canonical tag and let both URLs index independently.

Test: Removals-tool request is active and a permanent method is in place before it expires

• Test to run: check the request status in Search Console (Indexing → Removals), and separately confirm the permanent method (noindex, 404/410, or password) is live using the relevant test above.
• Expected result: the Removals entry shows “Removed” and the permanent method’s own test also passes.
• Failure interpretation: if the Removals entry is active but the permanent method test fails, the page will reappear once the ~6-month window expires.
• Monitoring window: check again well before the 6-month mark (set a reminder at ~4 months) so there’s time to fix a failed permanent method before expiry.
• Rollback trigger: if removal was a mistake, cancel the Removals request in Search Console and remove the permanent method (undo noindex, restore the content, or remove the password wall).

Deindexing choices in practice

Keep the page for users, remove it from search

Before: an internal account page returns 200, is publicly reachable, and can appear in search. After: keep it crawlable and return noindex; do not block it in robots.txt, because crawlers must fetch the page to see the directive.

Delete content with no replacement

Before: an expired page returns 200 even though its content is gone. After: return 404 or 410 and remove it from internal links and sitemaps. Do not redirect it to an irrelevant homepage merely to avoid a 404.

Consolidate a true duplicate

Before: two equivalent URLs are indexable and compete as separate candidates. After: redirect the duplicate when users do not need it, or keep it crawlable with a canonical pointing to the preferred URL. This is consolidation, not a noindex job.

Hide sensitive material immediately and permanently

Before: a private document is public and indexed. After: restrict access or remove it at the server, use the Search Console Removals tool for temporary search suppression, and keep the permanent access/removal control in place.

Test yourself: deindexing

Five questions on how to actually get a page out of Google. Pick an answer for each, then check.

Resources worth your time

My related writing

• How to Remove URLs From Google Search (5 Methods) — my own walkthrough of the five removal methods, including the Google-vs-Bing tool-duration comparison.
• “Indexed, though blocked by robots.txt” Can Be More Than A Robots.txt Block — why blocked pages still get indexed, and the four distinct causes of that Search Console status.
• Robots.txt and SEO: Everything You Need to Know — what robots.txt does (crawling) and doesn’t (indexing) do.
• The Beginner’s Guide to Technical SEO — where indexing controls fit in the bigger picture.

My speaking

• How Search Works (SlideShare) — my walkthrough of crawling, rendering, indexing, and serving, where deindexing controls live. (My standing disclaimer applies: “This is my understanding of systems… not going to be 100% complete or accurate.”)

Official

• Google — Remove a page hosted on your site from Google and Block Search Indexing with noindex.
• Google — A note on unsupported rules in robots.txt (Gary Illyes, 2019) — the noindex-in-robots.txt deprecation.

From around the industry

• When Unindexing Pages From Google Search Use Noindex, Not Robots.txt (Search Engine Roundtable, Schwartz) — John Mueller’s 2021 tweet on using noindex over robots.txt to unindex.
• Google’s John Mueller Clarifies How To Remove Pages From Search (Search Engine Journal, Southern) — Mueller’s two-step “allow crawling, then noindex” model and “it works very quickly” on the Removals tool.
• Google Might Change How NoIndex In Robots.txt Works (Search Engine Roundtable) — the origin of the 2019 deprecation, with Gary Illyes’s “nun in a mosh pit” line.
• John Mueller’s Site No Longer In Google Search (Search Engine Roundtable) — the March 2024 cautionary tale of Mueller’s own site vanishing over a robots.txt parsing edge case.
• What is a Noindex Tag? (Ahrefs glossary) — a plain-language reference on the noindex directive and why canonical (not noindex) is the tool for duplicate content.
• r/TechSEO — the community for crawl/index and deindexing debugging.

Deindexing

Deindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist.

Related: Noindex, Removals tool

Deindexing

Deindexing (also called “removing a page from Google” or “unindexing”) means getting a URL to stop appearing in search results. There is no single “delete from Google” button — the right method depends on three questions: do you own the page, do you want it gone temporarily or permanently, and should the content itself still exist for users?

The core toolbox:

• noindex — a meta tag or X-Robots-Tag HTTP header that tells search engines not to list a page. It’s the most effective way to remove a URL when crawling is allowed — the page must stay crawlable so Google can actually see the rule.
• 404 / 410 status codes — delete the content and the URL drops from the index once it’s recrawled (410 is a marginally more explicit “gone”).
• The Search Console Removals tool — fast (within a day) but only temporary, about six months; pair it with a permanent method.
• Password protection — blocks crawler access entirely; the most airtight option, and it works across every search engine, not just Google.
• Canonical consolidation (rel="canonical" or a redirect) — for folding duplicate URLs into one indexed version. This is consolidation, not true removal.

The persistent myth is that a Disallow rule in robots.txt removes a page from the index. It doesn’t. Robots.txt is a crawling directive, not an indexing one — Google can still index a URL it was blocked from crawling if that URL is discoverable through links, and it will never see a noindex rule on a page it can’t crawl. Robots.txt is, however, the sanctioned method for removing images specifically (via a Googlebot-Image disallow), because image indexing depends on crawling the image file directly.

Related: Noindex, Removals tool

Changelog