How to Get Indexed by Google

TL;DR — Getting indexed means Google has crawled your page, decided it’s worth keeping, and stored it so it can show up in search. You can’t rank without it. The things you actually control are helping Google find the page (internal links + a sitemap) and, for a few important URLs, clicking Request Indexing in Search Console. None of it forces indexing — if a page isn’t getting indexed, the reason is usually that Google doesn’t think it’s good enough yet.

What “getting indexed” means

When you publish a page, three things have to happen before it can appear in Google:

1. Crawl — Googlebot finds the URL and downloads it.
2. Index — Google processes the page and decides whether to store it in its index (the giant database it pulls search results from).
3. Serve (rank) — when someone searches, Google picks the best matches from the index and orders them.

“Getting indexed” is step two. And here’s the part that trips almost everyone up: a page can be crawled and still not indexed. Crawling and indexing are separate steps. Google will happily fetch your page and then choose not to keep it — you’ll see that in Search Console as “Crawled – currently not indexed.” Evidence for this claim Google separates crawling, indexing, and serving and states that not every page makes it through each stage. Scope: Google Search; crawling does not guarantee indexing. Confidence: high · Verified: 2026-07-14 Google: How Search works

What actually helps

There’s no button that forces Google to index a page. What you can do is make the page easy to find and worth keeping:

• Link to it. Every page you care about should be linked from at least one other page on your site. Pages that nothing links to (orphan pages) are a top reason things never get indexed.
• Put it in your XML sitemap and submit that in Google Search Console. A sitemap helps Google discover your URLs — it doesn’t guarantee they’ll be indexed, but it’s the right tool when you have more than a handful of pages.
• Request indexing for individual important URLs. In Search Console, use the URL Inspection tool and click “Request Indexing.” Use this for a few high-priority pages — not in bulk, and not over and over on the same URL. Evidence for this claim Google's URL Inspection request is for individual URLs, has a quota, does not crawl faster when repeated, and does not guarantee inclusion. Scope: Google Search Console request indexing workflow. Confidence: high · Verified: 2026-07-14 Google: Ask Google to recrawl URLs
• Make the page genuinely good. This is the real gate. Thin, duplicate, or low-effort pages are exactly the ones Google crawls and then declines to index.

The things people get wrong

• Clicking “Request Indexing” ten times doesn’t help. Google says re-requesting the same URL won’t get it crawled any faster, and its own docs literally say “no need to resubmit.”
• The “Google Indexing API” is not a universal fast-track. It officially works only for job-posting pages and certain live-video pages. For a normal blog post or product page, it’s not the tool.
• IndexNow does not tell Google anything. IndexNow is a Bing and Yandex protocol. Google doesn’t use it. Submitting via IndexNow will speed up Bing, not Google.
• There’s no secret paid service that forces Google to index a page. Anything legitimate is just doing what you can do yourself in Search Console — and none of it overrides Google’s quality bar.

Want the full version — exact quotas, when you actually need a sitemap, what the Indexing API is really for, and how to read “Crawled – currently not indexed”? Switch to the Advanced tab.

TL;DR — Indexing is crawl → judge → store, and the judgment is the hard part: Google indexes a URL only if it clears a quality/usefulness bar, so “why isn’t my page indexed” is usually a content problem, not a mechanics problem. The levers you control affect discovery and crawl priority, not the indexing decision: internal links (no orphans), a clean XML sitemap (discovery aid, not a guarantee), and Request Indexing in URL Inspection for a few individual high-priority URLs — which explicitly does not guarantee or speed a quality decision, and “no need to resubmit.” The two “fast-index” APIs are narrower than their reputation: Google’s Indexing API is officially only for JobPosting and BroadcastEvent-in-VideoObject pages (default 200 publish requests/day), and IndexNow is a Bing/Yandex protocol that Google does not use. Nothing guarantees indexing.

Crawling ≠ indexing — the reframe the whole page hangs on

Most “why isn’t my page indexed” frustration comes from collapsing two separate steps into one. Google’s pipeline is three stages — crawl, index, serve — and, in Google’s own words, “not all pages make it through each stage.” A URL can be crawled and then not indexed. Evidence for this claim Google separates crawling, indexing, and serving and states that not every page makes it through each stage. Scope: Google Search; crawling does not guarantee indexing. Confidence: high · Verified: 2026-07-14 Google: How Search works That’s not a bug; it’s Google exercising editorial judgment.

So indexing is not a mechanical queue you can force your way into. It’s a gate with a quality bar. Getting indexed is a prerequisite for ranking, not a ranking factor: an indexed page can still rank poorly or for nothing at all, but a non-indexed page can never appear. Everything below is organized around that: the parts you control (discovery, crawl priority) versus the part Google controls (the indexing decision).

Why a page isn’t indexed — a quick diagnostic

Before reaching for tools, rule out the mechanical blockers. A page won’t be indexed if:

• It carries a noindex meta tag or X-Robots-Tag header.
• It’s blocked in robots.txt (Google can’t read the content — and a robots-blocked URL can still show up indexed without content if links point to it; see canonicalization for that failure mode).
• Its rel="canonical" points at a different URL, so Google consolidates onto the canonical instead (GSC: “Alternate page with proper canonical tag” or “Duplicate, Google chose different canonical than user”).
• It’s an orphan — nothing internal links to it.
• It returns a 4xx/5xx or soft-404s.
• It depends on JavaScript for its main content or links, and Google’s renderer never successfully executes it — a blocked script/CSS file, a render error, or an app-shell page with no content in the initial HTML. Googlebot queues rendering as a separate step after the initial crawl, so a page can be fetched but still leave Google unable to see the real content or links inside it.
• The content is thin, duplicate, or low-value — the most common cause of “Crawled – currently not indexed.”

If none of the first six apply, you’re almost certainly in quality territory, and no amount of resubmitting changes that.

Request Indexing (URL Inspection) — for individual, high-priority URLs

The URL Inspection tool in Search Console lets you ask Google to (re)crawl one URL. Google: “To request a crawl of individual URLs, use the URL Inspection tool. You must be an owner or full user of the Search Console property to be able to request indexing in the URL Inspection tool.”

Three things to internalize:

• It’s rate-limited, and spamming it does nothing. Google: “There’s a quota for submitting individual URLs and requesting a recrawl multiple times for the same URL won’t get it crawled any faster.” Google does not publish the exact daily number, and has historically kept it opaque on purpose — it removed the on-screen “submissions remaining” counter from the old Fetch-as-Google tool back in 2018 while keeping the underlying limit. Practitioners informally cite a range around 10–15/day for lower-trust properties, but that’s observed, not official — treat any specific number as anecdotal.
• It doesn’t guarantee anything. Google: “Requesting a crawl does not guarantee that inclusion in search results will happen instantly or even at all. Our systems prioritize the fast inclusion of high quality, useful content.” And the timeline is vague by design — crawling “can take anywhere from a few days to a few weeks.” Evidence for this claim Google's URL Inspection request is for individual URLs, has a quota, does not crawl faster when repeated, and does not guarantee inclusion. Scope: Google Search Console request indexing workflow. Confidence: high · Verified: 2026-07-14 Google: Ask Google to recrawl URLs
• It’s for a handful of URLs, not bulk. Google’s own guidance pushes you to sitemaps for anything beyond a few: “If you want many pages indexed, try submitting a sitemap to Google.” And when a page is already stuck in “Crawled – currently not indexed,” Google’s status glossary is explicit: “no need to resubmit this URL for crawling.”

The through-line, which the reps have made repeatedly over the years, is to favor the non-manual channels — sitemaps and internal links — over compulsively clicking a button.

Submit and maintain a clean XML sitemap

A sitemap is a discovery aid, full stop. Google is blunt about the limit of what it buys you: “A sitemap helps search engines discover URLs on your site, but it doesn’t guarantee that all the items in your sitemap will be crawled and indexed.”

When you actually need one. Google’s rough guidance: you likely need a sitemap if your site is large, is new with few external links (Googlebot leans on already-known pages and backlinks to discover new URLs, so new sites are structurally slower to get found), or is rich in media/video/news. You likely don’t need one for a site of about 500 pages or fewer where every important page is reachable by following links from the homepage and there’s no significant media/news content.

Size limits. A single sitemap file is capped at 50MB uncompressed / 50,000 URLs; past that, split into multiple files and reference them from a sitemap index file.

Audit indexing rates from it. The most useful workflow: in the Page Indexing report you can filter by sitemap to see how many of a given sitemap’s URLs are actually indexed. A big gap between submitted and indexed is a quality/architecture signal, not a reason to resubmit. And keep sitemaps lean — stuffing them with non-canonical or low-value URLs wastes crawl attention and muddies your own reporting; list only canonical, indexable URLs.

Fix your internal linking

This is the most foundational and most overlooked lever. Google’s link best-practices doc states it directly: “Every page you care about should have a link from at least one other page on your site.” Orphan pages — those with no internal links pointing at them — are a top real-world cause of non-indexing on both brand-new and very large sites.

A few specifics that matter for indexing:

• Links must be crawlable. That means real <a href> elements, not onClick handlers or JS-only navigation. If a bot can’t extract the link, the page it points to may never be discovered.
• Keep important pages shallow. Crawlers don’t use search boxes or interpret nav menus the way people do — important pages should be reachable within a few clicks of the homepage.
• Link to the canonical URL, not a duplicate. Consistent internal linking to the preferred version reinforces Google’s understanding of which URL to index, and it’s one of the canonicalization signals.
• Anchor text helps. Google: “Paying more attention to the anchor text used for internal links can help both people and Google make sense of your site more easily and find other pages on your site.”

The Google Indexing API — what it’s actually for

This is the single most misunderstood tool in the indexing conversation, so be precise: the Indexing API is not a general fast-index endpoint. Google’s docs name the only two supported content types outright: “The Indexing API can only be used to crawl pages with either JobPosting or BroadcastEvent embedded in a VideoObject.”

• Scope: job-posting pages and pages with a livestream event (BroadcastEvent in VideoObject) only. A normal article, product, or category page is out of scope.
• Quota: the default is 200 publish requests/day per project (covering both URL_UPDATED and URL_DELETED), 180 read-only requests/minute, and 380 across all endpoints/minute. Google notes “the quota may increase or decrease based on the document quality,” and quota-increase requests are gated to the two supported use cases.
• Why using it for other content is risky. People do point it at unsupported pages and sometimes see it “work” temporarily — but Google’s Search Relations team has been clear this is unsupported and can be revoked without notice: the API may stop supporting unsupported content formats at any time, and access for non-supported verticals could be cut off overnight. Building your indexing strategy on an unsupported hack is building on sand.

If your site genuinely publishes job listings or livestreams, the Indexing API is excellent and fast for exactly those. For everything else, it’s the wrong tool.

IndexNow — real, useful, and not a Google tool

IndexNow gets conflated with “fast Google indexing” constantly, and it’s worth being blunt: IndexNow does not include Google. It’s an open protocol launched in October 2021 by Microsoft Bing and Yandex; you ping a participating endpoint when a URL changes and the engines share that notification with each other. Bing’s own IndexNow page lists adopters like Yandex, LinkedIn, Yahoo, eBay, Etsy, GitHub, Wix, Cloudflare, Yoast, and RankMath — Google is not among them.

So submitting via IndexNow will help Bing, Yandex, and other participating engines discover changed URLs faster. It has zero direct effect on Google indexing — Google was never in scope. “I submitted via IndexNow but Google still hasn’t indexed my page” is a category error, not a bug.

Even for the engines that do participate, IndexNow is a discovery notification, not an indexing guarantee. Its own documentation is careful about what a successful response means: “The HTTP 200 response code only indicates that the search engine has received your URL.” Received, not indexed.

Should you still implement it? Yes — it’s low-effort (host a key file, ping an endpoint) and it genuinely speeds Bing/Yandex discovery. Just don’t expect it to move Google.

The real gatekeeper: content quality

Strip away the tooling and the actual bottleneck is almost always quality. Google’s own language around the not-indexed statuses is editorial, not mechanical:

• “Crawled – currently not indexed: The page was crawled by Google but not indexed. It may or may not be indexed in the future; no need to resubmit this URL for crawling.”
• “Discovered – currently not indexed: The page was found by Google, but not crawled yet. Typically, Google wanted to crawl the URL but this was expected to overload the site; therefore Google rescheduled the crawl.”

The reps have consistently framed “Discovered/Crawled – currently not indexed” as a worthiness question. As John Mueller put it on the topic, moving pages from not-indexed to indexed basically means convincing Google it’s worthwhile to index more — since Google doesn’t yet have an understanding of an un-indexed URL, it pulls in the rest of the site to judge the URL’s potential context, and there’s nothing special or new about “discovered / not indexed.” His shorthand for what it takes, across several public statements, has been some variation of “awesomeness” — the page needs to be genuinely good.

In practice that means: thin content, near-duplicate content, low-effort or mass-produced pages, and pages that overlap heavily with better pages you already have are the usual suspects behind non-indexing. Consolidate or improve them rather than resubmitting them.

What about crawl budget?

For most sites, crawl budget is a red herring for indexing problems. Gary Illyes has said the vast majority of sites don’t need to worry about crawl budget at all — it becomes a real constraint only at large scale (roughly 1M+ pages changing weekly, or 10k+ daily). If you’re a small-to-midsize site and pages aren’t indexing, the culprit is far more likely quality or architecture than budget. (See crawl budget for the full treatment.)

How long does indexing take, and how to monitor

Honestly: a few days to a few weeks is normal, and new/low-authority sites are slower because Google relies on already-known pages and backlinks to find and value new URLs. A couple of quality backlinks and a clean sitemap matter disproportionately early on. Monitor with the Page Indexing report (aggregate, property-wide) and URL Inspection (single URL) rather than re-clicking Request Indexing — patience and improvement beat the button.

Where this fits

This is the actions companion to the Page Indexing report, which explains how to read the not-indexed statuses; this page is what to do about them. It sits inside the broader indexing stage of how search works, and leans on its siblings: canonicalization decides which URL from a duplicate cluster gets indexed at all, XML sitemaps and internal linking are the discovery channels, and crawl budget is the efficiency concern that occasionally (rarely) intersects. For the whole pipeline — discovery, crawling, rendering, indexing, serving — see the How Search Works cluster.

AI summary

A condensed take on the Advanced version:

• Indexing = crawl → judge → store. Getting indexed is a prerequisite for ranking, never a ranking factor. Crawling and indexing are separate steps — a page can be crawled and still not indexed.
• You control discovery and crawl priority, not the indexing decision. The levers: internal links (no orphans), a clean XML sitemap, and Request Indexing for a few individual high-priority URLs.
• Request Indexing (URL Inspection): rate-limited, doesn’t guarantee or speed a quality decision, and “no need to resubmit.” Google’s own exact daily number is unpublished; ~10–15/day is anecdotal. For bulk, use a sitemap.
• XML sitemap aids discovery only — “doesn’t guarantee that all the items in your sitemap will be crawled and indexed.” 50MB / 50,000-URL cap per file; audit indexed-vs-submitted in the Page Indexing report.
• Internal links: “Every page you care about should have a link from at least one other page.” Use real <a href>, keep important pages shallow, link to canonicals.
• Indexing API is officially only for JobPosting and BroadcastEvent-in- VideoObject pages; default 200 publish requests/day. Using it elsewhere is unsupported and can be cut off without notice — Google does not use it as a general fast-index tool.
• IndexNow is Bing/Yandex/others — Google does not participate. HTTP 200 there means “received,” not indexed. Implement it for Bing, not Google.
• Quality is the real gate. “Crawled – currently not indexed” is usually a quality signal; convince Google the page is worthwhile. Crawl budget is a non-issue for most sites (Illyes). Indexing takes days to weeks; new sites are slower.

Official documentation

Primary-source documentation from the search engines.

Google

• Ask Google to Recrawl Your Website — Request Indexing via URL Inspection, the quota, and why resubmitting doesn’t help.
• URL Inspection tool — Search Console Help: daily limit, non-guarantee, and “use a sitemap for many pages.”
• What Is a Sitemap — when you need one, and the “doesn’t guarantee indexing” line.
• Build and Submit a Sitemap — the 50MB / 50,000-URL cap and how to submit.
• Manage Your Sitemaps With Sitemap Index Files — splitting large sites across a sitemap index.
• SEO Link Best Practices for Google — crawlable links and “every page you care about should have a link.”
• How to Use the Indexing API — the scope: JobPosting and BroadcastEvent-in-VideoObject only.
• Requesting Approval and Quota (Indexing API) — the 200/day publish quota and the quality-based quota note.
• Page indexing report — Search Console Help: the not-indexed statuses, including “no need to resubmit.”

Bing / Microsoft / IndexNow

• Why IndexNow — Bing’s IndexNow overview and its published list of adopters (Google not among them).
• IndexNow Documentation — the protocol, and “the HTTP 200 response code only indicates that the search engine has received your URL.”

Quotes from the source

On-the-record statements from Google and Bing/IndexNow. Each link is a deep link that jumps to the quoted passage on the source page.

Google — Request Indexing (URL Inspection)

• “To request a crawl of individual URLs, use the URL Inspection tool. You must be an owner or full user of the Search Console property to be able to request indexing in the URL Inspection tool.” — Google Search Central docs. Jump to quote
• “There’s a quota for submitting individual URLs and requesting a recrawl multiple times for the same URL won’t get it crawled any faster.” Jump to quote
• “Requesting a crawl does not guarantee that inclusion in search results will happen instantly or even at all. Our systems prioritize the fast inclusion of high quality, useful content.” Jump to quote

Google — sitemaps and links

• “A sitemap helps search engines discover URLs on your site, but it doesn’t guarantee that all the items in your sitemap will be crawled and indexed.” Jump to quote
• “Every page you care about should have a link from at least one other page on your site.” Jump to quote

Google — the Indexing API scope (the accuracy spine)

• “The Indexing API can only be used to crawl pages with either JobPosting or BroadcastEvent embedded in a VideoObject.” Jump to quote

Google — “Crawled – currently not indexed” (don’t resubmit)

• “The page was crawled by Google but not indexed. It may or may not be indexed in the future; no need to resubmit this URL for crawling.” — Page indexing report, Search Console Help. Jump to quote

IndexNow — what a 200 means

• “The HTTP 200 response code only indicates that the search engine has received your URL.” — IndexNow documentation. Jump to quote

My page isn’t indexed — what do I do?

Work top-down. Rule out the mechanical blockers before you touch any “fast-index” tool, because none of those tools override a noindex, a robots block, a canonical, or a quality problem.

Get-it-indexed checklist

A pass to confirm a page can be found, crawled, and judged fairly:

• URL Inspection shows the page is not blocked by noindex or robots.txt.
• The page’s rel="canonical" points at itself (or you genuinely intend it to consolidate elsewhere).
• The page is linked internally from at least one other relevant page — not orphaned — with descriptive anchor text.
• Those internal links are real <a href> elements, not JS-only click handlers.
• If the page’s main content or links depend on JavaScript, they actually render — check the rendered HTML in URL Inspection, not just the raw server response.
• The page is reachable within a few clicks of the homepage.
• The URL is in a clean XML sitemap (canonical, indexable URLs only) submitted in Google Search Console.
• The server returns 200 fast and stably (no 4xx/5xx, no soft-404).
• The content is genuinely useful and unique — not thin, duplicate, or overlapping a better page you already have.
• For a single high-priority URL, you clicked Request Indexing once — and did not resubmit repeatedly.
• You are not relying on the Indexing API (unless it’s JobPosting / BroadcastEvent) or on IndexNow (which Google doesn’t use) to force Google.
• You’re monitoring via the Page Indexing report and URL Inspection, not by re-clicking the button.

SOP: getting a new (or stuck) page indexed

A repeatable procedure. Run it in order — earlier steps gate the later ones.

1. Inspect first. Open URL Inspection in Search Console and enter the exact URL. Note: is it indexed? What canonical did Google choose? Is it blocked?
2. Clear mechanical blockers. If there’s a noindex, remove it (if you want the page indexed). If it’s blocked in robots.txt, unblock it so Google can read the content. If the chosen canonical is a different URL, decide which URL should be the indexed one and align the tag, redirects, sitemap, and internal links to it.
3. Make it discoverable. Add at least one internal link from a relevant, crawled page (ideally a strong one), using descriptive anchor text and a real <a href>. Confirm the page is within a few clicks of the homepage.
4. Add it to the sitemap. Ensure the URL appears in a clean XML sitemap that lists only canonical, indexable URLs, and that the sitemap is submitted in Search Console.
5. Judge the content honestly. Is it unique and worthwhile, or thin/duplicate/ overlapping? If it’s weak, improve or consolidate it before expecting indexing — this is the actual gate.
6. Request indexing once (optional, for individual high-priority URLs). In URL Inspection, click “Request Indexing” a single time. Do not resubmit the same URL repeatedly — it won’t crawl faster.
7. Wait and monitor. Give it days to a few weeks. Check the Page Indexing report (filter by sitemap to see indexed-vs-submitted) and re-run URL Inspection. Do not re-click Request Indexing as a reflex.
8. If still stuck after weeks, treat it as a quality/architecture problem, not a tooling problem: strengthen internal links, earn a couple of relevant backlinks, and improve the content. Special cases (job postings, livestreams) can use the Indexing API; Bing/Yandex discovery can use IndexNow — neither affects Google indexing of normal pages.

Indexing anti-patterns

The mistakes that show up over and over — what’s wrong, and what to do instead.

Spam-clicking “Request Indexing.” Why it’s wrong: Google explicitly says re-requesting a recrawl of the same URL won’t get it crawled any faster, and its status glossary says “no need to resubmit.” Repeated clicks accomplish nothing and burn your daily quota. Do instead: Request once for individual high-priority URLs; for anything beyond a handful, use a sitemap and fix discovery/quality.

Treating the Google Indexing API as a universal fast-index hack. Why it’s wrong: It’s officially only for JobPosting and BroadcastEvent-in- VideoObject pages. Pointing it at normal pages is unsupported, can “work” briefly, and can be cut off without notice. Do instead: Use it only for those two content types. For everything else rely on sitemaps, internal links, and quality.

Expecting IndexNow to speed up Google. Why it’s wrong: IndexNow is a Bing/Yandex protocol; Google doesn’t participate. Submitting there does nothing for Google indexing. Do instead: Implement IndexNow for Bing/Yandex discovery, and use Google’s own tools (sitemaps, URL Inspection) for Google.

Assuming a sitemap guarantees indexing. Why it’s wrong: Google says a sitemap “doesn’t guarantee that all the items in your sitemap will be crawled and indexed” — it only aids discovery. Do instead: Keep the sitemap clean (canonical, indexable URLs only) and fix the underlying quality/architecture; audit indexed-vs-submitted in the Page Indexing report.

Buying “guaranteed indexing” services. Why it’s wrong: No legitimate mechanism forces Google to index a page outside its own tools, and none override the quality bar. These services do nothing you can’t do yourself. Do instead: Spend the effort on content quality, internal linking, and a couple of real backlinks.

Blaming crawl budget on a small site. Why it’s wrong: Per Gary Illyes, the vast majority of sites don’t have crawl-budget problems; for small-to-midsize sites the real culprit is quality or architecture. Do instead: Diagnose quality and orphaning first; only investigate crawl budget at genuinely large scale.

Bloating the sitemap to look bigger. Why it’s wrong: Stuffing low-value or non-canonical URLs in wastes crawl attention and muddies your reporting — more URLs isn’t more authority. Do instead: List only the canonical, indexable pages you actually want indexed.

Getting indexed — cheat sheet

Which tool for which job

What each channel actually does

Fast facts

• Indexing API scope: JobPosting + BroadcastEvent-in-VideoObject only; default 200 publish requests/day.
• Request Indexing: no official daily number published; ~10–15/day is anecdotal; “no need to resubmit.”
• Sitemap file cap: 50MB / 50,000 URLs — split with a sitemap index beyond that.
• IndexNow HTTP 200 = “received,” not indexed. Google does not participate.
• Typical indexing time: a few days to a few weeks; new/low-authority sites are slower.

Find orphan pages (pages nothing links to)

Orphans are a top cause of non-indexing. Compare the URLs in your sitemap against the URLs your own crawl actually reaches — anything in the sitemap that your crawler never found via internal links is effectively orphaned.

Shell — pull every URL out of an XML sitemap

# Extract <loc> URLs from a sitemap (works for a single sitemap file)
curl -s https://example.com/sitemap.xml \
  | grep -oE '<loc>[^<]+</loc>' \
  | sed -E 's~</?loc>~~g' \
  | sort -u > sitemap-urls.txt
wc -l sitemap-urls.txt

Export the “crawled/found via links” URL list from Screaming Frog or Ahrefs Site Audit as crawled-urls.txt, then:

# URLs in the sitemap that the crawl never reached = likely orphans
comm -23 sitemap-urls.txt <(sort -u crawled-urls.txt)

Check a page’s indexability signals from the terminal

Before blaming discovery, confirm the page isn’t quietly telling Google to stay away.

# noindex in the response header?
curl -sI https://example.com/my-page/ | grep -i x-robots-tag

# noindex / canonical in the HTML <head>?
curl -s https://example.com/my-page/ \
  | grep -iE '<meta[^>]+robots|rel=["'"'"']canonical'

Chrome DevTools Console — indexability at a glance

Paste into the Console on the live page to surface the signals that decide indexing:

// Meta robots, canonical, and whether this URL matches its canonical
(() => {
  const robots = document.querySelector('meta[name="robots"]')?.content || '(none)';
  const canonical = document.querySelector('link[rel="canonical"]')?.href || '(none)';
  console.log('meta robots:', robots);
  console.log('canonical:', canonical);
  console.log('this URL:', location.href);
  console.log('self-canonical?', canonical === location.href);
})();

Bookmarklet — jump straight to URL Inspection for the current page

Save as a bookmark; click it on any page to open that URL in your Search Console property’s inspection tool (replace the property prefix with yours).

javascript:(function(){var p='https://example.com/';var u=location.href;window.open('https://search.google.com/search-console/inspect?resource_id='+encodeURIComponent('sc-domain:example.com')+'&id='+encodeURIComponent(u),'_blank');})();

Tools for getting (and checking) indexing

• URL Inspection (Google Search Console) — the single-URL source of truth: is it indexed, what canonical did Google choose, and the Request Indexing button.
• Page Indexing report (GSC) — aggregate, property-wide view of indexed vs. not-indexed, grouped by reason; filter by sitemap to audit indexed-vs-submitted.
• Sitemaps report (GSC) — submit and monitor your XML sitemaps.
• Bing Webmaster Tools — Bing’s index coverage and URL submission, plus IndexNow.
• IndexNow — one-time-setup protocol to ping Bing/Yandex on URL changes (not Google).
• Screaming Frog SEO Spider / Ahrefs Site Audit — crawl your site to find orphan pages, broken internal links, and non-canonical/noindex URLs before Google does.
• Ahrefs Webmaster Tools — free crawl + audit for sites you verify.

Common indexing issues

Lookup table for what you’re actually seeing, in Search Console or in the wild — matched to the likely cause and the fix to try.

GSC shows “Discovered – currently not indexed”

Symptom: URL Inspection or the Page Indexing report shows the page as “Discovered – currently not indexed” — Google knows the URL exists but hasn’t crawled it.

Likely cause(s): Nothing (or too little) links to it internally, so it isn’t prioritized; or Google expected the crawl to overload the site and rescheduled it.

Fix: Add internal links from relevant, already-indexed pages with descriptive anchor text, and confirm the URL is in a clean XML sitemap. Confirm by re-running URL Inspection after a few days — resubmitting via Request Indexing doesn’t speed this up.

GSC shows “Crawled – currently not indexed”

Symptom: Google fetched the page (you can see it was crawled) but it’s still not indexed, sometimes for weeks.

Likely cause(s): Content is thin, duplicate, or overlaps heavily with a better page you already have — this is the most common cause of this exact status.

Fix: Improve or consolidate the page rather than resubmitting; Google’s own guidance is explicit that there’s “no need to resubmit this URL for crawling.” Confirm by checking whether the page’s unique value versus your other pages has actually changed, then re-check status after a few weeks.

”Request Indexing” seems to do nothing no matter how many times you click it

Symptom: Clicked Request Indexing multiple times on the same URL over days; status hasn’t changed.

Likely cause(s): Re-requesting the same URL doesn’t get it crawled any faster — you’re burning quota, not fixing anything. The real blocker is elsewhere (mechanical or quality).

Fix: Stop resubmitting. Run URL Inspection once to see what Google actually reports (blocked, canonicalized elsewhere, or a not-indexed status), then fix that root cause instead.

Page looks fine in “View Source” but content/links are missing from the index

Symptom: The raw HTML response looks thin or empty, but the page looks normal in a browser; internally-linked pages that only appear in JS-rendered navigation aren’t getting discovered or indexed.

Likely cause(s): The page relies on client-side JavaScript to inject its main content or links (an app-shell pattern), and Google’s renderer either hasn’t gotten to it yet or failed — commonly because a script, stylesheet, or API call the render depends on is blocked or erroring. Rendering happens as a separate, queued step after the initial crawl, so a 200 response doesn’t mean Google saw the rendered content.

Fix: Check the rendered HTML in URL Inspection (not just “View Source”), confirm no JS/CSS resources the page needs are blocked by robots.txt, and fix any console errors that stop the render from completing. Prefer server-rendered or statically-generated markup for content and links you need indexed quickly.

A robots.txt-blocked URL shows up in search results anyway

Symptom: A URL you blocked in robots.txt still appears in Google’s index — usually as a bare URL with no title/snippet.

Likely cause(s): Blocking crawling in robots.txt doesn’t remove a URL from the index if other pages still link to it — Google can index the URL itself without reading its content.

Fix: If you want it fully out, use noindex (which requires Google to be able to crawl the page to see the tag) or remove the incoming links, not just a robots.txt block. Confirm with the robots-txt-tester and a fresh URL Inspection.

Page is indexed under a different URL than expected

Symptom: URL Inspection shows “Alternate page with proper canonical tag” or “Duplicate, Google chose different canonical than user” for a URL you wanted indexed as-is.

Likely cause(s): Your rel="canonical", internal links, or both point at a different URL, so Google consolidated onto that one instead.

Fix: Decide which URL should be canonical and align the tag, internal links, and any redirects to point at it consistently. Confirm with the canonical-checker and re-run URL Inspection.

Sitemap submitted, but the Page Indexing report shows a big gap between submitted and indexed

Symptom: Hundreds or thousands of URLs in your sitemap, but the indexed count is much lower.

Likely cause(s): The sitemap itself doesn’t force indexing — it’s a discovery aid. A large gap usually points to quality or architecture problems across many URLs, not a submission problem.

Fix: Filter the Page Indexing report by that sitemap to see the specific not-indexed reasons, then treat it as a quality/architecture audit — not a reason to resubmit the sitemap.

Validation tests

Proof that an indexing fix actually took effect — run these after making a change, not instead of one.

Confirm a noindex was actually removed

Test to run: curl -sI https://example.com/page/ | grep -i x-robots-tag and check the page’s <head> for <meta name="robots"> (or use the render-gap tool if the tag is added client-side).

Expected result: No noindex in either the HTTP header or the rendered HTML.

Failure interpretation: If noindex still appears in the rendered HTML but not the raw response (or vice versa), a template, CMS setting, or CDN rule is still injecting it — check both source and rendered output.

Monitoring window: Immediate — this is a status check, not something that needs to propagate.

Rollback trigger: If removing noindex was intentional and the page still doesn’t index after several weeks, that’s not a rollback signal — move to the quality diagnosis instead.

Confirm robots.txt no longer blocks the URL

Test to run: robots-txt-tester against the exact URL path.

Expected result: The tool reports the URL as allowed for Googlebot.

Failure interpretation: Still blocked means a Disallow rule (yours or an inherited one) still matches the path — check for overly broad wildcard rules.

Monitoring window: Immediate for the robots.txt check itself; allow a few days for Googlebot to re-fetch robots.txt (it’s cached) before expecting a crawl.

Rollback trigger: N/A — this is a one-way fix; re-block only if you deliberately want the URL excluded again.

Confirm the canonical is self-referencing (or points where you intend)

Test to run: canonical-checker on the URL, and cross-check “Google-selected canonical” in URL Inspection.

Expected result: The declared canonical matches the URL you want indexed, and Google’s selected canonical agrees with it.

Failure interpretation: If Google’s selected canonical still differs from your declared one, internal links or duplicate content are overriding your signal — Google treats the canonical tag as a hint, not a directive.

Monitoring window: 2–4 weeks — canonical selection can take time to update after you fix the signals.

Rollback trigger: If, after 4+ weeks, Google’s selected canonical still disagrees, revisit whether the URLs are different enough that they shouldn’t be consolidated at all.

Confirm a requested URL actually gets indexed

Test to run: URL Inspection → Request Indexing (once) on the specific URL, then re-run URL Inspection later.

Expected result: Status moves from “not indexed” to “URL is on Google,” and a site: search (or the Page Indexing report) shows the page.

Failure interpretation: Still not indexed after a reasonable wait, with no mechanical blockers found, means it’s a quality/worthiness decision — resubmitting again won’t change the outcome.

Monitoring window: Days to a few weeks per Google’s own guidance; check again before assuming it failed.

Rollback trigger: Not applicable — there’s nothing to undo; if it stays not-indexed, address content quality instead.

Confirm a sitemap fix improved the indexed-vs-submitted rate

Test to run: Page Indexing report in GSC, filtered by the specific sitemap.

Expected result: The indexed count for that sitemap trends up relative to submitted URLs over successive checks.

Failure interpretation: A flat or worsening ratio after cleanup means the remaining not-indexed URLs share a quality or architecture problem, not a discovery problem.

Monitoring window: 2–4 weeks for a meaningful trend — Page Indexing data itself lags by a few days.

Rollback trigger: N/A — if the sitemap change was cleanup (removing non-canonical URLs), there’s nothing to roll back even if the ratio doesn’t move quickly.

How to measure indexing health

The standing KPIs for whether your discovery and quality work is actually landing — not one-time checks, but numbers to watch over time.

Indexed vs. submitted ratio

What it tells you: How much of your sitemap is actually making it into the index — the single best proxy for “is my discovery/quality pipeline working.”

How to pull it: Page Indexing report in GSC, filtered by sitemap (Indexing → Pages → filter by sitemap).

Benchmark / realistic range: Depends heavily on site type and age — a small, well-linked site can sit near 90–100%; large or newer sites commonly run lower. There’s no universal “good” number; establish your own baseline and watch the trend rather than chasing a fixed target.

Cadence: Monthly, or after any sitemap/content cleanup.

Not-indexed reason breakdown

What it tells you: Why pages aren’t indexed — separates mechanical problems (noindex, blocked, canonicalized elsewhere) from quality problems (“Crawled – currently not indexed”), which need completely different fixes.

How to pull it: Page Indexing report in GSC — the “Why pages aren’t indexed” table breaks URLs down by reason.

Benchmark / realistic range: No universal benchmark; the useful signal is the mix shifting toward mechanical (fixable in minutes) versus quality (needs real content work) over time.

Cadence: Monthly.

Time-to-index for new pages

What it tells you: How quickly new content clears discovery + the quality bar — a proxy for overall site authority and internal-linking health.

How to pull it: Note publish date, then check URL Inspection periodically (or watch for the URL appearing in the Page Indexing report / a site: search) to log days-to-index for a sample of new pages.

Benchmark / realistic range: Days to a few weeks is normal per Google’s own guidance; new or low-authority sites run slower because Google leans on existing backlinks and known pages to prioritize crawling. Depends on site authority — track your own trend rather than comparing across sites.

Orphan page count

What it tells you: How many pages have no internal links pointing at them — a leading real-world cause of non-indexing that’s entirely within your control.

How to pull it: Crawl the site with the site-audit-lite tool or Screaming Frog/Ahrefs Site Audit and compare against your sitemap URL list (see the Scripts tab for a comm-based diff).

Benchmark / realistic range: The honest target is zero for any page you want indexed — this isn’t situational the way the other metrics are.

Cadence: After every content push, or monthly for larger sites.

Prompts for indexing diagnosis

Use these with exported Search Console data, crawl data, or page details. Do not ask the model to guess whether a URL is indexed; verify that in Search Console.

Triage a set of excluded URLs

Classify these URLs by Search Console exclusion reason. For each class, separate discovery, crawl-access, canonicalization, and content-quality causes. Recommend the smallest test that would confirm each suspected cause. Do not treat Request Indexing as a sitewide fix. Data: [paste URL, status, canonical, sitemap, internal-link count].

Review one page before requesting indexing

Review this page’s indexability evidence: HTTP status, robots access, meta/X-Robots directives, rendered canonical, sitemap presence, and internal links. Return a pass/fail table, then identify anything that would make an indexing request premature. Evidence: [paste checks].

Turn a recurring issue into a template fix

These URLs share an indexing exclusion. Group them by page template and identify which fixes belong in the CMS/template rather than page-by-page edits. Rank the fixes by affected URLs and confidence, and specify how to validate each deployment. Sample: [paste rows].

Resources worth your time

My related writing

• How to Get Google to Index Your Website — Ahrefs’ practical guide to request indexing, sitemaps, internal links, and content quality (a good companion to this page).
• What “Crawled – Currently Not Indexed” Means In Google Search Console — why that status is fundamentally a quality signal.
• How to Fix “Discovered – currently not indexed” — the Ahrefs guide I reviewed: orphan pages, crawl prioritization, backlinks as a value signal.
• The Beginner’s Guide to Technical SEO — where indexing fits in the bigger picture.
• My author archive on the Ahrefs blog — more of my technical-SEO writing on crawling and indexing.

My speaking

• How Search Works (SlideShare) — my walkthrough of crawling, rendering, indexing, and ranking, and why crawled ≠ indexed. (My standing disclaimer applies: “This is my understanding of systems… not going to be 100% complete or accurate.”)

Official

• Ask Google to Recrawl Your Website and the URL Inspection tool — Request Indexing, its quota, and the non-guarantee.
• How to Use the Indexing API — the JobPosting/BroadcastEvent-only scope.
• Why IndexNow (Bing) and IndexNow documentation.

From around the industry

• Google: Sites Need To Be Worthwhile To Be Indexed (Search Engine Journal) — John Mueller on “Discovered – currently not indexed” as a worthiness question.
• Google Search Console request indexing limit removed from interface (Search Engine Land, 2018) — the history of Google keeping the manual quota opaque.
• Should I Worry About Crawl Budget? (Search Off the Record, Aug 2022) — Gary Illyes on why most sites don’t need to worry about crawl budget.
• What is “Crawled – currently not indexed” in Search Console? (Yoast) — a plain-language take on the same status.
• How To Fix “Crawled – Currently Not Indexed” in GSC (Onely) — a thorough troubleshooting walkthrough.
• r/TechSEO — the community for crawl/index debugging.

Test yourself: getting indexed

Five questions on how pages actually get indexed by Google. Pick an answer for each, then check.

Changelog