Sitemaps

TL;DR — A sitemap is a file that lists the pages on your site so search engines can find them. It helps Google and Bing discover your content — but it doesn’t force them to index it. Most sites should have one (it’s cheap insurance), it should be generated automatically so it stays current, and you submit it in Google Search Console and Bing Webmaster Tools.

What a sitemap is

A sitemap is a file where you list the URLs on your site — and sometimes your images and videos — so search engines have a tidy inventory to work from instead of having to discover everything by following links. Google’s own definition: “A sitemap is a file where you provide information about the pages, videos, and other files on your site, and the relationships between them.” Evidence for this claim A sitemap supplies search engines with information about pages, videos, and other files on a site and their relationships. Scope: Google's general definition of sitemaps; supported formats and extensions have additional requirements. Confidence: high · Verified: 2026-07-14 Google Search Central: Learn about sitemaps

The most common kind is an XML sitemap, which is written for crawlers rather than people. There’s also an HTML sitemap, which is a page of links meant for human visitors. They solve different problems.

Do you actually need one?

If your site is small (think a few hundred pages) and everything is well linked from your navigation, search engines can usually find it all on their own — Google says you may not strictly need a sitemap in that case. But it’s so cheap to have one that I’d generate one anyway. John Mueller put it well: a sitemap file is “a minimal baseline for any serious website.” So my rule of thumb is simple: always have one, and let your CMS or framework generate it automatically.

You especially want one if your site is large, brand new with few links pointing at it, or heavy on images, video, or news content.

The thing to understand up front

A sitemap helps discovery — it does not guarantee indexing. Putting a URL in your sitemap doesn’t force Google to crawl it, and it definitely doesn’t force Google to index or rank it. Google says it plainly: “A sitemap helps search engines discover URLs on your site, but it doesn’t guarantee that all the items in your sitemap will be crawled and indexed.” Think of it as a helpful list, not a magic “index me” button. Evidence for this claim A sitemap helps search engines discover URLs but does not guarantee that Google will crawl or index them. Scope: Google Search sitemap discovery; inclusion is not an indexing or ranking directive. Confidence: high · Verified: 2026-07-14 Google Search Central: Learn about sitemaps

How to find a site’s sitemap

Most sitemaps live at a predictable spot. Try:

• yoursite.com/sitemap.xml (the most common default).
• Your robots.txt file (yoursite.com/robots.txt) — sites often list the sitemap there with a Sitemap: line.
• Google Search Console (your own site) under the Sitemaps report.

One thing worth knowing: not every site advertises its sitemap. Some deliberately keep it out of robots.txt and submit it only inside Search Console and Bing Webmaster Tools, so competitors can’t grab a clean list of every URL they publish. So “I can’t find it at the usual spot” doesn’t mean a site has no sitemap.

How to make and submit one

Most platforms make one for you automatically — WordPress, Shopify, Wix, Squarespace, and Webflow all generate a sitemap out of the box (see the Cheat Sheets tab for the path on each one). Once you have it:

1. Make sure it lists only the URLs you actually want indexed.
2. Submit it in Google Search Console and Bing Webmaster Tools.
3. Add a Sitemap: line to your robots.txt (unless you’re deliberately keeping it private).

Want the full version — the size limits, the tags Google ignores, how to find a site that’s hiding its sitemap, and the different types? Switch to the Advanced tab.

TL;DR — A sitemap is a file listing your URLs (and optionally images/videos) so engines can discover them — a coverage and diagnostic tool, not a discovery silver bullet, because submitting never guarantees crawling or indexing. A single sitemap caps at 50,000 URLs or 50MB uncompressed (gzip is allowed; the cap is the uncompressed size); UTF-8; fully-qualified absolute URLs; entity-escape &, ', ", <, >. Google ignores <priority> and <changefreq> and only uses <lastmod> if it’s consistently accurate. Submit via Search Console, Bing Webmaster Tools, and the robots.txt Sitemap: line — the old ping endpoint has 404’d since Jan 2024, and IndexNow notifies Bing and others (not Google). Build it automatically, list only canonical indexable URLs, and use the submitted-vs-indexed signal in Search Console to find what’s missing.

What a sitemap is — and what it’s for

Google’s definition is the right anchor: “A sitemap is a file where you provide information about the pages, videos, and other files on your site, and the relationships between them.” Evidence for this claim A sitemap supplies search engines with information about pages, videos, and other files on a site and their relationships. Scope: Google's general definition of sitemaps; supported formats and extensions have additional requirements. Confidence: high · Verified: 2026-07-14 Google Search Central: Learn about sitemaps It’s an inventory you hand to search engines.

The most important framing I can give you: a sitemap is a coverage and diagnostic tool, not a discovery silver bullet. Google is explicit that “A sitemap helps search engines discover URLs on your site, but it doesn’t guarantee that all the items in your sitemap will be crawled and indexed.” Submitting a URL doesn’t index it. The real day-to-day payoff is the submitted-vs-indexed comparison in Search Console: split your sitemaps by section or content type and you can see which part of the site isn’t getting picked up. During a migration I’ll keep a sitemap of the old URLs around for a while on purpose, specifically so I can watch them drop out of the index in GSC.

Do you need one? Resolving the contradiction

You’ll see two pieces of advice that look like they conflict:

• Google’s threshold: you likely don’t need a sitemap if the site is small (around 500 pages) and well internally linked; you likely do if it’s large, new with few backlinks, or media/News-heavy.
• Mueller’s baseline: “Making a sitemap file automatically seems like a minimal baseline for any serious website, imo.”

Both are right, and here’s how I reconcile them: a sitemap isn’t strictly required for a tiny, well-linked site, but it’s cheap insurance and it’s now the expected baseline — so always generate one, automatically. Which brings me to the one rule I won’t bend on.

Automate it, or it rots

Sitemaps should be generated automatically from the pages you actually have. If someone hands me a manually-built sitemap, I already know it’ll fall out of date fast — somebody adds pages, nobody updates the file, and now it’s lying to Google. And there’s a subtler point: if you’re generating the sitemap from your real, crawlable pages anyway, search engines can usually reach those pages on their own through links. The sitemap earns its keep by being current and complete, not just by existing. Automation is what makes it current.

The types of sitemap

• XML sitemap — the workhorse, written for crawlers. A list of <loc> URLs, optionally with <lastmod>. This is what people usually mean by “sitemap.”
• HTML sitemap — a human-facing page of links. Useful for users and for internal linking, but it isn’t the file you submit to Search Console.
• Image sitemap — surfaces images search engines might otherwise miss (for example, images loaded by JavaScript or served dynamically). Can be a standalone file or image tags added to an existing sitemap.
• Video sitemap — gives search engines the metadata they need to understand and index your videos (thumbnail, title, description, and the media or player URL).
• News sitemap — for sites in Google News, listing recent articles.

And XML isn’t the only accepted format. Google also takes RSS/Atom feeds and a plain-text .txt file (one URL per line). RSS and text formats can only list page URLs — no image or video metadata — so XML is the most versatile.

XML vs HTML

These get confused constantly, so keep them separate: an XML sitemap is for crawlers — machine-readable, submitted to Search Console, the thing search engines parse. An HTML sitemap is for people — a regular web page linking to your sections, which can also help internal linking. If you only build one, build the XML sitemap; that’s the one search engines actually use.

How to find a sitemap (and why some sites hide theirs)

To find a sitemap, try, in order:

• /sitemap.xml — the default location on most platforms.
• robots.txt — a public sitemap is usually named here with a Sitemap: line, which is also how engines auto-discover it.
• site: operators / Search Console — for your own property, the Search Console Sitemaps report is the authoritative view of what’s submitted and how it’s doing.

But here’s the part most guides skip: not every sitemap is meant to be found. Some sites deliberately omit the Sitemap: line from robots.txt and submit the file only through Search Console and Bing Webmaster Tools. A sitemap referenced in robots.txt is readable by anyone — including competitors who’d love a tidy list of every URL you publish (unlinked pages, new launches, strategically important pages) and a way to watch how fast you ship. A sitemap submitted only through the consoles is effectively private. So if you can’t find a site’s sitemap at the usual paths, it may just be unlisted, not absent.

The trade-off is real, though: hiding the sitemap from robots.txt also means any engine that relies on robots.txt auto-discovery won’t see it unless you’ve submitted it in each console. If you go the private route, you have to do the submission legwork everywhere you care about.

How to create one

Almost every platform can generate a sitemap for you — and on most of them it’s automatic. I’ve put the per-platform paths (WordPress, Shopify, Wix, Squarespace, Webflow, Drupal, JS frameworks, and manual generators) in the Cheat Sheets tab so you can jump straight to yours. The short version for JS frameworks: search the framework name plus “sitemap” (for example, “Gatsby sitemap” or “Next.js sitemap”) — there’s almost always an existing module so you don’t hand-roll it. This site runs on Astro and uses @astrojs/sitemap, configured in astro.config.mjs.

How to submit it

The live methods:

• Google Search Console — the Sitemaps report (and the Search Console API). The primary method.
• Bing Webmaster Tools — submit there too; Bing fetches it immediately, then rechecks roughly daily.
• robots.txt Sitemap: line — works for any engine that reads robots.txt, and is how a public sitemap gets auto-discovered. Use the full absolute URL.

What’s dead: the old standalone ping endpoint (google.com/ping?sitemap=). It’s been deprecated since 2023 and returns a 404 since January 2024 — Google killed it because the vast majority of submissions were spam. If you’ve got a plugin or cron job still hitting it, strip that out. For fast, URL-level notification of single changes to Bing and others, use IndexNow — but note Google does not participate in IndexNow, so it won’t speed up Google indexing. Bing’s own framing is that the two are complementary: sitemaps for comprehensive coverage, IndexNow for fast per-URL pushes.

Submission problems

When something goes wrong, the Search Console Sitemaps report will tell you — but the error labels aren’t always self-explanatory. I’ve put a full error → meaning → fix table in the Cheat Sheets tab. The greatest hits: “Couldn’t fetch” (wrong URL, robots block, or just not processed yet — often transient), “Unsupported format” (you submitted an HTML page instead of a real XML/RSS/Atom/txt file), and “URL not allowed” (the classic HTTP-vs-HTTPS / www mismatch, or URLs above the sitemap’s own path). A couple of non-error gotchas worth knowing: Google may serve you a stale cached copy, so changes aren’t instant, and submitting both the children and the sitemap index is harmless but unnecessary.

Best practices — and “exclude ≠ noindex”

The rules that matter, all from Google’s own spec:

• Size: a single sitemap holds at most 50,000 URLs or 50MB uncompressed, whichever comes first. You can gzip it — the 50MB cap is the uncompressed size. Evidence for this claim Google limits a single sitemap to 50,000 URLs or 50 MB uncompressed. Scope: Google-supported sitemap files; larger inventories must be split across multiple sitemaps, optionally joined by an index. Confidence: high · Verified: 2026-07-14 Google Search Central: Build and submit a sitemap Over the limit, split into multiple sitemaps and tie them together with a sitemap index.
• Encoding: UTF-8.
• URLs: fully-qualified, absolute URLs. Entity-escape &, ', ", <, and >.
• Tags Google ignores: <priority> and <changefreq> do nothing — don’t bother with them.
• lastmod done honestly: Google only trusts lastmod if it’s consistently and verifiably accurate. Set it on significant updates (main content, structured data, or links) — not a blanket copyright-year or “today” stamp on every URL. Lie about it and Google stops believing the field. Done right, an accurate lastmod genuinely helps re-crawling (Bing leans on it even harder than Google does).
• Contents: list only canonical, indexable, 200-status URLs. No redirects, no non-canonical URLs, no noindex’d pages. Any page you want indexed should be in the sitemap; nothing you don’t.

Worked audit: valid XML, polluted inventory

A sitemap can pass XML validation and still send contradictory discovery signals. This illustrative crawl joins each <loc> to its live response and index controls:

The sitemap file itself is well formed. The pollution appears only after comparing its inventory with status, canonical, robots, and lifecycle evidence. Fix the generator or source query rather than deleting the same rows by hand every release.

That last point hides the single most common conceptual error: exclude ≠ noindex. Leaving a URL out of your sitemap does not deindex it. The sitemap is an advertisement, not a gate — dropping a URL just stops you advertising it; it doesn’t remove it from Google. If you want a page gone, allow crawling and use noindex. The sitemap is the wrong tool for that job.

Where to go next

This page is the overview. Each of these is its own deep dive nested under the discovery topic:

• XML sitemap — the format and anatomy: <urlset>, <loc>, <lastmod>, the ignored tags, what to include and exclude, and hreflang in sitemaps.
• Sitemap index — the “sitemap of sitemaps” for large sites, when to split, and the math on how many URLs you can cover.
• Image sitemap — surfacing images search engines might miss, the current (not deprecated) tag list, and cross-domain rules.
• Video sitemap — the required tags, accepted file types, and getting videos understood and indexed.

Sitemaps are one half of discovery — the other half is crawling, which is how engines actually fetch the URLs your sitemap points to. For the whole picture, see How Search Works. Every topic above is in the sidebar too.

AI summary

A condensed take on the Advanced version:

• A sitemap is a file listing your URLs (and optionally images/videos) so search engines can discover them. Google: “a file where you provide information about the pages, videos, and other files on your site.”
• Discovery, not indexing. Submitting a sitemap doesn’t guarantee crawling or indexing — “it doesn’t guarantee that all the items in your sitemap will be crawled and indexed.” Its real value is the submitted-vs-indexed diagnostic in Search Console.
• Do you need one? Not strictly for tiny well-linked sites (Google’s ~500-page threshold), but Mueller calls it “a minimal baseline for any serious website.” Always generate one — automatically, or it rots.
• Types: XML (for crawlers), HTML (for people), image, video, news; plus RSS/Atom and plain .txt formats (page URLs only).
• Finding one: /sitemap.xml, robots.txt, site: / Search Console. Some sites deliberately hide theirs (omit it from robots.txt, submit only in the consoles) to keep their URL list private — at the cost of losing auto-discovery.
• Limits: 50,000 URLs or 50MB uncompressed per sitemap (gzip allowed; cap is uncompressed); UTF-8; fully-qualified absolute URLs; entity-escape & ' " < >.
• Tags: Google ignores <priority> and <changefreq>; uses <lastmod> only if consistently accurate (significant updates, not copyright-year).
• Submit via Search Console, Bing Webmaster Tools, and the robots.txt Sitemap: line. The ping endpoint 404s since Jan 2024. IndexNow notifies Bing/Yandex/others — not Google.
• Exclude ≠ noindex. Leaving a URL out of the sitemap doesn’t deindex it; use noindex for that.
• Deep dives: XML sitemap, sitemap index, image sitemap, video sitemap.

Official documentation

Primary-source documentation from the search engines.

Google

• Sitemaps overview — what a sitemap is, do you need one, and the supported formats.
• Build and submit a sitemap — the 50,000-URL / 50MB limits, UTF-8, absolute URLs, lastmod, and how to submit.
• Manage your large sitemaps with sitemap index files — splitting big sitemaps and the index file.
• Image sitemaps — the current image tags and namespace.
• Video sitemaps — required/optional video tags.
• Sitemaps report help — every Search Console sitemap error and what it means.
• sitemaps.org protocol — the cross-engine XML sitemap standard.

Bing / Microsoft

• Keeping Content Discoverable with Sitemaps in AI-Powered Search (Jul 2025) — Bing’s current stance: XML preferred, lastmod matters, sitemaps + IndexNow are complementary.
• The Importance of Setting the lastmod Tag in Your Sitemap (Feb 2023) — Fabrice Canel on why lastmod is crucial.
• IndexNow / indexnow.org documentation — the push protocol for instantly signaling changed URLs (Bing, Yandex, and others — not Google).

Quotes from the source

On-the-record statements from Google and Bing. Each link is a deep link that jumps to the quoted passage on the source page.

Google — what a sitemap is and what it does

• “A sitemap is a file where you provide information about the pages, videos, and other files on your site, and the relationships between them.” — Google Search Central, sitemaps overview. Jump to quote
• “A sitemap helps search engines discover URLs on your site, but it doesn’t guarantee that all the items in your sitemap will be crawled and indexed.” Jump to quote

Google — discovery, not a guarantee

• “submitting a Sitemap doesn’t guarantee the pages referenced in it will be indexed. Think of a Sitemap as a way to help Googlebot find your content … if the content is shallow, it may totally happen it will not be indexed at all.” — Gary Illyes, Google. Jump to quote
• “One part of sitemaps is that Google has to be keen on indexing more content from the site. If Google’s not convinced that there’s new & important content to index, it won’t use the sitemap.” — John Mueller, Google. Jump to quote

Google — do you need one?

• “Making a sitemap file automatically seems like a minimal baseline for any serious website, imo.” — John Mueller, Google. Jump to quote

Google — the tags it ignores

• “we ignore those. It’s essentially a bag of noise.” — Gary Illyes, Google, on <priority> and <changefreq>. Jump to quote
• “Priority and change frequency doesn’t really play that much of a role with Sitemaps anymore … it is much better to just specify the time stamp directly.” — John Mueller, Google. Jump to quote

Google — lastmod done honestly

• “setting today’s date in a sitemap file isn’t going to be something that works in favor of anyone, it’s just lazy.” — John Mueller, Google. Jump to quote
• “an update to the main content, the structured data, or links on the page is generally considered significant, however an update to the copyright date is not.” — Google Search Central, build a sitemap. Jump to quote

Bing — lastmod and IndexNow

• “XML remains the preferred format for sitemaps, as it supports structured metadata like lastmod, which helps Bing assess content freshness and relevance more effectively.” — Microsoft Bing blog. Jump to quote
• “By combining sitemaps for comprehensive site coverage with IndexNow for fast, URL-level submission, you provide the strongest foundation for keeping your content fresh, discoverable, and visible.” — Microsoft Bing blog. Jump to quote

The standing KPIs for sitemap health

A sitemap doesn’t have a ranking metric of its own — its job is coverage, so the KPIs that matter track whether it’s actually doing that job over time, not a one-off audit.

Submitted vs. indexed, per sitemap

• Metric — Of the URLs in a given sitemap, what share Google reports as indexed vs. not indexed.
• What it tells you — Whether the URLs you’re advertising are actually landing in the index. A ratio that’s fine overall but bad for one sitemap (say, a blog sitemap vs. a product sitemap) tells you exactly which section of the site has a problem — that’s the whole reason to split sitemaps by section in the first place.
• How to pull it — In Search Console, open Page Indexing (“Why pages aren’t indexed”) and filter by sitemap — that view splits indexed vs. not-indexed for the URLs in that specific file. The Sitemaps report itself only shows what was submitted and when it was last read, so the indexed/not-indexed split comes from Page Indexing, not the Sitemaps report. Spot-check individual URLs with the Google Index Checker when you need the observable signals (status, redirects, noindex, canonical) behind a single URL’s non-indexing.
• Benchmark / realistic range — There’s no honest universal target — a site with lots of thin, near-duplicate, or genuinely low-value pages will never see 100%, and that’s not necessarily a bug. Establish your own baseline per section and watch the trend: a ratio that’s drifting down is the signal, not any specific absolute number.
• Cadence — Monthly for a stable site; weekly during a migration, a large content push, or right after you restructure your sitemaps, since that’s when the ratio moves fastest.

Sitemap coverage completeness

• Metric — Share of your actual canonical, indexable, 200-status URLs that are present in the submitted sitemap(s).
• What it tells you — Whether the sitemap is doing its job as a complete inventory, or whether it’s rotted — missing new sections, still listing redirected or deindexed URLs, or drifting out of sync with what your CMS actually generates. This is a data-quality check on the file itself, not on Google’s behavior.
• How to pull it — Fetch and validate the live sitemap with the XML Sitemap Validator to get the exact URL list plus any structural errors, then compare that list against your site’s real canonical URL set (a crawl export, or your CMS’s own page list) to find gaps in either direction — pages missing from the sitemap, or stale URLs still in it that shouldn’t be.
• Benchmark / realistic range — Depends entirely on how you define “should be indexable” for your own site, so there’s no defensible universal number here either — the right bar is simply “close to 100% match, with any gap explained” rather than a target percentage.
• Cadence — Whenever the sitemap is regenerated (ideally every deploy, if it’s automated) and as a manual spot-check after any large content migration or URL restructuring.

Ready-to-use AI prompts

Copy-paste starting points for sitemap tasks. Treat the output as a draft to verify against the actual sitemap file, Search Console, and Google’s own sitemap docs — not a final answer.

Audit a sitemap file against the spec

Here is my sitemap XML [paste the file or the first ~200 lines]. Check it against Google’s sitemap requirements: is it under 50,000 URLs / 50MB uncompressed, UTF-8, using fully-qualified absolute URLs, and are &, ', ", <, and > properly entity-escaped? Flag anything that looks like a non-canonical URL, a redirect target, or a duplicate. Don’t flag <priority> or <changefreq> as errors — Google ignores those tags, they’re not a validity issue.

Decide whether to split into a sitemap index

My site has approximately [N] indexable URLs across these sections: [list sections, e.g. blog, product pages, category pages]. Based on Google’s 50,000-URL / 50MB-per-sitemap limit, tell me whether I need a sitemap index, and if so, propose a sensible split by section (so I can also diagnose submitted-vs-indexed per section in Search Console afterward).

Write a lastmod update policy

I want a lastmod policy for my sitemap generator that only updates the date on genuinely significant changes (main content, structured data, or links) — not a blanket “today” or copyright-year stamp, since Google says it stops trusting lastmod once it’s inaccurate. Here’s how my CMS tracks edits [describe: publish date, last-edited timestamp, version history, etc.]. Draft the logic for when lastmod should and shouldn’t change.

Decide whether to keep the sitemap public or submit it privately

Here’s my situation: [describe the site — competitive niche? new unlinked pages you don’t want scraped? currently listing the sitemap in robots.txt?]. Walk me through the trade-off between listing my sitemap in robots.txt (public, auto-discoverable by any crawler) versus submitting it only through Google Search Console and Bing Webmaster Tools (private, but I lose auto-discovery for any engine I haven’t manually submitted to). Recommend one given my situation and explain why.

Triage a submitted-vs-indexed export by section

Here’s a CSV export of my sitemap URLs joined with their indexing status from Search Console [paste: URL, sitemap, indexed y/n, reason if not indexed]. Group the “not indexed” URLs by likely cause (thin content, near- duplicate, blocked, not yet crawled, deliberately low-value) and tell me which sitemap section has the worst ratio, so I know where to focus first.

Sitemap checklist

A pass to confirm your sitemap is helping rather than rotting:

• A sitemap exists and is generated automatically (not hand-maintained).
• It lists only canonical, indexable, 200-status URLs — no redirects, no non-canonical URLs, no noindex’d pages.
• URLs are fully-qualified and absolute, and special characters (& ' " < >) are entity-escaped.
• It’s UTF-8 encoded.
• No single sitemap exceeds 50,000 URLs or 50MB uncompressed — split + use a sitemap index if it does (gzip is fine; the cap is the uncompressed size).
• You haven’t bothered with <priority> or <changefreq> — Google ignores both.
• <lastmod> is only set on significant updates and is genuinely accurate (not a blanket copyright-year or “today” date).
• Submitted in Google Search Console and Bing Webmaster Tools.
• A Sitemap: line is in robots.txt (or it’s deliberately omitted to keep the sitemap private — in which case you’ve submitted it in every console).
• No dead google.com/ping?sitemap= calls left in any plugin or cron job (it 404s since Jan 2024).
• You periodically check the submitted-vs-indexed numbers in Search Console to spot under-indexed sections.
• You’re not using the sitemap to try to deindex anything — that’s noindex’s job.

The mental models

1. A sitemap is an inventory, not a discovery silver bullet. It’s a coverage tool — a clean list of what you want indexed. It does not force crawling or indexing. If pages aren’t getting indexed, the sitemap is where you notice the problem (submitted vs indexed), not where you fix it.

2. Exclude ≠ noindex. Leaving a URL out of the sitemap does not remove it from Google. The sitemap is an advertisement, not a gate. To remove a page, allow crawling and add noindex — the sitemap is the wrong lever entirely.

3. Automate it, or it rots. A manually-built sitemap goes stale the moment someone adds a page and forgets the file. Generate it from the pages you actually have so it stays current and complete. A sitemap’s value is in being accurate, not in merely existing.

4. Submitted-vs-indexed is a diagnostic. Split your sitemaps by section or content type and the Search Console coverage numbers tell you which part of the site is under-indexed. That’s the real, day-to-day payoff — far more useful than the act of submitting itself.

5. Discovery has two halves. Sitemaps are the pull-inventory half of discovery; links are the other pull half, and push protocols (IndexNow, the Indexing API) are a separate channel. If a page isn’t being found, ask which channel should be carrying it — and whether anything links to it at all.

How to create a sitemap, by platform

Each platform below either generates a sitemap automatically or has a standard module for it. The two final steps are the same everywhere: (1) make sure only the right URLs are included (canonical, indexable only); (2) submit it in Search Console + Bing Webmaster Tools and add the Sitemap: line to robots.txt (or deliberately omit that line to keep it unlisted).

Search Console sitemap errors → meaning → fix

Other real-world gotchas: an HTTP-vs-HTTPS or www mismatch between sitemap URLs and the verified property triggers “URL not allowed”; Google may serve a stale cached copy so changes aren’t instant; and submitting children and the index is harmless but unnecessary.

Tools for validating and diagnosing a sitemap

• Google Index Checker — spot-check a specific URL from your sitemap for the observable indexability blockers (status code, redirects, noindex, canonical signals) before you assume the sitemap itself is the problem. It routes you to Search Console’s URL Inspection for Google’s actual, authoritative answer — this tool only surfaces what’s visible from the outside.
• Google Search Console — Sitemaps report — where you submit a sitemap and see what Google last fetched and when; combine with the Page Indexing report (filtered by sitemap) to get the indexed-vs-not-indexed split used in the How to Measure tab above.
• Bing Webmaster Tools — Sitemaps — the equivalent submission and status view for Bing; submit here in addition to Search Console, since the two engines track sitemaps independently.
• A crawler (Ahrefs Site Audit, Screaming Frog) — the way to generate the “actual canonical URL list” you compare a sitemap against when checking coverage completeness — a sitemap validator confirms the file is well-formed, but only a real crawl of your site tells you if the file is complete.

Resources worth your time

My related writing

• Enterprise Technical SEO — where automated, current sitemaps fit into managing a large site.
• When Should You Worry About Crawl Budget? — using the sitemap as a clean inventory of your indexable set.
• Website Migration: The Ultimate Guide — including keeping a sitemap of the old URLs around to watch them drop out of the index.
• JavaScript SEO Issues & Best Practices — when sitemaps matter most (JS-loaded content and framework sitemap modules).

My speaking

• How Search Works (SlideShare) — my walkthrough of crawling, rendering, indexing, and ranking, where sitemaps live on the discovery side. (Standing disclaimer: “This is my understanding of systems… not going to be 100% complete or accurate.”)

From others

• Google’s Sitemaps overview and build-a-sitemap docs — the primary source for everything on this page.
• The sitemaps.org protocol — the cross-engine standard XML sitemaps are built on.
• Bing Webmaster Blog: Keeping Content Discoverable with Sitemaps in AI-Powered Search — Bing’s current take on XML format preference, lastmod importance, and sitemaps + IndexNow as complementary.
• Bing Webmaster Blog: The Importance of Setting the lastmod Tag in Your Sitemap — Fabrice Canel explains how Bing uses lastmod for freshness assessment.
• IndexNow documentation — the push protocol for instantly notifying Bing, Yandex, and others of changed URLs (Google does not participate).
• Google Search Console Sitemaps report help — full reference for every sitemap error and status in Search Console.
• Search Engine Journal: Google Explains Why It May Not Use a Sitemap — covers John Mueller’s explanation of when Google ignores submitted sitemaps.
• Search Engine Roundtable: Google: XML Sitemaps Are A Minimal Baseline For Any Serious Website — Barry Schwartz covering Mueller’s “minimal baseline” stance.

Test yourself: Sitemaps

Five quick questions on what sitemaps do and what belongs in them. Pick an answer for each, then check.

Sitemap

A sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.

Related: XML sitemap, Sitemap index, URL discovery

Sitemap

A sitemap is, in Google’s words, “a file where you provide information about the pages, videos, and other files on your site, and the relationships between them.” In practice it’s an inventory you hand to search engines so they don’t have to rely entirely on links to find your content.

The most common type is the XML sitemap — a machine-readable list of URLs, sometimes with a lastmod date. Large sites split their URLs across several sitemaps and tie them together with a sitemap index (a sitemap of sitemaps). There are also specialized image, video, and news sitemaps, an HTML sitemap meant for human visitors, and RSS/Atom/plain-text alternatives.

The key thing to keep straight: a sitemap helps with discovery, not with indexing or ranking. Submitting one doesn’t guarantee the URLs get crawled or indexed — it just makes them easier to find. Its real day-to-day value is as a coverage and diagnostic tool: comparing the URLs you submitted against the URLs Google actually indexed (in Search Console) tells you which parts of your site aren’t getting picked up.

A single sitemap can hold up to 50,000 URLs or 50MB uncompressed, whichever comes first. Google ignores the <priority> and <changefreq> tags entirely; an accurate lastmod is the metadata that still matters. And leaving a URL out of a sitemap doesn’t deindex it — exclusion is not the same as noindex.

Related: XML sitemap, Sitemap index, URL discovery

Changelog