Paywalls and SEO
How to keep paywalled and registration-gated content indexable without cloaking — flexible sampling, isAccessibleForFree/cssSelector markup, the JavaScript-paywall trap, and metering strategy.
A paywall doesn't inherently hurt SEO — Google has no bias against gated content, and the biggest paywalled publishers rank fine. What hurts is Google not being able to see enough content to understand the page. The supported fix is flexible sampling: let Googlebot crawl the full article, then declare the gated part with structured data (isAccessibleForFree plus a cssSelector). That's an explicit, sanctioned exception to cloaking — cloaking is about intent to deceive; this is a declared mechanism. Use metering (start around 6–10 free articles/month) or lead-in, gate server-side (not with JavaScript that just hides content in the DOM), give login pages unique copy, and never use robots.txt to hide private URLs.
TL;DR — A paywall (subscription, one-time payment, or just a registration/login gate) doesn’t automatically hurt your SEO. Google has a supported way to handle it called flexible sampling: you let GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. read the whole article, then use a bit of structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding. to tell Google which part is gated. Done that way, showing Google the full article while readers see a truncated version is not cloaking — it’s an approved exception.
Do paywalls hurt SEO?
Google supports paywalled content when crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. can access it and the implementation uses the documented paywall structured-data pattern. Evidence for this claim Official or primary documentation supporting the adjacent article claim, with scope limited to the source's published description. Scope: No ranking guarantee or undisclosed system mechanics are inferred beyond the cited source. Confidence: high · Verified: Google: Paywalled content structured data Google’s flexible-sampling guidance describes metering and lead-in approaches, not a ranking guarantee. Evidence for this claim Official or primary documentation supporting the adjacent article claim, with scope limited to the source's published description. Scope: No ranking guarantee or undisclosed system mechanics are inferred beyond the cited source. Confidence: high · Verified: Google: Flexible sampling
Not on their own. This is the first thing to get straight, because half the guides out there frame paywalls as an SEO problem to be minimized. They aren’t. Google has no bias against paywalled content — the New York Times, the Wall Street Journal, the Financial Times, and the Washington Post all sit behind paywalls and rank prominently for exactly the stories they gate.
What does hurt rankings is Google not being able to see enough of your content to understand what the page is about. If a bot only ever sees a two-sentence teaser, it can only rank you for those two sentences. So the whole game with paywalls and SEO is this: let the search engine read the full article, while normal visitors still hit the gate.
One thing this markup is not: a promise. Getting isAccessibleForFree and the
rest of the markup exactly right doesn’t guarantee indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., ranking, or a rich
result — Google’s own structured-data documentation says plainly it doesn’t
guarantee any feature will show up in search results. What the markup does is
remove the cloaking risk of showing crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. more than users see; it doesn’t
manufacture rankings by itself.
The supported way to do it: flexible sampling
Google’s model is called flexible sampling, and it has two flavors:
- Metering — visitors get a quota of free articles (Google suggests starting around 6–10 per month) before the paywall kicks in.
- Lead-in — you show the opening of an article, then gate the rest.
On top of whichever you choose, you add a small piece of structured data to the page that tells Google, “this section is behind a paywall.” That’s the label that makes everything legitimate.
Isn’t showing Google the full article cheating?
This is the question everyone asks, and the answer is no — because you declared it. Cloaking (the bad thing) is when you show search engines different content than users in order to deceive them and manipulate rankings. Flexible sampling is the opposite: you’re openly telling Google, through structured data, “hey, real users see something more limited than what you’re crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor..” Google’s own spam policy carves paywalls out of the cloaking definition by name, as long as you follow the flexible-sampling guidance and let Google see the full content.
The one mistake to avoid
Don’t build your paywall by shipping the entire article in the page’s HTML and just hiding it with JavaScript or CSS until someone logs in. It feels easier, but it backfires: anyone can turn off JavaScript and read your paid content for free, screen readers will read the “hidden” text aloud, and Google can’t reliably tell which part you meant to gate. The right way is to gate it on the server — only send the full article once you’ve confirmed the person is logged in or subscribed.
Want the full mechanics — the exact structured data, the metering numbers, the JavaScript trap, and why login pages cause their own problems? Switch to the Advanced tab.
TL;DR — Paywalls don’t inherently hurt rankings; Google being unable to see your content does. The supported model is flexible sampling — metering or lead-in — declared with structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding. (
isAccessibleForFree: falseplus ahasPart/cssSelectormarking the gated section, class selectors only). That declaration is what makes serving GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. the full article not cloaking: cloaking requires intent to manipulate and mislead, and Google’s spam policy explicitly carves paywalls out of that definition. Gate server-side (the 2025 doc update and Mueller’s screen-reader caution both target the same JS-hiding mistake), give login pages unique copy, neverrobots.txtprivate URLs, and usenoarchiveto stop a cached copy leaking the full text. Registration walls use the same markup as paid ones.
What actually causes ranking problems (it isn’t the gate)
Paywall eligibility depends on crawlable content and accurate markup; the presence of a paywall alone is not documented as a penalty. Evidence for this claim Official or primary documentation supporting the adjacent article claim, with scope limited to the source's published description. Scope: No ranking guarantee or undisclosed system mechanics are inferred beyond the cited source. Confidence: high · Verified: Google: Paywalled content structured data Sampling choices remain publisher decisions with user and business tradeoffs. Evidence for this claim Official or primary documentation supporting the adjacent article claim, with scope limited to the source's published description. Scope: No ranking guarantee or undisclosed system mechanics are inferred beyond the cited source. Confidence: high · Verified: Google: Flexible sampling
Google has no penalty for paywalled content, and this article’s parent hub says as much: gated content is fine as long as Google can read it through the supported approach. The failure mode is upstream of ranking — it’s comprehension. If GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. only ever sees a teaser, that teaser is all it can indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. and rank you for. Every technique below exists to solve one problem: let the engine read the whole thing, while unauthenticated humans still hit the gate.
Two boundaries worth stating plainly, since it’s easy to overreach in either
direction. First, this markup is a tool for content you want indexed under a
declared gate — not a mechanism for exposing content you don’t want indexed at
all. Genuinely private account/admin URLs are a different case (see the decision
tree below): those get noindex or an authentication redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't., not
isAccessibleForFree. Second, valid markup and full crawl access are not a
ranking guarantee. Google’s structured-data guidelines say directly that “Google
does not guarantee that features that consume structured data will show up in
search results” — the markup is the declaration that keeps you out of the
cloaking bucket, not a promise of indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., ranking, traffic, or a rich result.
Historically this is where the biggest cautionary tale comes from. When the Wall Street Journal pulled out of Google’s old First Click Free program in 2017, it reported a ~44% drop in Google search traffic — not because paywalls are penalized, but because Google could no longer see the articles at all. (More on First Click Free below; it is history, not current policy.)
Flexible sampling: metering and lead-in
The current, active model is flexible sampling, laid out in Google’s Flexible Sampling guidelines. Google describes two sampling types: “metering, which provides users with a quota of articles to consume before requiring users to subscribe or log in, after which paywalls will start appearing; and lead-in, which offers a portion of an article’s content without it being shown in full.”
The numbers that matter, all from Google’s own doc:
- Prefer monthly over daily metering. Google: “In general, we think that monthly, rather than daily metering provides more flexibility and a safer environment for testing.” A one-unit change is far less jarring at 10 monthly samples than at 3 daily ones.
- Start around 6–10 free articles/month. “As a starting point for your explorations, we encourage you to provide 10 articles per month… for most daily news publishers, we expect the value to fall between 6 and 10 articles per user per month.”
- Watch the exposure ceiling. “Our analysis shows that general user satisfaction starts to degrade significantly when paywalls are shown more than 10% of the time (which generally means that about 3% of the audience has been exposed to the paywall).”
- Lead-in is a good practice. Showing the first few sentences above the paywall lets users “experience the value of the content.”
None of these numbers is a mandate. Google says directly: “There is no single value for optimal sampling across different businesses” — the 6–10/month figure is a starting point Google gives specifically for daily news publishers, and even that comes with “we leave the exact number to the discretion of individual publishers, who are best positioned to understand the particular demands of their businesses.” Treat it as a tested starting range, not a rule to copy verbatim.
The under-appreciated point: metering is not purely a monetization dial. Google opens the doc noting that “even minor changes to the current sampling levels could degrade user experience and, as user access is restricted, unintentionally impact article ranking in Google Search.” Tightening the meter can quietly cost you rankings.
Why this isn’t cloaking — the reasoning, not just the rule
This is the load-bearing part of the whole topic, and most guides assert the conclusion (“paywalls aren’t cloaking if you use structured data”) without showing why. Here’s the actual reasoning, straight from Google’s spam policies.
Start with the definition. Cloaking is “the practice of presenting different content to users and search engines with the intent to manipulate search rankings and mislead users.” The load is on that clause — intent to manipulate and mislead. A paywall isn’t trying to trick anyone; it’s monetizing content, and it’s declaring the difference in treatment through markup.
Then the explicit carve-out, in the same policy: “If you operate a paywall or a content-gating mechanism, we don’t consider this to be cloaking if Google can see the full content of what’s behind the paywall just like any person who has access to the gated material and if you follow our Flexible Sampling general guidance.”
So the exception has two conditions: (1) Google sees the same full content a paid subscriber would, and (2) you follow flexible sampling — which in practice means the structured data below. Google’s flexible-sampling doc reinforces the same logic: “Enclose paywalled content with structured data in order to help Google differentiate paywalled content from the practice of cloaking, where the content served to Googlebot is different from the content served to users.” The structured data is the declaration that turns “different content for botsA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index.” from deception into a disclosed, sanctioned mechanism.
Implementing the structured data
The markup lives in Google’s Subscription and paywalled content doc. Two properties do the work:
isAccessibleForFree(Boolean, required) — whether the content is free or gated. Google’s own property reference marks this the required one; set it on the top-levelCreativeWork/NewsArticlenode and on each gated section.hasPart(recommended, not required) — an array ofWebPageElementobjects, one per gated section, each with its ownisAccessibleForFree: falseand acssSelectorpointing at the class you wrapped the gated HTML in. This is how you tell Google which part of the piece is gated when it’s a section rather than the whole thing; it’s the recommended way to get section-level precision, not a second required property alongside the top-level flag.
A minimal NewsArticle looks like this:
{
"@context": "https://schema.org",
"@type": "NewsArticle",
"isAccessibleForFree": false,
"hasPart": {
"@type": "WebPageElement",
"isAccessibleForFree": false,
"cssSelector": ".paywall"
}
}Three implementation details people trip on:
- Class selectors only. The
cssSelector“references the class name that you set in the HTML.” Use.paywall— not an ID (#paywall), not a descendant or attribute selector. - Multiple gated sections use an array of
hasPartobjects, each with its own class-based selector. Don’t nest the gated sections inside each other. - It’s not just for news. The markup is supported on any
CreativeWorksubtype —Article,NewsArticle,Blog,Comment,Course,HowTo,Message,Review,WebPage. The broader structured data guidance treatsisAccessibleForFreeas a generalCreativeWorkproperty, not a news-only one. - Correct markup doesn’t guarantee a result. Even fully valid, correctly-nested markup only makes Google eligible to understand your gating — it isn’t a ranking or rich-result guarantee. Treat the markup as the mechanism that keeps you out of the cloaking bucket, not a promise of any specific outcome.
Registration walls use the identical markup. Google doesn’t distinguish “pay to access” from “register to access” at the schema level. John Mueller said as much on Search Off the Record: the mechanism “could be maybe you require a login, maybe you require a payment, maybe after a certain number of iterations you’re like, ‘Oh, this is enough free content.’ Now you have to pay for it… It can just be something like a login or some other mechanism that basically limits the visibility of the content.” If you gate it, mark it — paid or not. He even flags A/B pricing tests as a valid reason: “if you have something like different thresholds where you say some people get to view five pages for free and others have the whole content available for free because you’re doing A/B testing… then you’d want to use a paywall structured data.”
The JavaScript-paywall trap
Here’s the single most common real-world mistake, and it’s distinct from “forgetting the structured data.” A lot of paywall solutions ship the full article in the HTML the server sends, then use JavaScript to hide it until subscription status is confirmed. Google explicitly warned against this in a 2025 addition to its JavaScript troubleshooting doc: “Some JavaScript paywall solutions include the full content in the server response, then use JavaScript to hide it until subscription status is confirmed. This isn’t a reliable way to limit access to the content. Make sure your paywall only provides the full content once the subscription status is confirmed.”
Why it’s bad on three fronts:
- It’s trivially bypassable. Disable JavaScript and the “hidden” article is right there in the source. You’re not actually gating anything.
- It muddies the cloaking exception. If the full text is sitting in the DOM for everyone, Google can’t cleanly tell which content was meant to be gated — which is the whole thing the structured-data declaration is supposed to make clear.
- It’s an accessibilityWeb accessibility means designing sites so people with disabilities can use them, per the W3C's WCAG guidelines. It overlaps with SEO in specific, checkable ways — alt text, heading structure, descriptive link text, captions, and page speed all serve both audiences — but Google has said accessibility itself is not a ranking factor, and most WCAG success criteria (keyboard focus order, ARIA live regions, form labels) have no SEO effect at all. problem. Mueller raised exactly this on Search Off the Record: “when a user looks at your page, you don’t load the content into the HTML, but rather you make sure that it’s really not loaded into the page’s DOM so that, if a browser has something like… a screen reader, that the screen reader doesn’t go off and read all of this text that you’re trying to hide… make sure you don’t load it into the browser and use JavaScript to turn it on, but rather that it’s really only served to the user when you want to make it available.” The 2025 doc update and Mueller’s caution are the same mistake seen from two angles.
The fix is server-side gating: confirm subscription/login status on the server,
and only include the full article in the response for authenticated users. Then
layer isAccessibleForFree/cssSelector on top so Googlebot — which is allowed
to see the full text under flexible sampling — still gets everything, while
unauthenticated humans genuinely don’t. This is also where paywalls intersect with
mobile-first indexingGoogle's practice of using the mobile version of a page's content — crawled by Googlebot smartphone — for indexing and ranking. It is not a separate index and not a ranking boost.: Google crawls and evaluates the mobile version, so the full
gated content has to be present in the mobile server response too, not just desktop.
Login pages and registration gates: the quieter pitfalls
Two distinct problems show up around login/registration, both from the same Search Off the Record episode.
Generic login pages get folded into duplicates. Mueller: “if you have a very generic login page, we will see all of these URLs that show that login page, that redirect to that login page, as being duplicates… We’ll fold them together as duplicates, and we’ll focus on indexing the login page… If someone is searching for your service… the only thing… they find in search is like, ‘Here’s how to log in,’ that might be a kind of a weird experience for them.” The fix is to give login pages unique contextual copy per service, so they’re not all identical.
Don’t robots.txt private URLs. This one contradicts a common intuition.
Mueller: “whether all of this should just be blocked by robots.txtA Google Search Console Page Indexing status: the URL was excluded from indexing because your robots.txt disallows crawling it. Usually intentional and benign — robots.txt blocks crawling, not indexing., which is another
common strategy… The problem, I think, with doing that is the URLs could become
indexable so we wouldn’t see the contents of the login page… if it’s private
content, serve it with a noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. or redirect it to a login page somewhere. Don’t
use robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere..” A robots-blocked URL can still be indexed as a bare, contentless
URL — often worse than a clean noindex. (This is genuinely-private content, which
is a different case from paywalled-but-should-be-indexed; don’t confuse the two.)
Testing and the “leaky” worry
Test with the Rich ResultsRich results (formerly 'rich snippets') are enhanced search listings — stars, images, prices, breadcrumbs, video thumbnails, and more — that Google and Bing build from structured data. They're a display feature, not a ranking factor, and eligibility never guarantees they'll show. Test. Google
added paywalled-content support
to the Rich Results Test in October
2023, so it validates isAccessibleForFree/cssSelector on a live URL, testing as
Googlebot desktop or smartphone. As Mueller put it back in a 2020 office-hours,
“you would use the rich results test, like any other kind of structured data… the
tricky part with some of these paywall implementations is that Googlebot, of course,
needs to be able to see the full content.”
The self-audit trick: open an incognito window (logged out of everything), search for your own brand or service, and see what shows. Mueller’s advice — “If the top result is something like a login page and there’s no information on this page at all otherwise, then probably that’s something that you can improve.”
Is showing Googlebot the full article “leaky”? No. Danny Sullivan, Google’s
Search Liaison, addressed the recurring worry that this exposes paid content:
“Our system is looking to be shown the full content, if a publisher wants to do
that. If they do, we understand more about it. If we understand more, then we might
be able to show it for more queries where it’s relevant,” and “Since only we are
seeing this, there’s nothing ‘leaky’ as you are suggesting.” The real leak vector,
he noted, is the cached copy — solved with noarchive, a separate control from
the paywall markup itself.
Sullivan’s remarks are relayed via Search Engine Roundtable’s coverage;
treat them as reported rather than a first-party transcript.
Bing’s approach
Bing’s subscription and paywall guidance
(Fabrice Canel, May 2022) is structurally similar but not schema-centric. Its
three points: (1) let BingbotBingbot is Microsoft Bing's primary web crawler — the bot that discovers, fetches, and renders pages to build the Bing index. That index also powers Yahoo, DuckDuckGo, Ecosia, and Microsoft Copilot, so Bingbot's reach is far wider than Bing's own search-market share. crawl the full gated content, (2) use
noarchive/nocache (or the X-Robots-Tag: noarchive header) so cached copies
don’t leak, and (3) verify the crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. is genuinely Bingbot by checking the
requesting IP against Bing’s published ranges — not by trusting the user-agent
string, which anyone can spoof. There’s no published Bing equivalent to
isAccessibleForFree/cssSelector; Bing’s model is crawl-access-plus-cache-control,
where Google’s is markup-centric. Don’t assume feature parity.
First Click Free — history, not policy
You’ll still see blog posts and forum answers describing First Click Free as if it’s current. It isn’t. Google retired it in October 2017, replacing it with flexible sampling. Richard Gingras, then Google’s VP of News: “First, Flexible Sampling will replace First Click Free. Publishers are in the best position to determine what level of free sampling works best for them.” First Click Free had required participating publishers to let Google-referred visitors read a set number of articles a day (commonly three) even past their own paywall. Flexible sampling handed that decision back to publishers. If you see FCF cited as something you can opt into today, that guidance is eight-plus years stale.
Where this sits in news SEO
Paywall handling is one piece of the broader News & Discover SEO picture — alongside news sitemapsA news sitemap is a sitemap (a standalone file or a section of an existing sitemap) that uses Google's news: XML namespace to tell Google about your recent articles — only those published in the last two days. It speeds discovery of breaking content; it isn't a ranking factor and isn't required for Google News eligibility., Google News/Top Stories eligibility, Discover, and syndication (canonical vs. noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed.). If you’re a publisher, get your paywall markup and your syndication policy sorted before either one quietly costs you indexation or attribution.
AI summary
A condensed take on the Advanced version:
- Paywalls don’t inherently hurt SEO. Google has no bias against gated content; the biggest paywalled publishers rank fine. What hurts is Google being unable to see enough content to understand the page — and the markup itself is a declaration, not a ranking guarantee (Google’s own docs say structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding. doesn’t guarantee any feature will show up in search results).
- Flexible sampling is the supported model (not the retired First Click Free, gone since Oct 2017): metering (start ~6–10 free articles/month, prefer monthly over daily) or lead-in (show the opening, gate the rest). Google is explicit there’s “no single value for optimal sampling across different businesses” — 6–10/month is a daily-news starting point, not a universal rule. User satisfaction degrades past ~10% paywall-exposure; tightening the meter can even cost rankings.
- Structured data is the mechanism:
isAccessibleForFree: false(the required property) on the article node, plus a recommendedhasPart/WebPageElementwith a class-basedcssSelectorfor section-level precision. Works on anyCreativeWorksubtype, not just news. It’s for content you want indexedStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. under a declared gate — genuinely private URLs getnoindexinstead, not this markup. - Why it isn’t cloaking: cloaking requires intent to manipulate and mislead; Google’s spam policy explicitly carves out paywalls when Google sees the full content and you follow flexible-sampling guidance. The markup is the declaration.
- Registration/login walls use the identical markup as paid paywalls — Google doesn’t distinguish pay-vs-register at the schema level (per Mueller).
- The JS-paywall trap: don’t ship the full article in the HTML and hide it with JS/CSS — it’s bypassable, muddies the cloaking exception, and screen readers read the “hidden” text. Gate server-side; the full content must be in the mobile response too (mobile-first indexingGoogle's practice of using the mobile version of a page's content — crawled by Googlebot smartphone — for indexing and ranking. It is not a separate index and not a ranking boost.).
- Login-page pitfalls: generic login pages get folded as duplicates (give them
unique copy); never
robots.txtprivate URLs (usenoindex/redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't.). - Cache leak is a separate control:
noarchive/nocachestops a cached copy exposing gated text (per Danny Sullivan). Bing’s model is crawl-access + cache control + IP verification, with noisAccessibleForFreeequivalent. - Test with the Rich ResultsRich results (formerly 'rich snippets') are enhanced search listings — stars, images, prices, breadcrumbs, video thumbnails, and more — that Google and Bing build from structured data. They're a display feature, not a ranking factor, and eligibility never guarantees they'll show. Test (paywall support since Oct 2023) and Mueller’s incognito self-audit.
Official documentation
Primary-source guidance from the search engines.
- Flexible Sampling — the core model: metering vs. lead-in, the 6–10 articles/month starting point, the 10%-exposure ceiling, and the cloaking-differentiation rationale.
- Subscription and paywalled content markup —
isAccessibleForFree,hasPart/WebPageElement, and the class-basedcssSelector. - Spam policies — Cloaking — the cloaking definition and the explicit paywall carve-out.
- Fix search-related JavaScript problems — the 2025 JavaScript-paywall guidance.
- Google common crawlers list — the actual crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. user-agentsA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target. (used below to debunk the fabricated “GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. Subscriber” claim).
- Driving the future of digital subscriptions — the 2017 First Click Free → Flexible Sampling transition.
- Rich Results Test — validates paywall structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding. on a live URL.
Bing / Microsoft
- SEO best practice for subscription-based and paywall content — Fabrice Canel, May 2022: crawl access, cache control, and IP-based BingbotBingbot is Microsoft Bing's primary web crawler — the bot that discovers, fetches, and renders pages to build the Bing index. That index also powers Yahoo, DuckDuckGo, Ecosia, and Microsoft Copilot, so Bingbot's reach is far wider than Bing's own search-market share. verification.
Quotes from the source
On-the-record statements from Google and Bing. Each link deep-links to the quoted passage where the source page supports it.
Google — why paywalls aren’t cloaking (the load-bearing quotes)
- “Cloaking refers to the practice of presenting different content to users and search engines with the intent to manipulate search rankings and mislead users.” — Google spam policies. Jump to quote
- “If you operate a paywall or a content-gating mechanism, we don’t consider this to be cloaking if Google can see the full content of what’s behind the paywall just like any person who has access to the gated material and if you follow our Flexible Sampling general guidance.” Jump to quote
- “Enclose paywalled content with structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding. in order to help Google differentiate paywalled content from the practice of cloaking, where the content served to GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. is different from the content served to users.” Jump to quote
Google — flexible sampling and metering
- “There are two types of sampling we advise: metering, which provides users with a quota of articles to consume before requiring users to subscribe or log in, after which paywalls will start appearing; and lead-in, which offers a portion of an article’s content without it being shown in full.” Jump to quote
- “In general, we think that monthly, rather than daily metering provides more flexibility and a safer environment for testing.” Jump to quote
- “As a starting point for your explorations, we encourage you to provide 10 articles per month to Google search users and iterate from there… for most daily news publishers, we expect the value to fall between 6 and 10 articles per user per month.” Jump to quote
- “Our analysis shows that general user satisfaction starts to degrade significantly when paywalls are shown more than 10% of the time (which generally means that about 3% of the audience has been exposed to the paywall).” Jump to quote
Google — the JavaScript-paywall trap
- “Some JavaScript paywall solutions include the full content in the server response, then use JavaScript to hide it until subscription status is confirmed. This isn’t a reliable way to limit access to the content. Make sure your paywall only provides the full content once the subscription status is confirmed.” Jump to quote
Richard Gingras, VP of News, Google (Oct 2017)
- “First, Flexible Sampling will replace First Click Free. Publishers are in the best position to determine what level of free sampling works best for them.” Read the announcement
John Mueller, Google — Search Off the Record (Sep 2025)
- On registration vs. payment gates: “It also doesn’t have to be something that’s behind a clear payment thing. It can just be something like a login or some other mechanism that basically limits the visibility of the content.”
- On the DOM/screen-reader caution: “you make sure that it’s really not loaded into the page’s DOM so that, if a browser has something like… a screen reader, that the screen reader doesn’t go off and read all of this text that you’re trying to hide.”
- On private URLs: “if it’s private content, serve it with a noindexNoindex is a directive that tells search engines to keep a page out of their index, so it won't appear in search results. It works only on pages a crawler can actually fetch — a page blocked in robots.txt can never be noindexed. or redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. it to a login page somewhere. Don’t use robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere..” Full transcript (PDF)
John Mueller, Google — SEO office-hours (Dec 2020)
- “Essentially you would use the rich resultsRich results (formerly 'rich snippets') are enhanced search listings — stars, images, prices, breadcrumbs, video thumbnails, and more — that Google and Bing build from structured data. They're a display feature, not a ranking factor, and eligibility never guarantees they'll show. test, like any other kind of structured data. I think the tricky part with some of these paywall implementations is that GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer., of course, needs to be able to see the full content so that we can understand what it is that we should be showing your site for.” Coverage (Search Engine Journal)
Danny Sullivan, Google Search Liaison — the “not leaky” clarification
- “Our system is looking to be shown the full content, if a publisher wants to do that. If they do, we understand more about it. If we understand more, then we might be able to show it for more queries where it’s relevant.” and “Since only we are seeing this, there’s nothing ‘leaky’ as you are suggesting.” Coverage (Search Engine Roundtable)
Which paywall setup do I need?
Paywall implementations differ mostly on how you gate and what you want indexedStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.. Walk through it — the leaf tells you which markup (if any) and which control applies.
Choosing the right gating + markup approach
What not to do with paywalls
1. Treating First Click Free as current policy. Plenty of stale posts describe First Click Free as if you can still opt in. Google retired it in October 2017 and replaced it with flexible sampling. Fix: design around metering/lead-in and structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding.; if you see FCF cited as live guidance, ignore it.
2. Hiding the full article with JavaScript/CSS instead of gating server-side. Shipping the whole article in the HTML and hiding it until login is bypassable (disable JS and it’s readable), muddies Google’s ability to recognize the paywall, and makes screen readers read the “hidden” text aloud. Fix: confirm subscription/login status on the server and only send the full content to authenticated users — then add the structured data on top.
3. Assuming any paywall counts as cloaking. Google’s spam policy explicitly carves paywalls out of the cloaking definition, conditioned on letting Google see the full content and following flexible-sampling guidance. Fix: don’t hide your content from Google out of cloaking fear — declare it with markup, which is the sanctioned mechanism.
4. Believing there’s a special “GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. Subscriber” crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index.. Several low-quality guides (likely one propagating to others) claim you must allow a “GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. Subscriber” or “Googlebot Registered User” crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index.. No such user-agentA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target. exists — Google’s published crawler list has Googlebot, Googlebot-Image, Googlebot-Video, and Googlebot-News, and nothing subscriber-related. Fix: ignore it; there’s no separate crawler to allow-list.
5. Blocking private/login URLs with robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere..
A robots-blocked URL can still be indexedStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. as a bare, contentless URL — often worse
than a clean noindex, and Mueller says as much. Fix: use noindex or a
redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. for private content; reserve robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. for crawl-budget control, not
deindexingDeindexing means getting a URL to stop appearing in Google's search results. There's no single delete button — the right method depends on whether you own the page, whether removal is temporary or permanent, and whether the content should still exist..
6. Citing an “80-word minimum lead-in” as Google policy. This figure circulates as if it’s official, but it doesn’t trace to any Google document. Google’s actual quantified guidance is about sampling frequency (6–10 articles/month), not lead-in word count. Fix: treat any word-count floor as an unverified practitioner heuristic, not policy.
7. Forgetting that showing Google the full text needs a cache control.
The paywall markup lets Googlebot see the full article, but a cached copy can leak
it to anyone who finds the cache. Fix: add noarchive/nocache (or the
X-Robots-Tag: noarchive header) if that’s a concern — it’s a separate control from
the paywall markup.
Snippets for checking a paywall setup
Practical checks for whether your gating and markup actually work. Swap
https://example.com/article and .paywall for your own.
1. Does the full article ship in the HTML? (the JS-trap test)
If your paid content is present in the raw server response, it’s not really gated — it’s just visually hidden. Fetch the HTML without executing JavaScript and search for a paid sentence.
macOS / Linux (curl + grep)
# Fetch the raw HTML (no JS execution) and look for a line that should be gated.
curl -s "https://example.com/article" | grep -i "a sentence only subscribers should see"
# Empty result = the gated text isn't in the raw HTML (good, server-side gated).
# A match = the full content is shipping to everyone and merely hidden (the JS trap).Compare what GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. vs. a logged-out user receives
# As a normal visitor:
curl -s "https://example.com/article" -o guest.html
# Emulating Googlebot's user-agent (only meaningful if you serve UA-based content):
curl -s -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
"https://example.com/article" -o googlebot.html
# Diff the visible article body — Googlebot should get the full text under flexible sampling.
diff <(grep -o '<p>.*</p>' guest.html) <(grep -o '<p>.*</p>' googlebot.html)2. Extract and sanity-check the paywall structured data
Pull the JSON-LDJSON-LD (JavaScript Object Notation for Linked Data) is a script-based structured data format, typically paired with the schema.org vocabulary to describe page content for search engines and AI systems. Google recommends it over Microdata and RDFa because it's the easiest format to implement and maintain at scale — but all three work, and structured data isn't a ranking signal. blocks with a Chrome DevTools Console snippet. Open the article, open DevTools → Console, paste:
// Dump every JSON-LD block and flag paywall properties.
[...document.querySelectorAll('script[type="application/ld+json"]')]
.map(s => { try { return JSON.parse(s.textContent); } catch { return null; } })
.filter(Boolean)
.forEach(obj => {
const json = JSON.stringify(obj);
if (json.includes('isAccessibleForFree') || json.includes('cssSelector')) {
console.log('Paywall markup found:', obj);
} else {
console.log('JSON-LD (no paywall props):', obj['@type']);
}
});Confirm the cssSelector actually matches an element (class selectors only —
#id and complex selectors are unsupported):
// Paste your declared selector; it MUST match at least one element, and be a .class.
const sel = '.paywall';
console.log('Matches on page:', document.querySelectorAll(sel).length);
console.log('Is a class selector:', /^\.[\w-]+$/.test(sel)); // true = supported form3. Bookmarklet: is this page marked as gated?
Drag-to-bookmark this one-liner (or paste in the address bar) to check any article
for isAccessibleForFree: false without opening DevTools:
javascript:(()=>{const b=[...document.querySelectorAll('script[type="application/ld+json"]')].map(s=>s.textContent).join('');alert(b.includes('"isAccessibleForFree":false')||b.includes('"isAccessibleForFree": false')?'Gated: isAccessibleForFree:false present':'No paywall markup found on this page');})();4. Confirm the cached copy isn’t leaking (noarchive check)
# Check for a noarchive directive in the meta robots tag or the X-Robots-Tag header.
curl -s "https://example.com/article" | grep -i 'name="robots"'
curl -sI "https://example.com/article" | grep -i 'x-robots-tag'
# You want "noarchive" (or nocache) present if you don't want a cached copy exposing gated text.After these pass, validate the live URL in the
Rich Results Test as GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. desktop
and smartphone — it’s the authoritative check that Google parses your
isAccessibleForFree/cssSelector markup.
Paywall symptoms, causes, and fixes
Rich Results Test does not validate the gated section
Symptom: The live URL does not show usable paywalled-content markup, or the reported
cssSelector does not identify the gated content.
Likely cause: isAccessibleForFree is missing or set inconsistently; hasPart is
malformed; the selector uses an ID or complex selector instead of a class; the HTML lacks
the declared class; or GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. receives only the teaser and cannot inspect the full work.
Fix and confirmation: Use isAccessibleForFree: false on the work and each gated
WebPageElement, point every cssSelector at an actual class such as .paywall, and
make the full subscriber-equivalent content available to GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. under flexible
sampling. Re-run the live URL in the Rich ResultsRich results (formerly 'rich snippets') are enhanced search listings — stars, images, prices, breadcrumbs, video thumbnails, and more — that Google and Bing build from structured data. They're a display feature, not a ranking factor, and eligibility never guarantees they'll show. Test as both smartphone and desktop
until the markup and gated section are parsed as intended.
Logged-out source contains the full paid article
Symptom: Disabling JavaScript, inspecting the HTML, or using a screen reader exposes text that the visible paywall claims is unavailable.
Likely cause: The server ships the full article to everyone and JavaScript or CSS merely hides it after the page loads.
Fix and confirmation: Move entitlement checking to the server and send the full text only after login/subscription is confirmed, while continuing to serve Googlebot under the declared flexible-sampling setup. Fetch the page logged out with scripts disabled and confirm the gated body is absent; then authenticate and confirm the full article arrives.
Search results lead mainly to a bare login page
Symptom: An incognito brand/service search surfaces a generic login page, or many private URLs collapse onto the same contentless login experience.
Likely cause: Private routes redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't. to one generic page with no service context, or robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. blocks the private URLs while still allowing bare URL indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed..
Fix and confirmation: Give legitimate login destinations unique contextual copy. For
genuinely private content, use authentication plus noindex or a purposeful login
redirect rather than robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. as an indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. control. Repeat the incognito search and
inspect representative URLs to confirm the result is informative and private URLs are
not appearing as bare entries.
Google can rank only the teaser
Symptom: The page is indexed but appears relevant only to the lead-in, not to the full article’s subject.
Likely cause: Googlebot receives the same short teaser as an unauthenticated reader, so the engine cannot understand the gated body.
Fix and confirmation: Implement flexible sampling so verified Googlebot can crawl the same full content a subscriber receives, declare the gated section with structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding., and validate the live page. Use URL InspectionA Google Search Console feature that reports how Google sees one specific URL on a property you own. By default it shows the last-indexed snapshot; a separate \"Test live URL\" mode fetches the current version. after recrawlCrawl frequency is how often a search engine comes back to re-fetch a page it already knows about. Popular pages that change often get refreshed many times a day; stable pages can go weeks or months between crawls — and you influence it indirectly, not by setting a dial. to confirm Google can render the intended article; ranking recovery is not an immediate validation signal.
Paywall launch checklist
Sampling and access model
- Choose metering or lead-in deliberately; do not inherit an arbitrary vendor default.
- If using a meter, test monthly sampling first and use Google’s 6–10 free articles-per-month range as a starting point, not a universal command.
- Monitor how often the paywall appears; Google says satisfaction degrades when it is shown more than 10% of the time.
- GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. can access the same full content an entitled reader receives under the declared flexible-sampling setup.
Markup
- The top-level
Article,NewsArticle, or otherCreativeWorkdeclaresisAccessibleForFree: falsewhen the work is gated. - Every gated section has a
hasPartWebPageElementwithisAccessibleForFree: false. - Each
cssSelectoruses an actual class selector such as.paywall, not an ID or complex descendant selector. - Multiple gated sections are separate, non-nested
hasPartentries. - Registration walls use the same paywall markup as paid access walls.
Delivery and privacy
- Entitlement is enforced server-side; logged-out HTML does not contain the hidden full article for JavaScript or CSS to reveal.
- The mobile response follows the same correct gating and sampling behavior.
- Genuinely private URLs use authentication and
noindexor a login redirectA redirect sends browsers and crawlers from a requested URL to a different one. An HTTP redirect specifically is a 3xx status code paired with a Location header; meta refresh and JavaScript redirects achieve a similar navigation without being a 3xx response themselves. Permanent redirects (301/308) are Google's signal the target should be canonical; temporary ones (302/303/307) aren't., not robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. as the privacy mechanism. - Login pages include useful, service-specific context rather than one generic page duplicated across every route.
-
noarchive/nocacheis present where cached copies must not expose gated text.
Pre-launch proof
- The live candidate validates in the Rich ResultsRich results (formerly 'rich snippets') are enhanced search listings — stars, images, prices, breadcrumbs, video thumbnails, and more — that Google and Bing build from structured data. They're a display feature, not a ranking factor, and eligibility never guarantees they'll show. Test as smartphone and desktop.
- A logged-out, JavaScript-disabled fetch does not reveal the full gated body.
- An authenticated session receives the complete article.
- An incognito brand/service search does not reduce the site to a bare login result.
- Analytics records meter consumption and paywall exposure without including private article text in event payloads.
Prove the paywall is declared and enforced
Paywalled structured-data test
- Test to run: Test the live URL in Google’s Rich ResultsRich results (formerly 'rich snippets') are enhanced search listings — stars, images, prices, breadcrumbs, video thumbnails, and more — that Google and Bing build from structured data. They're a display feature, not a ranking factor, and eligibility never guarantees they'll show. Test as smartphone and
desktop, inspecting
isAccessibleForFree,hasPart, and everycssSelector. - Expected result: Google parses the gated
CreativeWorkand each declared class maps to the intended gated section while GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. can access the full article. - Failure interpretation: Missing properties, selector mismatches, invalid nesting, or a teaser-only GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. response means the flexible-sampling declaration is broken.
- Monitoring window: Immediate after each template or paywall-vendor deployment.
- Rollback trigger: The production template stops declaring or exposing gated content correctly across the article set and cannot be fixed before broad rollout.
Server-side entitlement test
- Test to run: Fetch the same article logged out with JavaScript disabled, then fetch it in an authenticated entitled session; include a screen-reader check on the logged-out response.
- Expected result: Logged-out users receive only the intended sample and cannot find the gated body in HTML/DOM, while entitled users receive the complete article.
- Failure interpretation: Full text in the logged-out response means the paywall only hides content client-side; missing text after authentication means entitlement delivery is failing.
- Monitoring window: Immediate in staging and production after any paywall JavaScript, template, cache, CDN, or authentication change.
- Rollback trigger: Unauthenticated users can retrieve the full paid article, or entitled readers broadly lose access after the change.
Cache-control and private-URL test
- Test to run: Inspect the page’s robots meta and X-Robots-TagThe X-Robots-Tag is an HTTP response header that carries the same indexing and serving directives as the robots meta tag (noindex, nofollow, nosnippet, and the rest). Because it lives in the header rather than the HTML, it's how you control indexing for non-HTML files like PDFs, images, and videos. for
noarchiveornocacheas required, then inspect representative genuinely private URLs for auth andnoindexbehavior. - Expected result: Cached-copy controls are present on gated articles where intended; private URLs are protected and not relying on robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. alone to prevent indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed..
- Failure interpretation: Missing cache directives create a copy-leak risk, while a robots-only block can leave a bare private URL eligible for indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed..
- Monitoring window: Immediate after header, CDN, robots, or authentication changes; recheck the affected templates after deployment.
- Rollback trigger: A deployment exposes private content, removes access controls, or broadly makes private URLs indexable and cannot be corrected immediately.
Ongoing flexible-sampling metrics
Paywall display rate
- Metric: The percentage of eligible content views in which the paywall is shown.
- What it tells you: How restrictive the sampling model feels across visits; it is the exposure measure Google ties directly to user satisfaction.
- How to pull it: Divide server- or paywall-platform gate impressions by eligible article views, segmented by user cohort, acquisition source, and device.
- Benchmark / realistic range: Google says general satisfaction degrades significantly when paywalls are shown more than 10% of the time, generally exposing about 3% of the audience. Treat that as a caution ceiling and test against your own subscribers, business model, and article mix.
- Cadence: Weekly for abrupt configuration changes and monthly for the stable trend; this is a leading experience/monetization control.
Monthly free-article allowance and consumption
- Metric: The configured monthly free-article quota plus the distribution of how many free articles users consume before encountering the gate.
- What it tells you: Whether the meter gives readers enough sampling to understand the product while still reaching the subscription prompt.
- How to pull it: Use server-side meter or paywall-platform logs grouped by anonymous meter identity and month; report the configured quota alongside consumption percentiles.
- Benchmark / realistic range: Google recommends 10 articles per month as an exploration starting point and expects 6–10 per user per month for most daily news publishers. This is a starting range, not a mandate for every publication.
- Cadence: Monthly, matching the recommended meter period; review after deliberate quota experiments rather than reacting to daily noise.
Test yourself: Paywalls and SEO
Five quick questions on keeping gated content indexable without cloaking. Pick an answer for each, then check.
Paywalls (in SEO)
A paywall restricts some or all of a page behind payment, subscription, or registration. Google's supported way to keep that content indexable is flexible sampling: mark the gated part with structured data (isAccessibleForFree, cssSelector) so Googlebot can still read it — which is why it doesn't count as cloaking.
Related: Structured Data
Paywalls (in SEO)
A paywall, in the SEO sense, is any mechanism — a subscription, a one-time payment, or a registration/login gate — that restricts some or all of a page’s content from users who haven’t paid or signed up. The SEO question a paywall raises is simple: if regular visitors can’t see the full article, how does a search engine understand and rank it?
Google’s answer is flexible sampling. Publishers let GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. crawl the full text, then use CreativeWork/NewsArticle structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding. — isAccessibleForFree: false plus a cssSelector marking the gated section — to declare which parts are restricted. Because this is an explicit, disclosed exception under Google’s own guidance, serving GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. the full article while serving unauthenticated users a truncated or login-gated version is not cloaking. Cloaking is defined by intent to manipulate rankings and mislead users; flexible sampling is the opposite — a sanctioned, declared mechanism.
Flexible sampling has two flavors: metering (a quota of free articles before the paywall appears — Google suggests starting around 6–10 per month) and lead-in (showing a portion of the article, then gating the rest). It replaced the older First Click Free program, which Google retired in October 2017. Registration/login walls use the identical markup as paid paywalls — Google doesn’t distinguish “pay to access” from “register to access” at the schema level.
Related: Structured Data
Build-time retrieval analysis plus live signals for this exact article. The automatic chunk report includes a deterministic readiness score and is ready without a model download.
Search Console
sampleGA4 traffic (28d)
sampleCloudflare traffic (7d)
sampledCrUX field data (28d, phone)
sampleGoogle NLP entities
localChangelog
Updated Jul 18, 2026.
Editorial summary and recorded change details.Summary
Clarified that the paywall markup is a declaration, not a ranking guarantee, and corrected the required-vs-recommended split between isAccessibleForFree and hasPart/cssSelector.
Change details
-
Added explicit no-outcome-guarantee language (quoting Google's structured-data docs) so the markup isn't read as a promise of indexing, ranking, or a rich result — and clarified the markup is for content you want indexed under a declared gate, not a tool for hiding genuinely private URLs.
-
Corrected the structured-data section to state isAccessibleForFree is the required property and hasPart/cssSelector is recommended (not a second required property), per Google's live paywalled-content markup reference.
-
Added Google's own caveat that the 6-10 articles/month metering figure is a daily-news starting point, not a universal quota ("no single value for optimal sampling across different businesses").
Full comparison unavailable — no prior snapshot was archived for this revision.