On-Page SEO Checker · one finding
Noindex directive is present
Recommended action: Remove noindex from pages meant to be indexed, or keep it only when exclusion is intentional.
Fix principle: Remove every unintended noindex from its confirmed owner; absence of noindex is sufficient, while an explicit index directive cannot override another noindex because restrictive directives combine.
Implementation for your setup
Confirm first: The final HTML or HTTP response contains a noindex directive, including one added only by client-side script during rendering. The page is intended to be an indexable canonical URL. A robots meta tag or X-Robots-Tag intended to keep the page indexable is neutralized by a conflicting directive elsewhere, because directives combine using the most restrictive rule. Robots.txt disallows the URL itself, or disallows a resource the page depends on to render, so the crawler cannot reach or fully evaluate the intended indexable page.
Do not apply when: Exclusion is intentional for the page type, environment, or workflow state. The directive appears only on a noncanonical or otherwise ineligible URL. Robots.txt does not disallow the URL or the resources it needs to render, and the effective combination of every directive already resolves to the intended indexable state.
Likely control points: cms-app · server · edge-cdn
Choose the layer that owns the final response.
Generic HTTP / HTML
Steps
- Follow redirects, identify the exact owner of each unintended robots/googlebot meta or X-Robots-Tag noindex, and remove it only at that owner.
- Return the intended canonical page with a crawlable 200 response, align its canonical, and purge the caches that can retain the directive.
- If a robots meta tag or X-Robots-Tag is present but rendered ineffective by a conflicting directive elsewhere, remove the actual unwanted restrictive directive rather than adding a permissive one to compete with it — the most restrictive rule always wins. Google also honors a robots meta tag placed outside <head>, so an out-of-head placement is a specification and cross-crawler risk to correct, not by itself proof Google is ignoring the directive.
- If robots.txt disallows the URL itself, or disallows a resource (such as CSS or JS) the page needs to render its content, correct only the specific Disallow rule at the robots.txt file that owns it — keep the rule as narrow as its original intent, and remember Disallow blocks crawling, not indexing: a disallowed URL can still be indexed if discovered elsewhere.
- Keep the URL crawlable so Google can observe the removal.
Verify on this platform
- After redirects and cache purge, inspect every robots and googlebot meta tag plus every X-Robots-Tag value on the final response.
- Fetch and parse robots.txt for every group that applies to the relevant user-agent — Google consolidates every group matching the same user-agent token into one effective ruleset, though not every crawler does — and confirm the URL and its critical rendering resources are not disallowed.
- Use Search Console URL Inspection’s live test on the production URL, then request indexing when appropriate.
Rollback
- Restore the prior directive or robots.txt rule if the page or a resource it needs was exposed or blocked unintentionally, then correct the underlying publication rule before trying again.
Caveats
- Do not add index as a counter-directive: the most restrictive directive applies, so another noindex still wins.
- The live test checks current accessibility; it does not guarantee selection, canonical choice, or indexing.
- Google’s own parser combines every robots.txt group matching the same user-agent token into one effective ruleset, but many other crawlers and Robots Exclusion Protocol implementations honor only the first matching group — treat a directive split across groups as a real interoperability risk even when Google itself is unaffected.
- If the page combines a canonical to a different URL with a noindex directive, remove the noindex only if this URL itself should be indexable; if the canonical target is the one meant to be indexed, keep the noindex here and verify the canonical target separately rather than treating its indexability as confirmed by removing this noindex.
Official sources
- Google: robots meta tag and X-Robots-Tag specifications verified
- Google Search Console: URL Inspection verified
- Google: canonical URL guidance verified
- Google: robots.txt introduction (Disallow blocks crawling, not indexing; blocking rendering resources) verified
- Google: how Google interprets the robots.txt specification (group consolidation, root placement) verified
WordPress
Steps
- If WordPress core globally owns the directive, confirm the scope and change Settings → Reading so blog_public permits search-engine visibility.
- For a scoped page-type or URL rule, identify and change the plugin or theme callback filtering wp_robots or invoking the core noindex behavior; do not invent a plugin UI.
- Purge page, object, host, and edge caches that can retain the old output.
Verify on this platform
- While logged out, follow redirects and inspect all robots/googlebot meta plus X-Robots-Tag values on the final public response.
Rollback
- Restore the previous visibility or plugin setting if unintended pages become indexable.
Caveats
- Do not use the global Reading setting to solve a scoped plugin or theme rule.
- WordPress exposes the wp_robots API, but a third-party owner’s exact UI is not verified here.
- Robots.txt is a separate control point from this meta/header directive — correct a Disallow rule using the generic-http-html guidance above regardless of which CMS serves the site.
Official sources
- WordPress: Reading Settings screen verified
- WordPress: wp_robots_noindex() verified
- WordPress: wp_robots filter verified
- Google: robots meta tag and X-Robots-Tag specifications verified
Cloudflare Workers / Rules
Steps
- Use Cloudflare Trace and configuration review to confirm which Transform Rule or Worker adds the final X-Robots-Tag and account for response-header rule ordering.
- Narrow or remove only that confirmed edge emitter; in a Worker, clone the response before modifying its headers.
- Purge the affected cache entries and leave any valid origin-owned robots header intact.
Verify on this platform
- Compare origin and final Cloudflare responses after redirects, then confirm every final robots header and meta directive is free of unintended noindex.
Rollback
- Restore the prior edge rule if the scope change exposes excluded routes, then replace it with a narrower condition.
Caveats
- Never blanket-delete a valid origin X-Robots-Tag when Cloudflare is not the confirmed emitter.
- Later response-header rules can override earlier changes, so verify the final rule order and response.
- Robots.txt is a separate control point from this response-header directive — correct a Disallow rule using the generic-http-html guidance above.
Official sources
- Cloudflare: response header modification verified
- Cloudflare: Trace a request verified
- Cloudflare Workers: alter headers verified
- Google: robots meta tag and X-Robots-Tag specifications verified
Next.js / Vercel
Steps
- In the App Router, remove the unintended noindex from static metadata.robots or generateMetadata at the route that owns it.
- In the Pages Router, correct the owning next/head robots meta; for an X-Robots-Tag, correct the scoped next.config headers() rule.
- Deploy to the intended production environment and purge or bypass relevant caches before verification.
Verify on this platform
- Inspect all final robots/googlebot meta and X-Robots-Tag values after redirects on the current production deployment.
Rollback
- Revert the route metadata, Head output, or headers() rule if the deployment exposes a route that should remain excluded.
Caveats
- Vercel intentionally serves X-Robots-Tag: noindex for Preview Deployments and outdated Production Deployments; preserve that protection.
- Do not diagnose an intentional preview response as a production metadata defect.
- Robots.txt (a static file or a route handler such as app/robots.ts) is a separate control point from this metadata/header directive — correct a Disallow rule using the generic-http-html guidance above.
Official sources
- Next.js App Router: robots metadata verified
- Next.js Pages Router: Head verified
- Next.js: headers() verified
- Vercel: preview deployment indexing protection verified
- Vercel: response headers verified
- Google: robots meta tag and X-Robots-Tag specifications verified
Platform-managed / ask the vendor
Steps
- Ask the vendor to identify the exact global or scoped control that emits noindex and remove it only for the confirmed public canonical URL.
Verify on this platform
- Follow redirects and inspect all robots/googlebot meta and X-Robots-Tag values on the final public response, then use the Search Console live test.
Rollback
- Restore the vendor visibility control if an unintended page becomes public while the owning rule is corrected.
Caveats
- Do not add index to compete with a noindex emitted elsewhere.
- Keep the URL crawlable long enough for search engines to observe the removal.
- Ask the vendor to confirm whether it also owns robots.txt for the domain — a Disallow rule there is a separate control point from a meta/header directive.
Official sources
How to verify
- Follow redirects and confirm a crawlable 200 final response with no noindex in any robots/googlebot meta or X-Robots-Tag value.
- Confirm the canonical points to the intended URL and relevant caches no longer serve the directive.
- Fetch and parse robots.txt for the applicable user-agent group(s) and confirm the URL and its critical rendering resources are not disallowed.
- Rerun SEO Site Audit Crawler, On-Page SEO Checker, Google Index Checker, Render Gap, or Robots.txt Tester (whichever surfaced the finding), and use Search Console URL Inspection’s live test for Google’s current view.
Escalate when
Escalate when page ownership or publication intent is unclear, noindex or a robots.txt Disallow rule is applied by multiple layers, or removing it would expose private, duplicate, or staging content.