"Excluded by noindex tag": find and remove the noindex your app launched with

Updated Oct 27, 2026 · Pewterkit Guides

If Google won't list your homepage and Search Console reports the page as excluded by a "noindex" tag, your site is telling search engines not to index it, usually through a <meta name="robots" content="noindex"> line, an X-Robots-Tag: noindex response header, or a robots.txt rule left over from a staging setup. Remove the rule from the live site, then ask Google to recrawl the page with Search Console's URL Inspection tool.

Where a noindex hides

Google's documentation names two ways to say noindex:

  • In the HTML: <meta name="robots" content="noindex"> (or <meta name="googlebot" content="noindex">) in the page's <head>.
  • In an HTTP header: X-Robots-Tag: noindex, often set in hosting config (vercel.json, netlify.toml, _headers, a Cloudflare rule) or by a framework setting for previews.

Check both:

curl -sI https://www.example.com/ | grep -i x-robots-tag
curl -s https://www.example.com/ | grep -i 'name="robots"'

A related launch mistake is a robots.txt that blocks everything:

User-agent: *
Disallow: /

That stops crawling of the whole site. Google also notes a noindex only works when the page "must not be blocked by a robots.txt file", because a blocked page is never fetched, so Googlebot never sees the noindex.

The fix

  1. Delete the noindex meta line from index.html (or the layout or SEO component that adds it).
  2. Remove any X-Robots-Tag: noindex header rule for production. Keep it for preview or staging hosts if you want those out of Google.
  3. Replace a Disallow: / robots.txt with one that allows crawling and names your sitemap:
User-agent: *
Allow: /

Sitemap: https://www.example.com/sitemap.xml
  1. Redeploy, rerun the two curl lines, then in Search Console use URL Inspection on your homepage and request indexing. Google says that without this "it may take months for Googlebot to revisit a page."

In AI builders

Ask your builder: "Search the project for noindex, X-Robots-Tag and Disallow: / and remove them for the production site. Show me the lines you changed." A rule added while the site was in preview ships with the same code when it is published, so check after every launch.

What Pewterkit Check looks for

A free check reads your homepage's meta robots tag and X-Robots-Tag header, and your robots.txt. It flags a noindex in either place, and a robots.txt that disallows the whole site for every crawler.

FAQ

What does "Excluded by noindex tag" mean?

Google's help for the Page indexing report describes this status (listed there as "URL marked 'noindex'"): when Google tried to index the page "it encountered a 'noindex' directive and therefore did not index it."

I removed the tag. How long until Google lists the page?

Request indexing with URL Inspection to speed it up. Without a request, Google says a revisit can take months.

Should I also block the page in robots.txt?

Use one or the other. A page blocked in robots.txt can't be fetched, so Google never sees its noindex.

How do I keep previews out of Google but the live site in?

Send X-Robots-Tag: noindex only on preview hosts, and nothing on your production domain.

Sources

  1. Google Search Central: Block search indexing with noindex — read 27 Oct 2026: meta robots and X-Robots-Tag, robots.txt caveat, URL Inspection
  2. Search Console Help: Page indexing report — read 27 Oct 2026: noindex status explanation

How this page was made: drafted with AI from the sources listed, reviewed by Pewterkit. Docs checked Oct 27, 2026. Spotted something out of date? Tell us.

Related guides