How to Fix URLs with Noindex in HTML and HTTP Header

How to Fix URLs with Noindex in HTML and HTTP Header

Free Complete Site Audit

Access a full website audit with over 300 technical insights.

Something went wrong. Please, try again later.
Trusted by
Sitechecker trusted company

Free Website SEO Checker & Audit Tool

  • Scan the site for 300+ technical issues
  • Monitor your site health 24/7
  • Track website rankings in any geo

Noindex in HTML and HTTP header issue means that website has pages where a ‘noindex’ directive is specified in the meta robots tag and in the HTTP response header (X-Robots tag).

The Importance of the Issue

This guideline tells a search engine that you don’t want to index the current URL. Therefore, the page will not be indexed and will not take part in the ranking. When prohibiting indexing, you need to clearly understand that a user might need this page, however, it has no point for the search engine and you don’t plan on generating traffic from it.  For instance, a Thank you page after a form completion might become such a page.

How to Check the Issue

Using any browser is enough to check the issue. Open the source code of the flawed page. To do this, click the right mouse button at any spot of the page and choose “browse the code” option, or apply an online tool https://codebeautify.org/source-code-viewer.

Find a directive <meta name=”robots” content=”noindex” />. If the value of the content = noindex, there is an issue at the page.

To check the server’s response, use a tool https://redbot.org/ or any similar tool. The presence of an X-Robots-Tag: noindex in the server’s response means there is an issue at the page.

Detailed directive description 
https://developers.google.com/search/reference/robots_meta_tag?hl=en

Another way to check the problem is to use services like Sitechecker. It will scan your website and provide you with a list of issues, including pages where a ‘noindex’ directive is specified in the meta tag and in the HTTP response header (X-Robots tag).

Noindex HTML HTTP Header

Additionally, you will receive a list of all affected pages.

Noindex HTML HTTP Header Pages List

How to Fix this Issue

When using <meta name=”robots” content=”noindex” /> substitute noindex value for index.

When using the X-Robots-Tag: googlebot: delete noindex from the response

Note that it is necessary to check both options, as using one of them does not exclude the use of another one.

Please note! Suspending pages from indexation is common when promoting a website. A page can be suspended from indexation purposefully. Before changing the indexation settings, make sure that the current configuration is incorrect. Coordinate your activity with a SEO specialist, if necessary.

Detect pages with noindex directive

Crawl the website to collect all pages with noindex directive in HTML and HTTP header

Something went wrong. Please, try again later.
Fast Links

You may also like

View More Posts
Mobile Font Size Issue
Site Audit Issues
Mobile Font Size Issue
Roman Rohoza
Jun 5, 2024
How to Fix URLs Where Canonical Tag is Missing
Site Audit Issues
How to Fix URLs Where Canonical Tag is Missing
Ivan Palii
Dec 13, 2023
Query Strings Containing Search & Filter URL Parameters
Site Audit Issues
Query Strings Containing Search & Filter URL Parameters
Ivan Palii
Apr 19, 2024
close