How to Fix URLs with Noindex in HTML and HTTP Header

How to Fix URLs with Noindex in HTML and HTTP Header
Fast Links

Free SEO Audit

Crawl the website for technical issues and get a prioritized to-do list with detailed guides on how to fix.

Something went wrong. Please, try again later.
Sitechecker crozdesk rating Sitechecker crowd rating Sitechecker capterra rating
Sitechecker trusted company

Noindex in HTML and HTTP header issue means that website has pages where a ‘noindex’ directive is specified in the meta robots tag and in the HTTP response header (X-Robots tag).

The Importance of the Issue

This guideline tells a search engine that you don’t want to index the current URL. Therefore, the page will not be indexed and will not take part in the ranking. When prohibiting indexing, you need to clearly understand that a user might need this page, however, it has no point for the search engine and you don’t plan on generating traffic from it.  For instance, a Thank you page after a form completion might become such a page.

How to Check the Issue

Using any browser is enough to check the issue. Open the source code of the flawed page. To do this, click the right mouse button at any spot of the page and choose “browse the code” option, or apply an online tool https://codebeautify.org/source-code-viewer.

Find a directive <meta name=”robots” content=”noindex” />. If the value of the content = noindex, there is an issue at the page.

To check the server’s response, use a tool https://redbot.org/ or any similar tool. The presence of an X-Robots-Tag: noindex in the server’s response means there is an issue at the page.

Detailed directive description 
https://developers.google.com/search/reference/robots_meta_tag?hl=en

Another way to check the problem is to use services like Sitechecker. It will scan your website and provide you with a list of issues, including pages where a ‘noindex’ directive is specified in the meta tag and in the HTTP response header (X-Robots tag).

Noindex HTML HTTP Header

Additionally, you will receive a list of all affected pages.

Noindex HTML HTTP Header Pages List

How to Fix this Issue

When using <meta name=”robots” content=”noindex” /> substitute noindex value for index.

When using the X-Robots-Tag: googlebot: delete noindex from the response

Note that it is necessary to check both options, as using one of them does not exclude the use of another one.

Please note! Suspending pages from indexation is common when promoting a website. A page can be suspended from indexation purposefully. Before changing the indexation settings, make sure that the current configuration is incorrect. Coordinate your activity with a SEO specialist, if necessary.

Detect pages with noindex directive

Crawl the website to collect all pages with noindex directive in HTML and HTTP header

Something went wrong. Please, try again later.
Sitechecker rating on crozdesk Sitechecker rating on crowd Sitechecker rating on capterra

Check Your Website for SEO Errors

Something went wrong. Please, try again later.
close