Practical Guide on What 404 Error Is, Why It Appears and How to Track 404 Pages

Wednesday December 20th, 2017

510042

Practical Guide on What 404 Error Is, Why It Appears and How to Track 404 Pages

(29 votes, average: 4.38 out of 5)

Table of contents

What is a 404 error?
Why does it appear?
How do you track down broken links and their redirections?
Can it cause harm?
Your own 404 page

What is a 404 error?
Why does it appear?
How do you track down broken links and their redirections?
Can it cause harm?
Your own 404 page

What is a 404 error?

“Error 404” (or “Page Not Found”) – is a standard HTTP status code that occurs when the server can’t find the document requested by a user. Basically, this is a page that doesn’t exist or broken.

How can they look like:

404
404 Error
404 Not Found
Error 404
404 Page Not Found
Error 404 Not Found
HTTP 404 Not Found

What causes a 404 error?

Your page was deleted from the website
The user has typed the URL address incorrectly
Page was moved and the redirection was configured incorrectly
The server malfunctions (it happens very rarely)

Ideally, there shouldn’t be any links that return 404 errors on the site (or external links). But let’s be realistic if the site has more than a thousand pages, the occurrence of a 404 error can’t be avoided. It can be a bit surprising though when you find a site with a small number of pages (which means that all broken pages can be traced manually) that contain three broken backlinks to articles in the “Latest News” section of the front page.

Significant increase in the number of 404 errors

It’s one thing when a site is bypassed, and search engine robots report on several dozens of errors, but what if this number increases by hundreds or even thousands? A prompt intervention and solution should be initiated if the number of 404 errors exceed 10% of the total number of pages on the website. There can be many reasons for the appearance of such a larger number of non-existent pages, but mostly they are related to problems with the code (for instance, these pages were created with the help of scripts and styles, and the styles are in the code of the template, and not in a separate file).
Another reason why robots bypass pages that have never been on the site is because of virus attacks (for example, placing doorway pages on the website). Even after its prevention and removal, robots will continue to bypass the created pages for some time afterwards, and this can lead to the appearance of numerous 404 errors. But, thanks to various tools, you can easily track conversions on such links.

How to track down broken links and their redirections?

Yandex.Metrica
To track broken links and redirections in Yandex.Metrica you need to go to “Visitor settings”. Just place this string in the counter code:

params:window.yaParams||{ }});

Also you should place this JS-code somewhere on the 404 page:

<script>

var url = document.location.pathname + document.location.search

var url_referrer = document.referrer;

var yaParams = {error404: {page: url, from: url_referrer}};

</script>

Here url – is the current address of the 404 page, and url referrer is the address from which it was accessed. This way you can not only track all the broken links, but also the pages on which they are placed.

Google Analytics
To track the redirections, you should place this code on a 404 page:

<script>

ga('send', 'pageview', '404.html?page='+ document.location.pathname + document.location.search +'&from=' + document.referrer);

</script>

Where document.location.pathname + document.location.search – is the URL of the page not found on the site;
document.referrer – URL of the page which the user was redirected from.

Yandex.Webmaster
To find these pages you need to select the section “Indexing” – “Searchable pages” – “Excluded pages”. Here you can select the pages that have the error using the “Error HTTP: 404” filter.

Google Search Console
You can also find errors using “Crawl” – “Crawl errors” – “Error 404” in Google Webmaster.

There is one more way to find them out. Just launch website audit and get rid of the broken links.

What is the danger of a 404 error?

There is an opinion that pages with a 404 error directly influence the indexing of the site and its position in the search results negatively, but in most cases this statement is wrong. What happens to such a page when the robot finds it? It removes it from the index (if it’s there, if not – the bot simply doesn’t scan it). This is natural because it is no longer informative and useful. But you need to remember the existence of SOFT errors. SOFT 404 is not an official response from the server, but rather a tag, assigned to pages by search engines after a crawl.
What is wrong with them and why do search engines make such decisions?

1. There is little or no content on the page (Google can consider it as 404, Yandex as substandard quality).
2. There is a redirection to the page, which does not meet the users’ request.
3. The non-existent page doesn’t provide either a 404 or 410 server response.

What to choose: 301 or 404?
After deleting the page, a new question arises: what is the next step? Do you configure a 404 server response or a 301 redirect? To determine this, each case has to be analyzed separately. For instance, a method which was previously successfully used for a web shop might not be a good choice for a site that offers services. But how to fix 404 error? Configure a 404 server response if, for some reason, a page is deleted completely from the site.
301 redirect is better when:

The address of the page was changed (e.g. because of structural changes);
A page is relevant to the users’ request;
Redirection to another page with a similar content is possible (has the same theme or when going from an item page to a catalog);
There are external links to the page.

301 redirect to the homepage is not the best solution for both users (will cause confusion) and SEO (for example, Googlebot considers them as SOFT 404).

404 error and behavioral factors
Remember the feeling when you visit a site in search of something, and instead of finding what you’re looking for you discover a 404 error page. What does the user usually do in this case? Closes the tab and searches for a new site of course. Naturally, the more pages on the site that show a 404 server response, the more users experience this error, the less time they will spend on the site and more rejections will occur. The search engine algorithms function in a way that special attention is paid to behavioral factors as they play a very important role in the ranking process. To avoid worsening behavioral factors, you should motivate users to stay on the site. To do this you should approach the issue of creating 404 page in an interesting and creative way.

Your own 404 page

If you want users to stay on the site, you should think about creating and displaying your own 404 page once an error occurs. In most cases, webmasters do not pay much attention to this solution.

Page objectives:

Attract the users’ attention;
Explain what has happened;
Suggest a solution the problem.

When speaking about your own 404 page, search engines recommend the following:

Yandex:

Its design should look different from the other pages of the site;
You should use different colors or even do not use graphics at all when creating a page.

Google:

The page should be created in the same style (including navigation) as the main website;
You should place the links to the most popular articles and a comment section on this page;
Provide your users with the opportunity to report broken links.

General recommendations:

It should contain a link to the main page;
You should inform the user in a polite way that the page he/she wanted to visit is not available.

The recommendations above are not unconditional rules for 404 pages. It all depends on the theme of the website and the target audience. For a web shop it is relevant to add a search line, and for site that offer services – contact information.

Hotjar 404 error

GitHub 404 error

Facebook 404 error

Ahrefs 404 error

Semrush 404 page