When you make a request for an address in a browser, a series of events happen in a matter of microseconds:
- You type an address with http or https and hit enter.
- The http stands for hypertext transfer protocol and is routed to a domain name server. Https is a secure connection where the host and browser do a handshake and send data encrypted.
- The domain name server looks up where the domain is pointing to.
- The request is passed to the host of the website.
- The page is requested from the host.
- If the host has a content management system (CMS), the request is routed through a database and the page is looked up and prevented. If it's a static site, the page is just found and displayed.
- In any case, the webserver responds with a code… 200 is a valid page revealed with no internal issues, and a 404 is an error that tells the recipient that the page was not found on the server. (There are plenty of other codes that define other issues… but we're sticking to 404 errors here).
We've written quite a bit about 404 pages because they're often an afterthought by companies – but they severely impact both their ability to rank well in search engines as well as being very frustrating to users who took the time to click-through to your site.
What Produces 404 Errors?
There are a number of reasons that your site may produce 404 errors:
- You changed your URL structure or redesigned your site and all the pages moved. In this case, you must redirect these pages to update search engines and provide a great user experience.
- You removed a page from your site that wasn't useful. In this case, I'd also recommend redirecting to a page that does exist and is relevant. If that page had backlinks out on the web, that would re-establish the new page's authority in your search rankings.
- There are hackers, bots, and scripts that look for susceptible, known pages in content systems that may give them a back-door into your site. You can typically ignore monitoring these… but it's a good idea to keep your CMS and plugins up to date to avoid getting hacked.
- Someone may link to your site but use the wrong URL in their link. If you can't get them to update the link, add a redirect so that you fix the use experience and maintain that search authority.
The infographic below also provides some great tools – including Screaming Frog, Ahrefs, and SEMRush's tools for identifying links WITHIN your site that would produce a 404 Error Page. However, I'd highly recommend that you monitor externally produced links as well. You can do that with Google Search Console and Google Analytics.
Pro Tip: Read my article on how to use Google Analytics to send you a report on 404 Error Pages where you can identify the origin of the bad page, schedule weekly or monthly reports, and work to apply the optimal solution.
How to Correct 404 Errors
In any case… when it's a search engine or a user, you should redirect your searchbot or visitor to provide an exceptional user experience and to maintain your backlink authority. As the infographic below illustrates, you can do this by:
- Redirecting the 404 error page to another relevant page.
- Restoring the missing page.
- Correcting the link within your site or externally. Sometimes it's difficult to get someone to update their links on a third-party site… but it's worth a shot!
Here's the animated infographic from SEO Sherpa that walks you through the basics of 404 Error Pages.