Soft 404 errors

Usually, when someone requests a page that doesn’t exist, a server will return a 404 (not found) error. This HTTP response code clearly tells both browsers and search engines that the page doesn’t exist. As a result, the content of the page (if any) won’t be crawled or indexed by search engines.

In addition to returning a 404 code in response to a request for a page that doesn’t exist, the server will also display a 404 page. This may be a standard "File Not Found" message, or it could be a custom page designed to provide the user with additional information. The content of the page is entirely unrelated to the HTTP response returned by the server. Just because a page displays a 404 File Not Found message doesn’t mean that it’s a 404 page. It's like a giraffe wearing a name tag that says "dog." Just because it says it's a dog, doesn't mean it's actually a dog. Similarly, just because a page says 404, doesn't mean it's returning a 404. You can use Fetch as Google (or other tools available on the web) to verify whether the URL is actually returning the correct code.

Returning a code other than 404 or 410 for a non-existent page (or redirecting users to another page, such as the homepage, instead of returning a 404) can be problematic. Firstly, it tells search engines that there’s a real page at that URL. As a result, that URL may be crawled and its content indexed. Because of the time Googlebot spends on non-existent pages, your unique URLs may not be discovered as quickly or visited as frequently and your site’s crawl coverage may be impacted (also, you probably don’t want your site to rank well for the search query [File not found]).

We recommend that you always return a 404 (Not found) or a 410 (Gone) response code in response to a request for a non-existing page. You can improve the user experience by configuring your site to display a custom 404 page when returning a 404 response code. For example, you could create a page containing a list of your most popular pages, or a link to your home page, or a feedback link. You can also use the Webmaster Tools Custom 404 widget to add a search box and more site search options to your site. But it’s important to remember that it’s not enough to just create a page that displays a 404 message. You also need to return the correct 404 or 410 HTTP response code.