Wednesday, December 04, 2013

Smartphone crawl errors in Webmaster Tools

Webmaster level: all

Some smartphone-optimized websites are misconfigured in that they don't show searchers the information they were seeking. For example, smartphone users are shown an error page or get redirected to an irrelevant page, but desktop users are shown the content they wanted. Some of these problems, detected by Googlebot as crawl errors, significantly hurt your website's user experience and are the basis of some of our recently-announced ranking changes for smartphone search results.

Starting today, you can use the expanded Crawl Errors feature in Webmaster Tools to help identify pages on your sites that show these types of problems. We're introducing a new Smartphone errors tab where we share pages we've identified with errors only found with Googlebot for smartphones.

Not found errors and soft 404s: A page can show a "not found" message to Googlebot, either by returning an HTTP 404 status code or when the page is detected as a soft error page.

Faulty redirects: A faulty redirect is a smartphone-specific error that occurs when a desktop page redirects smartphone users to a page that is not relevant to their query. A typical example is when all pages on the desktop site redirect smartphone users to the homepage of the smartphone-optimized site.

Blocked URLs: A blocked URL is when the site's robots.txt explicitly disallows crawling by Googlebot for smartphones. Typically, such smartphone-specific robots.txt disallow directives are erroneous. You should investigate your server configuration if you see blocked URLs reported in Webmaster Tools.