Using robots to block Google News

We understand that news organizations publish lots of content and not all of it may be right for Google News. Google News crawls with the same robot as Google Web Search, called Googlebot.

Google Search and Google News support two different 'bots', namely Googlebot and Googlebot-News, that you can use as meta tags or in your robots entry to control where your content appears.

In other words:

If you block access to Googlebot-News, your content won't appear in Google News.

If you block access to Googlebot, your content won't appear in Google News or Web Search.

Note that Google respects the more restrictive interpretation of your bots choice.

Creating a robots.txt file

Using a robots.txt file gives you a high level of control over what parts of your site appear in Google Search and Google News. You'll find a comprehensive guide to creating and maintaining robots.txt files at our Webmaster Help Center.

Please note:

To prevent your site from appearing in Google News, block access to Googlebot-News using a robots.txt file.

To prevent your site from appearing in Google News and Google Search, block access to Googlebot using a robots.txt file.

Be careful to provide our crawler access to your robots.txt file, so we will know if you've specified certain sections of your site you don't want crawled.

Creating a meta tag

Rather than use a robots.txt file to block crawler access to areas of your site, you can add a meta tag to an HTML page to tell robots not to crawl specific pages. This standard is described at our Webmaster Help Center.

Please note:

To prevent specific articles on your site from appearing in Google News, block access to Googlebot-News using the following meta tag:

<meta name="Googlebot-News" content="noindex, nofollow">

To prevent specific articles on your site from appearing in Google News and Google Search, block access to Googlebot using the following meta tag:

<meta name="googlebot" content="noindex, nofollow">

To prevent specific articles on your site from being crawled by all robots, block access using the following meta tag:

<meta name="robots" content="noindex, nofollow">

To prevent robots from crawling images on a specific article, block access using the following meta tag:

<meta name="robots" content="noimageindex">

To inform us that an article will expire at a certain time, at which point it should be removed from the Google index, you'd use the following tag:

The date and time must be specified in the RFC 850 format. This information is treated as a removal request: it will take about a day after the removal date passes for the page to disappear from the search results. However, in order for the tag to function properly, it must be included with your article at the time that it is first crawled.

Using HTTP Header Specifications

You can also provide robots instructions in the HTTP header. Please visit Google Developers article on HTTP header specifications for more information.

How helpful is this article:

Contact us

The following articles may help you resolve your issue right away. If the articles don't address your issue, try the contact options below. You can quickly return to this page after visiting an article.

Still need help? To contact the support team, click below.

Oops! Looks like something went wrong. Please try again.If you still get an error, please try again in a day or two. We're on it.