Block access to content on your site

This article explains how to block access to content on your site.

Some of the content you publish may not be applicable for inclusion on Google News. You can restrict Google’s access to certain content by blocking access to Google's robot crawlers, Googlebot, and Googlebot-News.

Learn more about how to personalize Google News, including how to see more stories from preferred sources, and block news sources you don’t like.

Create a robots.txt file

Use a robots.txt file to get a high level of control over which parts of your site appear in Google Search and Google News. Learn more about robots.txt files.

You can block access in the following ways:

To prevent your site from appearing in Google News, block access to Googlebot-News using a robots.txt file

To prevent your site from appearing in Google News and Google Search, block access to Googlebot using a robots.txt file

Make sure you give our crawler access to your robots.txt file, so we can see if you've specified certain sections of your site you don't want crawled.

To prevent specific articles on your site from appearing in Google News, block access to Googlebot-News using the following meta tag: <meta name="Googlebot-News" content="noindex, nofollow">

To prevent specific articles on your site from appearing in Google News and Google Search, block access to Googlebot using the following meta tag: <meta name="googlebot" content="noindex, nofollow">

To prevent specific articles on your site from being crawled by all robots, use the following meta tag: <meta name="robots" content="noindex, nofollow">

To prevent robots from crawling images on a specific article, use the following meta tag: <meta name="robots" content="noimageindex">

To inform us that an article should be removed from the Google index at a certain time, use the following meta tag: <meta name="googlebot" content="unavailable_after: 25-Aug-2011 15:00:00 EST">

Specify the time and date in RFC 850 format. This meta tag is treated as a removal request. It takes about a day after the removal date passes for the page to disappear from the search results. However, for the tag to function properly, it must be included with your article when it’s first crawled.