Login

Crawl Track

Welcome to the second part of a two-part series that tests search engine reactions to the meta robots tag and robots.txt. In this part, the results of the experiment and testing we set up in the first part will be presented. For details about the experiment background, setup of the test pages and user-agents, please refer to the first part. You need to have read it very carefully to easily understand this one.Continue reading →