Computer repair, tech and Norton Internet Security renewals

Should I block the Yandex bot?

Should the Yandex Russian Search Engine Bot be blocked from spidering your pages? I believe it should. I did some serious testing using server logs and honeypots and this bot currently does not respect robots.txt files. Worse still it applies such a server load that it must be contained.

I initially had this code (amongst others) in the .htaccess file:

1

2

3

4

5

6

7

SetEnvIfNoCase User-Agent"^Yandex bot"bad_bot

&lt;Limit GET POST&gt;

Order Allow,Deny

Allow from all

Deny from env=bad_bot

&lt;/Limit&gt;

but because it’s a persistent little critter I now have this as the first line:

1

SetEnvIfNoCase User-Agent"^Yandex*"bad_bot

If it bothers me any more, I shall start to fight back and start a campaign against it. One of our team got so fed up he did this: