While investigating a traffic surge at The Quotations Page, I found something interesting: between 4:00 AM and 10:00 AM I received almost 10,000 hits from two particular IP addresses. Things like this happen every day, and it’s usually just a referral spammer or email address harvester. Today’s attack had two interesting qualities, though:

Every single request was for the robots.txt file. (9875 requests, about one every 5 seconds)