Early last month, webmasters here at The Reg noticed an unexpected spike in our site traffic. Suddenly, we had far more readers than ever before, and they were reading at a record clip. Visits actually doubled on certain landing pages, and more than a few ho-hum stories attracted an audience worthy of a Pulitzer Prize winner. Or so it seemed.

DAMN! Why couldn’t it have happened here? I’m about to get married, the extra traffic would translate to extra dollars and help me out quite a bit! :) The Register article continues:

Read more it gets MUCH BETTER…

As it turns out, much of this traffic was driven by the new malware scanner from AVG Technologies.

Then, in February, AVG paired LinkScanner with its anti-virus engine, which has about 70 million active users worldwide. The company estimates that 20 million machines have upgraded to its new security suite, AVG version 8, and this has already cooked up enough ghost clicks to skew traffic not only on The Reg but any number of other sites as well.

Adam Beale, who runs a UK-based internet consultancy, says that across his small stable of clients, traffic has spiked as much as 80 per cent on some sites. And this is more than just an inconvenience. After all, sites live and die by their traffic numbers. And net resources aren’t free.

This is ridiculous! On a site like ours, that feeds off of traffic, this is great (for me, not necessarily for ZDNet), but for most sites out there, this increase in traffic could lead to server downtime, network congestion, and might even force companies to by expensive load balancer devices and additional servers when the traffic really is NOT generating any more business for them. The Register continues:

“Although [the AVG LinkScanner] might be good for the security of users, it’s a real pain for website owners and webmasters,” Beale tells us, having blogged about this growing problem. “It’s causing people to think their traffic is increasing, costing those who pay for bandwidth, and wasting disk space with large amounts of unnecessary lines in log files.”

One of his clients, Beale says, normally pulls in 140GB of bandwidth a month, and for June, he predicts a 5 per cent jump.

When we spoke to AVG chief of research Roger Thompson earlier this week, he was unaware of these issues. But he defended the role of LinkScanner, which he designed while serving as CTO of Exploit Prevention Labs.

“There’s so much hacking activity going on the web. The only way to really tell what’s there is to go and have a look,” he told us. “I don’t want to sound flip about this, but if you want to make omelettes, you have to break some eggs.”

Holy crap, that is the single most irresponsible thing I have EVER heard a CTO of any company say. Unbelievable! On top of this, how much security is it really providing? It’s not like Anti-Virus or these fancy link scanners or anything like that have really lessened the impact of malware that much. We’ll see at DEFCON this year just how easy it is for attackers to morph malware into something that AVs do NOT pick up on. Funny… I just saw AVG’s corporate images:

Maybe it should be the other way around?

Back to the Register article:

But what about webmasters?

Webmasters deal with robot traffic and other rogue visits all the time. But this is a little different. In an effort to fool even the sneakiest malware exploits, LinkScanner does its best to imitate real user clicks - which means most webmasters are completely unaware of the problem.

At the moment, there is a way of filtering AVG traffic from log files. But it’s unclear whether this method would bag legitimate traffic as well. And Thompson suggests that - in the name of high security - AVG may make changes that prevent such filtering.

Can you believe the cojones on this guy? He’s basically flooding our servers with illegitimate traffic and then telling you that in the high name of security, you should bend over, present, enjoy it, and then PAY HIM FOR IT!

We’d like to thank our web community for bringing these challenges to our attention, as building community trust and protecting all of our users is critical to us. We have modified the Search-Shield component of LinkScanner to only notify users of malicious sites; this modified version will be rolled out on July 9th 2008. As of this date. Search-Shield will no longer scan each search result online for new exploits, which was causing the spikes that webmasters addressed with us. However, it is important to note that AVG still offers full protection against potential exploits through the Active Surf-Shield component of our product, which checks every page for malicious content as it is visited but before it is opened.