Google: The real problem with click fraud lies with the companies that report it

Google releases the results of an internal investigation into click fraud, and …

Google has stepped into the ring and started swinging. The company has for years been under assault by critics who contend that many of the clicks advertisers receive (and then pay for) are bogus. Third-party firms such as ClickFacts have sprung up to help companies cope with the problem of "click fraud," which by some estimates can range as high as 35 percent. The situation got bad enough that a group of advertisers sued Google last year, demanding that the search giant refund some of their money.

Google has just completed its own internal probe of the click fraud problem, and the report (PDF) pulls no punches. Google claims that its analysis found "pervasive reproducible problems" in the methodology used by these third-party testing firms, and goes so far as to state that "we have not yet discovered a single legitimate vulnerability as a result of a third-party click fraud auditing report."

Google claims that the main problem is so-called "fictitious clicks," events which are listed as fraudulent by the monitoring companies, but which do not show up as actual ad clicks in Google's own logs. The researchers attribute this to two problems. The first is that fictitious clicks sometimes appear because page reloads are interpreted as advertising clicks, even though Google tracks such things and does not bill for them. The second problem is that fictitious clicks may appear when a reporting firm conflates advertisers and ad networks, assigning clicks to the wrong reports.

While this addresses fictitious clicks, Google also looked at clicks that were marked as fraudulent but which do appear in Google's logs. What it found was that these so-called fraudulent clicks actually generated a similar conversion rate to "legitimate" clicks. "For example, in one case where 800 paid clicks were marked as 'fraudulent,' the rate of conversion for these clicks was 5.1%," the report says, "which compared favorably with the 5.8% overall conversion rate the advertiser achieved on approximately 24,000 paid clicks." This conversion rate suggests that few of these clicks were actually fraudulent after all.

Google goes so far as to name names. Appendix B of the report presents case studies of three third-party analysts—ClickFacts, Click Forensics, and AdWatcher. Google identifies major problems with the reports produced by all three firms. Regarding an Adwatcher report, for instance, Google says it "had such a huge number of fictitious clicks attributed to Google that the total number of fraudulent clicks reported by AdWatcher was ~12,000 for a period in June, while the total number of ad clicks that Google generated for the advertiser was only approx. 6,800 (of which 800 had been discarded by Google as invalid clicks, so [the] advertiser was charged for only 6,000)."

Mikhail Ledvich, Chief Strategy Officer of ClickFacts, tells Ars Technica that "Google is correct in citing the issues with our system. However, the report they dissect is based on February data, generated using the beta version of our system. After we submitted that report to them, they pointed out the issue to us, and we've since eliminated the problem."

Google is not a disinterested bystander, of course, but most of the criticisms in the report raise serious questions (as opposed to simply asserting that these third-party firms are inflating their claims). As Google notes in the report, advertisers have begun requesting refunds based on these click fraud studies, and Google hopes to reassure such advertisers that its own fraud detection systems are in good shape. While the report lays out a host of criticisms, it does not give Google's own estimate of average click fraud, only noting that third-party estimates show "much higher levels of click fraud than we believe could possibly be realistic."

Update

We were curious about the assertion that the ClickFacts data problems had all been cleared up this spring. Google's report made it quite clear that Google had examined several different reports from ClickFacts, and at least one of them appeared to be generated in June. We asked the company how that squared with the claim that their data errors had been fixed up long ago. They responded, "As soon as Google brought the issue to our attention we proceeded to adjust our software. All data is subject to interpretation, and that is the reason for third party companies, to give an objective evaluation of the situation based on facts."

We also asked why, if the system was in beta and prone to such errors, the results were given to news outlets like the Washington Post. ClickFacts responded by saying that they had no idea at the time that the reports were incorrect. Though the Google team expressed incredulity at the double-digit percentages of click fraud that ClickFacts and others were claiming, the company insists that click fraud is still a massive problem. Ledvich tells us that it's difficult to give a single estimate because the "fraud varies signifigantly accross different verticals and clients, and to give a single estimate would be misleading." But he does go on to say, "On average it's between 10 and 30 percent."