What Analytics can and can not do to help you find bot traffic

Understanding Analytics – every tool is as good as you know how to use it

Google Analytics (GA) has become a de-facto industry standard to analyse traffic. But many webmasters unfortunately don’t use GA to its full potential, and use it as the sole method to analyze their website’s performance. It’s important to both understand the limitations of GA and learn how to use it.

Website owners — or God forbid those buying or selling advertising space on a small scale — are often convinced they’re out of the woods and seeing the light because they’re well educated GA users or have an in-house GA guru (or at least a buddy who once met a guy that said he knew a lot).

Don’t fool yourself: GA will not solve all of your problems.

Use the right tool for the right job – as my grandpa’ used to say

Let’s take Steve.

Steve runs a blog about grafting cacti and succulents. He’s a nice chap and tries his best to engage his visitors. He pushes out a richly illustrated post based on his botanic experience every Monday afternoon.

Steve is quite happy with the data he’s learning from GA. Namely, his readers are mostly:

women in their mid 50s

with some college education

have an annual household income of 40-80K

interested in Home & Gardening / Home decor.

Life is good. Steve’s content is getting shared.

As months pass by and Steve becomes more familiar using GA. It shows ups and downs, with engagement levels periodically going downhill. He later on realizes that his site and later posts are in fact getting hit by Microsoft, Amazon and Google bots.

Now, let’s imagine Steve’s evil* twin. Scott is running a similar venture. But with a focus on making money.

His writings strongly revolve around generating traffic and then monetizing it. He does so through an advertising network that in turn sells this traffic to web-shops offering gardening tools and utensils. As per the popular model, the ad-network pays Scott based on the number of times advertisements are displayed to people on his gardening themed blog.

Scott is happy as long as people come to his site and view ads delivered through this advertising network. His business model implies that he relies heavily on the quality of these visits.

He’s also convinced that he’s safe from bot traffic. Steve explained him — while casually sipping a soy chai latte — how filtering bots with GA is simple!

But there are two issues with this. On the one hand side, the bots Steve may or may not see on his site can be very different from the bots Scott may see on his. On the other, Scott may not even realize his website is getting bot traffic, since he is not using the proper tool for that.

The common mistake – what GA can and more importantly can NOT do for you.

Google Analytics is a fantastic website analytics tool. Full stop.

We all use it and understand the ins and outs of it. And so does every fraudster. Or at least the ones born within the last hundred years and in this galaxy.

Any halfwit fraudster in online advertising knows how to fly under the radar, do a quick hit ‘n’ run, and monetize. Rinse and repeat. Since GA is a public tool, most of the time fraud will stay hidden from you as bots and methods keep evolving to make sure it stays that way. It was once true that all bots showed up as direct traffic that did not run any javascript. It no longer is.

Back to Scott and a real life example – the bad and the ugly

Let’s look a bit deeper into Scott’s operations. He decides to buy traffic from www.supercheap-high-converting-traffic-for-gardening-sites.com. It’s easy profit: it’s cheaper than the payouts he’ll get from his ad-network.

But Scott soon gets an email from the advertising network.

He’s baffled. The email complains that his traffic quality is sub-standard and threatens to cease partnership. The super-high-quality-gardening-traffic he’s buying seems (or so says GA) to originate from authority sites such as Pinterest or Better Homes and Gardens.

But they don’t. The ad-network realizes this and shows Scott the door. Scott learned the hard way that ticking the box in GA that reads “Exclude all hits from known bots and spiders” does not, in fact, protect him from harm.

With some homework, Scott could have learnt that www.supercheap-high-converting-traffic-for-gardening-sites.com doesn’t have the amount and quality of traffic they boast and that something’s up. And that isn’t even touching the reason why authority sites would send so much traffic to this www.now-not-so-supercheap-high-converting-traffic-for-gardening-blogs.com site.

Welcome to the world of online advertising – don’t be scared it’s just full of fraud…

This much is sure: wherever there is money to be made, there will be players who will find an easier way to make more. Fraud has spread its gnarly fingers in – wait for it – all aspects of online advertising where there is someone on the other end with an open wallet.

The types of advertising fraud ranges from unviewable stacked ads to full-fledged bot networks clicking ads like hard working bees, from affiliate fraud to re-marketing fraud with borrowed cookie profiles. The list just goes on, and on, and on.

How many of just the above-mentioned schemes do your analytics protection you from?

You can either do your due diligence manually if you have the time and know-how (which I really think Scott should practice more, mind you) or “find and use the sharpest tool for the given job”.

The best advice I can give is that when your adversary goes underground you should too. Use a sophisticated method to unveil sophisticated fraud schemes.

*Scott isn’t really evil, of course. He’s just a regular guy who is trying to make an extra coin or two from doing what he likes.