Competitive Intelligence Analysis: Why, What & How to Choose

We already have a lot of data at our disposal in the world of Web Insights. Clickstream, Outcomes, Surveys, Usability etc. That in itself is hard to wrap our brains around.

But there is one more really great source of data out there that I am sure some of you are already tapping into, perhaps not as many of as that could benefit from it. I am of course talking about the data that falls broadly into the category of “competitive intelligence”.

The most compelling reason to tap into competitive intelligence is that in many ways it allows you to step away from your silo, the existence that is pretty much defined by your web analytics tool or data warehouse or all things connected to your website. In optimizing our website and experience and expenses based on just our website data we might not be optimizing for the overall business landscape.

Why should you care?

Revenue for your e-commerce business, selling DVD rentals, is up 30% year over year (YOY). Cause for celebration? Maybe or maybe not, it would depend on how the overall DVD rental business is doing on the web. If it is up YOY by 100% then that is not so great.

You have a cute pet mascot and the TV ads for dog food you run have increased traffic by 100% in two months. What is the impact of that, your ads, on your chief competitor?

After spending half a million dollars on a SEO (search engine optimization) project over six months you have increased your traffic from top five key phrases by 20% (that is huge). If the number of people searching are the same is that increase the expense of your affiliates or competitors or the fact that in those six months traffic on the web increased in your category by yyy%?

An industry rag reports that your competitor has been eating your breakfast, lunch and dinner. You look bad. To save your job you need to find out if your competitor has started new kinds of campaigns (say affiliate) or they have started advertising on websites you don’t know of or have massively ramped up PPC spending or targeted certain new demographic.

You are looking to provide true business strategic insights in a culture where web analytics suffers its life as a “reporting” function. Competitive intelligence provides an option to get ahead of the business with some game changing actionable insights.

These are very simple scenarios but in each of these competitive intelligence is key to providing the answers that you need. It helps you understand your performance in the context of the greater web eco-system and allows you to better understand causality due to “eco-system trends” vs. your actions (or lack there of).

It is pretty easy to celebrate success (or sometimes failure) of our websites based on just our numbers (Omniture, ClickTracks, WebTrends, HBX etc), true delight comes knowing how you are doing vis a vis your competitors or industry as a whole.

A search of the phrase web competitive intelligence yields 45 million results. I am sure there are that many ways to get competitive intelligence. : ) The focus of this post will be on two of the “big boys” in this space: ComScore and HitWise.

[A poor man’s option for basic competitive data is Alexa. I use it on my blog goal tracking page. Alexa collects its data via folks who install its tool bar and the data is extremely basic for this reason I am not including it.]

HitWise and ComScore are radically different services, as we’ll outline below, and you should be very careful in your choice and ensure that you are choosing the right one. Success of your valuable dollars invested depends on choosing the right service for your company, and of course then diving in and playing James Bond. : )

How do they capture data?

At a summary level HitWise is “ISP based” and ComScore is “Panel based” in terms of data capture (vastly different ways of collecting data).

HitWise has agreements with ISP’s worldwide whereby the ISP’s share the anonymous weblog data collected on the ISP network with HitWise. This data is analyzed by HitWise. They also combine this data with a worldwide opt-in panel to get demographic and lifestyle information.

ComScore on the other hand has a panel of people who opt in to be 100% monitored as they surf the web (by ComScore installing monitoring software on their Panel Member’s computers and then funneling 100% of the surfing via their proxy servers). In exchange for being monitored the Panel Member gets one (or combo) of these benefits:

Server-based virus protection

Attractive sweepstakes prizes

Opportunity to impact and improve the Internet

ComScore’s service is very much modeled after the US Neilsen TV ratings system.

According to HitWise they have roughly 10 million US and 25 million worldwide users that they get data for.

Per ComScore their global network is 2 million (though the Media Metrix audience measurement is 120k US panelists and Media Matrix Global services is 500k outside the US).

Cornell University has a in-depth outsider view of how ComScore collects its data, please Click Here. I recommend the section titled: Exactly What Does MarketScore do? Quick five minute read and quite illuminating. (I looked for a rebuttal from ComScore to the Cornell University article / concerns, I could not find anything via google or on ComScore’s website. If you know of a webpage with such info please email me and I’ll add a link to it here.)

Benefits of using HitWise:

The sample size is a huge benefit, multiple times that of ComScore.

The basic data capture mechanism means they have a much more diverse pool of people in their data. (For example I would personally never agree to ComScore monitoring in exchange for sweepstakes or virus protection because they will take 100% of my data, including any credit card information and what I buy and what sites I surf and my social security number and logins and everything that they capture via their proxy servers. This means I, Mr. Valuable Internet Participant, will never be in ComScore, but there is no way for me to avoid HitWise but atleast it is 100% anonymous and HitWise has no access to my https, hence private, data.)

HitWise has much deeper and richer search traffic data.

The psychographic (demographic, lifestyle) data HitWise provides is via the “Prism” database which is a significantly better than self reporting of such data.

HitWise has a lot more on-demand reporting available through their web access interface, very much amenable to self service.

Benefits of using ComScore:

ComScore has deep data from their Panel and they can go really deep in terms of website data.

ComScore can provide metrics such as Conversion Rates or Purchasers. They have all the traffic for their panel and they apply aggregations and sophisticated computations to approximate these metrics.

ComScore can break down some websites into deep embedded pages, for example they can measure microsoft.com/office/avinash/awesome.html, a potential page that could be embedded deep into some directory and can’t be tracked effectively by others from the outside.

ComScore can do more custom work on your behalf because they have every page, http or https, from their Panel Members and all the data associated with that Panel Member and their surfing habits.

Both companies state that they have the highest of protections of customer data and their privacy policies explicitly state what they will do with the data.

Give me a sound bit for each service:

HitWise is more suited as a marketing tool: Acquiring new customers, benchmarking performance, measuring search campaign effectiveness and what competitors are doing.

ComScore is more suited for decision making in advertising: how many people go to each site each month for their panel, and from which site to which site and deeper site behavior (conversion).

Which one should I use?

It depends. : ) See immediately above.

But I do have one guidance: If your website gets more than one million Unique Visitors a month for similar metrics you can use ComScore and trust the data. If you get less than one million Unique Visitors a month you are better off with HitWise.

The reason is simple: Each service uses its ok statistical computation and aggregations to approximate “representative” all world internet usage. There is a much higher probability of error for smaller sites, less than a million, due to these statistics and weighting as they are applied to show “real world behavior”. You are better off with the HitWise sample of millions of visitors than the ComScore sample of hundreds of thousands.

As you decide please balance for sample bias in case of ComScore because of the kind of Panel Members who’ll be ok with 100% monitoring in exchange for sweepstakes or anti-virus protection vs. the benefit of depth at a site level data that ComScore certainly has.

Hopefully this post makes a case for using Competitive Intelligence data and the kind of value it can add to your organization. My next post on Thursday will go deeper into the specific kinds of metrics you can report and analysis you can perform once you have access to this data.

Agree? Disagree? Have you seen other benefits of Competitive Intelligence? Was it a waste of time? Did this post miss anything for or against this type of analysis or the two vendors? Please share your feedback via comments.

Comments

Avinash,
Great post. Thanks very much for the info on HitWise and ComScore. The link to Cornell's policy regarding ComScore is really instructive.

Given that ComScore has access to all kinds of secure data of its participants, Cornell's position seems like a no-brainer: It cannot allow members of the university community to use ComScore without violating its own mission of protecting that data. So it's surprising that every college and university has not taken the same position.

Following up on my own comment… (The full impact of ComScore use is still sinking in, I guess.) I'm thinking of the fact that my wife and I do roughly 90% of our banking online: bill payments, deposits, transfers, etc.

I cannot imagine that any of our banks would want us to use ComScore. Why aren't banks and credit card vendors — or any other institution whose members transmit secure information, for that matter — going ballistic over ComScore use?

Great post. This is the "new territory" of web analytics I have been exploring in the last year. This refers to what I call the "site ecology" or ecosystem. Since everybody knows that "competition is one click away" how does one go about analyzing that competition and its pressure on the site? Personnaly, I have a sweet spot for Hitwise; I am pretty impressed by the information they can provide. It's a fantastic addition to the portait we produce with the behavioral and attitudinal analyses.

I have found Hitwise to be a valuable service, but its something that we've only needed to look at a couple of times a year and therefore we could never justify purchasing it.

Panel based services have a whole host of accuracy issues associated with them. Mark Belkin had a nice post recently about panel based services and some of the difficulties they encounter. In my experience, if your site gets a lot of traffic from the at work crowd then you are likely to run into accuracy issues with panel based services regardless of how large you are.

Chris: We have had similar experiences to you with HitWise, so we didn't renew with HitWise this year.

However when we did have it we developed a pretty cool calculation by taking its Search Volume numbers plus Key phrase Search Visibility numbers from http://www.seireports.com to show clients the estimated traffic potential numbers.

Avinash,
Do you know of any products which provide accurate Search Volume numbers for keywords(specifically historical) besides HitWise and Overture?

Chris/Manoj: CIA is something that you have to have a fit for. (Sounds cool saying CIA, I mean Competitive Intelligence Analysis of course.)

There is massive amount of data in HitWise (or ComScore) that can add incredible value to any organization. I have found though that a critical self diagnosis is important before purchase. Two criteria:

1) Are there resources smart enough from a business perspective to dig into the data and find relevant gold (relevant being the important word in the world of CIA)?

2) Is the business at a level of maturity that they can action the insights that are found from the data?

If the answer to any of this is "maybe" then clearly the toll will not add any value and a company is better off spreading that around as a bonus to employees. : )

(Manoj: The answer to your search volume question is complex and the answer is "it depends". Data capture by each tool is done so differently that it is hard to answer the question without asking you a couple more questions. In my humble experience I have found HitWise to be most optimal for our search trend needs.)

I stumbled onto your post on Competitive Intelligence. Very well written, thank you. I am familiar with both services you mentioned. However, take a look at Compete at compete.com and competeinc.com. We have been using it extensively and feel their data quality and intelligence to be far superior.

Great post Avinash. This is an excellent assesment of both Hitwise and ComScore.

I must say that I am in agreement with Seb, the previous commenter. Compete combines the positive aspects of Hitwise and ComScore into one service. They also offer free analytics, check it out snapshot.compete.com/hitwise.com+comscore.com+compete.com+

Very interesting post. I am wondering how Compete.com and quantcast.com integrate to this market. Do they give enough information to allow us to give up comScore or Hitwise? Will they do it in the future? How do they going to make money if they don’t sell anything?

You might also want to look at Syntryx Executive Marketing Solutions for more affordable and yet more comprehensive online marketing intelligence option, especially for affiliate networks and SEO purposes. Oh, and you might also want to know that Syntryx reports will include contact details and what site/network each prospect is already working with. I don't think any other supplier in this field provides that type of rich info on over 25 million sites.

I like your point about using the data for personalisation (and qualitative studies). This is on my wishlist to present content to the customer (e.g. main benefits of a product) in a way that is tailored to their segment, rather than presenting/selling in the same way to all…

What about websites with less than 10,000 uniques a month? Hitwise reports show something around 15 people visiting our site. And our competitors are around there. Are there any speciallity 'niche' competitive intelligence sites for sites like ours?

We're needing to see trends with our competitors and Hitwise is just tracking yeilds are too small for us to base any worthwhile actions into it.

Regarding the use of ISP data (which is the basis of data from Compete, Hitwise and Quantcast), I want to point out some issues that have not been covered yet in this discussion. Here in the US, we are now in a situation where concerns about privacy (fueled in part by the flap over the use of ISP data for ad targeting purposes) have caused virtually all of the ISPs to decide to not sell their clickstream data to third parties. And, it's clear that this creates major problems of representivity in any database that is based on the limited ISP data that are now available to research companies. I covered this issue in a blog posting a while back http://www.comscore.com/blog/2007/06/lessons_about_isp_data.html
The topic was also covered in the Wall Street Journal http://blogs.wsj.com/numbersguy/?s=fulgoni&x=6&y=10

Some have suggested that United Online / NetZero is one of the only remaining ISPs to sell their data today. If so, this would be a problem because they are mainly a dial-up ISP and surely no-one would accept the behavior of dial up users as being representative of the Internet user population.

It would be helpful if Compete, Hitwise and Quantcast were to identify the source of their ISP data, but they refuse to do so.

Hi,
I am very interested in this thread of comments, as I have been myself involved in such a competitive analysis lately.

I understand the need for user centric methods like ComScore's, but though the Consumer may be very precisely depicted, I still cannot accept the absolute values as reliable, considering the gap between a panel of a few thousand homes and the amount of websites in the world (NetCraft records more than 170 Million of them in August 2008, 70 Million being active…)…

As a former CEO of Information Resources, Mr Fulgoni will certainly agree that nothing is better than a census data collection; hence, the Hitwise network centric method should be highly superior when it comes to measuring the web activity…

But why then is Hitwise so bad when it comes to depicting small-sized sites? in the UK, they do not even publish one single absolute figure…

Then they do not have census? or not a sufficient sample at least?

Is there a way then to create a more reliable network centric collection and reporting? Is it needed anyway?

A very intresting post and still very well ranked since the time it was written. After evolution of Hitwise and ComScore there are these other niche competitor/competitive intelligence tools like adthena.co.uk, Adgooroo.com, Spyfu.com and others. I wonder how accurate are the reports and analysis produced by these tools and if consumers have benefited from using them.

Overall though you won't really get good signal for small sites. The article above outlines how these tools collect data, and despite their very best effort (none of them are at any fault) they can't represent CI data for small sites to any degree of accuracy. You are better off using them to learn what bigger players in your space are doing, how well or how badly, and learn from them what you can do better.

Quantcast is another option – however, as already mentioned smaller sites are going to, either not be mentioned or be a projection.

One way you could estimate your ranking proxy is by comparing your traffic (that you know probably based on Google Analytics) to the numbers on the smaller sites that do get tracked in one of these ranking sites.

Thanks for this post Avinash and the older one comparing Hitwise versus ComScore, very helpful as I'm just looking into purchasing some CI software for my company.

Do you have a view in terms of the paid version of SimilarWeb versus ComScore (Hitwise doesn't cover the international markets I am interested in)? ComScore gets a lot of mentions around the web as one of the big players but SimilarWeb seems to be a lot newer and it is harder to find feedback on it.

Considering the quite different price points, my concern is that by going with the cheaper option, it isn't as reliable.

Trackbacks

Avinash Kaushik's posts are always well thought out and well written – Competitive Intelligence Analysis: Why, What & How to Choose is his latest post and has real gems of insight such as: " If your website gets more than one millio…

I have a hard time with all the measurements I get from different Market Research companies. Data from Nielson NetRatings appears to be measuring audience while HitWise appears to be looking at total search volume (in the US mainly). …

[…] There are numerous posts online about how the various online measurement firms present very different views on things like unique users and page views. A few of the better ones I've read include Fred Wilson's 'Whose Numbers are right?', Donna Bogatin's 'Data Attraction: Hard science or numbers game?' and Avinash Kaushik's detailed analysis of how Hitwise and Comscore get their data. I've even commented on Alexa's fallibility w.r.t. Judy's Book Traffic. […]

[…] • Alexa Traffic can be used as a competitive intelligence tool but what you should take into account, is the fact that the audience sample size can be easily labelled as small; Your only task is to enter your competitor's site URL in the "Compare Sites" section and assess the results of your web marketing efforts in comparison with your competitors'. […]

[…] One possibility for Blockbuster would've been to use its "employee discount" leek as mentioned at HackingNetflix.com. To capitalize on its competitor's conversion problems, they only needed to purchase PPC ads for "Netflix coupon," where they could've offered Netflix customers a Blockbuster coupon on a customized landing page. As of today, only five ads currently appear on Google for that search term.

Moral of the story: Follow your competitors. Look for their conversion mishaps, and capitalize on them. […]

[…] Alexa Traffic can be used as a competitive intelligence tool, but you should take into consideration the fact that the audience sample size is fairly small. Just enter your competitor's site in the "Compare Sites" section and measure the results of your web marketing efforts in comparison with your competitors'. […]

[…] Alexa Traffic can be used as acompetitive intelligence tool, but you should take into consideration the fact that the audience sample size is fairly small. Just enter your competitor's site in the "Compare Sites" section and measure the results of your web marketing efforts in comparison with your competitors'. […]

[…]
Avinash Kaushik is a web analytics expert, evangelist, blogger and author who has some really great posts on his blog about competitive intelligence tools. His blog is called Occam’s Razor. His postings are tutorials that can help you to extract some great insights about a competitor’s online activities.

I highly recommend that you read each of these postings. The comments in each of the postings are informative as well.

[…] In order to answer questions 1 and 2 it is critical to know how well other players (especially competitors) are doing, since comparing just to your own numbers will leave you in bubble without context for question 1 and without best practices for question 2. Find more details in Avinash's post on competitive intelligence. […]

[…]
Alexa Traffic can be used as a competitive intelligence tool but you should take into consideration the fact that its sample size of audience is too small; Just enter your competitor’s site in the “Compare Sites” section and measure the results of your web marketing efforts in comparison with your competitors’.
[…]