Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspxRecently Microsoft announced that they’re releasing a new XBox which was apparently big news and was reported on at length around the globe. One article in particular on the BBC News website caught my attention because it contained what I thought to beenCommunityServer 2.1 SP2 (Build: 61129.1)re: Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspx#49469Thu, 06 Jun 2013 09:06:47 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:49469Adam Adshead<p>Hi Jamie,</p>
<p>I logged on to What Does The Internet Think and searched for “xbox” like yourself and received quite different results:</p>
<p>Negative: 39.5% (93k+ hits)</p>
<p>Positive: 54.1% (128k+ hits)</p>
<p>Indifferent: 6.3% (15k+ hits)</p>
<p>If you ran this query the same day you posted that tweet (22nd May), it means the number of tweets has increased over 5 times in 26 days.</p>
<p>I think this wide variation adds to your point that you can’t just cherry pick tweets, as given such swinging opinion you could probably find a tweet that supports any argument. Also when data is aggregated over large volumes of data, allowing some drill down capability would provide the reader more confidence in the results.</p>
re: Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspx#49470Thu, 06 Jun 2013 09:33:55 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:49470jamiet<p>Hi Adam,</p>
<p>Ah that's interesting. I believe the screenshot above was taken the day after the XBox Reveal event, which would make it 22nd May, as you say. Now I also visited the same URL (<a rel="nofollow" target="_new" href="<a rel="nofollow" target="_new" href="http://www.whatdoestheinternetthink.net/">http://www.whatdoestheinternetthink.net/</a>xbox"><a rel="nofollow" target="_new" href="http://www.whatdoestheinternetthink.net/">http://www.whatdoestheinternetthink.net/</a>xbox</a>) less than a week ago (today is 6th June for anyone reading this in the future) and the numbers were exactly the same as what they were on 21st May. In other words, judging by your observation after visiting the site today there has been a massive change over the past week or so.</p>
<p>This suggests that <a rel="nofollow" target="_new" href="http://www.whatdoestheinternetthink.net/">http://www.whatdoestheinternetthink.net/</a> does not update their data on a regular cadence so perhaps its not the best source of data here. I guess that's what you get if you accept something for free. Pay peanuts, get monkeys...or something.</p>
<p>The takeaway here is that there must be a space for a service that will provide sentiment analysis, shows trends over time and, also as you say, allow some drill-down capability. I guess this is why Salesforce shelled-out $340m (<a rel="nofollow" target="_new" href="http://www.forbes.com/sites/ericsavitz/2011/03/30/salesforce-buys-social-media-tracker-radian6-for-340m/">http://www.forbes.com/sites/ericsavitz/2011/03/30/salesforce-buys-social-media-tracker-radian6-for-340m/</a>) for Radian 6 (which is a company that I believe provides this type of analysis - though I stand to be corrected on that).</p>
<p>Thanks for your comment. We've definitely learned something here.</p>
<p>regards</p>
<p>Jamie</p>
re: Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspx#49481Thu, 06 Jun 2013 21:32:19 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:49481Scott MacDonald<p>2 things - BIG difference between searching on 'xbox' and 'xbox one' (xbox had the #'s in the first comment, xbox one had 92% negative).</p>
<p>I searched on a few other things of interest to me - and BOY is the Internet negative! ;)</p>
<p>Scott</p>
<p>Tucson, AZ</p>
re: Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspx#49483Thu, 06 Jun 2013 23:49:34 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:49483Ian Yates<p>So true about noting it's the trend in the data, not just the numbers themselves. &nbsp;I've found quite a few Forbes articles to be worth reading although I only read them when they're linked to by something else (Ars Technica links to them occasionally).</p>
<p>We do hospital financial analysis as a mostly off the shelf product. &nbsp;Often it's hard to get things making sense in the client's mind until we've built up a few months worth of data so we can trend it and help them relate spikes (good or bad) to events they recall in recent memory. &nbsp;The numbers themselves, in the absence of anything to compare to, are rather difficult to interpret.</p>
<p>Similarly we've been asked to do benchmarking and problems exist there but in a different dimension (across businesses rather than time). &nbsp;Knowing how each business actually calculates the number is part of determining if we're really doing an apples to apples comparison. &nbsp;Then the numbers can be meaningful and the hospital in isolation, which may make money in spite of itself, suddenly doesn't look so crash hot when compared with other hospitals who also make money (just making a lot more) :)</p>
re: Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspx#49484Fri, 07 Jun 2013 07:40:20 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:49484jamiet<p>Scott,</p>
<p>Thanks for the comment.</p>
<p>I have 2 takeaways from what you said.</p>
<p>1) Stats don't lie, but they *can* mislead with careful consideration of what you capture stats upon</p>
<p>2) Given that the internet is a forum for whinging that emphasizes my point that its not the numbers that are important, its the trend</p>
<p>:)</p>
re: Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspx#49511Mon, 10 Jun 2013 12:05:34 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:49511Simon Sabin<p>A company I did some work for do exactly what you asked</p>
<p><a rel="nofollow" target="_new" href="http://www.onalytica.com/">http://www.onalytica.com/</a></p>
<p>but its not free.</p>
re: Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspx#49600Wed, 12 Jun 2013 11:33:10 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:49600Nathan Finley (@FinleyNathan)<p>Well It's not too smart to use my tweet to get the &quot;public opinion&quot; as I'm biased against Microsoft to begin with.</p>
<p>Though the BBC did give me free publicity. not that I wanted any.</p>
re: Want to prove something with data? Aggregate it!http://sqlblog.com/blogs/jamie_thomson/archive/2013/06/06/want-to-prove-something-with-data-aggregate-it.aspx#49638Wed, 12 Jun 2013 18:07:59 GMT21093a07-8b3d-42db-8cbf-3350fcbf5496:49638jamiet<p>Hi Nathan,</p>
<p>thanks for the comment. Out of interest, How did you stumble upon this blog post?</p>