A Partner at Web Analytics Demystified, Mr. Wilson has been working in digital analytics for over 12 years in a diverse range of environments and with a wide range of analytics platforms.

What Marketing/Analytics Can Learn from Mythbusters

Earlier this month, I gave a presentation at the Columbus Web Group meetup that I titled Mythbusters: Analytics Edition. The more I worked on the presentation — beating the same drums and mounting the same soapboxes I’ve mounted for years — the more I realized that the Discovery Channel show is actually a pretty useful analog for effective digital analytics. And, since I’m always on the lookout for new and better ways to talk to analysts and marketers about how to break out of the soul-sucking and money-wasting approaches that businesses have developed for barfing data and gnashing teeth about the dearth of “actionable insights,” this one seemed worth trying to write down.

Note: If you’re not familiar with the show…you can just bail on this post now. It’s written with the assumption that the reader actually knows the basic structure and format of the program.

First, a Mythbusters Episode Produced by a Typical Business

When I do a thought experiment of putting an all-too-typical digital marketer and their analytics team in charge of producing a Mythbusters episode, here’s what happens:

The show’s opening credits roll. Jamie and Adam stand in their workshop and survey the tools they have: welding equipment, explosives, old cars, Buster, ruggedized laptops, high-speed cameras, heavy ropes and chain, sheet metal, plexiglass, remote control triggers, and so on. They chat about which ones seem would be the most fun to do stuff with, and then they head their separate ways to build something cool and interesting.

[Commercial break]

Jamie and Adam are now out in a big open space. They have a crane with an old car suspended above it. They have an explosive device constructed with dynamite, wire, and a bunch of welded metal. They have a pole near the apparatus with measurements marked on it. They have a makeshift bomb shelter. They have high-speed cameras pointed at the whole apparatus. They get behind the bomb shelter, trigger the crane to drop the car and, right as it lands on the explosive device, the device goes off and blows the car up into the air.

[Commercial break]

Jamie and Adam are now reviewing the footage of the whole exercise. They play and replay videos in slow motion from different angles. They freeze-frame the video at the peak of the old car’s trajectory and note how high it went. Then, the following dialogue ensues:

Adam: “That was soooooo cool.”

Jamie: “Yeah. It was. What did we learn?”

Adam: “Well, the car was raised 7’2″ into the air.”

Jamie: “Right. So, how are we going to judge this myth? Busted, plausible, or confirmed?”

Adam: “Um… what was the myth we were trying to bust?”

Jamie: “Oh. I guess we didn’t actually identify one. We just came up with something cool and did it.”

Adam: “Hmmm. I don’t know. I don’t think how high the car went really tells us anything. How loud do you think the explosion was?”

Jamie: “It was pretty loud. Did we measure the sound?”

Adam: “No. We probably should have done that. But…man…that was a bright flash when it blew up! I had to shield my eyes!”

Jamie: “Aha! We have software that will measure the brightness of the flashes from the video footage! Let’s do that!”

[They measure the brightness.]

Adam: “Wow. That’s pretty bright.”

Jamie: “Yeah. So, have we now done enough analysis to call the myth busted, plausible, or confirmed?”

Adam: “Well…we still don’t know what ‘it’ is. What’s the myth?”

Jamie: “Oh, yeah. I forgot about that.” [turns to the camera] “Well, we’re about out of time. We’ll be back next week! You know the format, folks! We’ll do this again next week — although we’ll come up with something else we think is cool to build and blow up. Hopefully, we’ll be able to make a busted, plausible, or confirmed call on that episode!”

[Credits roll]

This is how we’ve somehow managed to train ourselves to treat digital analytics!!!

We produce weekly or monthly reports and expect them to include “analysis and insights.” Yet, like the wrongheaded Mythbusters thought experiment above, we don’t actually ask questions that we want answered.

Sure, We Can Find Stuff Just by Looking

Keeping with the Mythbusters theme and, actually, lifting a slide straight from the presentation I did, what happens — in reality — when we simply point a web analyst to the web analytics platform and tell them to do some analysis and provide some insights for a monthly report? Poking around, clicking into reports, correlating data, even automatically detecting anomalies, we can turn up all sorts of things that don’t help the marketer one whit:

To be clear, the marketer (Jamie) is complicit here. He is the one who expects the analyst to simply dig into the data and “find insights.” But, week in and week out, month in and month out, he gets the report, the report includes “analysis” of the anomalies in the data and other scattershot true-but-not-immediately-relevant findings, but he doesn’t get information that he can immediately and directly act on. (At which point we invoke Einstein’s definition of insanity: “doing the same thing over and over again and expecting different results.”)

“Insights” that are found this way, more often than not, have a perfectly logical and non-actionable explanation. This is what analysis becomes when the analyst is told to simply dig into the data and produce a monthly report with “analysis and insights.”

The Real Mythbusters Actually Gets It Right

The Mythbusters team develops a plan for testing that myth in a safe, yet scientifically valid, way.

They experiment/construct/iterate as they implement the plan.

They conclude with a one-word and unequivocal assessment of the result: “Confirmed,” “Plausible,” or “Busted.”

Granted, the myths they’re testing aren’t ones that lead to future action (just because they demonstrate that a lawn chair with a person on it can be lifted by balloons if you tie enough of them on doesn’t mean they’re going to start promoting a new form of air travel). But, aside from that, the structure of their approach is exactly where marketers could get the most value. It is nothing more and nothing less than a basic application of the scientific method.

Sadly, it’s not an approach that marketers intuitively follow (they’re conditioned not to by the legacy of bloated recurring reports). And, even worse, it’s not an approach that many analysts embrace and push themselves.

Outlining those same exact steps, but in marketing analytics terms:

A marketer has an idea about some aspect of their site that, if they’re right, would lead them to make a change. (This is a hypothesis, but without the fancy label.)

The analyst assesses the idea and figures out the best option for testing it, either through digging into historical web analytics or voice of the customer data or by conducting an A/B test.

The analyst does the analysis or conducts the test

The analyst clearly and concisely communicates the results of the analysis back to the marketer, who then takes action (or doesn’t, as appropriate)

So clear. So obvious. Yet…so NOT the mainstream reality that I see. I have a lot of theories as to why this is, and it’s becoming a personal mission to change that reality. Are you on board to help? It will be the most mundane revolution, ever…but, who knows? Maybe we’ll at least come up with a cool T-shirt.

Viva la revolution! Now…how we kick off la revolution? At the
outset of each month, rather than blindly digging in data, survey
marketers/stakeholders via email or meeting for their burning questions
or potential insights?

http://tim.webanalyticsdemystified.com/ Tim Wilson

I’ve got a case-study-in-the-making on that.

My best luck has been multi-pronged: 1) relationship/credibility/trust-building with stakeholders, 2) Education-education-education of business users (but in bite-sized and engaging ways — like talking about Mythbusters), 3) shifting the structure of legacy recurring reports to promote a dialogue rather than a data-puke. Every organization is different, but it starts with recognizing that the current way isn’t working and being open to shaking things up and doing thing differently. And, boy, getting to that starting point can be a BEAR — it’s against human nature to admit that what we’ve been doing month in and month out is actually not delivering value commensurate with the cost.

Recent Blog Posts

No, I’m not referring to SecondLife (which, BTW, is still around and, apparently, still has life in it). I’m referring to the fact that podcasts just turned ten, and there are a lot of signs that they might be one of the "next big things" in digital. Earlier this year, when I wrote a post announcing the launch of the Digital Analytics Power Hour podcast, I listed three examples as to how it seemed like podcasts were making a comeback ...

Vendors commonly pitch the need for “real-time” data and insights, without due consideration for the process, tools and support needed to act upon it. So when is real-time an advantage for an organization, and when does it serve as a distraction? And how should analysts respond to requests for real-time data and dashboards?

I had this come up a couple of weeks ago with a client, and I realized it was something I’d done dozens of times…but had never written down the “how” on doing. So, here we go. This is a post about one very specific application of Excel, but it is also implicitly a post about how, with an intermediate level of knowledge of Excel, with a little bit of creativity, and a strong aversion to manually parsing/copying/pasting anything, a spreadsheet can accomplish a lot! And very quickly!

One of the benefits of having a number of friends in the analytics industry is the spirited (read: nerdy) debates we get in to. In one such recent discussion, we went back and forth over the merits of "bounce rate." I am (often vehemently) against the use of "bounce rate." However, when I stepped back, I realized you could summarize my argument against bounce rate quite simply ...

Happy belated new year to everyone reading this blog — on behalf of everyone at Web Analytics Demystified and Team Demystified I sincerely hope you had a wonderful and relaxing holiday season and that you’re ready to wade back into the analytical and optimization fray! Since I last wrote a few cool things have happened ...

I’ll admit it: I’m a Nate Silver fanboy. That fandom is rooted in my political junky-ism and dates back to the first iteration of fivethirtyeight.com back in 2008. Since then, Silver joined the New York Times, so fivethirtyeight.com migrated to be part of that media behemoth, and, more recently, Silver left the New York Times for ESPN — another media behemoth.

In digital analytics, "Governance" is a term that is used casually to mean many different things. In our experience at Web Analytics Demystified, every organization inherently recognizes that governance is an important component of their data strategy, yet every company has a different interpretation of what it means to govern their data. In an effort to dispel the misconceptions surrounding what it means to truly steward digital data, Web Analytics Demystified has developed seven data governance principles that all organizations collecting and using digital data should adhere to.

Hiring in the competitive analytics industry is no easy feat. In most organizations, it can be hard enough to get headcount – let alone actually find the right person! These three foundational tips are drawn from successful hiring processes in a variety of verticals and organizations.

Those of you who follow my blog have come to know that when I learn a product (like Adobe SiteCatalyst), I really get to know it and evangelize it. Back in the 90′s I learned the Lotus Notes enterprise collaboration software and soon became one of the most proficient Lotus Notes developers in the world, building most of Arthur Andersen’s global internal Lotus Notes apps. In the 2000′s, I came across Omniture SiteCatalyst, and after a while had published hundreds of blog posts on Omniture’s (Adobe’s) website and my own and eventually a book! One of my favorite pastimes is finding creative ways to apply a technology to solve everyday problems or to make life easier.

One of the things customers ask me about is the ability to profile website visitors. Unfortunately, most visitors to websites are anonymous, so you don't know if they are young, old, rich, poor, etc. If you are lucky enough to have authentication or a login on your website, you may have some of this information, but for most of my clients the "known" percentage is relatively low. In this post, I'll share some things you can do to increase your visitor profiling by using advertising campaigns and other tools.

Some of you may have noticed that I don't blog as much as some of my colleagues (not to mention any names, but this one, this one, or this one). The main reason is that I'm a total nerd (just ask my wife), but in a way that is different from most analytics professionals. I don't spend all day in the data - I spend all data writing code. And it's often hard to translate code into entertaining blog posts, especially for the folks that tend to spend a lot of time reading what my partners have to say.

Do you used in-cell dropdowns in your spreadsheets? I used them all the time. It's both an ease-of-use and a data quality maneuver: clicking a dropdown is faster than typing a value, and it's really hard to mis-type a value when you're not actually typing!

Yesterday, an article in the Harvard Business Review provided food for thought for the analytics industry. In Tesco's Downfall Is a Warning to Data-Driven Retailers, author Michael Schrage ponders how a darling of the "analytics as a competitive advantage" stories, British retailer Tesco, failed so spectacularly - despite a wealth of data and customer insight.

Regardless of what type of website you manage, it is bound to have some sort of conversion funnel. If you are an online retailer, your funnel may consist of people looking at products, selecting products, and then buying products. If you are a B2B company, your funnel may be higher-level like acquisition, research, trial and then form completion.

This post has an unintentionally link bait-y post title, I realize. But, I did a quick thought experiment a few weeks ago after walking a client through the structure of a dashboard I'd built for them to see if I could come up with ten discrete tips that I'd put to use when I built it. Turns out…I can!

Back in 2012, I developed an Excel worksheet that would take post-level data exported from Facebook Insights and do a little pivot tabling on it to generate some simple heat maps that would provide a visual way to explore when, for a given page, the optimal times of day and days of the week are for posting.

While in Atlanta last week for ACCELERATE, I got into the age-old discussion of "Adobe Analytics vs. Google Analytics." I'm up to my elbows in both of them, and they're both gunning for each other, so this list is a lot shorter than it would have been a couple of years ago.

Last night as I was casually perusing the days digital analytics news - yes, yes I really do that - I came across a headline and article that got my attention. While the article's title ("Top 5 Metrics You're Measuring Incorrectly") is the sort I am used to seeing in our Buzzfeed-ified world of pithy "made you click" headlines, it was the article's author that got my attention.

As a digital analytics professional, you've probably been tasked with collecting business requirements for measuring a new website/app/feature/etc. This seems like a task that's easy enough, but all too often people get wrapped around the axle and fail to capture what's truly important from a business users' perspective. The result is typically a great deal of wasted time, frustrated business users, and a deep-seated distrust for analytics data.

I am delighted to announce that our Team Demystified business unit is continuing to expand with the addition of Nancy Koons and Elizabeth "Smalls" Eckels. Our Team Demystified efforts are exceeding all expectation and are allowing Web Analytics Demystified to provide truly world-class services to our Enterprise-class clients at an entirely new scale.

In one of my recent Adobe SiteCatalyst (Analytics) "Top Gun" training classes, a student asked me the following question: When should you use a variable (i.e. eVar or sProp) vs. using SAINT Classifications? This is an interesting question that comes up often, so I thought I would share my thoughts on this and my rules of thumb on the topic.

Next month's ACCELERATE conference in Atlanta on September 18th will be the fifth - FIFTH!!! - one. I wish I could say I'd attended every one, but, sadly, I missed Boston due to a recent job change at the time. I was there in San Francisco in 2010, I made a day trip to Chicago in 2011, and I personally scheduled fantastic weather for Columbus in 2013.

A Big Question that social and digital media marketers grapple with constantly, whether they realize it or not: Is "awareness" a valid objective for marketing activity?

I've gotten into more than a few heated debates that, at their core, center around this question. Some of those debates have been with myself (those are the ones where I most need a skilled moderator!).

As I have mentioned in the past, one of the Adobe SiteCatalyst (Analytics) topics I loathe talking about is Product Merchandising. Product Merchandising is complicated and often leaves people scratching their heads in my "Top Gun" training classes. However, many people have mentioned to me that my previous post on Product Merchandising eVars helped them a lot so I am going to continue sharing information on this topic.

When Eric Peterson asked me to lead Team Demystified a year ago, I couldn't say no! Having seen how hard all of the Web Analytics Demystified partners work and that they are still not able to keep up with the demand of clients for their services, it made sense for Web Analytics Demystified to find another way to scale their services. Since the Demystified team knows all of the best people in our industry and has tons of great clients, it is not surprising that our new Team Demystified venture has taken off as quickly as it has.

Lately, Adobe has been sneaking in some cool new features into the SiteCatalyst product and doing it without much fanfare. While I am sure these are buried somewhere in release notes, I thought I'd call out two of them that I really like, so you know that they are there.

I was reading a post last week by one of the Big Names in web analytics…and it royally pissed me off. I started to comment and then thought, "Why pick a fight?" We've had more than enough of those for our little industry over the past few years. So I let it go.

One of my newest clients is in a highly competitive business in which they sell similar products as other retailers. These days, many online retailers have a hunch that they are being "Amazon-ed," which they define as visitors finding products on their website and then going to see if they can get it cheaper/faster on Amazon.com. This client was attempting to use time spent on page as a way to tell if/when visitors were leaving their site to go price shopping.

One of the most valuable ways to be sure your recommendations are heard is to forecast the impact of your proposal. Consider what is more likely to be heard: "I think we should do X ..." vs "I think we should do X, and with a 2% increase in conversion, that would drive a $1MM increase in revenue ..."

I am delighted to share the news that our 2014 "Advanced Analytics Education" classes have been posted and are available for registration. We expanded our offering this year and will be offering four concurrent analytics and optimization training sessions from all of the Web Analytics Demystified Partners and Senior Partners on September 16th and 17th at the Cobb Galaria in Atlanta, Georgia.

In working with a client recently, an interesting question arose around cart additions. This client wanted to know the order in which visitors were adding products to the shopping cart. Which products tended to be added first, second third, etc.? They also wanted to know which products were added after a specific product was added to the cart (i.e. if a visitor adds product A, what is the next product they tend to add?). Finally, they wondered which cart add product combinations most often lead to orders.

As an analyst, your value is not just in the data you deliver, but in the insight and recommendations you can provide. But what is an analyst to do when those recommendations seem to fall on deaf ears?

If I could give one piece of advice to an aspiring analyst, it would be this: Stop showing your "math". A tendency towards "TMI deliverables" is common, especially in newer analysts. However, while analysts typically do this in an attempt to demonstrate credibility ("See? I used all the right data and methods!") they do so at the expense of actually being heard.

I'm always amazed (read: dismayed) when I see the results of an analysis presented with a key set of the results delivered as a raw table of numbers. It is impossible to instantly comprehend a data table that has more than 3 or 4 rows and 3 or 4 columns. And, "instant comprehension" should be the goal of any presentation of information - it's the hook that gets your audience's brain wrapped around the material and ready to ponder it more deeply.

This post (the download, really - it's not much of a post) is about dealing with exports from Facebook Insights. If that's not something you do, skip it. Go back to Facebook and watch some cat videos. If you are in a situation where you get data about your Facebook page by exporting .csv or .xls files from the Facebook Insights web interface, then you probably sometimes think you need a 52" monitor to manage the horizontal scrolling.

Having worked as an industry analyst back in the day I still find myself interested in what the analyst community has to say about web analytics, especially when it comes to vendor evaluation. The evaluations are interesting because of the sheer amount of work that goes into them in an attempt to distill entire companies down into simple infographics, tables, and single paragraph summaries.

Funnels, as a concept, make some sense (although someone once made a good argument that they make no sense, since, when the concept is applied by marketers, the funnel is really more a "very, very leaky funnel," which would be a worthless funnel - real-world funnels get all of a liquid from a wide opening through a smaller spout; but, let's not quibble).

Those of you who have read my blog posts (and book) over the years, know that I have lots of opinions when it comes to web analytics, web analytics implementations and especially those using Adobe Analytics. Whenever possible, I try to impart lessons I have learned during my web analytics career so you can improve things at your organization.

I am excited to announce that registration for ACCELERATE 2014 on September 18th in Atlanta, Georgia is now open. You can learn more about the event and our unique "Ten Tips in Twenty Minutes" format on our ACCELERATE mini-site, and we plan to have registration open for our Advanced Analytics Education pre-ACCELERATE training sessions in the coming weeks.

I recently had a client pose an interesting question related to their shopping cart. They wanted to know the distribution of money its visitors were bringing with them to each step of the shopping cart funnel.

Over the past year, I've run into situations multiple times where I wanted an Adobe Analytics segment to be available in multiple Adobe Analytics platforms. It turns out…that's not as easy as it sounds. I actually went multiple rounds with Client Care once trying to get it figured out. And, I've found "the answer" on more than one occasion, only to later realize that that answer was a bit misguided.

If your web analytics work covers websites or apps that span different countries, there are some important aspects of Adobe SiteCatalyst (Analytics) that you must know. In this post, I will share some of the things I have learned over the years related to currencies and exchange rates in SiteCatalyst.

In the last few years, people have become accustomed to using multiple digital devices simultaneously. While watching the recent winter Olympics, consumers might be on the Olympics website, while also using native mobile or tablet apps. As a result, some of my clients have asked me whether it is possible to link visits and paths across these devices so they can see cross-device paths and other behaviors.

I had the pleasure last week of visiting with one of Web Analytics Demystified's longest-standing and, at least from a digital analytical perspective, most successful clients. The team has grown tremendously over the years in terms of size and, more importantly, stature within the broader multi-channel business and has become one of the most productive and mature digital analytics groups that I personally am aware of across the industry.

As someone in the web analytics field, you probably hear how lucky you are due to the fact that there are always web analytics jobs available. When the rest of the country is looking for work and you get daily calls from recruiters, it isn't a bad position to be in! At Web Analytics Demystified, we have more than doubled in the past year and still cannot keep up with the demand, so I am reaching out to you ...

Whether you have a single toe dipped in the waters of social media analytics or are fully submerged and drowning, you've almost certainly grappled with "engagement." This post isn't going to answer the question "Is engagement ROI?" ...

Unless you've been living under a rock, you have heard (and perhaps grown tired) of the buzzword "big data." But in attempts to chase the "next shiny thing", companies may focus too much on "big data" rather than the "right data."