Don’t be average

I’ve written before about what I perceive to be an emerging, key trend among many new tech startups – the vast volumes of data companies now produce and the importance of having someone in your organization whose job it is to sift through these stacks of data and look for trends and patterns (I’ve even suggested to a few college students interested in startups and entrepreneurship that they make sure they’re taking plenty of math and stats classes as I see this as a great way for a young person with limited experience to pitch themselves to be quickly impactfull working with a start-up).

One word of caution about these data that I’ve been digging into a lot recently is the risk of using averages to represent what’s happening in your business. As the complexity of the trends businesses analyze increase, and as the volume of data produced increase along with this complexity, the value of looking at straight averages decreases. It’s a real danger in many cases to look at an average and think you know anything about what’s taking place inside a complex system (indeed, at one of the companies I work with we’ve instituted a “no averages” rule to force ourselves to do at least the next level of analysis of whatever it is we’re looking at).

I’m a huge fan of businesses collecting and analyzing the data they produce. And as the data get more complex, the need for more thoughtful analysis increases. So keep taking those stats classes…

I once had a client that was using ARPU (average revenue per user) to track their business. They were a SaaS product with monthly subscription plans. The entry level plan was less than $10/mo. Their biggest customers were spending well over $1000/mo. nnThey used average lifetime customer value, based on ARPU, to determine their marketing spend. They didn’t segregate their marketing efforts by revenue potential either (another form of averaging). nnAnd here’s why that was a problem. Less than 50 of their biggest customers drove the same revenue as more than 5000 of their smallest customers. Obviously, the cost of sales for 5000 customers was significantly higher than for 50 customers, and the margin on the 50 customers was significantly higher too. nnAnd yet they were spending the exactly the same amount to acquire both types of customer. Not good…

http://www.sethlevine.com sethlevine

great (if crazy) example nick. i can’t believe with that spread that theyrnwere looking at a lump average (to your point a few huge customers werernmasking the size and therefore profitability of a bunch of smaller ones).rnthanks for the example.

http://www.onlineaspect.com Josh Fraser

I’ve recently had a renewed appreciation for standard deviations. With massive datasets it’s not always practical to analyze or even store every single datapoint. I’ve found that an average can convey far more information when it is combined with the std dev. With that one extra piece of data, you can know how representative that average is of the data set as a whole. The standard deviation lets you know when it’s okay to use the average and when you need to dig in deeper to see what’s going on.

http://www.sethlevine.com sethlevine

it’s a good point josh – just the kind of thing i’m talking about…

http://twitter.com/Inderwies Brian

Much of what changes the world happens a few standard deviations away.nnA great read on this topic is Nassim Taleb’s “The Black Swan.” (absolutely no relationship to the popular movie.)

http://www.onlineaspect.com Josh Fraser

Ordered it this morning. Thanks for the recommendation.

http://www.sethlevine.com sethlevine

Exactly. Good point @brian

Salim madjd

It’s a problem we face with the data we gather from our AsthmaMD app. There is a lot of useful data that is not just interesting but has enormous medical and health benefit. However, we soon discovered we are not statistician and need other organizations help to dig deeper.

Salim madjd

It’s a problem we face with the data we gather from our AsthmaMD app. There is a lot of useful data that is not just interesting but has enormous medical and health benefit. However, we soon discovered we are not statistician and need other organizations help to dig deeper.

http://blogme.dium.com dgmandell

@Sether Great point, but extrapolation outside of data analysis to other business disciplines also holds true. I’ve never met a successful company that’s succeeded because they did the “average” of what everyone else is doing. Whether marketing, product design, or customer service. The average is easy to achieve, which, by definition, is why it is the average. Too many start-ups, and surprisingly, later stage companies, define their actions based on what everyone else does, not as opposed to what everyone else does. The edge of the curve is where the winners live.

http://www.sethlevine.com sethlevine

good point @dgmandell. you’re exactly right…

sbmiller5

Another good (and definitely easier read than the Black Swan) is Jeffrey Ma’s The House Advantage: Playing the Odds to Win Big In Business. Jeff is the guy the movie 21 was based on…easy read, not the best written in a few parts, but I’ve taken away some valuable lessons related to analyzing trends within your business and marketplace and just a different way to think about data.