Wednesday, June 04, 2014

Learning More About That Other Half: The Case for Cohort Analysis and Multi-Touch Attribution Analysis (Part 1 of 2)

Note: This article first appeared as a guest post on the popular KISSmetrics blog. Thanks to Hiten Shah and Sean Work at KISSmetrics for publishing it. I'm republishing the post here as a series of two shorter posts, with a few small edits.

Anyone who has ever worked in marketing or advertising has heard the
quote, “Half the money I spend on advertising is wasted; the trouble is I
don’t know which half.” It is from John Wanamaker and dates back to the 19th century.

Fortunately, the industry has come a long way since then, and especially in the last 10 to 20 years, new technologies have made
advertising more measurable than ever. However, there’s still a
considerable gap between what people could measure and what they actually are measuring, and that leads to significant under-optimization of advertising and marketing dollars.

In B2B SaaS, which we at Point Nine Capital focus a lot of our efforts on, there are two techniques that I feel are particularly important but not used widely enough – cohort analysis and multi-touch attribution analysis. In this series of posts, I’ll try to provide a brief introduction to both methodologies and explain why I think they are so important.

A Quick Primer about Cohort Analysis

If you're a reader of this blog or know me a bit, you know that I'm a huge fan of cohort analysis and have written about the topic before. If you’re new to the topic, a cohort analysis can be broadly defined
as a dissection of the activities of a group of people (such as users or customers), who share a common characteristic, over time. In SaaS, the most frequently used common characteristic for grouping
customers is “join date”; that is, people who signed up or became paying
customers in the same period of time (such as a month).

Let’s look at an example, and it will become much clearer:

In this cohort analysis, each row represents all signups that
converted to become paying customers in a given month. Each column
represents a month in your customer’s life. The cells show the
percentage of retained customers of the respective cohort in the
respective “lifetime month.”
So What?

Why is it so important to do a cohort analysis when looking at usage
metrics or retention and churn? The answer is that if you look at only
the overall numbers, such as your overall churn in a calendar month, the
number will be a blend of the churn rate of older and newer customers, which can lead to erroneous conclusions.

For example, let’s consider a SaaS business with very high churn in
the first few lifetime months and much lower churn from older customers – not unusual in SaaS. If the company starts to grow faster, the
blended churn rate will go up, simply because the percentage of newer
customers out of all customers will grow. So, if they look at only the
blended churn rate, they might start to panic. They would have to do a
cohort analysis to see what’s really going on.

What else can you see in a cohort analysis? Whatever the key metrics
are in your particular business, a cohort analysis lets you see how
those metrics develop over the customer lifetime as well as over what
might be called product lifetime:

If you read the chart above (which I've borrowed from my colleague Nicolas) horizontally, you can see how
your retention develops over the customer lifetime, presumably something
that you can link to the quality of your product, operations, and
customer support. Reading it vertically shows you the retention
at a given lifetime month for different customer cohorts. This might be
called product lifetime, an, especially if you look at early lifetime
months, it can be linked to the quality of your onboarding experience
and the performance of your customer success team.
The Holy Grail of SaaS!

Maybe most importantly, a cohort analysis is the best way to estimate CLT (customer lifetime) and CLTV (customer lifetime value),
which informs your decision on how much you can spend to acquire a new
customer. As mentioned above, churn usually isn’t distributed linearly
over the customer lifetime, so calculating it based on the blended churn
rate of the last month doesn’t give you the best estimate. A better way
is shown in the second tab of this spreadsheet, where I calculated/estimated the CLT of different cohorts.

A cohort analysis is even more essential when it comes to CLTV.
Looking at how revenues of customer cohorts develop over time lets you
see the impact of churn, downgrades/contractions, and
upgrades/expansions:

This chart shows a cohort analysis of MRR (monthly recurring revenue)
of a fictional SaaS business. As you can see in the green cells, it’s a
happy fictional SaaS business as it has recently started to enjoy negative churn, which many regard as the holy grail in SaaS.

Still not convinced that you need cohort analyses to understand your SaaS business? :-) Let me know in the comments.