This site is about everything digital, giving an update on new things as I learn

Book review: “Lean Analytics”

Data are key. Analytics are key. But what does one need to measure? And why? Experienced entrepreneurs Alistair Croll and Benjamin Yoskovitz wrote “Lean Analytics”, which came out last month. “Lean Analytics” is a great book for anyone who wishes to learn more about validating an idea, building the right product and measuring growth.

In early 2012, I went to a talk by Eric Ries. It was a great talk, with lots of insights that I have since tried to implement on a day-to-day basis. The only thing missing, however, was a steer on how to measure performance in a ‘lean’ context. I felt that Ries’ suggestions around metrics were very much geared to consumer facing products or business models.

One of the questions I had was around measuring the performance non-consumer facing products such as infrastructural or enterprise services. When I got in touch with Eric Ries to follow up on my questions, the short answer was: “wait for a book by Alistair Croll and Benjamin Yoskovitz, due in March ’13.” I kept patient and it was definitely worth the wait.

This is why:

What makes a good metric? –“Lean Analytics” starts off with a basic but very important question: “what makes a good metric?” Croll and Yoskovitz go into four different characteristics of a good metric (see Fig. 1 below) and look at different metric types. For example, I hadn’t yet heard about the difference between ‘leading’ (predictive) and ‘lagging’ (current situation) metrics. This provides an essential starting point for the rest of the book, which delves into which metrics to measure and why.

Be sensible about data – In Lean Analytics, Croll and Yoskovitz do a good job of putting some caveats around the use of data. Firstly, they stress the importance of combining quantitative and qualitative data, urging readers to “get out of the building.” Secondly, they point out the risks of becoming too data-driven. With the help of Monica Rogati, a well-known data scientist at LinkedIn, they outline 10 common data related pitfalls. For example, businesses should be aware of ‘data vomit’; having extensive data dashboards but loosing focus of those key metrics that really matter to your business.

The One Metric That Matters – Especially if you’re an early-stage startup, Croll and Yoskovitz stress the value of having the “One Metric That Matters” (OMTM). The reasons they provide to underpin this argument are pretty compelling (see Fig. 2 below). When I work with young startups and their often very driven and passionate founders, I tend to to focus their minds on a single metric or value consideration that is likely to really impact their business.

What business are you in? –At first I thought it a bit tedious for Lean Analytics to outline a variety of business models. Surely, most readers should be able to know the difference between an e-commerce model and a media site!? I backtracked very quickly on this thought: not only do Croll and Yoskovitz provide a comprehensive overview of business models to consider, they also offer a great overview of relevant metrics to track per business model. For example, I’m currently working on a free mobile app and the book offers some useful metrics to consider in this respect (see Fig. 3 below). For each business model the book offers a wealth of concrete suggestions on which metrics to consider and why (using real-life case studies to illustrate).

What stage are you at? The kinds of metrics worthwhile tracking are not only dependent on your business model, they are also likely to vary based on the stage your business is at. Lean Analytics breaks down the different stages that most startups are likely to go through (see Fig. 4 below). I believe the value of using these stages as a reference point is twofold. Firstly, each stage requires its own specific metrics. For instance, as a business you’ll most likely concentrate on different metrics at the empathy stage than when you’re at the revenue stage. Secondly, the book suggests you ask yourself at each stage whether you should move on to the next one, using the relevant metrics to validate your decision making.

Learning point: Lean Analytics is a great fit within the “Lean Startup” movement. The book provides a very comprehensive overview of the different metrics worth tracking and – most importantly – offers a clear rationale as to why certain metrics should be tracked for a specific business model at a given stage. The thing I liked most about Lean Analytics is that all of it makes perfect sense. The book provides a well thought through approach to making sense of something that can be complex and challenging at times: data.

[…] and will look at all the activities they can design and measure to increase the chosen KPI. Alistair Croll and Benjamin Yoskovitz have introduced the notion of the “One Metric That Matters” in this respect, urging […]

[…] improvements or new features have the desired impact. I’ve learned a lot from the book Lean Analytics in which its authors, Alistair Croll and Benjamin Yoskovitz, expand on when and how to best use […]

[…] range of specific goals or metrics. I’d always recommend to keep it simple and focus on a single goal or metric, understand what your (target) users’ needs are and how are they actually using your […]

[…] product is crucial if you wish to learn about your product and your customers. I’ve written before about the importance of spending time on defining the right metrics to measure, avoiding the risk […]