Category: Measure

Small changes to our product can lead to large changes in user behaviour.

The colloquial phrase Quantum Leap or Quantum Shift means to make a very large improvement or change. Ironically, this is the exact opposite of Quantum’s scientific definition, which refers specifically to the smallest possible change.

When operating at scale, we find that small changes to our product can create large changes to user behaviour. A good question to ask ourselves is: What’s the smallest possible change I can make to my product which will result in the largest possible returns?

The answer will give us a hypothesis: I believe that moving the advertisement into the sidebar will increase my email subscription rate by 10%.

Data can mean anything, therefore it means nothing. How, then, can we extract meaning from a dataset?

Two shoe salesmen were sent into “darkest Africa” to feel out the potential shoe market. The first telegraphed home saying: it’s hopeless stop nobody here wears shoes. The second telegraphed back saying: it’s wonderful stop nobody has any shoes.

As Product Managers, we work with a lot of data. Sometimes we even hire a data scientist to go through our data and tell us what it means.

Framing is the lens through which we view data. Since we all have different brains (and hence different frames) the same data will always represent something different to different analysts. Data can be objective. Recommendations based on that data can never be.

Data can mean anything, therefore it means nothing.

How, then, can we extract meaning from a dataset? One very effective method is to look hidden assumptions.

Suppose your analytics show a temporary downtick in traffic during February. One might assume that this is simply a natural ebb, another may assume that February must be a low month in your industry, a third may assume that there was a technical error has since been resolved.

To extract meaning from these analytics, ask yourself:

What assumption am I making,
That I’m not aware I’m making,
That gives me what I see?

Challenging this assumption will help you learn something new about your product (a competitor launched, an industry event, a political influence), which you can then leverage to your advantage.

Most marketing campaigns, aim to reach as many eyes as possible. Here’s another approach: Minimum Viable Marketing.

We’ve all heard about lean product development principles: Create a Minimum Viable Product (MVP), measure its performance, and iterate.

We could apply this same principle to many Product disciplines. Take marketing, for example. Often, a Product team will hire a marketing manager or consultant and launch a campaign, aiming to reach as many eyes as possible.

Here’s another approach: A Minimum Viable Marketing (MVM) campaign.

Define a small campaign targeted only at the early adopters amongst your market segment, using words like Innovative, Pioneer, Breakthrough, Private, Limited, and Now. Choose just one channel to reach them on.

No need to build out every asset for every medium. No need to get the alignment just so. No need for pixel perfection. No need to wordsmith.

Since you’re starting small, take the time to get to know your audience. Talk with them, without any hint of self-promotion. Show them your marketing materials and gauge their thoughts and reactions.