Menu

How to Deal with Dozens of Evaluation Metrics in High Tech Product Management

One of the primary challenges of product management in a high technology industry is dealing with all of the different variables that customers consider when they make a purchase decision. Generically, there’s price, performance, usability, visual appeal, portability, utility, etc. However, each tech industry has it’s own set of unique evaluation metrics and in some cases there can be a very high number – several dozen or more.

As an example, in the advertising technology industry, the recent Forrester DSP Wave report examined over 60 evaluation points to rank products that deliver automated ads for advertisers and ad agencies.

This warrants the question: in an environment where customers care about so many different things, how do you build a differentiated roadmap?

That’s the question I thought about this week.

As usual, to help answer question, I spent most of my time trying to figure out how Apple does it.

It’s hard to find a product category that has more variables than the personal computer. There’s monitor size, processor speed, input device, portability, reliability, speaker quality, keyboard, memory, storage, etc. And that’s just the hardware. There’s also operating system, available software, ease of use, speed of software, configurability, visual appeal, etc. etc. You don’t have to look too hard to find several handfuls of evaluation criteria that people consider when buying a new computer.

So how has Apple dealt with the multitude of evaluation metrics? Well – it’s printed right on the box.

Below is a picture of what the Macbook pro box has looked like for the last few years. Apple doesn’t want you to care about dozens of evaluation criteria – they want to draw your attention to just one thing: thinness and portability. That is the one most defining characteristic that differentiates Mac’s from other PC’s and it’s accentuated perfectly by Apple’s product marketing.

But then turn over the box and this is what you see:

Evidently thinness alone isn’t enough. They also have to check some of the other boxes. Processor speed, memory, storage, screen resolution, camera – all of these are listed here and some are mildly differentiated features, but none as prominent as the thinness and portability.

Notice how they don’t mention everything here. No sign of productivity software (which is a notorious weak spot for Mac), nothing about the ability to network easily with non-mac computers and printers. Also – nothing about a DVD/optical drive, because this computer doesn’t have one.

Now even in these weak spots, Mac is still “good enough”. Microsoft office applications are a pain to use, but they’re workable. If you need a DVD drive, you can get a portable one and plug it in. None of the “check the box” features are completely missing and none of the core functionality is so bad as to be a total deal breaker. But Apple definitely does not try to be the best at everything.

They focus.

I think that is the real lesson here. If you try to do everything better than your competitors you will inevitably find yourself spread too thin and unable to execute. You only have to be “good enough” at everything to satisfy basic use cases. Outside of that, pick a small set of features that you’re going to use as points of differentiation (items to list on the back of the box). These are items you can brand (e.g. FaceTime HD camera; Thunderbolt port) and position as differentiators.

Lastly, choose very wisely and select your one major point of competitive advantage. What is the one thing among the dozens of evaluation criteria that is going to drive customers to buy your product over your competitors?

Once you’ve chosen your one defining feature – go big. Be the best in the world at it and position it clearly on the front of the box.

How to Deal with Dozens of Evaluation Metrics in High Tech Product Management