Making Bad Analytics Good in China

The virtues of analytics are well known. From supply chain to marketing and even employees, if you can measure, you can improve. But how do you proceed when you have bad data?

Recently, I sat in a meeting in the adidas offices in Shanghai, listening to our market research firm present some puzzling information. We’d commissioned a market share study, and the numbers sounded way off. If what the firm said was true, then the sportswear industry — and therefore our business — was twice as large as we knew it to be based on what we shipped. We use a reputable firm, and its numbers were clearly capturing something we were missing.

In all the years that I’ve worked on similar analyses, I’ve never seen such a challenge. Used to being inundated by real-time analytics and having to be painstakingly deliberate about what data was used when, I’d never faced such uncertain information when it came to the basics of market sizing. In developed markets, Nielsen panel data, IRI reports, and other syndicated, standard metrics, are always there ready to buy.

The result is that if your company wants data, it’ll have to create it by commissioning large-scale market share studies intended to cover the entire industry — and country. And even when this data comes in, as highlighted above, it’s still tricky to figure out what it means.

So what to do? Well, like any analysis, it comes down to 1) what is measured and 2) the assumptions. We think the following tips also apply to those working in markets like India, where the coordination problem is even worse, with only 6% of organized retail.

What is measured 1) Ask ruthlessly simple questions. Focus on the number of users or simple purchase measures so when results come back, it’s easier to check your intuition. Keep things in absolute terms to help identify problems that get obscured when discussing percentages. For instance at P&G, Joe’s team simplified its goal to calculate the number of users who would buy a particular product in the first year, which focused the team’s research to the absolute number of consumers to target and the number of stores they needed to have distribution in. Percentage growth year-on-year of one retail channel over another was interesting, but not as critical. Simple absolute terms also kept tangible metrics at the forefront of every discussion.

2) Calculate the counterfeit. If your business is in certain consumer goods products, there is likely a significant counterfeit element. This turned out to be an important factor in our analysis at adidas. Our market research firm asked consumers what they recently purchased, what the brands were, and at what price points. Often in lower-tier cities and the further you go west, consumers believe they’re buying official adidas or Nike merchandise, but it is in fact counterfeit. Since they still report it in the study as being the official brand, the result is that recorded purchases far exceed actual shipments. It’s difficult to unravel, but you can use the average reported price points versus sales in each tier or region to try to understand if that’s what’s at play.

The assumptions3) Explore every piece. With any research, accurate answers hinge on a myriad of assumptions and factors. Age, gender, income, population are the basic ones. But even with these, you use and modify a statistical range. In China, with seven tiers of cities and sample sizes of 10,000 to represent over a billion people, each factor has a big impact. Dig into all of them until you understand how even a small tweak can transform the final number.

4) Question the data. Culturally, the Chinese are not comfortable with or used to intensely scrutinizing or debating what’s presented in these cases. Things are often taken at face value. It takes coaching and patience to overcome this resistance to disagreement. Help local managers who understand the market to ask detailed questions thoughtfully and systematically.

We as managers are often counseled to balance data and intuition. But in China, it takes a fair bit of intuition just to get to the data. More than anywhere else, you’ll need to rely on strong partnerships, flexible thinking, and persistent digging to get even close to the “truth” of the market.

Avni Patel Thompson is a senior strategy manager at adidas China. Joseph Thompson leads marketing for P&G Male Grooming in Greater China. They currently live in Shanghai. Follow more of Avni’s brand observations at brandsundae.com.