The questions we receive run the gamut, from “What should we be measuring?” to “How do we get these numbers?” to “What do we do with the data once we get it?” What companies are in essence asking is, “How do we get to world class supply chain measurement?”

Debra first lists out some of the recounted obstacles with supply chain measurement:

Too many metrics

We see companies that are looking at hundreds, sometimes thousands of metrics.

Perhaps, that is a symptom of too many managers and the need for measuring their performance in order to justify the exercise of having so many managers in the first place. Or perhaps, it could very well mean that no one has a clue of why each metric needs to be measured in the first place.

Endless debate over metric definition

There is a definitional problem when there are no accepted underlying truths that everyone can agree upon. When it is more important to define one’s success by a metric, then definitely a definitional game is going to be played.

Clearly, some debate is healthy and, more importantly, necessary.

and,

There is a point, however, where debate becomes a form of resistance, providing a way to put off change.

Constantly changing metrics

One company we talked to described this as the “metric-of-the-year syndrome.”

One wonders if there is correlation to office holder/manager of the year as well that is also behind the metric of the year or quarter. I’m sure that everyone understands that metrics measure i.e. report back how things are going and they don’t innovate new ways of doing things. Metrics reside on the feedback loop and not on the forward loop of acting.

Old data

Or the only available data. I have found frequently that this is the case. Like the old adage, “What you don’t measure is not important to you!”, those companies that don’t have a handle on their data are a goldmine for consultants.

Gaming the system

For example, in our benchmarking studies at AMR Research, we measure the perfect order, which is defined as an order that’s complete, accurate, on-time, and in perfect condition. An order that is split because product is not in stock for some of the lines on the order is considered incomplete, and therefore the whole order is imperfect. What some companies have found when they instituted this metric is that in these situations, people were canceling the original order and replacing it with two new orders that could be filled, thereby keeping the perfect-order rating up.

I don’t think that I need to say more!!

An aside first:
As a budding engineer a long time ago, fluid dynamics was one of my favorite subjects and therein I had my fair share of numbers (very metric like issue) – dimensionless numbers. It even became a hobby of sorts i.e. to make up new dimensionless numbers which absolutely had no relevance to the world whatsoever except perhaps of proclaiming your name behind the number. If you don’t know anything about dimensionless parameters, see the box for more information.

Dimensionless Numbers
The definition of a dimensional number taken from Wikipedia is as follows:In dimensional analysis, a dimensionless quantity (or more precisely, a quantity with the dimensions of 1) is a quantity without any physical units and thus a pure number.Such a number is typically defined as a product or ratio of quantities which do have units, in such a way that all units cancel.
An example of a widely used dimensional number would be %.
So the next question that should pop into anyone’s mind is – “Why should people use it?”
Again, from Wikipedia: People work with dimensionless numbers in reading measuring instruments and manipulating (changing or calculating with) even dimensionful quantities.
One of the most important dimensionless numbers is the Reynold’s number, that describes the type of fluid flow – laminar, turbulent or something in between laminar and turbulent.
If you’re familiar with financial analysis, you would find the P/E (Price/Earning ratio where Price is $/Share and Earnings is Earned $/Share) ratio to be a dimensionless number which is used rightly or wrongly to compare firms within the same industry.

The question then that Debra tries to answer in this article is:

So what can companies do to address these challenges and get to world-class measurement? First, we’ll describe the dimensions of good supply chain measurement, and then we’ll identify some best practices to follow and pitfalls to avoid.

Debra first introduces the concept of “measurement maturity”:

What differentiates the leaders are two dimensions: first, their ability to measure, and second-and just as important-their ability as an organization to act on the results.

The concept of “measurement maturity” is founded on two dimensions – one related to the firm’s ability to measure (measurement aptitude) and the other related to a firm’s execution capability based on the measured reality (results actionability). This is basic control theory – the forward loop constitutes the execution capability while the feedback loop has to do with the measurement of the reality that the forward loop is trying to address.
Debra further delves into what characteristics comprise measurement aptitude:

Know what to measure.

Have a program in place to measure it.

and results actionability:

The ability to accept the results

The ability to act on the results

Now, based on these characteristics of measurement aptitude and results actionability, one can draw up a 2X2 matrix that describes measurement maturity. The four general types of firms that lie in the quadrants are:

As the quadrant titles suggest in and of themselves, any benchmarker worth his/her salt is going to tell you that a firm should be making efforts to get into the “Excellence Addicts” quadrant. But that is not by itself sufficient to ensure sustainable competitive advantage. See box on the left.

In the Innovator’s Dilemma. Professor Clayton Christensen describes a curious paradox about how outstanding firms can fail “by doing everything right” or how the very successes and capabilities that a company has invested an inordinate amount of time and effort developing becomes obstacles in the face of changing markets and technologies. (A more detailed look at the Innovator’s Dilemma)

The reason can be quite simply stated and I will employ the same control theory analogy for the purpose. A control system consists of a forward and feedback loop. However, a control system works for a particular environment for which it is designed. If that environment for some reason changes in a significant way, the control system will have to be changed, redirected or upgraded as the situation warrants. And that more or less is the problem highlighted by Professor Clayton. When the system is challenged or changed, a new benchmarking is needed and today’s Excellence Addicts might find themselves suddenly in another quadrant altogether. And that needs to be always kept in mind when one talks about best practices and benchmarking.

So what are the best practices when it comes to metrics definition? There are six practices that Debra has identified in her article when it comes to the task of metrics definition:

Design different metric portfolios for different goals

Keep it small: avoid the “mushroom effect.”

Address the Basics: Balanced, Cross Functional, Practical

Align execution and strategy

Understand the interdependencies

Balance the need for standards versus customization

Each of the above practices are a field to themselves and I refer you to the article to get a clearer picture of how to go about selecting/fashioning the right supply chain metrics.
Next, Debra outlines certain best practices when it comes to Metrics implementation:

1. Develop a Metrics Strategy and Time Frame

Implementing a measurement process is an organizational change, often of great magnitude, and as such it warrants a plan.

And furthermore,

The issue of constantly changing metrics noted above is a symptom of lacking a metrics strategy that clearly states when metrics definition is over and defines when metrics may or may not be changed.

2. Define Scope

Most companies have more than one supply chain, and the definition of what constitutes a supply chain is often independent of a company’s organizational structure.

I wonder if this notion takes into account the fact that a firm is a different supply chain player in multiple supply chains that a firm finds itself in. What’s more, that requires multiples of metrics as well.

3. Pay Attention to Roles, Responsibilities, Structure, and Process

Make sure you have the structure and processes in place or at your disposal with which to act on the results of your measurement exercise.

And,

It’s also important to choose the right resource to manage the data collection effort. The actual data will obviously come from multiple people in the organization who are responsible for different areas (for example, order management, production, and procurement). The person responsible for coordinating the effort must have influence in the organization and be well-respected by his/her peers in order to be successful in mobilizing their effort.

4. Manage the Culture

The way people respond to the results is tied to how the results will be used and, even more importantly, how people believe the results will be used.

I don’t think that you can get away from this and that means that you really need to get into it and set the rules rather than the rules being set ad hoc which you then have to get about changing.

5. Ongoing measurement is key

That just means that this is not a once a year exercise. But remember the Innovator’s Dilemma as well and how to solve it.

This article by Debra Hofman of AMR Research hits the problem at a high level and I think it supplies the framework within which one can go about setting up the right metrics and working with them. However, a supply chain is a firmwide complex thing to begin with and the details of how the supply chain is executed can often overwhelm the metrics effort i.e. the forward loop of supply chain execution sucks up all the available resources and time that the feedback loop just limps along. And thus a closed loop system is turned into a largely open loop system and if you’re familiar with control theory, you’d also know how dangerous that can suddenly get.