The Perils of KPI Abuse

As I write more and more blog articles, I am amazed at how all these different ideas seem to come together at some point. About a month ago I wrote about our over-reliance on facts, factoids and sound bites to learn about concrete. We are always looking for a way to get a better grasp of the massive amounts of information that comes our way. Rather than read an entire article or report, we often skim just the abstract or executive summary.

Then I throw fuel on the fire by conducting a survey about KPIs (key performance indicators) used in concrete quality control to reduce a lot of information into a single number that is easy to digest. It now occurs to me that we should also be asking, “Just because I can KPI, should I KPI?”

What is a KPI? As I said, they are numbers that are designated as “key performance indicators.” Rather than wade through tables of numbers or endless graphs, management tries to distill an issue down to a single number that answers the question, “Am I performing well or not?” If you want a passing grade of at least 70, and you get a 75, then you are passing the course. If you get a 95, then you are doing great!

Whether you know it or not, you are probably already using KPIs. What are some typical concrete QC KPIs?

Cementitious content for a given design strength

Average strength for a mix

Average strength above design or target strength for a mix

Standard deviation of a concrete mix

Standard deviation for all mixes in a plant

Variability of cement batched in a plant

So what is wrong with using a KPI? Let’s look at the first one: cementitious content for a given design strength. Back in the merger heyday of the 1980s and 1990s, I would often hear complaints from QC managers like this: “Now that the accountants are comparing my division with the new company we just bought, they are complaining that we use more cement for a 3000 psi mix than the new company. They want us to cut our cement content to the same amount used by the new company.” From an accounting or management viewpoint, this is a great idea. Let’s save money by cutting cement. From a QC perspective this idea is very, very, very wrong.

Why is cutting cement a bad idea? There are many factors that determine what cement content is necessary to produce a given strength. These include:

Mortar cube strength of the cement

Water demand of the cement and aggregates (water required to produce a given slump)

Interaction between the admixture and cementitious materials

The impact of air entrainment at different levels on strength

Tendency of local contractors to add water to concrete, thereby reducing strength

There are also other factors that impact the cement content in a concrete mix. If a 3000 psi (20 MPa) design mix also has a maximum w/c requirement of 0.45, for severe exposure conditions, the cement content will need to be increased. Even though the design strength is only 3000 psi, this mix will probably produce 5000 psi (35 MPa). We know we are getting excessive strength, but we are doing it for a reason. (This gets back to a blog article I did months ago on the double duty that the abbreviation f’c must perform.) If we reduce the cement content and the strength, we don’t comply with the exposure requirement of a maximum 0.45 w/c ratio and the concrete gets rejected. In short, just because one plant or division can produce concrete using a certain cement quantity doesn’t mean that all plants can produce the same strength with the same cement quantity.

So how do we solve the problem of inappropriate use of KPIs? We need to educate people in management and accounting, and probably QC as well. Since it seems unlikely they will sit down for a class on KPI etiquette, maybe a disclaimer sheet or a group of footnotes should come with each KPI report. The characters “*, &, #, @, +, and ~” should be mandatory in a KPI report, with corresponding footnotes indicating the conditions under which the KPI should be used.

How should we use concrete QC KPIs? A primary use of QC KPIs should be to study the variability of a single item over time. Is the cement batching variation better or worse today than last week (see Figure 1, which shows the change in variation following a plant upgrade)? Is the cement efficiency for Mix A better than, or at least equal to that of Mix B? Why does one mix have a lower efficiency than the other? In many cases KPIs should be used to help us ask questions and not just answer them. Is Lab A producing more consistent test results than Lab B for the same mix? While it is possible to compare data from different plants, with different materials, it is important to understand the effect of those differences on the KPI being studied.

In summary, KPIs can be a valuable tool in the hands of someone who understands the significance and limitations of each number. In the wrong hands a misunderstood KPI can lead to confusion and improper action or goals. Like any other tool, a KPI is only useful in the hands of a person who knows how to use it.

James M. “Jay” Shilstone Jr. is a concrete technologist for Command Alkon Inc. He is a Fellow of ACI and an active member of ASTM and NRMCA committees. E-mail jshilstone@command alkon.com or follow Shilstone’s blog at www.commandalkonconnect.com.