Re: What are you rewarding

It is nearing the end of the year and we at LeanDog are wrapping up our fiscal year. We’re looking at the potential tax benefits of spending some of our reserve and we’re mulling over other ideas related to the spend of money. We are not, however, discussing our bonus objectives. We aren’t discussing them because we don’t have them. I, for one, am happy that we don’t.

He points out the problem with bonus programs he’s seen, spills some ink (properly) criticizing the metrics that were used and how those incentivized dysfunctional behavior. He then closes with:

My advice to managers, directors, and executives who want to pay bonuses? Don’t. If you must do so, then break it down based on each employee’s income. If you have a top performer, don’t give them a bigger bonus, give them a raise. Think of a raise as a bonus that lasts forever.

This seems absurd to me. His title got it right, but the conclusion misses the mark. It’s not the form of the reward, it’s what you are rewarding. The problem isn’t with the bonuses (although those can exist, too), it’s with the metrics. If you gave raises based on the same metrics he described, you would get the same dysfunctions. You might get the same dysfunctions without bonuses or raises, simply by posting the metrics publicly. People do what you incentivize them to do. You are left with a choice between not measuring anything (with respect to incentive) or being very careful about what you are going to incentivize.

Now, I’m no expert on incentives, either, but here are some things I’ve observed.

Value, not behavior

You should incentivize the value you hope to achieve, not the behavior that you theorize will get you there. Measuring behavior can have great value, but should not be used to reward or punish (within the scope of normal expected behavior). I’m not talking about not punishing somebody for stealing. I’m talking about things like the classic KLOC’s. This is a classic bogeyman metric because everyone knows that more KLOC’s may mean more productivity at first, but it’s very easy to inflate your lines of code to the detriment of readability and quality. You may want to measure KLOC’s along with other things (you may want to see it going down in a particularly spaghettified module), but you need to be careful not to incentivize it.

Sphere of Influence

I’m a developer, so I’ll keep with the developer examples. Say Joe Developer is on team of 6 developers in a division with 12 teams in a company with 800 people. Tying some of Joe’s compensation to company profitability is certainly fair, but it doesn’t do much to incentivize him because it is very difficult for Joe to see how his individual contribution affected the bottom line. However, if you have some metric that measured the throughput or quality of the output of Joe’s team of 6 developers, now Joe can see how he affects things (to the benefit or detriment of the team). I don’t want to minimize the challenge of finding reliable measures at this level. (I can’t think of any that I really like for incentives.) But the point is that if Joe can’t see how he can affect it, Joe won’t do anything differently.

Avoid Individual Performance

I also believe firmly that any incentive should be tied to team metrics rather than individual metrics. You want this team working together, not competing with each other for their share of the pie.

Difficult to Game

A classic problematic metric is bug fix rate by the same individuals that can introduce bugs in the first place. If you tried to fix that by subtracting bugs for which the individual is responsible for introducing, get ready to spend a lot of energy trying to track down the ‘real’ responsible person and arguing over whose fault something is. And when a new bug is found, was it just introduced or just discovered? A better example might be if you had a mature legacy application in maintenance mode and a team that was responsible for that maintenance. Bug fix rate might make sense there, but again, you would want to incentivize the team, not the individuals, lest you get a bunch of people that don’t have time to help each other because they need to make their own numbers.

The incentives themselves

So far I have talked about metrics and started off poo-pooing criticism of bonuses. But that doesn’t mean incentives can’t be poorly designed. The problem, in my view is that any metrics can be problematic, given enough incentive. People are ingenious animals and will find very creative ways to game the system, given the ‘right’ incentive.

When people talk about deterrence of behavior, you will often see the phrase “swift, sure and severe”. I prefer “swift, sure and substantial” because that allows for carrots as well as sticks, to misuse a metaphor. What they mean is that the most effective way to change behavior is if the consequence of that behavior is swift – happens immediately – sure – always happens – and substantial – you will really care that it happened. Imagine if every time you went over the speed limit, your car sent a message to the highway department and you were issued a ticket that printed out of your dashboard. Suddenly, the speed limit would be an actual limit. You would almost never exceed it and almost surely drive below the speed limit to avoid accidentally exceeding it. (I’m not advocating such a police state, just pointing out that such a police state would be effective at controlling your behavior.)

Another case in point is smoking. The consequences of smoking can be severe (premature death) but they are neither swift nor sure, so those consequences aren’t enough to make most people quit. On the other hand, for people that enjoy smoking, the consequences are much less substantial (the enjoyment of the act of smoking) but still enough, combined with swift (immediate gratification) and sure (works every time). That’s good enough to sell a lot of cigarettes for a very long time.

So what does this mean for bonuses and other incentives? First, if you want to change behavior, keep those things in mind. If want to cause people to really get creative gaming the system, keep those things in mind. If you pick metrics that are very easy for someone to see for themselves how they are doing, they can count on a bonus (or lack of one) based on those metrics) and you make it very high stakes (for example 50% of base salary) you can bet people will find a way to make their numbers, both in ways you appreciate and probably ways that you do not.

If you don’t trust your metrics completely, dial back the consequences. Poorly chosen metrics can cause dysfunctional behavior even if the consequences are just that the metrics are displayed on a public screen. However, if the metrics are pretty good, but not perfect, put them up on a screen so people can see how they’re doing. If you really like the metric, consider tying a small bonus (or even a variably substantial one) at the end of the quarter or year based on it.

But if you’re going to give people a 50% bump (that they’re used to getting every year) you better love that metric to death, because people will maximize it one way or another.