How to Manipulate Statistics and Create Organizational Dysfunction

September 15, 2010
|
By David Xia

Mark Twain once said, “There are three kinds of lies: lies, damned lies and statistics.” This is
wonderfully exemplified in last week’s radio program from This American Life. The show
described how the New York Police Department’s 81st Precinct distorted crime statistics. So
what did police officer Adrian Schoolcraft do when his supervisors told him to downgrade real
crimes into lesser ones? He began secretly recording his commanding and fellow officers on the job
– for 17 months. Ever since Schoolcraft blew the whistle on these abuses (helped by this
five-part series in the Village Voice), the NYPD’s 81st has been hit with four
investigations and numerous class action lawsuits.

But this is more than just a story of unscrupulous police and careerist commanding officers. It’s
a story of how workplace statistics and performance metrics can create pervasive organizational
dysfunction. CompStat is the name of the NYPD’s accountability process. According to the
Police Chief Magazine, CompStat’s purpose is to

Collect, analyze, and map crime data and other essential police performance measures on a
regular basis, and hold police managers accountable for their performance as measured by these
data.

After the NYPD adopted CompStat in 1995, crime dropped by a whopping 60%. But despite CompStat’s
goal of holding individuals accountable, like any human system, there are loopholes. Police
departments have partially manipulated it into a huge numbers game. Adrian Schoolcraft’s
recordings reveal how commanding officers in the 81st Precinct pressured police to downgrade
serious crimes and manufacture petty ones. Rapists became trespassers while Halloween
trick-or-treaters became suspicious “roving bands.”

Why did this happen? Police supervisors began to expect a continuous decrease in crime
year-over-year. Otherwise their personal performances wouldn’t look so hot, especially when
they’re explaining to their own bosses. You try explaining to your boss that despite lousy stats,
you’re actually doing a better job than your predecessor because this time you’re being honest.
This would work in an ideal world where people are rational and everyone’s incentives line up. But
let’s face it, many people will choose immediate, selfish gratification over prudent, long-term
stewardship.

I don’t know enough about CompStat to say whether it’s more beneficial than harmful. But I do know
from personal experience and David Simon’s The Wire (only the best TV series ever) that
metrics are only good to an extent. Individual accountability and organizational performance
should be based on numbers to a degree. Focus solely on the numbers, and you’ll oversimplify a
complex system and blind yourself to unmeasurable factors like trust, morale, and happiness.

Ira Glass hosts This American Life on Chicago Public Radio. TAL is a terrific work of
journalism, public service, and authentic human stories rolled into one hour of pure awesomeness.
Incredibly, all their shows are archived and free to anyone who has Internet access. This requires
a shit load of bandwidth, and bandwidth does not grow on trees. So please donate money to them
here. I have.