Mostly about games, but with occasional detours into other nerdy territories.

Tuesday, January 19, 2010

Checklist Manifesto

I finished the previously mentionedChecklist Manifesto, by Atul Gawande, a few days back. It's a short book, slim and copiously footnoted, and I suspect I've spent more time thinking about it than I actually spent reading it. It's a good book, a really good book, and well worth a read if you deal with complex activities (be they for work or gaming) but it's due for a rocky reception.

The book is pretty much what it says on the tin: an argument for using checklists. Gawande's own interest in them came out of their application in surgery, but he does research in places like construction and aviation where checklists are an essential part of maintaining safety and quality in situations that are simply too big to fit in any one person's head.This is summarized particularly well in the chapter on construction, where checklists are used to manage the ocmplexities of building a real building that's not going to fall down on anyone. In the past, this complete vision was the job of the Master Builder, the one guy who knew enough about what was required that he could personally issue orders and review work to make sure everything was going as it should. As building got more complex, that model stopped being viable, and expertise got distributed, with checklist systems evolving as the means for keepig it all together.

Gawande saw a lot of similarities between the historical master builder and the modern day surgeon. Historically, the surgeon is the rock star, the all-knowing figure who has the rest of the room dance to his tune. But, Gawande argues, surgery has grown much more complex over time, and depends on more factors than any one person can reliably handle, no matter how skilled they are. To make sure that stupid details aren't missed, and to help the team in the OR work together, he made the case for using a surgical checklist.

But that was easier said than done. Construction checklists are huge affairs, concerned with minute details on projects that may take place over years. There's no time for that in the OR, so Gawande looked to examples in another complex, high pressure field where lives are on th eline every day: aviation. Like surgery, modern planes are incredibly complex, and pilots have developed a culture of using checklists to deal with this complexity. There are basic checklists for everyday tasks, but there are also checklists for a huge array of potential problem scenarios.

The research and methodology that goes into these lists is intense. Whenever there's a problem on one of the commercial airlines, part of the post mortem is to put together a checklist to help a future pilot deal with it if it should ever come up. Given that these may be life or death situations with little or no time, they have a lot of very specific guidelines and restraints on the designs of a checklist. It must be short enough to be easily referenced , but comprehensive enough to actually be useful. What's more, there also needs to be a culture of use - checklists save planes because pilots are trained to use them in every situation.

This brevity and clarity seemed to match Gawande's needs in the OR, but his attempts to create a model that will work (in his hospital and in others) demonstrates that it's much more complicated to make a good checklist than it appears. The process of how he builds it, an dhow it is deployed and tests it is pretty interesting in its own right, but it also highlights a lot of issues he's addressed along the way.

This is the kind of book that usually drives me nuts. The core argument is a pretty simple one; the sort that would make an excellent article in the New Yorker or The Atlantic, and when things like that turn into books they usually get puffy and diluted. This proves the odd exceptions for two reasons: first, as noted it's a pretty short book. Second, and perhaps more importantly, getting this simple message across is such a profoundly uphill battle that I don't begrudge him the space to make every argument he can.

This is a good, practical book with concretely useful advice, well presented and well argued, and I think it's going to be well received, but I also think it's primarily going to get treated as something that will be most useful for other people. The benefits are so clear and obvious that they can't be ignored, but it's very easy to say "oh, they don't apply to me" because, as we know, we are all special snowflakes.

If I sound a little cynical here, I admit my experience from talking with people about every other organizational discipline, from Getting Things Done to project management[1], has lead me to that perspective. The majority of smart people hate lists because they're either too smart to need them, or because they're a straightjacket, a limiter on creativity. Gawande addresses this much more politely by talking about our heroic ideal of the guy who carries it all on his back. We don't celebrate collaboration or procedure nearly as much as we do the heroic individual, and we treat things like checklists as a threat to ideal.

This is why I hope people will read the book, if only to realize that the checklists they envision are nto what we're talking about. Yes, there are absolutely production-line checklists out there which are used in place of any skill or knowledge. They have a place in business for things like building a Big Mac, and when you say "checklist" that's the first thing that people think of. But not only is that far form the case, that sort of checklist skips past the two biggest strengths of the idea.

The first is that the person reading the list knows what they're doing. A pilot's checklists does not tell them how to fly the plane. No checklist could. it takes years of training and experience to do so, and the checklists are written with the assumption that that's the case. The purpose of the list is not to tell the pilot (or surgeon or whatever) how to do his job, it is simply to run through the stupid, mundane stuff that it's easy to overlook because you know what you're doing too well.

The second is that the list is a social object (though Gawande is never a big enough dork to use that term) that gives the team working on things a common reference[2]. This is powerful ina number of ways. In situations of unequal authority, like an operating room or cockpit, the checklist gives the lower-status person the means to speak up because they're addressing the checklist, not challenging the alpha. The checklist also demands points to pause and review, if only for a minute, to get everyone on the same page.

It is only natural to focus on the idea of the checklist as automation, since it definitely has a role in that. That's the bogeyman of our time, and we all have incentive (both for our pride and our hopes of keeping a job) to consider our jobs to be things that cannot be automated. They depend on our talent, knowledge, skills and expertise, and that cannot be boiled down to a simple list.

But even so, the question is whether there is no part of your job that could benefit from this sort of review? Do your complex, brilliant plans not also need to be implemented? Do mistakes made on obvious things cost your company money? Are other people stupid? If you answer 'yes' to any of these, then this book is almost certainly worth a read.[3]

1 - 3o say this book has useful applications to project management is a vast understatement, but that's definitely not Gawande's perspective on things, so there's no handholding for PMs.2 - Some lists also include explicit social elements. The big surprise for me was that the construction plans include explicit tasks for having people talk. The checklist does not dictate anything about the discussion besides that A and B need to talk about X. If that discussion produces more things to do, they go into the process. If not, then it's just checked off and moved on.3 - And yes, gaming is a complex, social activity with lots of moving parts. There are a lot of lessons that translate interestingly to RPGs, but that's very much fodder for its own post.

3 comments:

One thing I've been considering is the utility of a system to build your own story within an RPG.

The idea is create a quick and easy way for people to build a story and inject their creativity within an effecient pre-constructed structure. This would obviously rub a lot of GM's the wrong way for reasons you've already cited, so I think the way to do it is put a bit of tongue inside the cheek and keep the tone light, campy and self depreciating. No one likes the idea that their stories are formulaic, but formulas do have their place. If you create simple story formulas but keep the pretensions nonexistent I think it would be acceptable.

@jesse Complimentary. The big underlying idea with GTD (that the system lets you get the uncreative stuff out of the way so you can focus on what's important) is in full force here, simply with complex tasks like flying a plane being the end goal rather than personal goals.