A Long Tail

One of the key books that started my interest in risk, uncertainty and ultimately complexity theory was Against the gods: The remarkable story of risk written by Peter L Bernstein, and published in 1996 when Peter was aged 77! This book explained much of the history behind the development of risk management in a way that I could understand and is a recommended read for anyone involved in managing projects. Despite his success, Peter Bernstein never retired, and at the time of his death last month, aged 90, he was working on another book on risk. As authors go, a very long and distinguished career.

The limitations of the risk framework built in the 18th century and so clearly described in Bernstein’s book have been defined and expanded in recent times in The Black Swan by N.N. Taleb (another recommended read). Taleb’s ideas are discussed in my post Risky Business.

The major failing of traditional risk models is the issue of ‘boundaries’. Rules of probability such as The law of large numbers work if the population is bounded. The problem with project data is that there are no limits to many aspects of project risk. Consider the following:

You plot the distribution and average the weight of 1000 adult males. Adding another person, even if he is the heaviest person in the world only makes a small difference to the average. No one weighs a ton! The results are normal (Gaussian-Poisson) and theories such as the Law of Large Numbers and Least Squares (Standard Deviation) apply.

You plot the distribution and average the net wealth of 1000 people. Adding Bill Gates to the group causes a quantum change in the values. Unlike weight, wealth can be unlimited. Gaussian-Poisson theories do not apply!

Most texts and discussion on risk assume reasonable/predictable limits. Managing variables with no known range of results is rarely discussed and many project variables are in this category. For more on this see Scheduling in the Age of complexity.

Fortunately our colleague, David Hillson’s latest book Managing risk in projects will be published by Gower on 11 August 2009. This book is part of the Gower Foundations in Project Management series, and will provide a concise description of current best practice in project risk management while also introducing the latest developments, to enable project managers, project sponsors and others responsible for managing risk in projects to do so effectively. I would suggest another ‘must read’ if you are interested in project management.

This is the method we use to classify risk profiles. This work is guided by the DoD Risk Managers Handbook. Upper and Lower limits of a triangle distribution are used for the Monte Carlo Simulation tool (Risk+ or @Risk for Project are two we use).

The long tails are replaced by the truncated 10%/90% limits of the PDF. The results are “conversation starters” rather than models of the underlying project.

The programmatic risk are modeled in this way. The technical risks (in our world) are modeled and managed with Active Risk Manager. These two paradigms are joined in the Integrated Master Schedule through the guidance of DID 81650.

I think the key message is understanding the difference between useful insights that help wise managers prepare for the uncertain future and a blind faith in data generated by predictive models. As George E.P. Box said, “All models are wrong, some are useful”.
Pat

When people accept these models with blind faith – and I’ve even seen it in A&D – it scares the hell out me. ;>)

I use Box’s quote on the introduction of every Programmatic Risk Management breifing we use.

Many organizaitons we work with have a deep understanding of the output of Risk+ and @Risk for Project as well as Crystal Ball. Others plan their next decisions based on a specific number at the 80% confidence level.