Murray Cantor

I know. It has been a long time since my last posting. Over the last few months, nothing much happened that prompted an entry. I was thinking of writing something sort of philosophical on the nature of estimates, but never got to it. Then there was the tsunami and the associated nucler reactor failures at the Fukushima power plant. Suddenly, the topic became more urgent. This is relevant to this blog, because our domain includes the engineering and economics of safety critical systems. Presumably the nuclear reactor industry uses the state of the art methods. I have been exploring what is going on there and while, I am far from an expert, I have found out some things worth sharing in a blog.

We have been told that reactor failure is a 1 in over a hundred thousand year event. Sounds reasurring.Yet, in my lifetime, there have been three that I know of: Three mile island, Chernoble, and now Fukushima. Discounting Chernobyl which apparently was greatly under-engineered, something must be wrong for the there to be two meltdowns in what has been estimated to be a one in over 100,000 year event. Apparently, there have been many near misses, e.g. the loss of coolant at the Brown's Ferry plant, something one would not expect from such safe systems. This raised some questions. What does a 'one is N year event' mean? Does it mean that we should not expect the event until N years has passed or that we can be certain one will occur within N years? More importantly, if there are K systems, each having a 1 in N year safety rating, what is the rating of the poputation? As I have pointed out in previous blogs, we do not need estimates, we need probablility distributions to get any practical understanding.

Here are some of the sources I found. This New York times article helps explain what is going on. A few points in the article caught my attention. First, they carried out 'deterministic' risk analysis (see this NRC page) because probabilistic methods are "too hard. A good summary of the difficulty of the problem and the history of how it is addressed is found in Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis by M. Granger Morgan and Max Henrion. Briefly there is little to go on to estimate the likelihood of individual events and their dependencies in event chains. So the distribution of estimated time to failure must have huge variance. This article by M. V. Ramana summarizes and criticizes the current practice including probablistic risk assessment (pra). The key idea is that failure results from a sequence of component failures. Each component is reliable and so the probability of a system failure is the joint probability of the component failures which is very low. This assumes that the component failures are independent events. However, as parts of thr system, the joint probabilities are hard to estimate. For example, one component may fail which results in a second component running out of spec which might result in a number of other component failures. Getting the joint probabilies right entails a very faithful system model and data collected from thousands of simulations with varying inputs and Monte Carlo methods to take into account the variability of the components. The output of such a simulation could be used to improve the system design.

I want to focus on another pra challenge: estimating the likelihood of the devastating single cause event, such as earthquake and the tsunami. Clearly some sort of data are needed for the estimate, but what sort of data? As pointed out in the New York Times article, there was a deep historical data search of the size and frequency of the earthquakes in the relevant geographic region. That led to the conclusion that planning for 18 feet of water was sufficient. Recall that the reactor was inundated by 40 feet of water. So past performance was not predictive. In retrospect, that is not surprising given that tectonic plate movements are hardly a stationary process. An alternate approach is to use modern geologic models and plate measurements. Then one could and should run simulations to get a distribution of the flood depths. One could argue that this approach is also suspect, since they depend on the quality of the models which introduces a subjective element. However, using historical data also is based on an assumption that earthquake generation is based on a stationary process, a very dubious model and its adoption is equally subjective. To be fair, presumably one could not run the needed simulations in the 1960's and so the frequency model may have been the best available. It can be argued that earthquake prediction is notoriously difficult, especally pinpointing when an earthquake will occur. However, using Monte Carlo methods and simulations, it seems reasonable that one can create a probility distribution of the time to an earthquake above a certain size and use this to estimate the likelihood of the event over the lifespan of the plant.

The point of this discussion is that frequency model data is no more 'objective' than data used to build and apply models. Both involve subjective assumptions of the validity of the model. Note Baysian data analysis methods can be used to validate the various models and so we can assess their usefulness in the estimation process.

Finally, these safety estimates are used to set policy and, in particular, to make economic decisions about nuclear energy. The cost of a failure is huge. For example, an estimate of the cost of Fukushima failure is $184 Billion. The proponants of nuclear energy argue that they make economic sense, assuming they are safe enough and the new designs are much safer. Maybe so. But knowing they are safe enough will take much better analytics than we have seen to date.