Garbage In, Money Out

Wall Street firms, which have made substantial investments based on complex computer models, are rethinking the limits of these models as they have watched a number of their peers sink into bankruptcy. Most experienced CEOs of non-financial firms would not invest $2 based solely on computer models prepared by one of their divisions seeking funding. The seasoned executive has seen too many hockey-stick market forecasts that oversimplify complex market dynamics to be anything but skeptical of such models.

Unfortunately, the U.S. Congress has already spent nearly a trillion dollars using computer models of similarly dubious accuracy. And if the Democrats have their way with cap-and-trade, we may find ourselves spending trillions more based on the same fallacies relative to climate.

Last week the Council of Economic Advisors (CEA) released its congressionally commissioned study on the effects of the 2009 stimulus. The panel concluded that the stimulus had created as many as 3.6 million jobs, an odd result given the economy as a whole actually lost something like 1.5 million jobs in the same period. To reach its conclusions, the panel ran a series of complex macroeconomic models to estimate economic growth assuming the stimulus had not been passed. Their results showed employment falling by over 5 million jobs in this hypothetical scenario, an eyebrow-raising result that is impossible to verify with actual observations.

Most of us are familiar with using computer models to predict the future, but this use of complex models to write history is relatively new. Researchers have begun to use computer models for this sort of retrospective analysis because they struggle to isolate the effect of a single variable (like stimulus spending) in their observational data. Unless we are willing to, say, give stimulus to South Dakota but not North Dakota, controlled experiments are difficult in the macro-economic realm.

But the efficacy of conducting experiments within computer models, rather than with real-world observation, is open to debate. After all, anyone can mine data and tweak coefficients to create a model that accurately depicts history. One is reminded of algorithms based on skirt lengths that correlated with stock market performance, or on Washington Redskins victories that predicted past presidential election results.

But the real test of such models is to accurately predict future events, and the same complex economic models that are being used to demonstrate the supposed potency of the stimulus program perform miserably on this critical test. We only have to remember that the Obama administration originally used these same models barely a year ago to predict that unemployment would remain under 8% with the stimulus, when in reality it peaked over 10%. As it turns out, the experts' hugely imperfect understanding of our complex economy is not improved merely by coding it into a computer model. Garbage in, garbage out.

To their credit, many people have recognized of late that these model results are at best unverifiable, and at worst totally absurd. Unfortunately, Congress is now considering legislation of a very different type--cap and trade limits on CO2 from utilities--whose academic justification bears an incredible resemblance to that of the stimulus.

While we have been bombarded with hockey sticks and forlorn polar bears, our focus in climate should really be on the computer models. The primary scientific case for man-made CO2 as the main driver of global temperatures is made in exactly the same way that the stimulus was determined to have created 3.6 million jobs: computer modeling. No one yet has been clever enough to structure a controlled experiment to isolate the effect of rising CO2 levels from other changing variables in the complex global climate. So, just like the CEA did in scoring the stimulus, climate scientists use computer models to run virtual experiments, running the models backward over the last century with varying assumptions for CO2 levels.

This modeling approach yields amazingly circular logic. Like macroeconomic models built by devoted Keynesians, climate models are constructed by academics who passionately believe that a single variable, CO2 concentration, is the dominant driver of the whole complex climate system. When run retrospectively, the models they create unsurprisingly give the result that past temperature increases are mainly attributable to CO2. The problem with these models is that when run forward, as in the case of the Washington Redskins election model, they do a terrible job of predicting the future. None of them, for example, predicted the flattening of global temperatures over the last decade.

The climate modeling approach is so similar to that used by the CEA to score the stimulus that there is even a climate equivalent to the multiplier found in macro-economic models. In climate models, small amounts of warming from man-made CO2 are multiplied many-fold to catastrophic levels by hypothetical positive feedbacks, in the same way that the first-order effects of government spending are multiplied in Keynesian economic models. In both cases, while these multipliers are the single most important drivers of the models' results, they also tend to be the most controversial assumptions. In an odd parallel, you can find both stimulus and climate debates arguing whether their multiplier is above or below one.

Our common sense about government stimulus tells us that the government is highly unlikely to invest money more productively than the private entities from whom the government took the money. Unfortunately, we have allowed this common sense to be trumped by computer models. Once our imperfect understanding the economy was laundered through computer models and presented with two-decimal precision, smart people somehow lost their skepticism.

We are now facing what is potentially an even more expensive decision: to regulate CO2 based mainly on computer models that claim to be able to separate the effects of trace concentrations of CO2 from a hundred other major climate variables. If your common sense is whispering to you that this seems crazy, listen to it. Otherwise all we get is garbage in, money out.