Macroeconomics and Confirmatory Bias

Greg Mankiw has written an outstanding new essay that reviews the history of thought in macroeconomics. I don't know where he is planning to give this paper, but I wish I could be a discussant. Here is a brief excerpt.

Among the leaders of the new classical school, none (as far as I know) has ever left academia to take a significant job in public policy. By contrast, the new Keynesian movement, like the earlier generation of Keynesians, was filled with people who would trade a few years in the ivory tower for a stay in the nation’s capital. Examples include Stanley Fischer, Larry Summers, Joseph Stiglitz, Janet Yellen, John Taylor, Richard Clarida, Ben Bernanke, and myself. The first four of these economists came to Washington during the Clinton years; the last four during the Bush years. The division of economists between new classicals and new Keynesians is not, fundamentally, between the political right and the political left. To a greater extent, it is a split between pure scientists and economic engineers.

In Mankiw's view, scientists are those concerned with the logical consistency and empirical verifiability of macroeconomic theory. Engineers are concerned with making economic policy. One point he makes is that dramatic developments and debates took place among the scientists, particularly in the 1970's and 1980's, but the engineers are still operating much as they did before all this intellectual ferment took place.

My view of macroeconomics is that it operates under conditions of a chronic and severe data shortage. We don't have a sample of a thousand Great Depressions to provide a testbed for theories. We just have one.

Because of this data shortage, there is an "observational equivalence" among otherwise incompatible macroeconomic models. Two (or more) models that have very different theoretical underpinnings and policy implications can nonetheless both fit the data.

In such an environment, confirmatory bias rules the day. That is, all it takes to believe in a particular model is a slight tendency to elevate the relevance of confirming evidence and a slight tendency to ignore contrary evidence.

For the engineers, there has in fact been plenty of contrary evidence over the past two decades. For example, in predicting the impact of the Reagan tax cuts on output and inflation, the engineers got the direction wrong--forget about magnitude. The standard prediction in 1981 was that the Reagan tax cuts would be highly inflationary. The engineers say that in retrospect monetary policy offset the tax cuts. But I say that one should keep in mind confirmatory bias.

In the early 1990's, the engineers under-estimated the capacity of the economy to grow and forecast much larger Budget deficits for the latter part of the decade than actually occurred. Again, the engineers say that they can explain what happened in the 1990's--after the fact. Again, I worry about confirmatory bias.

I am not saying that I, Arnold Kling, possess the true model of macroeconomics. I am saying that it is impossible to know who, if anyone, is the possessor of that model. I believe that because of confirmatory bias, the engineers trust their models much more than I think is warranted.

For policy purposes, I think that if another Great Depression came along, I would say, "try something Keynesian." I know that if inflation took off, I would say, "try something monetarist." In between, I think I would focus fiscal policy on long-term growth and tell the Fed not to do anything too crazy.

Greg is troubled by the fact that the engineers still use pre-1970 approaches to modeling, which might indicate that the scientists have not accomplished much since then. My take is that both groups are deceiving themselves about the value of their work. The scientists over-estimate the practical significance of their innovations, and the engineers over-estimate the reliability of their models.

And the reason that macro issues do not generate as much intellectual excitement as they did a generation ago is that the popular intellectual focus has shifted to a new arena in which protagonists hold highly-charged policy debates in the context of a chronic and severe data shortage: climate change.

Comments and Sharing

Economists carry too many business acounting practices into macroeconomics. They need to treat us like squirrels and take lessons from the biological study of evironments and habitats.

Perhaps someone should just ignore unit of money altogether and just look at the flow of actual goods. Maybe examining what happens when one species of intelligent squirrel invades the habitat of another species of squirrel is a better model for globalization.

"My view of macroeconomics is that it operates under conditions of a chronic and severe data shortage. We don't have a sample of a thousand Great Depressions to provide a testbed for theories. We just have one."

But theres more than just time series variation. We also have enormous variation in the cross section. I think one of the most intriging observations in macro is the disparity of economic performances between, say, North and South Korea and ditto for Germany. If we cant use these observations to infer implications about economic policy, really, I dont know what can. The case of Hong Kong is also striking.
Furhter, in the cross section, we have had similar policy experiments, say, in Britain and the US under Thatcher and Reagan; in both cases, the results, I think, were fairly similar and, for me, anyhow, lent considerable credence to supply side theories. Of course, you could say likewise about the US and Europe today.
In my view, policy makeres should just stick with Adam Smiths advice; the rest seems for of a distraction.

Perhaps I am naive, or I misunderstand the topic; but I would think that econometric analysis and the tools and models of microeconomists could go a long way toward solving this problem, and models such as agent-based models of the market could complete the picture.

You say "We don't have a sample of a thousand Great Depressions to provide a testbed for theories. We just have one."

This is true, but you do have 1000 tax cuts or at least 100. Using data points from the major OECD nations over the past 50 years, you can analyze how each tax cut affected output, productivity, employment, poverty, etc. You can hold GDP/capita constant, along with the starting tax rate. It may not be perfect but it is a decent way to evaluate your model. You can see whether the tax cuts seem to help in all circumstances - when the GDP/capita is low or the economy is in recession, or only when the economy is doing well - and you can do the same kind of analysis for other policy measures.

Microeconomists use these kind of analytic tools all the time, they look at empirical data and make predictions before making a business move; why can't macroeconomists do the same? And congress does look at this kind of data (eg on the minimum wage) before passing legislation. So, am I missing something?

Agent based models can help to fill the gaps that empirical data has and confirm what is seen in the real world, so as to be sure that other facts did not affect the outcomes.

>If we cant use these observations to infer implications about economic policy, really, I dont know what can.

Exactly! Well said.

We must learn from the Socialism experiment. There is so much data there!

There is no data shortage for basic economic analysis and testing of basic theories and how policy plays out in the world given the laws of supply and demand and incentives.

Every microeconomic assumption and prediction is tested, along with economic models from Smith to Hayek. Reading a book on the problems in the Soviet economy (such as "The Soviet Economic System" by Nove) you can see every one of these basic economic realities confirmed. Marxian and Galbraithian objections and re-evaluations are shot down left and right.

If we cannot use this data then we are not economists. And we have 20 countries with every gradient of public sector between socialism and free market in the test bed as well, especially considering all of the past 75 years.

All we have to do is categorize each country in each period and begin to analyze the data with comparisons.

Shortage of data? No, just a shortage of imagination.

The exciting work is all in the area of transition economics - and that is where the scientists should be.

Im afraid many economists have learned little, if anything, from the socialist experiments. It defies comprehension that some economists (such as say Robert Frank) recently wrote laudatory columns on Galbraith (who -lets face it - was basically a socialist); Frank even suggested he should have received a Nobel Prize, notwithstanding everything he said was wrong and many of his proposals were positively dangerous. And this is a man who has coauthored a textbook with the current chair of the Federal Reserve!

The problems are not as simple as Mankiw makes out; the issues are not between theorists and engineers, as he puts it. The thing is, many economists find the results of economic theory very distasteful and, instead, go on to rationalize some hypothesis that dicates the theory is "wrong" and - of course - call for some interventionist policies. That forms the corpus of much work in economics today. I think the real dichotomy is thus; you either take economic theory seriously or you dont.

As you say, the data speak very loud on the primacy of free market economics; laissez faire. Yet, many economics -perhaps a majority of academic ones - still loudly advocate the same policies that have caused European economies to flounder and vehemently oppose and disdain policies that have caused Hong Kong to flourish. A non-scientist would find this state of affairs most bizarre coming from an empirical "science". Again, as I said the fundamental problem is enslavement to political ideology. In this sense, economics is no science at all; economics is really politics elevated to scientifism.

And consider this: In almost every textbook the "Keynesian cross" is presented to students. Now, everyone - and I mean everyone - knows this to be a completely flawed depiction of reality. Its wrong, plain and simple. Yet, as I write, students are being questioned on this in exams. Students who will never again take an economics course leave being indoctrinated with this piece of nonsense. Again, the reason is simply politics; it dovetails very well with the ideology of liberal professors who forever call for greater intervention.

I am glad that I am not the only one who notices this. It was very bizarre to go from a microeconomic exam, where the assumptions are clearly defined and obvious and true and the results tested and proven correct; to a macroeconomics exam where I regurgitated what answers the professor wanted but could clearly see that assumptions were obviously not in line with reality, the model flawed and the results in contradiction of the predictions.

I have very little respect for today's macroeconomics. My book was written by Mankiw and many of the theories presented (like the Keynsian cross) were either explicitly described as being no longer considered accurate or in my opinion were obviously wrong and contradicted by data. From the Phillips curve to the Solow Growth Model, they are all far too simplistic, based on zero-sum, have flawed assumptions or all of the above. They discard differences in government spending and private spending - when that literally means pretending that the USSR and the USA must be economically equal. It is so absurd as to make economics more a game than a serious endeavor.

And all this while microeconomics has made serious contributions, has models that can be useful on a macro scale, while we have almost endless data from the experiment with socialism and now have computer power to allow crunching of this data or many simultaneous micro-based equations or models that are not zero sum, simplistic nor assume away important distinctions between agents (for example agent based models).

I must assume that you are correct - that it is political considerations that keep so many economists working with long since debunked modeling techniques.

Agreed, perhaps a bit over the top, though. I appreciate Mankiw and others (NOT Krugman) for the professional approach to their job, but until somebody comes up with a model with inputs for asset prices, trade flows, tax codes, weather, political events, etc. that can predict, say, core inflation two years out, economics is not a science in the sense physics is, but it is a social science. Scientific in the sense of how the study of the field is approached. But the results, ie theories and models, are not comparable to Newton's Laws, for example.

I really don't think that is the issue. I would be very happy to have model that says "all else equal" and doesn't take into account a freak hurricane that may be handled by policy makers in various ways - and political events which can be very unpredictable - so long as otherwise the model makes perfect rational sense, follows known assumptions, etc.

The concern is that macro-economists in particular have spent much of the past century ignoring rational models, making unrealistic assumptions and creating models that have not been able to predict an outcome based on even the most limited set of inputs.

The problem is that they have not approached the field scientifically.

very interesting posts, perhaps you can clarify some points for me - and perhaps others who are not so versed in the actual debate in macroeco..

mpvz

In almost every textbook the "Keynesian cross" is presented to students. Now, everyone - and I mean everyone - knows this to be a completely flawed depiction of reality. Its wrong, plain and simple.

Does the term "Keynesian cross" refer just to the simple closed economy IS/LM diagramm or / and the complete general equilibrium framework, including labour & money markets and production function? (Sorry- perhaps this is really obvious to anybody who studied in USA or UK or read the english language textbooks)

If this is now considered wrong "by everyone", what has taken / what should take its place? New Keynesian, New Classical or - "New Neoclassical Synthesis (Goodfried/King)" ???

liberty

microeconomic exam, where the assumptions are clearly defined and obvious and true and the results tested and proven correct

Results of microecomomic models follow clearly and mathematically correct from their assumptions. But the assumptions obvious? (did you ever asked your fellow students if they would agree?) And even if this is not considered important, are they "proven and tested???" - perhaps better one should ask: can they be falsified and do they have predictive power? I am not so sure....

>Results of microecomomic models follow clearly and mathematically correct from their assumptions. But the assumptions obvious?

Yes. The only necessary assumptions are that given the exact same product, the public will buy more if it costs less. Every single person may not buy more but those who cannot buy any at the higher price may now buy one at a lower price. This follows from the existance of scarcity. People have limited resources so they cannot buy everything that they want; hence when prices fall, the people will buy more of the things that they want (both in variety and quantity). These are the basic assumptions of microeconomics. Scarcity provides the framework, supply and demand outline the model and allow for scientific investigation. Supply and demand, like predator/prey relationships in ecology, lead to many unintended consequences when there is intervention - the simple laws are much more powerful than they seem and much can de deduced from studying them.

>(did you ever asked your fellow students if they would agree?)

Sure. We discussed objections and considered exceptions (such as kidney transplant) but the fact is that the relationship exists no matter what. You may not want more than one kidney transplant if its cheaper, but those who could not afford one when the price was too high can now afford one. Whether government pays for it or there is supply of only a single doctor in the whole country - no matter how exceptional the circumstance - the basic law still applies. This can be seen clearly by study of socialist systems where they did everything they could to change the law, but a law of nature is just that. They had a shortage economy because of the laws of supply and demand.

> And even if this is not considered important, are they "proven and tested???" - perhaps better one should ask: can they be falsified and do they have predictive power? I am not so sure....

Absolutely. They have predictive power in business, where they are used and proven every day as those in business calculate demand, marginal cost, marginal revenue and model what an increase in 5c on their product would do to their revenue. They would not use these tools if they did not work - they do work because the assumptions are correct and the models reflect reality.

This simple modeling is used not only for a single business, firms and industry analysts use it to model whole industries and multiple industries that affect each other. The labor market, like any other market, can and is modeled this way and predictions about minimum wage increases tend to be confirmed, along with predictions about the changing nature of industry as wage increases in one industry pull workers from other industries.

Fnally, comparisons between different countries and policies follow predicted patterns that confirm these assumptions and this model.

And the biggest experiment in the history of economics - socialism - proves beyind a shadow of a doubt that the assumptions and all of their implications are true, regardless of how an economy is organized. Read "The Soviet Economic System" by Nove (or just about any other book on socialist economics - I could suggest twenty) to see what I mean. In every way that they tried to implement Marx's ideas, they ran into the precise problems that a microeconomist would predict.

Now, I see one other possible objections you may have:

Supply and demand are not enough to model a whole economy - they are fine on the micro scale but cannot help to determine any macro level phenomenon.

Well, I would suggest this. It is the year 2006. Back in 1930 with the tools we had you might possibly have a point - how many simultaneous equations based on supply and demand can you really churn out in an hour? You may never be able to accurately predict total growth using such things.

Three things have happened since:

1. We have learned how poorly any macro-economic model that has tried to side step microeconomics has performed in these 75 years.

2. We have seen in action the micro concerns affecting the macro-economy of the Soviet Union, proving their importance on a macro scale.

3. We have gained immense computing power and can now build complex models that use only micro assumptions and yet can model the whole economy easily and run them without manually crunching any equations.

So I ask you: why would you create a model that makes assumptions that have not been proven when you can easily build a model that might actually be predicative?

Blogging software: Powered by Movable Type 4.2.1.
Pictures courtesy of the authors.
All opinions expressed on EconLog reflect those of the author or individual commenters, and do
not necessarily represent the views or positions of the Library of
Economics and Liberty (Econlib) website or its owner, Liberty Fund,
Inc.

The cuneiform inscription in the Liberty Fund logo is the
earliest-known written appearance of the word
"freedom" (amagi), or "liberty." It
is taken from a clay document written about 2300 B.C. in the Sumerian city-state of Lagash.