In "Understanding Economic
Forecasts," the editors candidly start right off acknowledging the
weaknesses of the econometric models used for forecasting. This constitutes a vast improvement in professional attitudes since the
1970s, when many economists were boasting nearly scientific precision for their
knowledge and econometric models.&

The models are
"mis-specified" (missing essential variables - based on the wrong
variables - and/or mathematically misrepresented) and are plagued by substantial unanticipated economic
shifts.&New theories and methods of econometric
forecasting now acknowledge economic dynamics and sudden shifts, and the
imperfections inherent in forecasting models. The editors assert that this
enables economists to "account for the different results of competing
forecasts."& The book's editors and contributing economists explain these new
developments - their capabilities and continuing limitations - in a clearly
written, commendably candid, handy little book, with a minimum of professional
jargon and econometric equations, so that the material is readily accessible and
useful to
the intelligent lay reader and professional economist alike. However, the lay
reader must be careful when interpreting the professional jargon that is used -
especially when it consists of common words that have professional meanings.&

Unfortunately, major unpredictable
"shocks" occur all too often - resulting in major forecasting
failures, especially for the near-term forecasts that are so important for
policy-making, business and investment purposes.

"Structural breaks appear to
explain why it is so hard to reduce forecast error standard deviations: the
outcome is sometimes very far from the forecast."

Forecasting uncertainties arise from
known factors that can only be established as probabilities, and factors that
"we don't know that we don't know." Unpredictable events will always
intrude on the future.& Fortunately, shocks and shifts come in
both favorable and unfavorable varieties, and so to a certain extent -
especially over longer periods - will average out. Forecasting models are
constantly checked to see how well they "characterize the existing
evidence," and are solved for average outcomes. They seek to reveal
"the average future." Unfortunately, in economics, major unpredictable
"shocks" occur all too often - resulting in major forecasting
failures, especially for the near-term forecasts that are so important for
policy-making, business and investment purposes.& Forecasters must deal with:

"Stochastic trends" - changes that
are regular and persistent;

"structural breaks" - changes that
are large and sudden;

inaccurate data; and,

"mis-specification" of model
elements.

"Structural breaks appear to
explain why it is so hard to reduce forecast error standard deviations: the
outcome is sometimes very far from the forecast. The 1929 crash and ensuing
Great Depression is the classic example of when large forecast errors
occur."

The Great Depression was not
unpredictable. There were those who provided reasoned
predictions - albeit uncertain as to timing. In fact, the financial press was full of reports
of widespread and increasing nervousness among investors and others
throughout the eight weeks prior to the crash. British investment trusts
quietly abandoned the N.Y. market in September, 1929. See, Great
Depression Chronology,
"The Crash of '29."& However, the bulk of the
cognizant officials and most prominent economists simply ignored increasing
financial distress abroad, and refused to evaluate
the weaknesses in the government's trade war economic policies, private over-expansion
- especially in agriculture, and, as always, the reckless extent of private
leveraging that had grown in some economic and financial sectors during the
prior period of prosperity.

Data that is frequently subject to substantial
subsequent revision presents additional problems for shorter term forecasts.

Forecasting models are far from accurate
representations of complex, dynamic modern economic and commercial systems. Even
models that attempt to closely represent the economy and that provide reasonably accurate
forecasts for one or more periods may suddenly prove unreliable thereafter.
Abrupt shifts arise from technological developments, political turmoil,
legislation and regulation, and similar factors.& Recent examples for England of developments with major impacts on
economic data and outcomes include the elimination of exchange controls -
privatization - and the introduction of interest-bearing checking accounts. Data that is frequently subject to substantial
subsequent revision presents additional problems for shorter term forecasts.& Thus, "naive predictors" that simply
extrapolate from previous results may more rapidly reflect such shifts and
provide more accurate forecasts over time.&

Forecasting
failures under or over the expected result are not unusual, but they should not
be permitted to persist for a significant duration.

Forecast models thus must "adapt" to
sudden shifts to minimize the duration for forecast failures. Forecasting
failures under or over the expected result are not unusual, but they should not
be permitted to persist for a significant duration.& Alternative forecasts or forecasts of average expected
outcomes can be used to deal with periodic but rare events.

The editors refer to oil shocks. When these are caused
by conflict, they may indeed be unpredictable, but when they are caused by
ordinary supply constraints, any competent economist should be able to see
them coming. FUTURECASTS had no trouble predicting in its February, 2001 "Near
Futurecast" that this current economic revival would be accompanied and
constrained by substantial energy shortages and high oil prices. It was an
obvious call.

Forecasting methods:

How forecasts are derived, their
degree of accuracy and precision, and the duration covered vary widely.&

"Because of the things we don't know [that] we don't
know," and because the future is subject to known but apparently random
uncertainties, the future is "unpredictable."

Professional economists don't necessarily provide
"predictions," David F. Hendry of Oxford Univ. (UK) explains. He
distinguishes between "predictions" and "forecasts." The
latter are the results of established techniques, but with no assurance of
accuracy for any individual period. "Anything can be forecast, but not
everything can be predicted."& A forecast is more probabilistic than a prediction as
those terms are used in this book. Like weathermen, professional economists
"forecast" - they don't "predict." Economists forecast
"unpredictable" events.

But weathermen at least attempt to create and apply valid
concepts in their forecasting models. They accept their responsibility to
forecast weather changes and storms. All too many economists have been mired in
unreliable statistics and obviously invalid, ideologically based concepts, and
now admit their inability to forecast economic changes and crises.

"Because of the things we don't know [that] we don't
know," and because the future is subject to known but apparently random
uncertainties, the future is "unpredictable." The latter uncertainties
are called "measurable" uncertainties. It is the former that pose the
most intractable problems.

Many macro-econometric models have been
"mis-specified" by reliance on Keynesian concepts that are clearly,
grossly invalid. Such models cannot even be said to provide
"forecasts." They merely - blindly - yes, stupidly - "project"
invalid results of invalid concepts.

Forecasts may be thrown off by major disturbances that
are inherently unforeseeable. A major earthquake or the sudden outbreak of war
are factors that can cause forecast failures without implying weakness in the
economic forecasting effort.& However, Hendry includes major changes in equity prices
and exchange rate crises as perhaps "inherently unpredictable."

While the exact timing may be unpredictable, such
events are frequently quite predictable within reasonable timeframes - and
should be included as possibilities in forecasts by competent economists
during those timeframes. What is required is professional analysis and opinion
- not blind reliance on mathematical technique and inherently limited and
often inaccurate data. Economics is, after all, a "profession" -
like law and accounting - a "practical art" - not
a "science." Total reliance on mathematical forms of reasoning
constitutes professional incompetence.

Econometric models depend on the accuracy and
proper inclusion of significant "deterministic terms" (average levels and
yearly or quarterly or other periodic trends) - "observed stochastic
variables" (known variables) - and "unobserved errors" (estimates
or averaging out of unknown variables).

"Relationships involving any of these three
components could be inappropriately formulated or inaccurately estimated, or
could alter over time in unanticipated ways. Each of the resulting nine types
of mistake could induce poor forecast performance through either
inaccurate - i.e., biased - or imprecise - i.e., high-variance -
forecasts."

The most important task is to develop "forecasting models
that are more robust to shifts," rather than making
improvements in the models themselves.

Actually, the worst econometric forecasting failures
are caused by unforeseen shifts in the values of average levels or period trends, rather
than shifts in variables. Thus, the unsophisticated models that do not depend on
data that may be erroneous or trends that can shift can often perform better
than more sophisticated models that get tripped up by major variances in the
wide variety of factors that they include.& All models must be subject to frequent adjustment
to reflect the economic shifts that impact the models. There is much less excuse
for a sequence of poor forecasts after an unforeseen shock than for the initial
forecast error caused by the shock.& Hendry thus asserts that the adequacy of the underlying
economic theory is less important than simple errors caused by shifts in yearly,
quarterly or other period trends. The most important task is to develop "forecasting models
that are more robust to shifts," rather than making
improvements in the models themselves. (However, faulty economic theory may
be what renders such shifts "unpredictable.")&

Extrapolation is good only as long as the tendencies persist. It
must always fail to predict the turning points in those tendencies, which
are inherently the most useful of forecasts.

Leading indicators (such as interest rate movements
and surveys of business and consumer sentiment) are good only so long as they
remain leading indicators. The need for frequent changes in the indicators
highlights "their inability to capture" many of the underlying
economic changes.

Surveys provide useful information about interviewee
plans. However, such plans can change. The reasons for such changes are not
always evident from the surveys.

Time-series models (that generally assume that past
relationships will persist) have provided forecasts with
reasonable accuracy. These are statistical models based on "past data
of the variable in question [that are extrapolated] into the future."

Econometric forecasting models based on empirical and
theoretical knowledge of the functioning of complex economic systems "provide
a framework for a progressive research strategy." They provide
forecasts and policy advice, and can even "help explain their own
failures."

To repeat, economics is a profession - not a science. Professional
analysis and opinion is not mere "guesswork" and "hunches,"
and can be far more reliable in the hands of a knowledgeable professional than
mathematical forms of analysis based on the faulty theory and the inaccurate
statistics generally available to the mathematical economics technicians that
rely on them.

Professional economists generally rely on time-series and
econometric models. These are "causal" models based on the study of
underlying causes.

Significantly, Hendry fails to include professional
analysis of the impacts of faulty government economic policies or even
analysis of ordinary business cycle phenomena like excess leveraging,
over-expansion, and continued use of marginal facilities during prosperous
times. Of course, much of this will accumulate in the data - but all too
frequently, the data in which it accumulates is not adequately represented in
forecasting models because it comes into play at infrequent and irregular
intervals.& Accounting for such data requires an effort similar to that being
developed for forecasting volcanic eruptions - as Hendry himself notes in a
somewhat different context. Unfortunately, much has changed in how statistics
were derived in the 1970s. Thus, current statistics are not strictly
comparable with those of the 1970s and are less likely to provide any reliable clues about
the similar rumblings now beginning to be felt under the economic surface.

Economic forecasts generally are both multi-period and
multi-variable. They will sometimes miss a rate of change even when they get
the forecast levels right - and vice versa. Therefore, judging the quality of a
forecast depends on what the forecast is being used for.&

"Confidence intervals" are established
according to the calculation of likely outcomes during a certain period of time.

Fan charts and other displays of confidence
intervals usefully dispel
the illusion of precision when a forecast is presented as a particular number.
However, even fan charts fail to reflect uncertainties that "we don't know
that we don't know." All forecasting is tentative in nature.

An estimate of
the uncertainty involved will frequently be included in economic forecasts. These "confidence intervals" are established
according to the calculation of likely outcomes during a certain period of time.
(The chance of rain tomorrow will be x%.) Confidence intervals generally widen as the
forecast time horizon lengthens.& The Bank of England provides inflation forecasts with "fan charts"
displaying the range of uncertainty widening into the future. Fan charts and other displays of confidence
intervals usefully dispel
the illusion of precision when a forecast is presented as a particular number.
However, even fan charts fail to reflect uncertainties that "we don't know
that we don't know." All forecasting is tentative in nature.& For several decades, economists have been testing and
refining their models against computer simulations and actual economic outcomes.
The primary problem remains unanticipated shifts in trend rates.
Fortunately, many shifts don't have significant impact on forecast results. Even
inaccurate data and model errors will frequently have little impact. This is a
reflection of the vastness and complexity of modern economic systems. However,
these weaknesses working together instead of individually or averaging out can cause forecast
failure.& Again, Hendry refers to major stock market or exchange
rate moves or various crises as inherently unpredictable variables that can
undermine forecasts. Sometimes, a forecast can undermine itself - as when
changes in sales taxes or interest rates or some other trouble are forecast and
impact consumer action or policy response.& Hendry demonstrates the impact of the start of the
Industrial Revolution on a 50 year forecast covering 1800 to 1850 using several
simple forecasting methods. The sudden growth acceleration threw the models off,
but those that included adaptive mechanisms fared much better. With adjustments
every 10 years, the 50 year forecast becomes much better. Hendry concludes:

"While economic forecasts from econometric systems
have a poor historic track record and face many potential and real problems,
the recently extended theory of economic forecasting offers a vehicle for
understanding and learning from failures, and for consolidating our growing
knowledge of economic behavior. Consequently, despite their present travails,
econometric systems provide the best long-run hope for successful economic
forecasting, especially as suitable methods are developed to improve their
robustness to unanticipated breaks."

Economic models:

The comparative advantages
of theoretically based "structural" econometric models and more simple
time-series models are discussed by Paul Turner, Sheffield Univ. (UK).&

Models based on economic theory can analyze shifts
in economic policy if the policy variables are included in the model. This is a
major advantage over pure time-series models that merely assume that past
relationships will persist.& Disadvantages of such "structural" models
include the greater time and effort involved, and professional disputes over
theory that may impact the analysis.

"Moreover, the forecasts generated by structural
economic models are often no 'better' than those generated by time-series
models, at least when assessed statistically in terms of variability and
degree of bias from the actual outcomes. Because structural economic models
are so costly and the resulting gains in forecasting accuracy are likely to be
small, the question naturally arises as to why anyone would adopt a structural
modeling approach rather than a time-series approach."

Although time-series forecasts are quick and easy
to compute, a structural approach has significant advantages by being able to
cope with a variety of different scenarios."

Simple time-series forecasts will not provide
explanations for their results, however. Such explanations are often demanded by business or
political or media consumers of the forecasts. When a consumer asks how policy
changes or major shifts will affect a forecast, only the structural model will
provide answers. By providing a series of forecasts reflecting a number of
possible shifts in major exogenous factors, the conditionality of the forecast
is highlighted.

"Forecasts are necessarily conditional on a set of
assumptions about the future. These assumptions should be recognized when the
forecasts are constructed, and they should be stated clearly when the
forecasts are presented. So, although time-series forecasts are quick and easy
to compute, a structural approach has significant advantages by being able to
cope with a variety of different scenarios."

The two methods can be used together. Variables outside -
"exogenous to" - the structural model can be forecast by time-series
methods and the values then included in the structural model, which then forecasts the
variables included - "endogenous" - within the model.& Turner demonstrates how a basic econometric model of the
United Kingdom economy can be used to reflect the results of two different disinflation
policies - either the adoption of a low, stable growth rate for the money supply
as Milton Freedman advocated, or active adjustment of interest rates to target
inflation directly as advocated by J. B. Taylor. Both policies arrive at the
same result, but the Taylor approach was shown by the model to take effect and stabilize more
quickly.

The Taylor approach, however, is more vulnerable to
changes in the definition of "inflation" and the methods by which it
is calculated. Although not without problems of their own, money supply data
are far less vulnerable to manipulation than inflation data. This is a matter
of professional judgment, not economic modeling.

Interpretation of economic forecasts:

Forecast uncertainties
are emphasized by Diane Coyle, Enlightenment Economics (UK), in her
explanation of how economic forecasts should be interpreted and used.&

"Journalists could
improve their own presentations of forecasts, as by avoiding spurious
precision and by explaining the uncertainties involved in forecasting."

Coyle developed the "Golden Guru" forecasting award
for the most precise forecast of the UK's "misery index" composed of
the rates of inflation, unemployment and economic growth. Unsurprisingly, the
winners have been established by very narrow margins. More revealingly, no
economist has won more than once. Coyle candidly acknowledges some of the
weaknesses of her rough evaluation method.

"Forecasters face a difficult task; and journalists could
improve their own presentations of forecasts, as by avoiding spurious
precision and by explaining the uncertainties involved in forecasting."

Because forecasters tend to "herd together," the average
forecast is always in the top few positions. So far, forecasts at one extreme or
another have never won. They almost always make a big error on at least one of
the three variables.&

"We don't know the true structure of the
economy, the true values of parameters in econometric models, or the
exact shocks that hit the economy; and we may never obtain accurate data
measurements either."

Uncertainty is inherent in economic forecasts. With commendable
candor, Coyle perceptively explains some of the most prominent aspects of the
problem.

"Part of that uncertainty is not knowing what has happened,
even long after the event. We don't know the true structure of the economy,
the true values of parameters in econometric models, or the exact shocks that
hit the economy; and we may never obtain accurate data measurements
either."

Forecasts are generally reported with "spurious precision," Coyle
points out. Words like "around," "approximately,"
"probably," and "might" are not favored. Instead, dramatic
terms are employed that routinely overstate the implications of events. Unfortunately, the media generally fail to reflect or explain the
uncertainty inherent in economic forecasts.

The publisher of FUTURECASTS has
been explaining just that in articles published over the last 40 years. The mass
media is all too often nothing but a conduit for authoritative misinformation.

Ranges of uncertainty are provided with UK Treasury forecasts but are
never reported by the mass media. The Bank of England provides fan charts of inflation and GDP
forecasts displaying the confidence intervals widening into the future.
Unfortunately, these charts are used by the media all too infrequently.& When an economy is expanding at about trend rates, forecasts tend to
be good. (A straight line graph would work just as well at such times.) It is when the economy accelerates or decelerates or turns that
economic forecasts generally fail. Unfortunately, the trend shifts and turning
points are precisely the times when good forecasts are most needed. Forecast
errors delay appropriate government policy responses.&

There are inherent ambiguities in economic and government
statistics, and officials are prone to game the system to provide the figures -
the "body counts" - desired by their superiors. There have also been
substantial changes in recent times in the way various economic statistics are
derived.

The figures lie. (See, Economic
Statistics.) There are inherent ambiguities in economic and government
statistics, and officials are prone to game the system to provide the figures (the "body counts")
desired by their superiors. There have also been
substantial changes in recent times in the way various economic statistics are
derived (especially for dollar productivity and inflation statistics).& Benchmarks used to evaluate the quality of government services are
particularly prone to manipulation (as experience with U.S. educational performance
statistics currently demonstrates). All too often, official forecasts are
treated by officials as targets - that must be achieved - by hook or by crook.
Coyle explains how surgical waiting periods were reduced by the UK Public Health
Service by rushing through a mass of the less complex operations to meet targets.&
Macroeconomic policy suffers from lags that can vary
substantially, and lack of precision in the impact that can be achieved. The
lags and impact of interest rate policy changes, for example, are particularly
critical uncertainties for economic forecasters.&

The reasons for the forecast uncertainty
of econometric models is further explained by Neil R. Ericsson, Federal
Reserve System (USA).

"Economic forecasts typically differ from the realized
outcomes, with discrepancies between forecasts and outcomes reflecting
forecast uncertainty. Depending upon the degree of forecast uncertainty,
forecasts may range from being highly informative to being completely useless
for the tasks at hand."

Measures of forecast uncertainty have been devised to clarify
the expected range of outcomes and assess forecast reliability. They reflect the
dispersion of possible outcomes relative to the forecasts being made.& Ericsson provides examples from forecasts of the U.S. trade balance
and the Bank of England's inflation forecast fan charts and probability density
charts that show the range of outcomes within which the results will fall with a
stated degree - 90% in the example - of confidence. He explains how to interpret
several types of "histogram" probability density charts.&There are several ways to measure forecast uncertainty. These
measures are typically useful for decisions involving insurance and investments.&

Forecast uncertainty varies with the variable being forecast - the type
of forecasting model being used - the economic processes involved - the
information available - and the time horizon of the forecast.

Five major causes of uncertainty are identified.

Changes in the structure of the economy during the forecast period

Error - "mis-specification" - in the model.

Errors in the base period data.

Inaccurate estimates of such factors as the rate of change in basic
interest rates.

Unexpected "shocks" to the economy.

The first three are "what we don't know that we don't know."
The degree of uncertainty for the last two can be recognized and even
calculated.& Forecast uncertainty varies with the variable being forecast - the type
of forecasting model being used - the economic processes involved - the
information available - and the time horizon of the forecast.& For example, trade balances are relatively small quantities arising
from variances in imports and exports - two much larger quantities. Thus, even
if imports and exports are fairly accurately forecast, minor deviations can
result in major changes in trade balances.& Some models are simply inherently less precise than others. Some
economic processes result in more precise data than others. (Market data is more
precise than accounting data.) The limits of available data will vary. The
length of the forecast period will impact forecast reliability. Sometimes,
lengthened forecast periods will make forecasts less reliable, but sometimes
they will reduce uncertainty by permitting cyclical movements to average out.& Examples are provided by Ericsson with static trend models and dynamic
models. A graph of a static trend model of UK real net national income has 95%
fixed width confidence intervals extended indefinitely into the future, while a dynamic random walk model shows a
substantial fan shaped graph of 95% confidence intervals. Dynamic models frequently
imply that forecast uncertainty depends on the forecasting horizon, because
dynamic models can accommodate increased possibilities of economic shocks for
subsequent periods.

"More generally, static models commonly imply forecast
uncertainty that is time-invariant or nearly so, whereas dynamic models
typically imply time-dependent forecast uncertainty, often increasing in the
forecast horizon. The trend and random walk models above present static and
dynamic relationships as black and white, but in practice a whole spectrum of
models exists with both static and dynamic features."

A random walk dynamic model for forecasting pound/dollar exchange
rates for 1971 to 2000 varies from high confidence levels for one month
forecasts to vast forecast uncertainty for two year forecasts. The result of the
pound exchange rate crisis of 1992 feeds substantially greater forecast
uncertainty into subsequent periods even though in the event exchange rates
subsequent to 1992 proved increasingly stable. In this example, the dynamic
model proved less reliable than a static model. Redesign of models to reflect
more recent data "is a topic of much current research in economics."

How about research into professional evaluation of government
budgetary and monetary policies and payments balances? The degree of monetary
stability is, after all, not an act of nature like the weather. It is a
product of the policy of the monetary authorities.& The restoration of UK exchange rate stability and strength was no accident. Acknowledgement of
Conservative Party government policies after 1992 that were clearly supportive
of exchange rate stability and strength could have served to greatly narrow
pertinent forecast uncertainty margins.

Information about forecast uncertainty can be as important as
the forecast itself, Ericsson emphasizes. It provides information about likely
outcomes and qualifies the forecasts. It provides help in evaluating the models
and assists in efforts at improvements. It reveals the importance of the unknown
unknowns that impact an economy during a forecast period by indicating if the
results were thrown outside the acceptable confidence interval.

"Measures of forecast uncertainty also provide economists with
a way of assessing the importance of unmodeled features of the economy, both
directly through the calculated forecast uncertainty, and indirectly through
comparison of that calculated uncertainty with the realized distribution of
forecast errors."

Evaluation of forecasts:

"Forecasts are made for a purpose."

&

Economic models must be evaluated in
light of their specific purposes, Clive W. J. Granger, U. Cal. (USA) tells
us. Then, the forecast must be evaluated with respect to how
well it fulfills its specific purposes.

"Forecasts are made for a purpose, with those forecasts
typically providing the basis for economic decisions and with the resulting
forecast errors entailing economic costs. Different models generate different
forecasts, and the resulting economic costs have different distributions,
which can be compared across models."

Because of the complexity of the task and the limits of the art, some
probability of error is always involved.

Generally, models are either theoretical or empirical.

"[Theoretical] models are constructed from logic, mathematics,
and sets of generally agreed-upon behavior by economic agents in the form of
axioms and assumptions. The empirical models arise mainly from the analysis of
economic data, possibly at least partially based on economic theory."

The variety of models is useful since no single model can represent
all the outcome-determinative variables in a vast modern economic system. The
varying results inform about the range of possibilities that can be forecast.
Many models are designed to deal with just specific segments of an economy.
Evaluation should determine how well a model represents the main economic
features of interest and how well it is performing compared to other models.& Models can be designed to assist with investment or policy decisions
or to provide general or sector forecasts or to test an economic hypothesis.
Because of the complexity of the task and the limits of the art, some
probability of error is always involved.& Granger provides the example of a central bank decision to take action
against inflationary expectations. Because of lengthy lags in the impact of
monetary policy, it may decide to act when inflation rates are forecast to
exceed
acceptable limits within some reliable confidence level.&

The question is thus: What is the cost of the error rate to the
decision makers who use the forecast? The importance of forecast errors that
miss the mark in one direction may not be the same as when they miss the mark in
the other direction.

A forecast of the start of a bear market that is too late is
generally worse than one that errs on the side of being too soon - except for
the bear investor. In forecasting economic turns, FUTURECASTS strives to be
within six months and to err on the early side.& The current FUTURECASTS market forecast is - and has been for
several years - that the market will be range-bound, with a bottom more than one third below
its top. An
austerity recession is now essential to overcome the inflationary impact of
recent Keynesian policies in the U.S. - and the longer it is put off, the worse the
inflationary problems and the deeper the ultimate austerity recession will
have to be.& Timing is a subject of
political forecasting as much as of economic forecasting, since timing is
dependent on government policy.

The evaluation of error costs is discussed in some detail by Granger.
He provides a mathematical formula for that purpose that takes into
account the asymmetrical nature of error costs. Of course, there may be many
consumers of a particular forecast - some of whom may be unknown to the forecast
provider - and each with its own error cost function. Each consumer must thus
evaluate the error costs of different forecasts for itself - based on the costs
to that consumer of working with the forecast's uncertainty range and error
rate.& Thus, evaluating competing forecasts depends critically on the needs
of each consumer, and there may be no "best" forecast model. The
higher the cost of decision error, the more critical the forecast confidence
level becomes.&

Business cycle forecasting:

&

The forecasting of business cycles using
econometric models is particularly difficult due to the (fortunately) infrequent
nature of the event. Denise R. Osborn and Marianne Sensier, Univ. of Manchester
(UK), and Paul W. Simpson, Sheffield Dep't. of Ed. (UK), discuss
"regime-switching models" using short-term interest rates as a leading
indicator of such economic switches. The interest rate decisions of the Bank of England are thus of
primary importance.

"Large increases in the interest rate raise the probability of
switching out of an expansion into a recession a year later, whereas, in a
recession, even small decreases in the interest rate help start a
recovery."

Inability to predict business cycle booms and
busts has resulted in widespread criticism of professional economists. Written just before
2000, the book notes that economic forecasters
missed the extent of the boom in the UK of the late 1980s and the related onset and
duration of the UK recession of the early 1990s.& The respected, independent National Institute of Economic and Social
Research failed to forecast declines in real GDP for the UK even as late as August,
1990, and when decline was recognized thereafter, forecast steady recovery throughout the next two
years although the recession lingered on in England for some time. Thus, the authors question whether
forecasting tools - particularly forecasting models - are capable of predicting
economic turning points. However, models that generally fail to forecast
economic turning points may nevertheless be the most accurate for periods that
do not involve turning points.& Between WW-II and the end of the century, there were only three
periods when the UK was in recession -
suffering at least two quarters of actual decline in GDP.
These occurred in the mid-1970s, the early 1980s, and the early 1990s. Not only
has the event been comparatively rare, each recession has had different
characteristics. (The methods by which the relevant statistics have been derived
have also been changing.) Nor is there universal agreement on the precipitating causes in
the chain of events leading up to the recessions.

Were the first two of these
recessions "caused" by sharp increases in oil prices - as so many
economists stupidly assert - or by the decade of Keynesian economic policies in
the U.S. that undermined the value of the dollar, made those sharp increases in oil prices
possible, and forced less accommodative monetary policy? It is no mere coincidence that current oil price spikes have come
after renewed substantial resort to inflationary Keynesian policies - and that now Fed
monetary policy is becoming cautiously less accommodative while commodity prices
are soaring .

The beginnings of recoveries, too, are thus rare events. The economic
policies adopted to encourage recovery have varied in type and impact.&

However,
interest rate policy is always an important factor, and so that is the variable
relied upon by the authors. They recognize that the impact of interest rate
changes may be different when moving from an expansion to a recession than when
moving from a recession to an expansion. They highlight the impacts of Bank of
England interest rate decisions. The yield on the 3 month UK Treasury Bill is
the interest rate used.& The authors discuss both univariate (single variable) linear models and univariate
"regime-switching" models. The former can't recognize shifts between
economic expansion and recession, and the latter has been generally unsuccessful
at regime-switching forecasts. The latter "recognizes" these shifts after they occur,
but then fails to forecast the next shift. The problem is the application of a
very small - 4% - likelihood of a shift in any particular quarter due to the
comparatively rare occurrence of such shifts.& The authors add short-term interest rate policy to these models.
Interest rates and other leading indicator "explanatory" variables are
used in multivariate (multi-variable) linear models in the attempt to forecast recessions. Lags
for the interest rate indicators of 5 to 7 quarters are also included.
"Thus, current or past observations on such variables are used to forecast
future output growth." Interest rate increases of at least 3 percentage points within a
short period are needed to move the referenced regime-switching model from
likely continued expansion to probable decline.&

The model still does poorly when applied to the 1990s recession, but
at least recognizes the recession when it occurs. However, it's a year early in
its recovery forecast.& Because there is so little data from the three relatively short post
WW-II recessions, the data reflects anomalous and clearly useless results for
evaluating recession-to-expansion probabilities from declining interest rates.
Several large anomalous interest rate policy shifts during the 1970s and 1980s
recessions have to be disregarded to get meaningful results. With that, the data
seems to indicate that interest rate declines have a more powerful impact on
recession-to-expansion probabilities than interest rate increases have for
expansion-to-recession probabilities.& "[Changes] in the interest rate alone do not appear to have
been sufficient to forecast the onset or length of the 1990s recession."
Interest rates are thus being supplemented with other financial variables as
leading indicators to help with the "exacting task" of forecasting the
business cycle.&

Bank of England forecasts:

&

The interest rate decisions of the Bank
of England are supported by various modeling and forecasting tools that
forecast economic output and inflation rates and estimate the impact of interest rate policy. The
forecasts are published in the Bank's Inflation Report.&

The
forecasts are published in the Bank's "Inflation Report."

Efforts to develop models of increasing sophistication and
complexity reflecting ever more closely the complexities of the economy have
been abandoned, Neal Hatch of the Bank of England explains. The results were
invariably poor and did not justify the effort.& Current efforts concentrate on developing smaller and more compact
models. They seek to achieve clarity by simplifying the analytical process. They
are used to help explain current conditions and how conditions develop as well as to
provide forecasts - insights into future probabilities. They are not good enough
to be relied upon mechanically.&

Judgment is required at every step of the process. Other
information - such as survey data and reports from regional agents - are vital
components of the decision-making process. (This sounds like a professional
evaluation process by professional economic policy makers.)&The Bank uses its "core model" supplemented by other
models to produce its probabilities forecasts - its fan charts. Various
assumptions and judgments are included in these models. Consideration of other
issues and policy judgments along with the forecasts go into the formulation of
policy. Thus, the forecasts are that of the Monetary Policy Committee (MPC) -
not of its staff.& The August, 1999 Inflation Report went even further. It
included the differing views within the Committee - and "explained how
alternative assumptions would shift the profile of forecast
inflation." It included a table showing the impact of "alternative
assumptions about variables such as earnings, profit margins, exchange rates,
and oil prices."

The inclusion of earnings and profit margins in the complex of
variables being considered is another vast improvement over 1970s economic
forecasting efforts.

The staff produces a preliminary central projection that begins
an iterative process of discussion and analysis of results, risks and
uncertainties that ultimately produces the final result - including the Bank's
probability fan charts of inflation and growth forecasts.

The forecasting effort is "intensive." Initially,
"provisional assumptions are made about variables exogenous to the core
model." To save time, some assumptions are automatic. Hatch notes that the
starting point for exchange rates is its average during the 15 working days
prior to the meeting, and the short term interest rate is as set by the MPC just
prior to publication of the Inflation Report. However, all assumptions
can be discussed and altered.& The core model is continuously adjusted in line with actual data.
However, this requires judgment as to whether unexpected data results are mere
random variation - perhaps involving measurement errors - that may subsequently
unwind. Judgment is also required when including the impacts of unexpected
exogenous events - such as a surge in energy prices, a financial crisis, or a
major change in taxation. Pertinent assumptions will be revealed in the Inflation
Report.& The staff then produces a preliminary central projection that begins
an iterative process of discussion and analysis of results, risks and
uncertainties that ultimately produces the final result - including the Bank's
probability fan charts of inflation and growth forecasts. These reflect the
forecasts and their "forecast error uncertainty" based on prior
experience.

"They allow the MPC to express judgments about the skewness and
variance of the forecast errors, and not just about the central case -- i.e.,
the most likely single outcome."

The core model is supplemented by various other models including
narrower models of such variables as the labor market showing impacts of such
policy shifts as minimum wage or tax credit changes. Other models incorporate
differing degrees of sophistication based on such factors as the aggregation or disaggregation of
data. Some models are data-driven, while others incorporate economic theory.& The core model itself incorporates more theoretical aspects for longer
range forecasts. It is supplemented with survey information - such as consumer
and business sentiment - and information from the Bank's regional agencies.
These are vital because they are more current than official output data and are
considered leading indicators of output.& The Inflation Report also includes other forecasts with various
graphs and charts displaying ranges and probabilities. This forecasting exercise
is constantly evolving, and still too recent for meaningful judgment about
accuracy. Such judgments are rendered more difficult when economic outcomes are
impacted by unpredictable shocks.&

Since the economy takes time to adjust, short-run tradeoffs do
exist between inflation and output as reflected by Philips-curve theory.

The core model is not designed to reflect the damaging impact
of inflation on long-run forecasts of unemployment and output, although Hatch
notes that persistently high levels of inflation will indeed have damaging
results. However, since the economy takes time to adjust, short-run tradeoffs do
exist between inflation and output as reflected by Philips-curve theory.& As a
model of a relatively small, open economic system, the core model reflects the strong
influence on domestic output and inflation of exchange rates and international
trade, growth and prices.&

These models are just tools. The forecasting task still remains an
"art" as well as a "science," "especially when
iterating between models, data, and economic judgment."

Supplementary forecasting models are of various types.

Philips-curve models relate price and wage inflation to estimated
employment or the output "gap" or other measures of real
disequilibrium. They can be important tools for analyzing short-run
disequilibria.

Small-scale macroeconomic models are complete models of the economy in
highly aggregated form. They are useful as "test beds."

Vector autoregressive models can capture the dynamic interactions between
particular variables. They are data driven - generally not dependent on
theoretical assumptions. Structural versions can be used to investigate the
impacts of shocks - both positive and negative - on pertinent variables.

Optimizing models assume that individuals act to optimize their economic
outcomes. They are useful in evaluating the impact of shocks - both positive
and negative. The demutualization of Building Societies in England provided
widespread windfall gains that could be analyzed by these models.

The latest econometric modeling approach relies on the "microfoundations"
- individual households and firms - of macroeconomic behavior. It relies on an
evaluation of how the "representative" household or firm makes its
choices. A new generation of macroeconomic models based on microfoundations is
now in use by central banks. They include Canada's Quarterly Projection Model,
the Bank of England Quarterly Model, and the IMF's Global Economic Model. See,
"The Economist," 7/15/06, pp. 67-69.

"The Bank uses a suite of models because specific issues often
call for new or different tools. It is not possible to construct a single
model that will perform all tasks. The Bank's models range in size,
complexity, and the role of economic theory. Although a core macroeconometric
model helps the MPC make its projections, that model's use depends on various
judgments, many of which are informed by other models."

These models are just tools. The forecasting task still remains an
"art" as well as a "science," "especially when
iterating between models, data, and economic judgment." (That's what the
publisher of FUTURECASTS has been saying and writing about for 50 years.)

It does, indeed, sound like an
exercise in professional judgment - not mere "guesswork" or "hunches." To repeat, economics
is a "practical art" - like law
and accounting - requiring professional forms of understanding and analysis -
resulting in statements of professional opinion on which the professional
stakes his reputation.

Forecasting the world economy:

The characteristics and uses of the
world econometric model of the National Institute of Economic and Social
Research (UK) is explained by Institute economist Ray Barrell.&

Uncertainty constitutes a vital component of these forecasts, so
"confidence intervals" are reported with them in the Institute's
quarterly "National Institute Economic Review."

Widely used short and longer term forecasts,
and analyses of the likely impacts of shocks - both positive and negative - and
changes in economic policy, are provided by the Institute.& Uncertainty constitutes a vital component of these forecasts, so
"confidence intervals" are reported with them in the Institute's
quarterly National Institute Economic Review. Some of this uncertainty is
decomposed by source - such as uncertainty about exchange rates, consumption
rates and
investment rates. Unexpected shocks that impacted forecasts in the 1990s
included the Asian Contagion crisis, the reunification of Germany, the sudden
growth spurt in the U.S., and the collapse of the Long Term Capital Management (LTCM)
hedge fund.&

There is a great deal of judgment involved in determining
how such changes should be reflected in econometric models.

The National Institute Global Econometric Model (NiGEM) is
widely used. It covers "all OECD countries individually and all non-OECD
countries individually or in blocks." It is constantly being revised to
reflect the continuous flow of EU structural changes. Globalization - the
collapse of the Soviet Union - economic integration and changes in trade policy
- are recent examples of structural changes with worldwide impacts. There is a great deal of judgment involved in determining
how such changes should be reflected in econometric models.& Examples provided by Barrell include labor market data through 2000
with forecasts thereafter for Germany, Finland and Sweden.

The gradual improvements forecast for Germany failed to materialize
when expected, but subsequent labor market reforms now seem to have reached
the point where they can have some favorable impact. The forecast for the beneficial impacts of labor market reforms
in Sweden and Finland have proven closer to the mark through 2005 for
Finland, but too pessimistic for Sweden into 2006.

The Institute's analysis and forecasts of the impacts of the Asian
Contagion crisis were far less pessimistic - and thus far more accurate - than
that of many other forecasters. Rapid adjustments and recoveries were accurately
forecast. The global market capitalist system demonstrated its resiliency and
ability to absorb shocks.& However, only a "bimodal" 60% - 40%
forecast favoring avoidance of widespread financial collapse could be provided
for the LTCM collapse since the outcome depended crucially on
appropriate government policy responses.&

Low unemployment and inflation rate declines in the U.S. in the 1990s
puzzle Barrell.

"Standard economic theory suggests that inflation should
increase when the unemployment rate falls below its long-run sustainable
level, which might also be the long-run average if markets work well and do
not change too much."

Barrell here exhibits the limits of his professional
understanding. Standard economic theory is obviously wrong! Inflation is
ultimately a monetary phenomenon - not a labor market phenomenon. Nor is
there more than a short-run tradeoff between inflation, output and
employment. Ultimately, inflation ALWAYS causes rising rates of
unemployment. See, Capital as
Purchasing Power, segment on "The determinants of purchasing
power," and Understanding
Inflation.& The reason why U.S. unemployment declined in the 1990s was
precisely because of the success in squeezing out the last of the 1970s
inflationary pressures. The reward was lower nominal interest rates, a strong dollar
and reduced risks which, along with a reasonable degree of labor market
flexibility, permitted rising employment and growth rates and price
stability all at the same time.
Nothing "unusual" about that at all!

However, Barrell correctly attributes much of this economic success to
a strong dollar and the rising demand levels of prosperous consumers. A strong
currency does much to dampen inflation (and fend off a variety of other economic
and financial ills - like the Asian Contagion). The fly in this picture
(actually, the proverbial 800 pound gorilla), is the rising U.S. trade and international
payments deficits that Barrell correctly identifies as unsustainable - a
significant problem for the future. (It is much worse, now.) However, he
expresses puzzlement over the strength of the dollar in the late 1990s.

Just another benefit of declining rates of inflation and the
substantial improvement in the federal government's budget in the 1990s due to major
reductions in military spending. Again, nothing "unusual" at all.

Another benefit is the "productivity miracle" enjoyed by the
U.S. The author asks why other nations are not enjoying similar benefits from
modern technology.

The multiple factors of economic flexibility provide much of
the answer. Moreover, the change in the way productivity is calculated may
explain more than the government wants to admit.

Barrell presciently feared the early demise of the 1990s period of
prosperity. He explains how the NiGEM model helps evaluate forecast uncertainty
by running simulations of various kinds of "shocks" and corresponding
interest rate shifts. He demonstrates this analysis using a sudden increase in
inflation above tolerated levels as the "shock." He notes the
importance to the small open economy of the UK of a stable exchange rate for the
pound. Stable exchange rates also substantially increase confidence levels for
economic forecasts by eliminating an important element of risk.&

Forecast errors:

&

Complaints about econometric forecasting failures
are summarized by Terence Burns, House of Lords (UK). A sketch of historic
forecast errors provides substance for these complaints, and the causes of
forecasting failures are perceptively explained.&

Sophisticated models can be large and difficult to understand - can be
based on assumptions that are implicit rather than explicit - and thus can
hide dangerous weaknesses.

"[The] profession appears to have made very little progress
in reducing the size of forecast errors over the past 30 years or so, whether
for the United Kingdom, the United States, or other industrialized
countries."

"The profession appears to have made very
little progress in reducing the size of forecast errors over the past 30
years or so."

Econometric forecasters apply "rigid and mechanistic
models of the economy" to produce forecasts that are thrust upon the
economic policy-making process.

Sophisticated models can be large and difficult to understand - can be
based on assumptions that are implicit rather than explicit - and thus can
hide dangerous weaknesses from policymakers and other customers. It may be
difficult to demonstrate the various properties of such models or to
understand the roles of the main forces behind a forecast.

Models may over-emphasize variables that have greater impacts in the short
run and inadequately consider variables that have significant longer-term
impacts.

There are suspicions of bias - a tendency to err on the conservative side
- so as not to expose the forecasters - so as not to put them out on a limb.

"Forecasters are also criticized for paying too much attention to
their models and not enough to what is actually happening in the
economy."

Burns, too, emphasizes that the greatest risk of forecast error occurs
when the economy most sharply shifts course - at turning points or moments of
substantial acceleration or deceleration - just when accurate forecasts would be
most useful. Forecast errors tend to be multiple - across multiple variables -
because of the relationships among variables. Thus substantial errors in
forecasting output will produce related errors in government tax receipts,
unemployment, and price movements.& Unexpected shocks may have greater than expected impacts. Burns, too,
uses the oil price increases of the 1970s as an example of such an unexpected
shock.

Again, this is a gross misstatement and over-simplification of the
causes of the economic problems of the 1970s. More than a decade of
accumulated inflationary forces - kept temporarily in check by the then fixed
dollar exchange rate - were at work behind those problems and behind the oil
price increases themselves.

However, sometimes - as after the 1987 stock market crash and during
the Asian Contagion crisis of the late 1990s - shocks can have surprisingly
little apparent economic impact.

A strong national currency provides significant shielding against such
shocks, and provides the government with the strength to deal with any problems
they may cause - just two of the many reasons why it is impossible to prosper with a
weak currency.

Inaccurate data can result in forecast errors. For short term
forecasts, recent data on which forecast models rely can be notoriously
inaccurate and subject to sometimes large subsequent revisions. These revisions
are often unfortunately largest precisely when the economy is shifting gears.

"[From] my interpretation of the research evidence as well as from my
own investigations, the profession appears to have made very little progress
in reducing the size of forecast errors over the past 30 years or so, whether
for the United Kingdom, the United States, or other industrialized
countries."

The old quantitative controls on credit and lending created useful data
for forecasters but were obstacles to efficiency and are now gone.

The floating exchange rates of
the modern world constitute major and difficult to forecast additional variables
that didn't exist in the fixed exchange rate period after WW-II.

Some of the causes of these difficulties are accurately noted
by Burns.

The increasingly free market global economic world is increasingly
complex.

The old quantitative controls on credit and lending created useful data
for forecasters but were obstacles to efficiency and are now gone.

Data for the service sectors
- which are growing - remain
less accurate than for the manufacturing sectors - which are shrinking as a
percentage of national output.

Private sector decision making has become
increasingly sophisticated and difficult to model.

The floating exchange rates of
the modern world constitute major and difficult to forecast additional variables
that didn't exist in the fixed exchange rate period after WW-II.

Business survey data is frequently useful for qualifying short term
forecasts - especially when evaluating the impact of shocks that may not as yet
have shown up in the economic data. However, it is hard to include such data in
a forecasting model, and survey data, too, has sometimes been in error. Burns
mentions the increasing pessimism of Autumn 1998 and winter 1998-1999 that
failed to reflect the booming conditions of 1999.

However, by 2000, there was
indeed a recession in the U.S. Perhaps interpretations of these surveys were just a bit
premature. Perhaps the expected recession was artificially delayed by the loose
monetary policies leading up to the mythological advent of the millennium bug.

When forecasts are used as one of the bases for economic policy
shifts, they must cover the additional complication of what the impact of
various policy shifts will be and how long the "lag" time will be
before that impact manifests itself. Interest rate changes, for example, have
historically had their maximum impact between 6 and 8 quarters after the change.
Interest rate changes impact many other variables such as exchange rates,
housing markets, inventories, investments and consumer spending.& Forecasting errors can thus lead to policy errors with consequences
throughout the economy. Because of the lags, these consequences may not get
adequately addressed for many months or even for more than a year.

"As mentioned earlier, forecasting is more difficult when
variables move sharply relative to trend. If a significant policy error takes
the economy well away from its trend, the initial disturbance can persist for
a long time."

A major problem with interest rate policy is that higher
interest rates are essential to eventually curb inflation - yet the rise in
interest costs is immediately reflected in inflation statistics. Mechanical use
of this data - that fails to adjust for this anomaly - has caused interest rate
policy to overshoot the mark - staying high until the inflation data has visibly
peaked - by which time the economy is already headed in the opposite direction.

A more gradual approach is currently being attempted by the U.S. Fed
- but the longer it takes for interest rates to rise sufficiently to deal with
inflation, the higher they have to go to achieve that objective. Inflation
rates have actually risen substantially during the two year period of
gradually rising interest rates. If they were calculated using 1970s methods,
they would be much higher.& Commodity price inflation data available from commodity markets
can't be fudged - and commodity price inflation has been running at multiple
double digit rates for several years. Industrial commodity inflation is currently
running astoundingly in excess of 50%.

Forecast errors can contribute to policy errors that increase
volatility with widespread economic consequences. Volatility generally creates
major problems for public finances and undermines confidence in both forecasts
and policy.&

New approach to econometric modeling:

The theory that the "best" model should provide the
"best" forecast has not yet been demonstrated.

&

Confidence in
econometric models remains elusive, Hendry and Ericsson note in a final
chapter. The theory that the "best" model should provide the
"best" forecast has not yet been demonstrated.

"Many econometric models for forecasting are known to be
seriously mis-specified, and the actual economy has been subject to important
but unanticipated shifts, so forecast failure has been a relatively common
phenomenon. Also, the implications of the above theory are inconsistent with
the results of empirical forecasting competitions between many models on
numerous time series. Simple methods often outperform better-fitting ones, and
pooling of forecasts -- i.e., using an average of forecasts -- can pay."

The ability to rapidly recognize and reflect forecast errors -
especially shifts in deterministic terms - data on average levels and period
trends - is essential to reduce the impact of
such errors and improve subsequent forecasts.

Simpler more adaptive models thus often outperform
more elaborate models, and long term forecasts often turn out to be more
accurate than short term forecasts. Business cycle shifts generally average out.

"A realistic alternative is to construct forecasts that adapt
quickly after any mistake is discovered, so that systematic forecast failure
does not ensue. Thus, econometric models might be redesigned to capture some
of the robustness of the simple models that win forecasting
competitions."

The ability to rapidly recognize and reflect forecast errors -
especially shifts in deterministic terms - data on average levels and period
trends - is essential to reduce the impact of
such errors and improve subsequent forecasts.

This type of "robustness" will indeed reduce the
size of long-term forecast errors, but does absolutely nothing to
improve the ability to forecast the shifts and turning points. That
must still be done as a matter of professional analysis and opinion -
not mathematical doodling often based on "mis-specified" - invalid
- theory and unreliable statistics.

Nevertheless, sophisticated representations of causal variable
connections remain essential for policy purposes. Only they can offer "rigorous"
estimates of the impacts of policy shifts.

"Rigorous," yes - as a matter of what is widely expected
by the profession. But how accurate do those estimates turn
out to be? "Rigor" in the application of invalid concepts is
useless. This otherwise commendable book fails to evaluate this important
point. Indeed, the accuracy of policy estimates is almost never subject to
subsequent evaluation.& Economist Timothy Kehoe, Univ. of Minn., points out that
econometric models drastically underestimated the NAFTA impact on trade flows.
They failed to reflect the explosion of exports of goods not previously substantially
traded. See, "The
Economist," 7/15/06, pp. 68-69.& Economist Ross McKitrick, Univ. of Guelph (Canada), demonstrated
how the way household and company responses are entered can drastically alter
an econometric forecast of the response to a tax increase. Id. In short, an economist can easily make an
econometric model deliver the result he wants. So much for
"rigorous" mathematical economics.