Forecasting The Economy: Do Presidents Get It Right?

by Randolph H. Boehm

Randolph H. Boehm is an editor at University Publications of
America.

Executive Summary

The growing significance of federal budget politics in
the American political dialogue cannot be dismissed. The
sums involved are fairly described as staggering and will
likely have a major impact on the performance of the American
economy. Consequently, budget policies have become a topic of
considerable concern. Supply-side economists commanded national attention when they questioned the revenue (tax-raising)
policies of the federal government, which they say pose a
serious threat to economic prosperity. Many of their critics, in turn, point to potentially disastrous effects of
federal deficits running into the hundreds of billions of
dollars. Special interest groups with a major stake in some
aspect of government outlays voice vigorous opinions on the
alleged shift in federal priorities from social services to
defense with some of the partisans warning of dire consequences should the shift continue and others of dire consequences
should it be checked.

The Reagan administration has helped force budget politics to the top of the American political agenda with its
stated determination to check a number of fiscal trends which
have been gaining momentum for at least a decade. Less appreciated, perhaps, but no less significant than the new administration in focusing attention on the federal budget is the cumbersome congressional budget process itself. It ensures that
budget politics consume an inordinately large portion of each
congressional session to the inevitable exclusion of many
other issues.

One important aspect of the federal budget that is frequently overlooked, however, is the reliability of forecasts
of major budget components such as the deficit, outlays and
receipts, Gross National Product (GNP), unemployment, and
inflation. In examining previous forecasts on these economic
variables in federal budgets between 1971 and 1982, it is
evident that the forecasters' records have been very poor,
that many indicators such as GNP and unemployment are seriously deficient as mirrors of economic reality, and that the
uncritical acceptance of aggregate budget figures by the mass
media, policymakers, and citizens alike is wholly unwarranted.

Aside, perhaps, from compiling the historical tables in
order to monitor the results of previous economic forecasts
used in the budget, much of the following analysis is not
original. Academic economists as well as economists in the
federal agencies who compile the data making up each of the
aggregates under discussion regularly point to their deficiencies. Most opine that these deficiencies can be overcome in
time with more complete statistical information and greater
conceptual refinement. Others are not sure that it will ever
be possible to make the aggregates accurately reflect economic
reality.

As an academic pursuit, the efforts to quantify data and
make predictions may be unexceptionable. Serious errors in
academic pursuits often prove beneficial so far as they serve
to educate. But such errors obtruding upon public policymaking or codified in law have pernicious effects far beyond
their educational value. More attention needs to be focused
on the limitations of these various budget components and of
efforts to forecast changes in them.

Policymakers' attitudes toward these concepts, however,
seem to reflect a trend that is moving in just the opposite
direction. President Reagan recently proposed "triggering"
stand-by income tax increases on a ratio between two of the
aggregates under discussion here -- the deficit and the GNP.
The Humphrey-Hawkins full employment bill of the last decade
proposed the unemployment rate as a "trigger" for wideranging public works projects. Other instances of this sort
could be noted. In addressing this trend of relying upon
aggregate economic statistics, a recent study by the Government Accounting Office discusses GNP figures:

The accounts help Federal policymakers pursue
the goals of the Employment Act of 1946 -- full
employment, price stability, and economic growth.
Federal economists use the accounts for short term
fiscal, monetary, and wage-price policy analysis,
for managing the nation's employment and anti-inflation goals, and analyzing long-term demands for
skilled labor and financing for capital formation.
Major users include the Council of Economic Advisers, the Federal Reserve Board, the Office of Management and Budget, and the Departments of Treasury
and Commerce....

Although the analytical uses of the accounts
are primary, additional uses are being made. The
Trade Act of 1974 (P.L. 93-618) specifies the use
of annual GNP estimates in determining limitations
on preferential treatment extended to countries
exporting goods to the United States. The GNP price
deflator is used as a component in the inflation
adjustment factor in the Natural Gas Policy Act of
1978 (P.L. 95-621) and the Crude Oil Windfall Profits Tax Act of 1980 (P.L. 96-223) for determining
the ceiling price on certain types of natural gas
and the windfall profits on crude oil, respectively....

Proposed legislation in the 96th Congress would
have further extended the use of GNP beyond that of
an analytic tool and could have generated concern
about what it represents and how well it does so.
Federal spending would have been affected by the
definition and accuracy of GNP.

One proposed amendment to the Employment Act
of 1946, H.R. 2314, would have limited Federal outlays in the President's Budget to equal the Council
of Economic Advisers' estimated Federal receipts.
The receipts would have been based on real economic
growth using real GNP estimates as part of the formula for the calculation.

Another proposed bill, H.R. 4610, would have
limited Federal outlays to a specific percentage of
GNP for the last complete calendar year occurring
before the beginning of the fiscal year. Fiscal
year 1982 spending, for instance, would have been
limited to 23% of calendar year 1980 GNP.

Lastly, H.R. 7112 proposed an antirecession
assistance program for aid to State and local governments to be triggered by two consecutive quarterly
declines in real GNP and real wages and salaries.
Allocation of funds to States and local governments
were to be based in part on the aggregate real wages
and salaries component.[1]

An equally long recitation might also be made for policy
uses of other aggregates like the unemployment rate or the
Consumer Price Index (CPI). Clearly, the tendency to accept
official economic projections at face value cuts across party
and ideological lines. It is common in the executive bureaucracy and is becoming more so within Congress. The honest
disclaimers from government agencies that compile the data
and from economists who have studied the concepts are lost in
the rush, or perhaps it comes closer to the truth to suggest
they are ignored under pressure to seize upon "official"
statistical data on the economic forces policymakers hope to
affect.

A common belief, which may blunt the concern over the
limitations of making forecasts from economic aggregates, is
that forecasting is a scientific procedure which is slowly
but surely undergoing refinements which make its measurements
and projections more exact. Such scientific exactitude, however, must exhibit two criteria: 1) a high probability of
accurate projections and 2) concepts that are themselves clear
and objectively meaningful. By way of assessing the "scientific" status of budget forecasts, therefore, we will assess
each of the selected components on the basis of the accuracy
of past projections and the clarity and meaning of the concepts themselves.

Evaluating Forecasts

There are several ways to evaluate the federal forecasting record. All have strengths and weaknesses. The
first approach would be to compare the size of the forecasting error with the size of the total budget. This tends
to yield only a small percentage error. However, these small
percentages should not detract from the dollar magnitude of
the error, which despite being a small percentage of the total
budgets, can be extremely significant as gross figures in
themselves. That is to say, a small percentage error produced
by this formula can disguise a massive error in forecasting
as a glance at the dollar amount of the errors in forecasting
deficits makes clear.

A second approach is to evaluate the error in forecasting the percentage change. This is a stiffer criterion
and would seem to be the more appropriate in assessing the
claim of forecasters to scientific rigor in their calculations.
It may be unrealistic to expect that forecasters can come
close to the precise figure when the size of the aggregate in
question, such as the deficit or the unemployment rate, is so
large. It should, however, be possible to assess the magnitude of the impending change -- that is, whether the economic
indicators are pointing toward a large change or a small one.

But here again the formula has its drawbacks. With the
first formula meager percentage errors tend to detract from
significant total dollar errors; the percentage change formula suffers from an almost opposite sort of quirk. It can be
overly sensitive to insignificant errors when both the predicted and actual changes are of small magnitudes. For example, if the forecast predicts a small change, say, a $1 billion increase in the federal deficit, and there is a small
increase of, say, $2 billion, then the error in forecasting
the percentage change would be an impressive 100% despite a
relatively minor error in the dollar magnitude.

On the other hand, small changes in some of the concepts
we will analyze -- the GNP, the unemployment rate, and the
CPI -- actually translate into huge changes in the magnitude
of economic activity. A change of a couple of points in
these calculations is significant, and a formula which is
sensitive to small changes would seem to be required.

A final criterion is somewhat like the assessment in the
error of percentage change, and it enjoys the same attractions
and drawbacks. It is to assess the error as a percentage of
the forecasted change. Put colloquially, it means that "so
much" change was forecast, and the actual change was "twice
as much" or "half as much." As a rule of thumb in assessing
forecast accuracy, this formula is probably the most easily
comprehensible to the nonspecialist layman or policymaker.

As with the previous criteria, it does not suffer from
the problem of being overly sensitive to comparisons of small
magnitude. But, in some cases, as we pointed out, this can
be a virtue. Each of the proposed criteria is useful for
asking different questions about forecasting reliability,
but the results of each still need to be considered in the
light of common sense.

There is yet another criterion of evaluation -- a
proposed margin of possible error -- to evaluate the
forecasters' accuracy. For present purposes, however, the
three formulas outlined above are adequate.

The Deficit

Apparent from an inspection of previous forecasts on the
federal deficit is that not only is there a poor record of
accuracy, but there is not even a consistent direction of
error. Between 1971 and 1982, the errors swing by wide margins both in overestimation and underestimation of the deficit. As for the margin of error, in only two of the 12 years
under scrutiny were projections even close to accurate (1977
and 1978). Also, beginning with the 1978 budget published in
early 1977, forecasters ventured long-range projections beyond
the budget year in question. The results are equally, if not
more, egregious.

Table 1
Federal Budget Deficit Projections, 1971-1982
(in billions)

' 71

' 72

' 73

' 74

' 75

' 76

' 77

' 78

' 79

' 80

' 81

' 82

1971 Budget

+1.3

1972 Budget

11.6

1973 Budget

25.5

1974 Budget

12.7

1975 Budget

9.4

1976 Budget

51.9

1977 Budget

43

1978 Budget

47

11.6

1979 Budget

60.6

37.5

8.6

1980 Budget

29

1.2

+36

1981 Budget

15.8

+ 5

1982 Budget

29.7

Actual deficit

23

23.4

14.9

4.7

45.2

66.4

44.9

48.8

27.7

59.6

57.9

110.6

Error in billions of $

24.3

11.8

10.6

8

35.8

14.5

1.9

1.8

32.9

30.6

42.1

80.9

Error as % of total budget

11.2%

5%

4.3%

2.9%

11%

3.9%

.4%

.3%

6.6%

5.2%

8.6%

9%

Error as % of year-to-year change

99%

2950%

125%

78%

88%

68%

8%

46%

156%

96%

2476%

154%

Error as % of projection

1869%

102%

42%

63%

381%

28%

4%

4%

54%

106%

266%

272%

Viewed as a percentage of the total budget, the error
will almost always seem smaller, since gross errors in each
of the 12 years are a relatively small fraction of the entire
federal budget. However, the gross dollar figures can hardly
be viewed as insignificant, particularly for FY 1971, 1975,
and every year since 1979. Except for a brief interlude between 1977 and 1978, budget projections did not improve.
Large errors are evident throughout the 12-year period. If
any trend is apparent, it is that the projections have been
chronically unreliable since 1978 (the year in which projections were made for FY 1979).

Assessing the forecasters' record on the magnitude of
percentage changes, the results are even more disappointing.
In seven of the past 12 years forecasters deviated from the
actual percentage change in the deficit by at least 96%.
Twice they deviated by more than 2,000%. Only in 1977 was
the forecast respectably close to predicting the magnitude of
change in the deficit (although for 1978, the flaw inherent
to the formula biases the result). They erred widely both in
underestimating the magnitude of change (FY 1971, 1975) and
in overestimating it (FY 1972, 1973, 1979, 1981). The average error in forecasting the magnitude of change in the deficit over the 12 years is almost 530%.

Turning to the third criterion, error as a percentage of
forecasted change, we see that in six of the past 12 years
the percentage of error exceeded 100%. In four of those years
it exceeded 200%. In 1981 and 1982, it was more than three
times higher than projected. In 1975, it was close to five
times higher than projected. In 1974 and 1979 it was only
about one third and one half, respectively, of the original
projection. The average error between projected and actual
deficits as a percentage of the projection is 266% over the
12-year period. Excluding 1971, the average error is 120%.
This is to say, on average, deficits have been either more
than twice as much or less than half as much as forecast over
the past 11 years. Again, it must be emphasized that the
errors run in both directions over the whole course of study,
without the slightest trace of improvement in accuracy or
even consistency in direction of error in the later years.
Certainly this should give policymakers pause in considering
budget projections as an element of law or policy.

Unlike the other concepts, there can be little dispute
over the meaning of "deficit."[2] It is simply an accounting
sum whose definition is arithmetically clear. The major problem with it as an instrument of fiscal or economic policy is
that it is highly unpredictable.

Outlays And Receipts

Outlays and receipts would seem to be, like the deficit,
simple accounting concepts. But this is not quite so because
they are aggregates of many lesser accounts. The aggregate
totals tend to obscure what happens in the component accounts,
as we shall see. And the component forecasts definitely detract from apparent, forecasting successes of the aggregate
totals.

Studying Tables 2 and 3, it is apparent that, in general,
the forecasting errors are not so far off as those for the
deficit. The highest error in estimating percentage change
of outlays occurred in FY 1971 -- 72%. There were two very
close forecasts in FY 1973 and 1974, but that trend was quickly reversed. The average error in forecasted percentage
change in outlays for the 12 years was 29%. When the 12-year
average is taken as the ratio between the forecasted change
in outlays and the actual error, the forecasts were off by an
average of 59.5%; that is, the change in outlays was on average 59% higher than that forecast from the previous year.

Turning to receipts, the record is far worse. This
should not be surprising since receipts are dependent to a
much greater extent on general economic conditions that affect
the tax base. As we will see, federal efforts to forecast
economic conditions have had poor results. The worst forecasts from the perspective of percentage change are the first
and the last years of the 12-year survey (1971, 1982). Undoubtedly, the case of 1982 was in part the result of the Reagan
administration-sponsored tax rate cut passed in 1981 after
the FY 1982 forecast was published. But the possibility of
significant legislative changes is but another reminder that
forecasts are likely to prove unreliable outside an academic
context where other factors can remain "constant." Both in
terms of the dollar magnitude of the errors and in terms of
the error in percentage change, there is no discernible pattern to the size of the mistakes. The average error in forecasting percentage change in receipts is 91%. Excluding the
1982 forecast, the average error remains at 53%.

There are factors behind the aggregate totals for outlays and receipts that make even these figures too generous
an assessment of forecasting accuracy. In the case of outlays, for example, a considerable portion of the total is, in
theory, uncontrollable. Generally, there are two types of
uncontrollable outlays. One is the entitlement programs,
with expenditures triggered to economic conditions -- unemployment insurance in time of recession, for example, or indexed increases in Social Security, Medicaid, and other

Table 2
FEDERAL BUDGET OUTLAYS

1971

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

Estimated

200.8

229.2

246.3

268.7

304.4

349.4

394.2

440

532

616

739

Actual

211.4

231.9

246.5

268.4

324.6

366.5

402

451

494

579.6

657.2

728.4

Error(billions of $)

-10. 6

-2.7

-.2

+.3

20.2

-17.1

-7.8

-11

+6

-47.6

-41.2

+10.6

Error(estimating % change)

72%

13%

1%

1%

36%

41%

22%

22%

14%

56%

53%

15%

Error as % of
projected
change

252%

15%

1%

1%

56%

69%

28%

29%

12%

125%

113%

13%

Table 3 FEDERAL BUDGET RECEIPTS

1971

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

Estimated

202.1

217.6

220.8

256

295

297.5

351.3

393

440

503

600

711.8

Actual

188.4

208.6

232.2

264.9

281

300

357.8

402

466

520

599.2

617.8

Error in billions of $

+13.7

+9

-11.4

-8.9

+14

-2.5

-6.5

-9

-26

-17

+.8

+94

Error in % change

258%*

45

48%

27%

87%

13%

11%

20%

41%

31%

1%

505%

Error as % of projected change

163%

31%

93%

37%

47%

15%

13%

26%

68%

46%

1%

83%

*Based on actual change of -5.3 billion from $193.7 FY 1970 receipts.

entitlements during inflationary periods. This sort of uncontrollable makes forecasting outlays more difficult because
it hinges on general economic conditions. On the other hand,
the second type involves accounts whose budget authority has
already been fixed by previous Congresses. Since the spending
scheme for many accounts runs over several years, federal
forecasters begin their task with a major portion of outlays
already known. The task should be much more easy, but still
the errors are significant.

Part of this paradox is the fact that many of these uncontrollable accounts have also proven uncontrollable in a pejorative sense. Many are subject to serious cost overruns. In
recent years the Government Accounting Office has tried to
monitor the cost overrun phenomenon. Even though the GAO had
to rely on the agencies themselves to provide the mass of
data necessary to audit the multibillion-dollar cash flows,
it assembled some startling statistics. (See Table 4, originally published in 1982, and valid through September 30,
1981.)[3]

Even though this is only a partial list of federal accounts and the GAO was not able to validate the data for accuracy, the cost overrun on projects depicted by this partial
list is almost $317.7 billion. Not all of this is charged as
an outlay for FY 1981. It represents cumulative overruns on
accounts which the federal government was servicing as of
September 30, 1981. Still, given the magnitude of the over-runs and the unavailability of validated data, the difficulty
of drawing reliable forecasts for outlays should become apparent.

Another point to keep in mind in looking at the aggregate
outlays figure -- which does inadvertently enhance the
forecasters' record -- is that many of the erroneous
estimates for the individual accounts that add up to total
outlays cancel each other out. Errors of underestimation for
many accounts are mitigated or canceled out by errors of
overestimation for other accounts.

Studying 50 large accounts for fiscal years 1977 and
1978, for example, the GAO notes that there were actually $27
billion in combined over- and underestimates of FY 1977 outlays. This contrasts sharply with the outlay error depicted
by Table 2, where the gross outlay error for all government
accounts shows up as merely $7.8 billion. Of the original
mistakes in the 50 accounts, however, $10.4 billion were overestimates and $16.4 were underestimates -- canceling out to
a $6 billion aggregate error for the 50 large accounts. The
same 50 accounts contained $35 billion in wrong estimates for
FY 1978, but this was disguised by the cancelling effect
of $12.3 billion in overestimates and $22.3 billion in under-estimates.[4]

If the gross dollar error of forecasted outlays for 1977
were changed from $7.8 billion to $27 billion, then the error
in forecasting percentage change in outlays would jump from
22% to 76%. Doing the same for FY 1978, the error jumps from
22% to 99%. Thus, the apparent improvement in forecasting
outlays, compared with either receipts or the deficit, is in
large part the result of random cancelling-out rather than
demonstrative forecasting skill.

With respect to receipts, the cancelling-out phenomenon
proves to be much less of a factor. There are far fewer accounts, and the major accounts -- individual income,
corporate income, and Social Security taxes -- all generally
react the same to upturns or downturns in the performance of
the national economy.[5] Forecasting receipts from the
national tax base, then, inevitably points toward forecasting
GNP, unemployment, and inflation.

Gross National Product

No aggregate computation is larger or more problematic
than Gross National Product. It is one of the best reasons
to distrust economic aggregates and forecasts of aggregate
changes. The media, however, report official announcements
of GNP calculations without a trace of reservation, and
policymakers increasingly embrace the figures as guides to
policy and legislation.

Efforts to assess national product or national wealth
have been a lure to academic economists at least as far back
as Adam Smith's Wealth of Nations. Since the 1930s governments such as the United States have drawn on academic
efforts to assess national product as a means, at the very
least, of assessing their tax base, but also to collect other
data that might help in formulating economic policies. Despite the fact that GNP has been carried afield from academia,
however, academicians are still not agreed on the correct
definition of the concept, or on the way to compute it, or in
some cases, whether it can be meaningfully computed at all.
As a result, GNP figures are subject to repeated "revisions,"
not only because of the need to have the most accurate data,
but also because of chronic disputes over the concepts under
which the data is grouped. To those who will take heed, the
government's own agencies issue disclaimers on this point:

Prior efforts in addressing the issues surrounding
GNP's accuracy and reliability, methods and concepts have brought changes to the estimates....
Nevertheless, issues and problems concerning the
estimates still exist....
The lack of precise error measures for the
estimates -- even though there are valid reasons
why they cannot be provided -- limits users' and
researchers' capability to judge how accurate the
estimates are and should be....
Given the size and complexity of the accounts,
we are not certain if the questions surrounding the
issues can ever be fully answered.[6]

Because the gross dollar amount of GNP figures has been
subject to revision every few years between 1971 and 1982,
and because it normally takes up to two years before the relevant data can be gathered and factored into the equation for
any given fiscal year, it would be misleading to compare actual
figures computed under one conceptual framework with estimates made under another. The focus instead will be on the
forecasts of percentage change from year to year which, according to Commerce Department economists, are less subject to
fluctuation as a result of conceptual revisions than the
gross dollar figures.[7,8]

While none of the forecasts depicted by the table is
quite accurate, the margin of error does not appear to be
very wide. But as economist Alex Rubner points out, the margin of change in GNP accounts for Western industrialized nations since the close of World War II has never been more
than a few points in either direction.[9] More violent fluctuations are simply not to be expected. This narrows the
acceptable margin of error considerably. Of the five years
covered by the table, the percentage change has never been
greater than five percentage points. Suppose we assume, generously and along lines suggested by Rubner, that the most a
forecaster could possibly be amiss is by five percentage
points -- that is, five points would constitute the largest
possible error, or a 100% error. Each percentage point of
error, then, corresponds to an actual forecasting error of
20%.[10]

Assuming that the "actual" GNP figures are reliable, we
see an array of errors from 4% to 54%, with an average error
over the five-year period of 24%. Also worth noting is the
fact that the percentage of error is much greater for the
long-range forecasts aimed beyond the upcoming fiscal year.
Taking the forecasts for only the second year ahead, where
data are available (1978-1981), the errors range from 30% to
100%, with an average error of 57.5%. Except for 1981, all
of the forecasts are shown to have been overly optimistic,
and wildly optimistic projections for that year were only
brought down in the immediately preceding year. Given the
standard we have imposed, the forecasting errors are quite
large. But whether an average error of one percentage point
(20%) of change in GNP is tolerable depends in large part on
the purpose of the forecast. As an academic exercise, it may
be considered acceptable. As a factor in shaping public policy or determining federal expenditures, a one-percentage-point error can certainly have unfortunate ramifications.

As to what it tells about the growth of the national
economy, the concept itself is open to objections that it
factors more than it can explain and also needs to factor in
much more than it does in order to fulfill its ambitious goal.
It does both too much and too little. In factoring vast collections of statistics and data, it loses sight of the meaning
of many particulars making up the GNP. Department of Commerce
economists recognize, for example, that measurements of (and
hence errors in) many types of data are simply not commensurate with measurements of many other types of data.[11] What
brings coherence to the disparate collection of economic data
is not scientific measuring procedures but the fortuitous
mutual cancellation of different errors within the aggregate,
much like the federal outlays discussed above. Thus, even if
it may prove useful, the GNP distorts and obscures component
parts of the economy that make up the aggregate.

Yet granting the formidable level of aggregation, GNP
also fails to take significant aspects of the economy into
account. Or where it does so, the interpretation is controversial. The eminent economist, Oskar Morgenstern, focused
critical attention on conceptual and statistical difficulties
in computing vast economic aggregates such as the GNP in his
book, On the Accuracy of Economic Observation, published 20
years ago.[12] The problems raised by Morgenstern continue
to diminish the reliability of GNP calculations and thus bear
repeating.

A major omission of long dispute is that the GNP limits
itself to computing cash flows (either income or expenditures)
and thus either ignores economic growth where there is no
money transaction or imputes its own value to that growth.
Cash flows are the focus of GNP accounts because they are
more easily countable. This does hold the compilers to a
more rigorous standard, but it excludes major contributions to
economic wealth such as the work of housewives, child home
labor, and volunteer work, as well as unrecorded cash flows
in the underground economy. In other cases of accrual of
value without cash flow, the Commerce Department economists
venture to impute their own assessment of value, as in the
case of assigning rent values to all residential housing in
America. This latter is, as Morgenstern observes, "a tricky
affair."

Then there is the problem of how to factor in government
enterprise. Some argue that many government expenditures
actually diminish GNP. Others feel that government expenditures are neutral or "intermediate" in that they do not create
wealth themselves but only lay the groundwork. Others, including those who currently define official GNP components, simply
calculate all government expenditures as an addition to GNP.

Objectively assessing capital depreciation is yet another
nettlesome issue. Standard concepts of depreciation, such as
"double declining balance," "sum of the years digits," LIFO,
FIFO, etc. are merely accounting fictions and frequently unrealistic guides to true capital depreciation. Moreover, the
Commerce Department has to rely on private businesses' own
estimates of capital depreciation. Beyond the fact that even
the most meticulous calculation using one of the standard
accounting procedures can prove deceptive, many businesses
are likely to have unrealistic assumptions about the impact
of inflation or taxes on their capital stock.

In this vein, we turn to the seemingly inevitable sampling errors in the collection of original data. Much of the
data comes from government agencies such as the Bureau of
Labor Statistics, the Commerce Department, and the Census
Bureau. It may be prudent not to accept the data on face
value. For example, by the Census Bureau's own recent estimate it missed over 4 million people in the census of
1970.[13] This was but a small percentage of the vast aggregate total U.S. population, but it was still more people
than counted for the entire state of Maryland or the city of
Chicago. Certainly the four million discrepancy has a major
impact on the national product. The influx of millions of
illegal aliens in the 1970s -- many of whom surely contribute
to the GNP --further damages the credibility of the census
figures used in computing GNP. Other sources of federal data
are also suspect, as we shall see as we turn to the unemployment rate and the CPI.

An important issue that Morgenstern did not consider is
the "underground economy" in the United States. It is made
up partly of illegal economic activity such as trade in drugs
and stolen property, but it also encompasses legal economic
activity conducted on a cash basis such as self-employment,
moonlighting, and employment of illegal aliens. The Commerce
Department does not factor all of this potentially huge cash
flow into its GNP calculation. Estimates as to the size of
the underground economy vary widely, but even conservative
estimates placed the figure as high as $100 billion as long
ago as 1976. Other estimates are as high as $369 billion for
1976, which is 22% of the entire Gross National Product
calculated for that year.[14]

It is difficult to determine exactly how much of this
underground economic activity is reflected in the cash flows
used to measure the GNP. No illegal transactions are factored
into GNP, but legal transactions made with earnings from underground activity will show up in the cash flows which the Commerce Department studies. By any account, however, the large
amount of underground economic activity alone is enough to
question official GNP calculations (to say nothing of other
official economic aggregates such as the unemployment rate).
A Congressional Research Service economist underscores this
point:

An active, expanding underground economy can
present serious problems for economic policymakers.
Data purporting to show the state of the economy
will actually provide a misleading picture of existing conditions. An example of this may have
occurred in 1978 and 1979. At that time many analysts were predicting that an economic downturn was
imminent. They also expected such a recession to
last about one year. Concern over imbalances in
the consumer sector, particularly a sizable increase in consumer debt, was largely responsible
for these forecasts. The recession occurred much
later and was much shorter than predicted. Because
of underground activity consumer incomes may have
been considerably higher than reported, thus consumer ability to take on new debt obligations may
have been greater than believed, accounting for the
actual course of events and misleading forecasters
and policymakers....

However, it is abundantly evident that knowledge of the magnitude of underground activity is
not very advanced. Precise statistical indicators
which can be explicitly factored into the decisionmaking process do not exist. Perhaps the best
course of action is to follow the advice of economist James Henry. In testimony before the Joint
Economic Committee Henry counseled, "Overall, we
need to think much more rigorously about conducting
economic policy in an environment where most economic variables -- not just target variables like
unemployment, employment, and inflation, but also
planning variables like money supply, effective tax
rates, and the Federal deficit -- are measured with
error."[15]

This should make it clear that along with the disappointing efforts to forecast supposed changes in the Gross
National Product, the reliability of the concept itself as an
accurate portrait of economic reality is also in question.
Similar questions arise with respect to other official economic aggregates.

Unemployment

Upon cursory inspection, unemployment projections appear
to have an accurate track record. A closer look, however,
indicates an inability to forecast dramatic shifts in the unemployment rate. As in the case of GNP changes, a small percentage change actually counts for a large number of unemployed. Following Rubner's suggestion of imposing a standard
of error on GNP, we will do the same for the unemployment
rate. Two or three percentage points over the course of a
year is about the largest swing one way or the other that we
are likely to experience. Taking 3% as our widest possible
margin of error, the rule of thumb would be that forecasts
are off by 33.3% for every percentage point they fall off
from the actual count.

Thus, the 1978 budget forecast erred by 17%, underestimating the cyclical turn toward economic recovery. As
the brief recovery aborted in 1980, budget forecasters again
missed the cue; the 1980 budget projection was amiss by approximately 30%. As the recession deepened in 1982, projections this time were off approximately 73%.

Because federal budgets didn't publish unemployment forecasts prior to 1976, it may be unfair to comment on unemployment assumptions between 1971 and 1975. Nevertheless, this
period was the era of President Nixon's much-vaunted "full
employment budget" concept. This was an ambitious attempt to
manage the performance of the national economy -- particularly
to smooth out cyclical turns -- through the impact of federal
fiscal policy.

According to President Nixon, the federal government
would simply spend the same amount of revenue that was assumed
to be collectible with the economy at "full employment" --
then defined as 4% unemployment. If the economy was "fully
employed," the fiscally responsible federal government would
spend no more than it took in. If the economy was tending
toward a higher level of unemployment, however, the resultant
deficit spending would stimulate the economy, nudge it toward
4% unemployment, and ultimately close the deficit with full
employment revenues. The president called this scheme a "self-fulfilling prophecy."[16] The statistics appear to contradict these optimistic claims. The administration managed to achieve the deficit but not the resulting reduction in unemployment.

Table 6
Federal Budget Unemployment Projections

71

72

73

74

75

76

77

78

79

80

81

82

1971 Budget

N/A

1972 Budget

N/A

1973 Budget

N/A

1974 Budget

N/A

1975 Budget

N/A

1976 Budget

7.9

7.5

6.9

6.2

5.5

1977 Budget

6.9

6.4

5.8

5.2

4.9

1978 Budget

6.6

5.7

4.9

4.8

4.7

1979 Budget

5.9

5.4

5.0

4.5

1971 Budget

6.2

5.7

4.9

1971 Budget

7.4

6.8

1971 Budget

7.5

Actual

5.9

5.6

4.9

5.6

8.5

7.7

7.1

6.1

5.8

7.1

7.6

9.7

Error

.2

.2

.5

.1

.9

.2

2.2

Error in % change

25%

33%

50%

33%

69%

40%

105%

Error as % ofprojected change

33%

25%

100%

50%

225%

67%

2200%

Margin of error3=100%

6%

6%

17%

3%

30%

6%

73%

From 1969, when the full-employment scheme was officially
embraced, to 1970, unemployment rose by 1.4% to 4.9% (from
3.5% in 1969). The rate rose another full point to 5.9% in
1971 and remained between .9% and 1.6% above the full-employment assumption for three more years before leaping to 8.5%
in 1975. Treasury forecasters relying on the 4% full-employment assumption for 1975 would have been amiss by some 150%,
given our margin of error (3 points = 100%). Indeed, the
full-employment episode offers a profitable case study in the
pitfalls of relying on economic aggregates in setting fiscal
policy.

Returning to the post-1975 statistics, Table 5 indicates
that forecasters of unemployment were fairly accurate as long
as the economy remained on a plateau. When significant turns
occur for better or worse in the economy, forecasters begin
to err by wide margins.

With regard to the accuracy of the concept itself as a
measure of economic reality, we find that as with GNP, the
unemployment rate is too easily taken at face value. Changes
in the rate in one direction or the other are frequently the
subject of alarm or self-congratulation among economic policymakers. A rising unemployment rate is often portrayed by the
mass media as a stagnant pool whose membership is being added
to with each month's figures. In fact, the pool is substantially changing, with many leaving to take up new work even
if there is a net increase. The rise in the rate could simply
reflect economic change as certain occupations are abandoned
for others. The change may be necessary and economically
healthy -- perhaps not something that government should attempt to reverse.

New jobs may not be opening up as fast as unprofitable
older jobs are closing, but even this can be relative to the
length of unemployment. It is certainly conceivable that a
rise in the total unemployment rate could simply be a function
of a temporary time lag between the opening of new jobs and
the abandonment of older ones. Whatever the cause, there is
a large and continuing segment of the unemployed who are finding work each month but only after a longer than average
period of unemployment. A Census Bureau economist cautions
against viewing the unemployed as a long-term homogeneous
group. Upon closer analysis of the duration of unemployment
among those making up the aggregate statistic, he notes that
even at the height of the 1982 recession in November, the
number of people who had either been looking for work for
more than 49 weeks or who had become totally discouraged and
ceased looking was 1.5% of the labor force, contrasted with
an aggregate unemployment rate of 10.7%.[17]

How the ranks of the unemployed change, and how during
periods of a lower aggregate unemployment rate massive numbers of workers might be temporarily unemployed but find work
relatively quickly, can be illustrated by the following example. For 1981, the aggregate unemployment rate for the year
was 7.6%, yet the total percentage of the workforce experiencing unemployment during the year amounted to 18.4%. The average duration of unemployment during 1981 was 13.3 weeks.[18]
It is not difficult to imagine that an extension in the duration of average unemployment by even a few weeks could create
enough of a bottleneck in the transition from job to job to
have a very significant impact on the unemployment rate. But
even an average layoff of, say, 20 weeks need not necessarily
evoke the concern that is incited by a 10% aggregate unemployment rate.

The aggregate unemployment rate also disguises the fact
that even during what are relatively low unemployment years,
there are pockets of hardcore unemployment. Many people in
this category do not even enter the Labor Department's calculations, as they have given up the search for work and also
may not have a permanent address where phone canvassers can
reach them. As a result, the Labor Department may be unable
to ascertain the true level of unemployment, let alone depict
the nation's employment levels via aggregate rates.

Probably the most challenging obstacle to assessing the
true level of joblessness is, again, the existence of a significant underground economy. Estimates of the size of the
underground labor force are inconclusive, but figures run as
high as 2.35 million workers.[19] This figure includes
workers who moonlight, or take for-cash pick-up work while on
strike or unemployment. It includes those engaged in criminal
activity, as well as self-employed contractors and professionals who report no taxes to the IRS. Illegal aliens are
probably its largest component.

Until such issues as the time span of unemployment, the
underground labor force, and the pool of permanently discouraged workers are dealt with comprehensively, the media
and policymakers both would do well to look at this concept
-- and predicted changes in its magnitude -- more critically.

Inflation

The record of the budget forecasts for changes in the
Consumer Price Index is obviously very poor. For 1979 and
1980 the rate of inflation as measured by the CPI was virtually twice that forecast. Again following Rubner's suggestion, if we assign a possible margin of error as a 5% change
in year-over-year figures, then each percentage point the
forecasts err counts for a 20% deviation. Using such a measure, the forecasting errors range from 10% in the 1977 budget to 136% in the 1980 budget. The average over the seven-year span for which forecasts were made was 63.4%. Forecasts
beyond the upcoming budget year are, with few exceptions,
much worse. The average error in assessing the percentage
change in the CPI was 130%. And as a percentage of what was
originally forecast, the average error was 759%, or 186% if
1982 is excluded.

This unreliability is overshadowed, though, by the widespread disenchantment with the CPI itself as an accurate
measure of price changes and inflation. To a far greater
extent than either the GNP or the unemployment rate, the CPI
has been the object of well-merited scorn by professional
economists. Many of the arguments against the concept were
discussed in a 1981 article by Prof. Robert J. Gordon.[20]
In reviewing these objections, it becomes apparent that the
main problem, like that of the GNP and the unemployment rate,
is that the aggregate concept is insensitive to the thousands
of possible components that it is supposed to represent.

The CPI is based on a hypothetical household expenses
list -- a shopping basket, as it's colloquially known. Precisely what items are included and how they are weighted are,
of course, matters of contention. But the issue is made more
dramatic by the fact that major revisions in the shopping
basket are made only on the average of once a decade. By the
time the new survey is taken and the arguments settled over
what to include, and how much to weight each item, several
years can pass. Prof. Gordon notes, for example, that despite
the fact that America became an automobile-oriented society
in the 1920s and 1930s, autos were not added to the CPI until
1940. Computers are still not a component.

Next, an issue arises regarding the weighting of specific items known as the "substitution effect." Not all prices
for the items chosen for the shopping basket are likely to
rise at the same rate. Indeed, some may actually fall over
time. But as selected items do rise substantially in price,
it seems reasonable to assume that the typical consumer substitutes other items for the more costly ones. Thus, if
the price of beef rises, consumers may buy chicken as a substitute and then move back to beef when relative prices change again. If fuel prices rise dramatically, people may substitute smaller autos. But the CPI's rigid system of weights is insensitive to such changes. If when the survey was taken it was established that the average household needed enough beef for six meals a week and enough gasoline to fuel a V-8, that is how the weights will remain. Of course, it is probably impossible to develop an index that is truly sensitive to the thousands of patterns of substitutions involving millions of families and goods.

Another aspect of the CPI's gross insensitivity to real
economic change is that it fails to account for important
qualitative changes in goods whose prices have demonstrably
risen. The improvements can be so vast, that, in essence, it
is not even the same good that originally entered the CPI
hypothetical list. Hospital costs have risen sharply, for
example, but by most accounts the quality of care has improved
dramatically. Airline tickets rise in price, but computerized
reservations make the entire process of booking a flight more
convenient. Gordon cites a study of auto tires, where it is
claimed that improvements between 1935 and 1978 have actually
decreased the tire-mile cost of tires by 9%; the CPI shows a
140% price increase. A similar study on the oil-mile price
of motor oil shows a 52% decrease versus a 234% increase registered over the same period by the CPI. Like the substitution
effect, the ignoring of product quality appears to be an inherent defect in any scheme to index the thousands of varieties
of consumer goods.

Until just recently, the CPI showed a large weighted
bias toward housing costs, with average housing costs assumed
to be 30% of household income. This may be the case for some
purchasers, but it certainly does not hold for households
generally. The CPI assumed that everyone was a recent homebuyer. To make matters worse, it failed to depreciate even
this artificially high computation with federal income tax
savings and real estate appreciation. On the contrary, it
actually double-counted housing expenses by calculating the
cost as the list purchase price and then adding all the mortgage payments in later years.

So obviously flawed was the treatment of housing costs
that a chorus of economists protested that it be changed.
Indeed, a commission of experts recommended changes in the
assigned weight of housing in the CPI as long ago as 1977.
It wasn't until January 1983, however, that the CPI housing
component was finally revised along lines suggested by the
commission. It was apparently for political reasons that the
change was refused for so long by the Bureau of Labor
Statistics. Certain special interest groups -- particularly
organized labor, Social Security recipients, and federal and
military retirees -- had a definite stake in maintaining the
upward bias of the CPI.

The situation is a good case study in the danger of
attaching government policies to vague economic aggregates.
Since many federal entitlement programs are indexed to the
CPI, federal outlays in selected areas have increased dramatically. In one year alone, according to Professor Gordon,
cost of living increases based on rises in the CPI were
awarded to 31 million Social Security beneficiaries, 2.5
million retired federal and military employees, 20 million
food stamp recipients, and 25 million recipients of subsidized school lunches. As with their uncritical acceptance
of other official economic aggregate figures, private businesses have been no wiser than the government in many cases:
Collective bargaining agreements governing 8 million unionized workers are also tied to changes in the CPI.

Summary

In analyzing the accuracy of federal budget projections,
there is ample evidence that official figures for the deficit,
outlays, and receipts, the Gross National product, the unemployment rate, and the Consumer Price Index should be examined
more critically. The accuracy of the predictions has generally
been poor, and the meaning of several of the concepts is subject to serious misinterpretation.

The forecasts of the impending federal deficit show an
average error in assessing percentage change of 529%. Only
three times between 1971 and 1982 has the estimate been
within 75% of the actual deficit.

Forecasts of federal outlays are shown to err in the
assessment of percentage change by an average of 28.8%, but
the aggregate disguises even more egregious forecasting errors
that cancel each other out. Forecasts of receipts err by an
average of 91% in assessing the percentage of change.

GNP forecasts display an average error in predicting
percentage change of 36% and an average gross margin of error
of 24%, according to the margin of error we have used. In
addition, the concept of Gross National Product is beset with
seemingly inherent conceptual and data gathering problems.
It relies very often upon imputed values and incomplete data
and excludes large segments of the economy such as domestic
work and underground economic activity.

By comparison, unemployment forecasts have been fairly
accurate, but they show serious errors in assessing cyclical
turns in the economy. As with GNP, more serious problems
arise with the concept of the unemployment rate itself. It
fails to show the changing composition of the unemployed and
provides an inaccurate reflection of hardcore unemployment
areas and of the underground workforce.

Budget forecasts of changes in the Consumer Price Index
erred by an average of 63% over the 12 years studied. Moreover, the CPI as a concept has been widely discredited as a
measure of economic reality. Its survey changes are sluggish, making the index chronically obsolete. It fails to
account for substitution effects and quality improvements in
the goods it purports to reflect, and it has been thrown out
of balance by its idiosyncratic weighting of housing costs.

The problems of both predictive accuracy and conceptual
coherence tend to diminish the claim that economic forecasting
is truly a scientific endeavor. In fact, it is little more
than an effort to ascertain present trends in the economy and
project them into the future. Concluding just what the trends
are in the first place is in itself an extremely controversial
effort, as the debate over problems in counting the GNP, the
CPI, or the unemployment rate makes clear. Yet academic efforts to gauge the economic past and present are valuable
since they add to our store of knowledge. As the precondition
to forecasting the future, however, uses of such aggregates
are bound to be misleading. Even if the aggregates were correctly assessed for the past and present, we would still need
to assume that no substantial unforeseen changes would enter
into the countless relationships (including potential relationships) of consumers, products, competitors, etc. that exist
in our modern global economy. This would certainly be an
impossible, even foolish assumption.[21]

These problems are not peculiar to the government. Private forecasts have been shown to err widely and regularly.
None of the private models are consistently right, and most
are consistently wrong.[22] It may seem that since private
firms make use of economic forecasts in planning their future,
government is equally obliged to make use of this tool in its
own planning. But there are good reasons why the tendency to
rely on forecasts can have far more mischievous effects in
government than in the private sector. First, private firms
relying heavily on erroneous forecasts accept the responsibility when they fail. The government, on the other hand,
takes its losses out on the tax base. Also, private industries tend to adapt quickly and change their plans as forecasts are gradually proven wrong. Governments, by contrast
act sluggishly in adapting to changing economic conditions,
whether it be a matter of trimming expenditures, adjusting
taxes, reformulating policies, or revising laws. The forecasting mistakes of private firms are likely to be changed by
managers sensitive to potential losses, while government
forecasts can become rallying points for vested interests
that often lobby against changing policies predicated on
mistaken assumptions. Thus, any legislation or policy that
is linked to questionable changes in economic aggregates
is more resistant to revision than private-sector planning.

There may be an even more important reason to be wary of
predicted changes in economic aggregates as a part of policy
formulation. The reason has to do with government candor.
In the interest of government accountability, we rightly expect that the federal government will explain in detail what
its budget plan is and describe how it calculates its plan.
Unfortunately, the use of complicated forecasting techniques,
together with unwieldy concepts of economic aggregates, runs
the risk of lulling people out of critical thinking on budget
assumptions. Voters and taxpayers are thereby encouraged to
leave such matters to "the experts," which can create an illusion of official omniscience.

This is not to suggest that the federal government should
cease trying to establish long-range budget plans or cease to
reveal its assumptions about those plans. It is rather to
suggest that the government show more candor in what it cannot do and then take its own admonition seriously. It is to
suggest making explicit that economic forecasts have usually
been wrong in the past and that they will probably prove erroneous in the future; to make clear that macroeconomic concepts are too easily taken as simple "buzzwords" when, in
fact, they are very imprecise and potentially misleading.
The government, like everyone else, must operate in a future
fraught with uncertainties that no one, least of all its experts, can foresee clearly. This would mean shunning legislation that links economic policy to forecasts or even to the
historical computation of macroeconomic aggregates. Above
all, it would mean rejecting those policies -- whether industrial policies, foreign trade plans, or budget formulas --
which assume an ability of the government to successfully
forsee and plan economic conditions. Economic policymakers
should be more honest with themselves as well as with us.

FOOTNOTES

[1] U.S. General Accounting Office, A Primer on Gross National Product Concepts and Issues, April 8, 1981, pp. 6-7.

[2] Even this simple accounting term has been subject to
recent controversy over its definition in the federal budget,
but this controversy seems easily resolved. Many federal
accounts are "off budget" and if included in the budget,
would add substantially to the size of the deficit and to the
overall size of the budget. The off-budget accounts are so
grouped because, in theory, most are self-financing, such as
the Post Office, the Federal Financing Bank and the Synfuels
Corp. Many have been running seriously in the red, however.
The U.S. Comptroller General notes that inclusion of these
figures in the 1981 Budget would have increased the total for
federal outlays by $120 billion, for receipts by $102
billion, and the size of the deficit by $18 billion.
Although this situation does not pose an insurmountable
problem in defining the deficit -- the off-budget accounts
can simply be added to the overall budget -- it does tend to
make the figures reported by the President's budget misleading to citizens, the mass media, and policymakers alike. The
Comptroller General warns that "the excluded amounts are now
so large that they limit the unified budget's usefulness as a
controlling device." See U.S. Comptroller General, Federal
Budget Totals Are Understated Because of Current Budget Practices, December 31, 1980, p.2.

[3] U.S. Comptroller General, Status of Major Acquisitions
as of September 30, 1981: Better Reporting Essential to Controlling Cost Growth, April 22, 1982, p. 8.

[5] A study by the Congressional Budget Office notes that of
these three major components of aggregate federal receipts,
corporate income taxes were the most difficult to estimate in
budgets between 1963 and 1978. CBO, "A Review of the Accuracy of Treasury Revenue Forecasts, 1963-1978," Staff Working
Paper, February 1981, p. 13.

[10] Economists specializing in the measurement of GNP have
been reluctant to suggest a tolerable margin of error with
which to assess their computations. They point out that such
a gauge may be theoretically impossible because so many of
the individual components making up the GNP are subject to
different kinds of errors that can't be measured in the same
way. See GAO, A Primer, pp. 21-28.
However professionally unsound it may seem for academic
economists, the fact that GNP figures are increasingly used
as instruments of government policy requires, at least in the
interest of government accountability, that some standard of
measure be used to assess the government's record. Thus,
following Rubner, we impose a margin of error drawn from the
record of recent historical changes in GNP. See Rubner,
Three Sacred Cows of Economics, pp. 117-118.

[11] GAO, A Primer, pp. 21ff.

[12] The major passages concerning GNP computation can be
found in Oskar Morgenstern, National Income Statistics (Washington, D.C.: Cato Institute, 1979).

[13] U.S. Bureau of the Census, Coverage of the National Population in the 1980 Census by Age, Sex, and Race, February 2,
1982, p. 6.

[21] A review of the factors that militate against our ability
to forecast future economic activity can be found in Ralph
Harris, "A Skeptical View of Forecasting in Britain," in James
B. Ramsey, Economic Forecasting -- Models or Markets? (Washington D.C.: Cato Institute, 1980).