Flawed Federal Land Use Report Encourages Unnecessary Spending

About the Authors

The
recent discovery of serious flaws in a U.S. Department of
Agriculture (USDA) land use survey, which compiled data by sampling
national land use trends, illustrates why extreme care must be
exercised by the federal government in processing, compiling, and
reporting the data it derives from the decennial census now
underway. Because the 2000 census utilizes both enumeration and
sampling techniques, it is essential that both the process and the
results obtained are screened to ensure that the type of errors
plaguing the USDA's National Resources Inventory (NRI) are
not replicated in the 2000 census. As the immediate response to the
NRI's release shows, the skewed results of error-plagued surveys
can induce government to react with costly programs and regulations
that address problems which may not, in fact, exist.

PROBLEMS FROM A FLAWED SURVEY

On
March 27, 2000, the U.S. Department of Agriculture warned visitors
to its Web site that a "problem" had been discovered in a land use
report released by Vice President Al Gore just three months
earlier. A revised version of the survey, it advised, would be
available in June 2000. Although the March advisory stated that the
"revised statistical processing is not expected to significantly
change any previously announced findings,"[1] by late April, USDA
officials were admitting publicly that they "do not know how
significantly" the findings would change.[2]

The
flawed land use report, the 1997 National Resources
Inventory,[3] was removed from
the USDA Web site shortly thereafter,[4] but not before
dozens of elected officials in the states and Washington, D.C., had
rushed forward with a variety of costly land preservation schemes
based on the survey's erroneous data. Vice President Gore, claiming
that there was a land use crisis, took the unusual step of
releasing the report a day before its formal release by the USDA
and Agriculture Secretary Dan Glickman.

At
the press conference, the Vice President remarked: "These new
figures confirm what communities across America already know--too
much of our precious open space is being gobbled up by sprawl." He
went on to advocate spending more federal money to preserve
farmland and to support the land purchase provisions in his
"Livable Communities" plan, which heretofore Congress had largely
ignored.

The
well-publicized release of these flawed data may have had its
intended effect: Congress has scheduled a May 2000 vote on H.R.
701, the Conservation and Reinvestment Act (CARA), which would
authorize the spending of $3 billion per year to buy up
environmentally sensitive land and to fund other conservation
programs.

Potential victims of this needless
spending will include more than just the federal taxpayer. The NRI
report claimed that the amount of land lost to development in
Pennsylvania was extraordinarily high, and it ranked that state
second in the nation for land lost to development. This ranking may
have led the Pennsylvania legislature to enact, and Governor Tom
Ridge to sign into law, a $650 million, five-year land preservation
program within two weeks of the survey's release. Similar concerns
were raised in other states where the NRI survey also reported
above-average land losses.[5]

The
flawed NRI survey claimed that Texas led the nation in land used
for development, with a reported loss of 2.1 million acres of
farmland. In Georgia, where the metropolitan area of Atlanta is
subject to the draconian growth-control demands of the
Environmental Protection Agency (EPA), the NRI claimed that more
than 1 million acres of previous agricultural land and open space
had been developed, while Virginia was reported to have lost nearly
half a million acres of rural land to development--doubling its
rate for the previous five years.

Meanwhile, the report's results assured
Oregon's state officials that their highly restrictive
anti-suburban land regulation policies for the Portland area had
greatly reduced their development rate. (Oddly, however, the NRI
survey also reported that rapidly suburbanizing Colorado and
Arizona had developed land at a lower rate per new resident than
Oregon had, despite its rigid growth-control boundaries).

CHECKING NRI DATA AGAINST OTHER FEDERAL
DATA

Had
officials at the USDA taken any time to review the NRI findings in
advance of the report's release, they most likely would have found
that some, if not all, of the data were of questionable
accuracy.

Indeed, the authors of this Backgrounder, who at the time were completing an urban
sprawl and smart growth report on Pennsylvania's development
trends,[6] had contacted the
USDA on December 17, 1999, just 11 days after the report's release,
to seek an explanation for some of the obvious inconsistencies
between the just-released NRI report and a USDA report of similar
scope, the Census of Agriculture, which had been released
nine months earlier.[7] Specifically, the
authors questioned the NRI's finding that the amount of land lost
to development between 1992 and 1997 in Pennsylvania (1.1 million
acres, of which 901,200 was farmland) was second only to that in
Texas, which reportedly lost 2.1 million farm acres.

Such
a finding for Pennsylvania made little sense and was markedly
inconsistent with the state's development trends. The U.S. Bureau
of the Census reported that between 1990 and 1998, Pennsylvania had
one of the slowest rates of population growth of any state: Its
population grew by only 1 percent over the period, compared with
8.7 percent for the country as a whole. Only three other
states--Connecticut, North Dakota, and Rhode Island--had slower
population growth rates.[8]

Because housing production and other forms
of development closely track demographic trends, it seemed highly
unlikely that one of the slowest growing states could have one of
the fastest rates of land converted to real estate development. As
the Commonwealth Foundation, a Pennsylvania-based think tank,
noted, the NRI survey could be correct only if all of the homes
constructed in Pennsylvania during that period had been built on
5.3-acre lots.[9]

Apparent discrepancies of this magnitude
between related federal data series should have given pause to some
of the statisticians and officials at USDA and encouraged further
review of the NRI numbers to ensure accuracy before the report was
released. Unfortunately, this seems not to have occurred; nor,
apparently, was the USDA concerned with disturbing inconsistencies
between the findings of its NRI and those of its Census of
Agriculture released in March 1999, which covered some of the
same land use patterns and was conducted over the same period of
time.[10]

Importantly, because the Census of
Agriculture is based on an enumeration of farmland (in which
every farm and its acreage is counted) and the NRI findings are
drawn from a national sample, the Census data would have provided
USDA officials with an excellent benchmark to confirm the accuracy
of the NRI results before their release. Indeed, had they done so,
they would have discovered that the results of the two surveys were
vastly different on a state-by-state basis. (See the Appendix for a
state-by-state comparison of the USDA Census of Agriculture and NRI findings.)

Pennsylvania and Texas offer the most
obvious inconsistencies between the two reports. In the case of
Pennsylvania, the NRI survey claimed that the state had lost
901,200 acres of farmland between 1992 and 1997, while the Census
reported that only 21,600 acres of farmland had been lost--a figure
much more consistent with Pennsylvania's half-century history of
very low population growth.

For
Texas, the differences are more extreme. Whereas the NRI survey
found that Texas had lost 2,105,400 acres of farmland between 1992
and 1997, the Census of Agriculture reported that Texas had gained 421,600 acres of farmland during the same period.
Assuming that this is correct, the USDA Census of
Agriculture would rank Texas fifth among the states that gained
farmland, just behind Wyoming, Nebraska, Oklahoma, and Utah.
Because the USDA announced that the NRI results were subject to
possible error but made no mention of any similar problems
associated with the Census of Agriculture, it must be
assumed that Texas most likely did experience a gain in farmland,
not the massive loss claimed by the NRI survey.

Surely, someone at USDA should have
noticed the extreme differences in results for Texas, with one USDA
survey indicating it a top gainer of farmland and the other
reporting it as the nation's biggest loser. Moreover, the
conflicting activities (farmland lost to development and land
becoming farmland) allegedly occurred over the same period of time.
Obviously, something is profoundly amiss in data collection at the
USDA, as its subsequent withdrawal of the NRI survey from its Web
site indicates.

Other notable discrepancies between the
NRI and Census involve Georgia. While the NRI reported a
loss of 720,000 agricultural acres, the Census of
Agriculture found that Georgia added nearly 650,000
acres of farmland. A comparison of the data with population growth
would have shown that the number of new acres the NRI alleged had
been developed would require the dedication of nearly four acres
for each new household in Georgia--an extremely large lot size even
for sprawling Atlanta.

Virginia's NRI-reported loss of 300,000
farmland acres is well in excess of the 70,000 the Census of
Agriculture reports. The anti-suburban, growth-control model
for Oregon did not fare as well in the Census: 32 states had
better performances in terms of farmland preservation. Another
questionable implication of the NRI findings is that California's
1.3 million new residents required 700,000 acres of new
development--less than the 1.1 million acres consumed by
Pennsylvania's 30,000 new residents.

CONTROLLING THE RISKS IN SAMPLING

Mistakes, of course, can be made when
dealing with large volumes of data, but errors of this magnitude
are hard to explain, given the many opportunities to cross-check
the results with other federal data and surveys. Apparently, such
essential cross-checking was not done by the USDA for the National Resources Inventory of land use, and the
consequence of this failing is that states such as Pennsylvania
have already committed to vast expenditures of money to solve a
problem that may not exist.

Although sampling can be a powerful and
cost-effective tool for deriving accurate information about large
populations and collections of data, that same power can also serve
to magnify and exaggerate relatively small errors in the sampling
process and/or in the procedures used to process and compile
sampled information. Because of the possibility that small errors
can lead to big mistakes, it is essential that samples be properly
designed, professionally managed according to generally accepted
scientific principles, relentlessly cross-checked against
alternative sources of data, and subjected to simple common sense
and good judgment.

The
NRI debacle not only illustrates the pitfalls of what can happen if
careless errors creep into the sampling process, but also
illustrates the risks the federal government confronts in ensuring
the absolute accuracy of the 2000 decennial census, for which
sampling is an important component in identifying the many
descriptive characteristics of the U.S. population. It also
illustrates the importance of conducting a full companion
enumeration to benchmark the accuracy of the sample. Had it not
been for USDA's companion census of land use conducted by
enumeration, these sampling flaws in the NRI might never have been
discovered. For this reason, a detailed, independent investigation
should be conducted to determine the nature of the errors and
ensure that such errors are not repeated in the decennial census
now being conducted by the Census Bureau.

WHAT CONGRESS SHOULD DO

To
ensure that the flaws of the sampling process used in the 1997 National Resources Inventory survey are not replicated and
the causes of the errors are fully understood, Congress should:

Hold oversight hearings with the USDA's
professional staff and leadership to determine how the mistakes
were made, why they went undetected prior to the NRI's release, and
why they remained unacknowledged even after outside information had
been received within two weeks that pointed out a discrepancy. At
the same time, Congress should postpone any further consideration
of H.R. 701, the Conservation and Reinvestment Act, until such
hearings can be held and the USDA provides a revised and accurate
inventory of land use later this summer.

Hold oversight hearings with U.S.
Bureau of the Census professional staff and leadership to
determine the extent to which similar errors may occur in the
decennial census and what measures are being taken to prevent
them.

Request that the U.S. General
Accounting Office (GAO) conduct an independent review of the
process by which the USDA conducted the initial NRI survey and
other federal data collections. Congress should also request the
GAO to suggest appropriate safeguards to ensure that similar
problems do not emerge in the compilation of the 2000 decennial
census.

Give serious consideration to
restructuring all federal data collection, compilation, and
reporting activities within a single independent and apolitical
federal agency. This has been proposed by Senator Daniel Patrick
Moynihan (D-NY) in S. 205, which would create an independent
commission to study such restructuring, and by Representative
Edward Royce (R-CA) in H.R. 2452, which would create a federal
statistical service as part of a comprehensive restructuring of the
U.S. Department of Commerce. During the 104th Congress,
Representative Steve Horn (R-CA) introduced H.R. 2521 to
consolidate all federal statistical functions in a single,
independent government agency. If data collection and reporting
were removed from political influence, higher standards of
professionalism could prevail and public confidence in the
integrity of the reports and results would increase.

Given the many suspicions and concerns now
surrounding the 2000 decennial census and its declining rate of
citizen compliance, it is essential that every effort be made to
maintain high confidence in the integrity of the nation's basic
statistics and data. Had an independent institution been created
earlier, much of the acrimonious debate and diminished public
confidence in the current census could have been avoided.

-- Wendell Cox, Principal of the Wendell
Cox Consultancy in St. Louis, Missouri, is a former Visiting Fellow
at The Heritage Foundation. Dr. Ronald D.
Utt is Senior Research Fellow in the Thomas A. Roe Institute
for Economic Policy Studies at The Heritage Foundation.

10. Data from the NRI
survey and the Census of Agriculture overlap in several
areas and can be used to cross-check each other. Historically, the
NRI data for the agriculture classifications of cropland,
agriculture reserves, pastureland, and rangeland track
comparatively closely with those of the Census of
Agriculture's farmland classifications. At the national level,
the NRI data were within 0.3 percent in 1992 and 0.2 percent in
1997. However, very significant differences are found in the 1997
state-by-state data. Because the NRI survey, and not the Census
of Agriculture, has been withdrawn for revision, it can be
presumed that the USDA considers the Census data to be
largely correct.