Below is a raw (and likely hideous) rendition of the
original report.
(PDF)

United States General Accounting Office
GAO Report to Congressional Requesters
September 1999
ENVIRONMENTAL
PROTECTION
Assessing the Impacts
of EPA’s Regulations
Through Retrospective
Studies
GAO/RCED-99-250
United States
GAO General Accounting Office
Washington, D.C. 20548
Resources, Community, and
Economic Development Division
B-283198
September 14, 1999
The Honorable Bud Shuster
Chairman
Committee on Transportation and Infrastructure
House of Representatives
The Honorable Frank R. Lautenberg
United States Senate
Over the past three decades, the Environmental Protection Agency (EPA)
has issued and enforced regulations, pursuant to federal statutes, to
enhance environmental quality. These regulations impose costs, such as
compliance expenses for the regulated entities, and produce benefits, such
as improved human health and recreational opportunities. These costs and
benefits are substantial. In 1997, the Office of Management and Budget
(OMB) estimated that federal environmental regulations cost about
$144 billion annually and produced benefits of about $162 billion.1
Under Executive Order 12866, EPA prepares detailed cost-benefit analyses
for all economically significant regulations—including, among others,
those expected to have an annual impact on the economy of $100 million
or more.2 These economic assessments inform and improve the regulatory
process by prospectively estimating the future costs and benefits of
feasible alternatives. Also, under this order, EPA periodically reviews
existing regulations that are economically significant to determine if they
can be made more effective or less burdensome. The order, however, does
not specify the nature or the extent of this review. One tool that can be
used in considering such regulatory changes is a retrospective study—an
assessment of the actual costs and benefits.
Concerns have been raised, however, that agencies have made only limited
attempts to assess the actual impacts of federal regulations. For example,
in May 1998, the Senate Committee on Governmental Affairs reported that
the review of existing rules, as directed by executive order, “has met with
varying degrees of failure. Clearly, getting agencies to review existing rules
1
In a 1998 report, OMB estimated that the costs of federal environmental regulations were between
$120 billion and $170 billion and the benefits could be as high as $3.3 trillion annually. The controversy
surrounding this estimate is discussed in our report, Regulatory Accounting: Analysis of OMB’s
Reports on the Costs and Benefits of Federal Regulation (GAO/GGD-99-59, Apr. 20, 1999).
2
The executive order was issued on September 30, 1993. EPA is also directed by the Unfunded
Mandates Reform Act of 1995 to conduct regulatory cost-benefit analyses under certain circumstances.
Page 1 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
is much easier said than done.”3 In light of these concerns, you asked us
to (1) determine to what extent EPA’s regulations have been the subject of
retrospective studies, (2) identify in what ways retrospective studies are
viewed as useful or difficult to do, and (3) identify ways to foster such
studies in the future.
Assessments of the costs and the benefits of EPA’s regulations after they
Results in Brief have been issued have rarely been done. Of the 101 economically
significant regulations issued by EPA from 1981 through 1998, only five
were the subject of retrospective studies.4 Various sources—including the
Congress, EPA, and academia—have provided the impetus for these studies,
which were all completed from 1997 through 1999. These studies covered
regulations designed to, among other purposes, control acid rain, phase
out chlorofluorocarbons, and reduce air pollution through improved
vehicle inspection and maintenance.
Authors of retrospective studies, EPA officials, and other experts told us
that these studies have been or can be useful but also pose a number of
difficulties for researchers. In terms of usefulness, for example,
retrospective studies have provided insights into a new, market-based
regulatory approach to reduce emissions that cause acid rain. These
studies found that the actual costs of reducing emissions were lower than
initially estimated. Based on these findings, a legislative proposal to
further limit emissions has been introduced. Retrospective studies are
viewed as difficult to do because, among other reasons, it is difficult to
obtain valid cost data from regulated entities. For example, corporate
accounting and financial records do not typically capture the information
necessary to determine the incremental costs of complying with federal
environmental regulations, and companies have little incentive to isolate
and monitor these costs. Also, it is extremely difficult to quantify actual
benefits. One reason is that a significant time lag usually exists between
the elimination or major reduction of pollutants and any corresponding
change in illness rates.
We found that a systematic approach by EPA would foster retrospective
studies in the future. While all economically significant regulations, by
executive order, are subject to a prospective economic assessment, EPA is
3
The Congress has considered a legislative proposal that would require agencies to review or “look
back” at existing regulations. To see our testimony on this proposal, see Regulatory Reform:
Comments on S. 981—The Regulatory Improvement Act of 1997 (GAO/T-GGD/RCED-97-250, Sept. 12,
1997).
4
According to EPA, another 14 economically significant regulations have been issued in 1999.
Page 2 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
not expected to calculate the actual costs and benefits for all of these
regulations. Accordingly, EPA officials told us that they have used the
agency’s limited resources on prospective economic assessments rather
than on retrospective studies. Based on discussions with study authors,
EPA officials, and other experts, we found that the resource constraints and
the difficulties in conducting retrospective studies could be better
addressed if EPA were to develop a plan to foster studies of the actual costs
and benefits of environmental regulations. We are recommending that EPA
develop a plan based on a systematic (1) identification of candidate
regulations for study, (2) assessment of the feasibility of conducting
retrospective studies of those regulations, and (3) identification of the
resources needed.
EPA’s regulations have rarely been the subject of a retrospective study.5
Few of EPA’s According to EPA, the agency issued 101 economically significant
Regulations Have regulations from 1981 through 1998,6 and five of these regulations have
Been the Subject of been the subject of retrospective studies. In addition, of the more than
2,600 environmental regulations issued during this period that were not
Retrospective Studies economically significant, 23 were the subject of retrospective studies.
As listed below, the retrospective studies we identified ranged from broad
studies, such as a report on the impacts of the entire Clean Air Act, to
studies of individual regulations, such as the phaseout of
chlorofluorocarbons (CFC). All were completed from 1997 through 1999.
Most of the studies had prospective cost estimates against which actual
costs could be compared. Several also had information on actual benefits.
• Section 812 of the 1990 amendments to the Clean Air Act required EPA to
provide information about the economic costs and benefits and the health,
welfare, and environmental impacts of the Clean Air Act. EPA issued the
Section 812 study in October 1997.7
• The 1990 amendments to the Clean Air Act directed an interagency group
to report every 4 years, beginning in 1996, on the costs, the benefits, and
the effectiveness of the acid rain program. The group, known as the
5
We considered a study to be a “retrospective” study if, at a minimum, it contained information on the
costs of existing regulations and focused on a specific regulation or a set of regulations with common
objectives.
6
We used 1981 as the starting point because EPA began tabulating the number of its regulations at that
time.
7
When counting the number of regulations that were the subject of retrospective studies, we excluded
the regulations covered in the Section 812 study because it aggregated the costs for a number of
existing regulations and did not identify the incremental costs of specific regulations.
Page 3 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
National Acid Precipitation Assessment Program, issued the first
retrospective study under this mandate in 1998.8
• Other retrospective studies involving EPA’s regulations include 23 studies
on pesticides and individual studies on the phaseout of CFCs, sulfur dioxide
reductions, vehicle inspection and maintenance, and reformulated
gasoline. The sulfur dioxide report was the only one of these studies to
include information on benefits.
Various sources provided the impetus for the retrospective studies we
identified. The Congress mandated the Section 812 and acid rain studies.
The latter study was funded under interagency agreements with EPA and
with other federal agencies.9 EPA’s Office of Policy signed a research
cooperative agreement with Resources for the Future (RFF),10 11 which, in
turn, initiated the 23 pesticide studies and the vehicle inspection and
maintenance study. EPA’s Office of Air and Radiation’s Acid Rain Division
partially funded the sulfur dioxide study through cooperative agreements
with the Massachusetts Institute of Technology and RFF. Two retrospective
studies were completed without funding from EPA. The CFC phaseout study
was conducted by a faculty member of Harvard University and funded by
Harvard University, and the reformulated gasoline study was prepared at
the researchers’ expense.
In addition to the retrospective studies that have been completed, others
are underway. EPA’s Offices of Policy and Ground Water and Drinking
Water have contracted for a retrospective study of EPA’s drinking water
standards for atrazine and nitrate.12 EPA expects the study to be completed
by September 30, 1999. Also, under a cooperative agreement with EPA, RFF
is conducting another series of retrospective studies on pesticide
regulations.
8
Title IV of the Clean Air Act amendments seeks to reduce the adverse effects of acid rain through
reductions in annual emissions of sulfur dioxide and nitrogen oxides from utilities burning fossil fuels.
9
Agencies in the National Acid Precipitation Assessment Program include EPA; the departments of
Interior, Energy, and Agriculture; the National Aeronautics and Space Administration; and the National
Oceanic and Atmospheric Administration.
10
Generally, under this type of cooperative agreement, EPA agrees to a research agenda proposed by
the recipient and provides funds without specifying how the research should be done. Also, the
recipient shares in the cost of the research.
11
RFF is an independent, nonprofit organization engaged in research and public education on natural
resources and environmental issues.
12
Atrazine is a herbicide and nitrate is an inorganic chemical present in fertilizers and animal wastes.
These chemicals can enter drinking water by runoff into surface water and leaching into groundwater.
Page 4 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
According to the experts we interviewed, while retrospective studies have
Retrospective Studies provided valuable insights they also have been difficult to perform.
Are Viewed as Useful
but Also Difficult to
Do
Retrospective Studies Are Authors of retrospective studies, EPA officials, and other experts we
Seen as Useful interviewed identified three primary areas where retrospective studies
have been or can be useful by providing (1) information for consideration
of legislative or regulatory modifications, (2) insights into the need for
methodological improvements in prospective economic assessments, and
(3) information for EPA’s use in implementing the Government
Performance and Results Act (the Results Act).
Modifying Legislation or Retrospective studies provide information used in considering legislative
Regulations and regulatory modifications—especially when a new regulatory approach
has been taken. Some time after implementation of a regulation, a
comparison of post- and pre-issuance estimates of the costs and the
benefits provides insights into whether that regulation is working as
planned, whether the burden on regulated entities is appropriate, and
whether unanticipated consequences are occurring.
For example, title IV of the 1990 amendments to the Clean Air Act
authorized EPA to adopt a new regulatory approach to the acid rain
problem instead of the traditional “command and control” approach,
under which pollution sources are required to install certain pollution
control technologies or meet specific performance standards. The new
approach provided market-based incentives. It allocated a number of
allowances for sulfur dioxide emissions to each utility under the program.
Each allowance represents the limited authority of a utility to emit one ton
of sulfur dioxide. The Act set an overall goal and capped the number of
allowances to ensure an overall reduction in emissions. Each utility may
choose to reduce its own emissions or to purchase unused allowances
from another utility. Furthermore, a utility may choose to “bank” its
allowances for sale at a later date. The allowances trade freely, as stocks
do.
Retrospective studies indicate that the actual cost of reducing emissions
under title IV has been lower than initially estimated. One reason for the
cost overestimation has been the reduced cost of pollution control
technologies. Based on these findings and reports that some ecosystems
Page 5 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
are not yet recovering from acid rain, a bill (S. 172) was introduced in the
U.S. Senate in January 1999 that would, among other things, further limit
sulfur dioxide emissions. The retrospective studies also provided
information on whether the market-based approach used for addressing
acid rain could be applied to other regulatory areas, such as climate
change and greenhouse gases.
Improving Methodologies of Retrospective studies can also provide agencies with insights on how to
Prospective Analyses better conduct prospective cost-benefit analyses and prevent
methodological biases in these analyses. For example, some analysts have
suggested that EPA tends to overestimate costs because, among other
reasons, it does not anticipate and account for cost-saving technological
innovations. However, in its analysis of the accuracy of prospective
cost-benefit estimates, RFF found that two studies (on reformulated
gasoline and the Dinoseb pesticide) had overestimated costs; two (on the
CFC phaseout and sulfur dioxide) had accurately estimated costs; and one
(on the Aldicarb pesticide) had underestimated costs.13 Additional
retrospective studies would enable EPA to more fully explore concerns
about what methodological factors, if any, contribute to the over- or
underestimation of costs.
According to an EPA official, even the best models for predicting the costs
and the benefits could be based on erroneous assumptions because it is
difficult to forecast technological improvements and economic changes.
The official explained that one conclusion from the retrospective studies’
results might be to include a “technology improvement case” when
estimating the costs of regulations.
Providing Information for Retrospective studies of the costs and the benefits can also help EPA in its
Implementing the Results Act efforts to implement the Results Act. The Results Act requires agencies to
set goals for program performance and to measure results. Retrospective
studies are one means of measuring results. Such analyses aim to identify
relevant results—usually expressed in monetary terms—of existing
regulations. The studies that identify actual benefits can provide insights
into the outcomes of EPA’s regulations. We reported in 1998 that the
Results Act recognizes and encourages the linkage between program
evaluations—retrospective studies are one type of such evaluations—and
performance measures. The Act directs federal agencies to provide a
13
RFF labeled a prospective study as “accurate” if the estimated costs in the retrospective study were
no more than 25 percent higher or no less than 25 percent lower than the prospective estimates.
Page 6 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
summary of program evaluation findings along with performance
measurement results in the required annual performance reports.14
Retrospective Studies Are Researchers face a number of difficulties in conducting retrospective
Difficult to Do studies: (1) determining the baseline against which the changes can be
identified and measured, (2) isolating the reasons for the actions taken,
(3) obtaining valid cost data from the regulated entities, and
(4) quantifying the benefits.
Determining the Baseline According to OMB’s 1998 Report to the Congress on the Costs and Benefits
of Federal Regulations, one of the primary problems with estimating the
incremental costs and benefits associated with implementing a federal
regulation is determining the baseline against which the costs and the
benefits should be compared. A baseline—how things would have been if
the regulation had not been issued—is elusive because so many changes
could have occurred while the regulation was being implemented. As OMB’s
report states, “what would have happened in the absence of regulation can
only be an educated guess since it never happened.”
In a 1996 report on companies’ regulatory burden, we found that it can be
extremely difficult for company officials to determine what actions their
company would have taken in the absence of the identified regulations.15
EPA and RFF officials took a more optimistic view but contended that
modeling would be needed to establish a baseline. Such modeling can be
costly and time-consuming and requires making assumptions about
economic behavior that may or may not approximate reality.
Determining the Reasons for According to OMB’s report, it is often difficult to attribute the actions taken
Regulated Entities’ Actions by regulated entities to specific federal regulations or other external
factors. These entities may be responding, for example, to regulations
from state and local governments. Voluntary standards from organizations
and public pressure also cause firms to protect the public in the absence
of federal regulations. In our 1997 report on the governmentwide
implementation of the Results Act, we found that regulatory agencies have
had difficulty in sorting out the effects of external forces.16 One expert
14
Program Evaluation: Agencies Challenged by New Demand for Information on Program Results
(GAO/GGD-98-53, Apr. 24, 1998).
15
Regulatory Burden: Measurement Challenges and Concerns Raised by Selected Companies
(GAO/GGD-97-2, Nov. 18, 1996).
16
The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be
Uneven (GAO/GGD-97-109, June 2, 1997). Also, see Managing for Results: Regulatory Agencies
Identified Significant Barriers to Focusing on Results (GAO/GGD-97-83, June 24, 1997).
Page 7 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
commented that external factors often influence technological innovation
and that isolating the impacts of regulations from these other factors is
difficult.
In 1997, EPA’s Office of Water found these multiple factors to be a problem
when it attempted to conduct a retrospective study of effluent guidelines.
The Office never implemented the study because it was too difficult to
isolate the costs specifically related to compliance with effluent guidelines
from either self-initiated technological improvements or state and local
government requirements.
Obtaining Valid Cost Data From A number of studies have reported difficulties in obtaining valid cost data.
Regulated Entities For example, our 1996 study on regulatory burden also found that
companies’ accounting and financial records did not capture the
information necessary to determine incremental compliance costs and that
there was little incentive to isolate and monitor these costs because such
information had little business value. Furthermore, companies were
concerned about the disclosure of information that is proprietary (e.g.,
information on their operating expenses) or could potentially involve
regulatory enforcement actions. Similarly, in 1996, we found it difficult to
determine the effects of regulatory compliance on research grantees at the
National Institutes of Health.17 Although members of the research
community were confident that compliance costs were high, they were
unable to provide cost data directly attributable to compliance activities.
Other research organizations have also had difficulties in data collection.
According to an expert in cost-benefit analysis at the Massachusetts
Institute of Technology, the university conducted a telephone survey to
collect actual cost data on scrubber technology, but despite having “very
good” industry connections, the survey’s results were somewhat
disappointing because some companies declined to participate. The
Organization for Economic Cooperation and Development—an
international agency sponsored by industrialized countries—is conducting
a business survey to determine the costs associated with implementing
regulations in three general areas: environmental compliance, tax
administration, and employment. Its survey is limited to small- and
medium-sized businesses. According to Organization officials, it is more
difficult for large companies that operate at multiple locations to
determine the cost impacts of regulations on their far-flung enterprises
and to isolate these costs from the costs attributable to other business
decisions. They also told us that asking businesses to self-report capital
17
Regulatory Compliance for NIH Grantees (GAO/HEHS-96-90R, Mar. 25, 1996).
Page 8 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
costs would not be valid because the data would not be verifiable or
consistent. In their view, such data would have to be collected through
very detailed structured interviews at selected businesses.
Some data on the costs of environmental regulatory compliance were once
available. Until 1994, the Department of Commerce administered the
Pollution Abatement and Control Expenditures (PACE) survey. When it
discontinued the survey to focus its resources on higher priorities,
Commerce collected data annually from manufacturers on the aggregate
cost of implementing pollution control regulations. Commerce, in
collaboration with EPA, is working to resume the PACE survey in 2000.
According to EPA, reinstatement of this survey is dependent on the
availability of funds. While one expert we interviewed suggested restarting
the PACE survey, another raised questions about the data’s validity and
usefulness. An official in EPA’s Office of Policy told us that his agency is
aware of these questions and, in developing the new survey, will attempt
to address them. EPA also pointed out that the agency’s Science Advisory
Board and others have recommended reinstatement of the survey.
Another attempt at data collection is currently being undertaken. A
contractor is collecting cost data for the retrospective study of the
drinking water standards for atrazine and nitrate. EPA officials told us that
they believe the drinking water treatment facilities will be able to isolate
the costs of compliance for these chemicals. They also expected both
public and private treatment facilities to cooperate.
Quantifying Benefits The experts we interviewed agreed that quantifying the benefits, as part of
a retrospective study, is extremely difficult. One cited reason is that data
on benefits, which generally accrue outside the regulated entities (e.g., to
protected species or society at large), are not available. Another reason,
which was highlighted by the author of the CFC retrospective study, is the
difficulty in assigning the benefits to a particular regulatory action versus
other factors. For example, he pointed out that although the benefits from
a reduction in occurrences in skin cancer could be attributed to the
phaseout of CFCs, other factors could have played a role. Finally,
considerable time may elapse—sometimes decades—before many benefits
are realized. For example, some diseases, such as cancer, can have latency
periods of decades. We emphasized this difficulty in our 1997 report on the
Results Act, citing EPA officials who noted that a significant time lag
usually exists between the elimination of a chemical hazard and any
corresponding change in illness rates. This lag makes it difficult to track
the yearly progress of some of the agency’s regulatory actions. EPA’s 1997
Page 9 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
Strategic Plan noted how difficult it is to assess the risks from pollutants;
measure the consequences of pollution to persons, plants, and animals;
and assign a dollar value to the health and environmental effects that are
remote in distance and time.
Based on discussions with study authors, EPA officials, and other experts,
A Systematic we found that a systematic approach by EPA would foster retrospective
Approach Would studies in the future. The agency, however, has not targeted regulations for
Foster Retrospective review after systematically (1) identifying candidate regulations for study
based on selection criteria, (2) assessing the feasibility of conducting
Studies retrospective studies of those regulations, and (3) identifying the
resources needed.
Identifying Candidates for While all economically significant regulations are subject, by executive
Review order, to a prospective economic assessment of their costs and benefits
before issuance, EPA is not required to calculate the actual costs and
benefits retrospectively for all of its rules. Program offices within the
agency have, on an ad hoc basis, initiated the sulfur dioxide study and the
ongoing study of the drinking water standards. EPA has not, however,
reviewed the full array of environmental regulations and, based on criteria,
selected certain rules as candidates for study.
In identifying candidates for review, one criterion cited by the experts we
interviewed is the potential for learning something useful that would
improve the efficiency and the effectiveness of a significant regulatory
program. For example, the significance of acid rain for the northeastern
states and the concern that existing regulations were not adequately
controlling the problem played a role when the Congress mandated a
retrospective study. Other criteria cited are the potential for gaining
insights into a new regulatory approach to environmental protection or the
methodology for prospective economic assessments. A retrospective study
can, for instance, examine assumptions made in prospective assessments
where uncertainty about regulatory costs exists because a new pollution
control technology is being used.
When identifying candidate regulations, timing is another consideration.
EPA would examine whether it would be more useful to study the actual
costs and benefits of certain regulations shortly after implementation or to
wait until a number of years or even decades have elapsed. A major factor
Page 10 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
in the timing decision would be the value of gaining insights into a
regulation’s impacts immediately or after a more extended period of time.
Assessing the Feasibility of The next action would be an assessment of the feasibility of conducting a
a Retrospective Study retrospective study of the candidate regulations. This assessment would
focus on the potential difficulties facing a researcher. For example, to
obtain valid cost data, EPA might find it easier to address those regulations
that rely on market forces because price and cost information is in the
public domain rather than those regulations based on “command and
control,” for which collecting the original data is especially challenging. In
some command-and-control cases, however, the regulated entities could
take steps during the implementation of the regulation to help researchers
develop data on compliance costs. These steps could include asking a
sample of regulated entities to establish financial and accounting
mechanisms to measure the impact of the regulation as it is implemented.
In addition, to attribute changes in the actions of regulated entities to
specific EPA regulations, RFF officials suggested that researchers employ
the same models used in conducting prospective economic assessments
and, therefore, make the same assumptions about regulated entities’
taking actions in response to the regulations. Of course, the researchers
would attempt to re-estimate the costs and the benefits after updating the
model, for example, with any data on the actual steps taken by regulated
entities.
Identifying Needed A planned approach would identify the resources needed to support the
Resources retrospective study. Funding is, of course, critical. EPA receives funding for
analytical work in general. While EPA has occasionally funded
retrospective studies, EPA program officials noted that they have used their
analytical resources almost entirely for prospective analyses—those that
provide information for decisions on proposed regulations. A 1997
Congressional Budget Office study found that EPA incurred an average cost
of $662,000 and a median cost of $376,000 (in 1995 dollars) on 65 economic
assessments.18 The program officials told us that they had limited
discretionary funds and resources and needed to use them in developing
new regulations. One official said that once a regulation is issued, his
office gears up to develop any other regulations required by law.
18
Regulatory Impact Analyses: Costs at Selected Agencies and Implications for the Legislative Process
(Mar. 1997). Costs included time spent by EPA personnel. The majority of the economic assessments
in the study were completed between 1990 and 1996. Because the scope and the complexity varied
substantially, the costs ranged from as low as $48,000 to as high as $6 million.
Page 11 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
The funding needed to do retrospective studies would be substantial if a
large proportion of the 101 economically significant regulations issued
since 1981 were the subject of retrospective studies. Where we were able
to obtain information on study costs, they varied substantially—from
$150,000 for the drinking water standards study, to $433,000 for the acid
rain report,19 to $4 million for the Section 812 study.
The method of funding as well as responsibility for a study can have
implications for its credibility. Authors of retrospective studies told us that
the credibility of these studies is enhanced when EPA funds an independent
analysis. The authors stated that it is better not to have the same EPA
offices that were responsible for conducting the prospective cost-benefit
analysis also responsible for conducting the retrospective study. OMB
officials stated that the best estimates of the costs and the benefits of
regulation are likely to be retrospective studies done by individuals who
do not have vested interests but do have reputations as objective analysts
to uphold. One study author believed that the cooperative agreement
arrangement helped ensure that the researcher had freedom to maintain
independence. As discussed earlier, EPA has used cooperative agreements
with organizations, such as RFF and the Massachusetts Institute of
Technology. Another author suggested that if EPA’s program offices were
directly involved in the study, the agency could use some type of advisory
committee or peer review process to oversee the studies and ensure their
credibility.
Resources also take the form of technical assistance, and EPA has provided
some. For example, the Office of Policy has built a library of EPA’s
economic assessments of major regulations and related reports and
provides a limited number of them on the Internet. Furthermore, EPA is
now soliciting comments on a major revision of its Guidelines for
Performing Economic Assessments.20 The agency first issued these
guidelines in December 1983 and has made a few modifications to them
over the past 15 years. The draft revision provides, among other things, a
summary of analytical methodologies, empirical techniques, and data
sources that can assist analysts in performing economic assessments of
environmental policies. While the guidance is targeted more to prospective
analyses, its principles can also be applied to retrospective studies.
19
From 1991 through 1996, the federal government spent $140 million to underwrite the scientific
research contained in the acid rain report. This research included an extensive network of monitoring
stations that provided scientists with data sets.
20
The Environmental Economics Advisory Committee of EPA’s Science Advisory Board has a major
role in reviewing these guidelines.
Page 12 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
To foster retrospective studies, the experts we interviewed also suggested
that the availability and the accessibility of the prospective economic
assessments and supporting documentation would have to be improved.
For example, some of these authors who had used the models employed in
the prospective analyses told us they had difficulty with the availability
and the usefulness of the models. One researcher wanted to run the model
used for the prospective economic assessment with different inputs. This
would have served as a useful way to determine to what extent and why
the estimate of actual costs and benefits differed from the prospective
estimate. In conducting his study, he had difficulty obtaining the complete
model from EPA. He was, however, able to obtain the complete model from
the contractor who performed the assessment. Another author found that
although he had access to the original model and supporting
documentation, it was very difficult to extract information.
Acknowledging concerns about the accessibility of models, the Office of
Air and Radiation maintains a web site that has placed all model inputs in
the public domain.
Finally, one author focused on the importance of capturing data on the
incremental costs of regulatory compliance from the regulated entities. He
noted that EPA would enhance its and others’ capacity to evaluate
environmental regulatory programs if the agency collected and maintained
data on the costs immediately after issuing those regulations it has
targeted for retrospective study.
EPA’s regulations, issued pursuant to federal statutes, cause this nation to
Conclusion make a substantial investment in protecting the environment. They also
result in considerable benefits. While EPA devotes substantial resources to
cost-benefit analyses when developing new regulations, the agency seldom
looks back at the actual costs and benefits after those regulations have
been implemented. An executive order and “good government” principles
recommend that EPA periodically reassess selected regulations to
determine if they are still needed and, if so, whether and how they could
be improved. Retrospective studies are difficult to do, but they are a
potentially valuable tool for such reassessments. We believe that EPA’s
current ad hoc approach gives no assurance that economically significant
regulations will be subjected to review. A systematic, planned approach
would help EPA to foster retrospective studies in the future.
Page 13 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
To help determine if existing environmental regulations need to be
Recommendation retained or improved, we recommend that the Administrator, EPA,
establish a plan to study the actual costs and benefits of such regulations,
including when and how to use retrospective studies as an integral part of
its approach. The plan should target selected regulations for review based
on a systematic identification of candidate regulations, an assessment of
the feasibility of studying those regulations retrospectively, and an
identification of the resources needed.
We provided a draft of this report to EPA for its review and comment. EPA
Agency Comments commented that the report is accurate, provides a fair characterization of
the difficulties in performing retrospective studies, and offers some useful
suggestions on how the agency could approach such studies in a more
systematic manner.
EPA also suggested that the budget implications of conducting
retrospective studies need to be emphasized in the report because the
scarcity of budget and internal resources limits the number of such studies
that the agency could support without the Congress’ providing significant
additional resources. In this report, we recognize that EPA concentrates its
analytical resources on prospective economic assessments, which EPA is
expected to perform, rather than on retrospective studies. Executive
Order 12866 states that EPA’s review of existing regulations should be
consistent with the agency’s resources and regulatory priorities. We
believe that if EPA were to develop a plan, as we are recommending, the
agency would be better able to identify the additional resources needed to
retrospectively review selected regulations. EPA also provided specific
technical comments and clarifications, which we incorporated, as
appropriate. EPA’s comments are contained in appendix I.
To accomplish our objectives, we interviewed officials in EPA’s Office of
Scope and Policy; Office of Air and Radiation; Office of Water; Office of Solid Waste
Methodology and Emergency Response; Office of Prevention, Pesticides, and Toxic
Substances; and Office of the Chief Financial Officer and obtained and
reviewed related legislation, memorandums, and reports. We also
interviewed all of the authors of the retrospective studies identified in this
report and several other experts in cost-benefit analysis, including RFF and
OMB officials, and representatives from Harvard University, the
Massachusetts Institute of Technology, and the Organization for Economic
Cooperation and Development. In documenting the difficulties in
Page 14 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
B-283198
obtaining actual cost data from companies, we relied heavily on our
November 1996 report, Regulatory Burden: Measurement Challenges and
Concerns Raised by Selected Companies. To identify the universe of
retrospective studies, we used data developed by RFF for its January 1999
report, On the Accuracy of Regulatory Cost Estimates. We also added the
Section 812 report, the acid rain report, and pesticide studies that RFF
chose to exclude because they met our criteria for containing information
on the actual costs of existing regulations and focusing on a specific
regulation or a set of regulations with common objectives.
We conducted our review from October 1998 through September 1999 in
accordance with generally accepted government auditing standards.
We are sending copies of this report to Senator Fred Thompson and
Senator Joseph Lieberman, as Chairman and Ranking Member,
respectively, of the Senate Committee on Governmental Affairs;
Representative Dan Burton and Representative Henry Waxman, as
Chairman and Ranking Member, respectively, of the House Committee on
Government Reform; Representative James Oberstar as Ranking Member
of the House Committee on Transportation and Infrastructure; the
Honorable Carol Browner, Administrator, EPA; and the Honorable Jacob
Lew, Director of OMB. We will also make copies available to others on
request.
If you have any questions about this report, please contact me on
(202) 512-6111. Major contributors to this report were Jay Cherlow, Ellen
Crocker, Bob Levin, and Bruce Skud.
Peter F. Guerrero
Director, Environmental Protection Issues
Page 15 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
Appendix I
Comments From the Environmental
Protection Agency
Page 16 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
Appendix I
Comments From the Environmental
Protection Agency
Page 17 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
Appendix I
Comments From the Environmental
Protection Agency
(160453) Page 18 GAO/RCED-99-250 Retrospective Studies of EPA’s Regulations
Ordering Information
The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.
Orders by mail:
U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013
or visit:
Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC
Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.
For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:
info@www.gao.gov
or visit GAO’s World Wide Web Home Page at:
http://www.gao.gov
PRINTED ON RECYCLED PAPER
United States Bulk Rate
General Accounting Office Postage & Fees Paid
Washington, D.C. 20548-0001 GAO
Permit No. G100
Official Business
Penalty for Private Use $300
Address Correction Requested