National Research Council's Evaluation of EPA STAR
Program

For the past 30 years, the U.S. Environmental Protection Agency
(EPA) has been the government's
primary agency for environmental sciences. The EPA's goal "is
to protect human health and to safeguard the natural environment -
air, water, and land - upon which life depends." To assist the
agency in its widening arena of tasks, the EPA created the Science
To Achieve Results (STAR)
program in 1995. The STAR program is an independently reviewed, competitive
research grant program aimed to attract the nation's best scientists
and engineers into the environmental science field. The goal of the
program is to produce high-quality relevant research that can aid
the agency's decision makers. This program accounts for 10-20% of
the EPA's Office of Research and Development's (ORD) budget. The president's
budget for the fiscal year (FY) 2003 completely eliminated the program's
funding. Congress approved a bill to supply one-half of the programs
requested funding. This budget dilemma has brought about questions
on the significance of the program. This budget dilemma has caused
the EPA to request the National Research Council (NRC)
to conduct an independent review of the program.

The NRC, whose Committee Chairman was Dr. Harold A. Mooney, concluded
that the STAR program plays an important role within the EPA. This
program produces a strong research base for the EPA to keep up with
the ever-changing and increasing environmental problems society faces.
Although the NRC committee believes it is too early to fully evaluate
the long term importance of the STAR program, the committee concluded
that the program has supported research that has already aided in
the EPA decision making process. The committee also concluded that
the STAR program needs to make use of external expertise -- from academia,
the private sector, federal agencies, state, tribal, and local governments
-- to assist in areas where the agency does not have expertise and
to "concentrate its focus of research agendas to identify the
highest-priority research." The STAR program should also use
ORD and EPA to aid in distributing its results to the appropriate
audience. The committee concluded that the STAR program has proven
to be a valid part of the EPA and it made a recommendation that its
"funding should be increased to 15-20% of the ORD annual budget."
A copy of the NRC report, The Measure of STAR: Review of the U.S.
Environmental Protection Agency's Science to Achieve Results (STAR)
Research Grants Program, is available on-line.

Specific Findings

The NRC's committee agreed to review the STAR program and assess its
merit. To evaluate the program, the committee examined the program's
quality, relevance, and performance. The committee produced a number
of questions in addressing the three previous points.

"Should the STAR program continue to be part of the ORD research
program?"

The committee concluded that the program should continue to be part
of the EPA's research program. They concluded that the EPA needs a
strong research program to meet its goals. The STAR program provides
excellent caliber researchers, a broad community of research, up to
date scientific methods, and it aids in addressing issues that the
agency does not cover internally.

"What is the unique contribution of the STAR program?"

The research done by the STAR program is directly related to the
EPA's goals. Most of the research conducted would not have been done
without the STAR program's funding. Additionally, the STAR program
has been successful in collectively working with other government
and nongovernmental organizations to achieve their mutual goals.

"Does the STAR program have adequate processes to ensure
that it is sponsoring high quality and relevant research?"

The STAR program uses a "grant-awarding process that is similar
with other agencies such as the National Science Foundation and the
National Institute of Environmental Health Sciences." The committee
concluded that the program needs to focus its research agendas and
make better use of its "outside experts to assist in identifying
the highest-priority research."

"Is the STAR program producing high-quality research results?"

The committee believes it is too early to evaluate this question
because of the short time period that the program has been in place.
The program, however, has already conducted research that has been
"published in highly respected, peer-reviewed journals."
Additionally, some research conducted by STAR researchers has contributed
to "understanding the biological mechanisms by which inhaled
ambient particles cause health problems."

"Are the STAR program results useful for EPA decisions and
processes?"

Again the committee believes that this question cannot be fully answered
because of the short time frame the program has been running. The
committee's initial review found that the STAR program is conducting
research that has been useful in environmental decision making. The
program has produced research on particulate matter (mentioned above),
endocrine disrupters, and ecological assessment. The ecological indicator
research has led to the "development of an economically linked
model to evaluate the driving forces and ecologic consequences of
land-use." Furthermore, the program needs to bring in outside
experts to assist in areas that the agency does not have expertise.

"Is the STAR program effective in providing results relevant
to the appropriate audiences?"

The program has tried to reach the many appropriate audiences but
it has had only minimal success. The program has excelled in providing
results to the science community and to those who are responsible
for the technical environmental-management decision. Its weakness
has been in not providing its results to other "federal agencies,
industry, non-governmental agencies, state, tribal, and local governments."
The committee stated that the STAR program needs to improve in these
areas. Additionally, the committee said the responsibility falls under
the STAR program to distribute its results but in "some cases
the primary responsibility falls with the ORD or EPA." As a whole
ORD and EPA need to assist the STAR program when appropriate, to provide
its results to the relevant audiences.

"Should the fellowship program continue to be part of the
ORD research program?"

The STAR fellowship program was established to "encourage promising
students to obtain advanced degree and then work in the environmental
field." This goal has been achieved. "Nearly 90% of
former fellowship recipients have remained in the environmental
field."

"Are the STAR program's funds adequate to achieve its objectives?"

The STAR program's funding is only able to support between 15-10%
of all applications. In addition, its funding has not increase with
inflation. The committee recommends that the funds be increase to
15-20% of the ORD's overall budget in hopes of having a funding acceptance
of 25-30% like the NIH and NSF.

"How should the STAR program be evaluated?"

The STAR program should be evaluated by a structured system of reviews
by panels of experts. The program should be "evaluated for the
quality, relevance, and performance of the program as set forth by
the Office of Management and Budget." In planning for reviews
the committee recommends a structure of four levels:

"level 1 should examine the individual research project

level 2 should focus on the topics or groups of research projects
on the same subject

level 3 should address the STAR program as a whole, and

level 4 should tackle the question of how the STAR program relate
to the broader institutions of ORD and EPA?"

Sources: NRC report, The Measure of STAR: Review of the U.S. Environmental
Protection Agency's Science to Achieve Results (STAR) Research Grants
Program.