DESCRIPTION OF EFFICIENCY MEASURES FOR PART

Efficiency Measures

While outcome measures provide valuable insight into program achievement, more of an outcome can be achieved with the same resources if an effective program increases its efficiency. The President’s Management Agenda (PMA) Budget and Performance Integration (BPI) Initiative encourages agencies to develop efficiency measures. Sound efficiency measures capture skillfulness in executing programs, implementing activities, and achieving results, while avoiding wasted resources, effort, time, and/or money. Simply put, efficiency is the ratio of the outcome or output to the input of any program. Because they relate to costs, efficiency measures are likely to be annual measures.

Outcome efficiency measures: The best efficiency measures capture improvements in program outcomes for a given level of resource use. Outcome efficiency measures are generally considered the best type of efficiency measure for assessing the program overall. For example, a program that has an outcome goal of “reduced energy consumption” may have an efficiency measure that shows the value of energy saved in relation to program costs.

Output efficiency measures: It may be difficult to express efficiency measures in terms of outcomes. In such cases, acceptable efficiency measures could focus on how to produce a given output level with fewer resources. However, this approach should not shift incentives toward quick, low-quality methods that could hurt program effectiveness and desired outcomes.

1

OMB 2006.

The National Academies of Sciences, Engineering, and Medicine 500 Fifth St. N.W. | Washington, D.C. 20001

Citation Manager

"Appendix I: PART Guidance on Efficiency Measures."
Evaluating Research Efficiency in the U.S. Environmental Protection Agency.
Washington, DC: The National Academies Press, 2008.

Please select a format:

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 128
Appendix I
PART Guidance on Efficiency Measures1
DESCRIPTION OF EFFICIENCY MEASURES FOR PART
Efficiency Measures
While outcome measures provide valuable insight into program achieve-
ment, more of an outcome can be achieved with the same resources if an effec-
tive program increases its efficiency. The President’s Management Agenda
(PMA) Budget and Performance Integration (BPI) Initiative encourages agen-
cies to develop efficiency measures. Sound efficiency measures capture skillful-
ness in executing programs, implementing activities, and achieving results,
while avoiding wasted resources, effort, time, and/or money. Simply put, effi-
ciency is the ratio of the outcome or output to the input of any program. Because
they relate to costs, efficiency measures are likely to be annual measures.
• Outcome efficiency measures: The best efficiency measures capture
improvements in program outcomes for a given level of resource use. Outcome
efficiency measures are generally considered the best type of efficiency measure
for assessing the program overall. For example, a program that has an outcome
goal of “reduced energy consumption” may have an efficiency measure that
shows the value of energy saved in relation to program costs.
• Output efficiency measures: It may be difficult to express efficiency
measures in terms of outcomes. In such cases, acceptable efficiency measures
could focus on how to produce a given output level with fewer resources. How-
ever, this approach should not shift incentives toward quick, low-quality meth-
ods that could hurt program effectiveness and desired outcomes.
1
OMB 2006.
128

OCR for page 128
129
Appendix I
Meaningful efficiency measures consider the benefit to the customer and
serve as indicators of how well the program performs. For example, reducing
processing time means little if error rates increase. A balanced approach is re-
quired to enhance the performance of both variables in pursuit of excellence to
customers. In these instances, one measure (e.g., increase in customer satisfac-
tion) may be used in conjunction with another complementary measure (e.g.,
reduction in processing time).
In all cases, efficiency measures must be useful, relevant to program pur-
pose, and help improve program performance. An efficiency measure for a Fed-
eral program tracks the ratio of total outputs or outcomes to total inputs (Federal
plus non-Federal). Leveraging program resources can be a rational policy deci-
sion, as it leads to risk or cost sharing; however, it is not an acceptable effi-
ciency measure, because the leveraging ratio of non-Federal to Federal dollars
represents only inputs. Although increasing the amount leveraging in a program
may stretch Federal program dollars, this does not measure improvements in the
management of total program resources, systems, or outcomes.
3.4: Does the program have procedures (e.g., competitive sourcing/cost
comparisons, IT improvements, appropriate incentives) to measure and
achieve efficiencies and cost effectiveness in program execution?
Purpose: To determine whether the program has effective management
procedures and measures in place to ensure the most efficient use of each dollar
spent on program execution.
Elements of Yes: A Yes answer needs to clearly explain and provide evi-
dence of each of the following [see Box I-1]:
• The program has regular procedures in place to achieve efficiencies and
cost effectiveness.
• The program has at least one efficiency measure with baseline and
targets.
BOX I-1 Measures and PARTWeb
To receive a Yes answer, the program must include at least one effi-
ciency measure, baseline data/estimates, and targets in the Measures
screen in PARTWeb.
Only measures that meet the standards for a Yes should be entered
in PARTWeb.
Please ensure that the proper characterization of measures is se-
lected in PARTWeb (that is “efficiency”). Make sure to indicate the term of
the measure in PARTWeb too (that is, long-term, annual, or long-
term/annual).

OCR for page 128
130 Evaluating Research Efficiency in EPA
There are several ways to demonstrate that a program has established pro-
cedures for so improving efficiency. For example, a program that regularly uses
competitive sourcing to determine the best value for the taxpayer, invests in IT
with clear goals of improving efficiency, etc., could receive a Yes. A de-layered
management structure that empowers front line managers and that has under-
gone competitive sourcing (if necessary) would also contribute to a Yes answer.
For mandatory programs, a Yes could require the program to seek policies (e.g.,
through review of proposals from States) that would reduce unit costs. Also con-
sider if, where possible, there is cross-program and inter-agency coordination on
IT issues to avoid redundancies. The program is not required to employ all these
strategies to earn a Yes. Rather, it should demonstrate that efforts improving
efficiency are an established, regular part of program management.
An efficiency measure can be the per-unit cost of outcomes or outputs, a
timing target, and other indicator of efficient and productive processes germane
to the program. Efficiency measures are likely to be annual measures since they
relate to cost.
The answer to this question should describe how measures are used to
evaluate the program’s success if achieving efficiency and cost effectiveness
improvements.
Elements of No: A No must be given if the agency and OMB have not
reached agreement on efficiency measures that meet PART guidance.
Not Applicable: Not Applicable is not an option for this question.
For more detailed discussion on defining acceptable efficiency measures
please see the section called “4. Select Performance Measure” of this document
or visit OMB’s PART website.2
Evidence/Data: Evidence can include efficiency measures, competitive-
sourcing plans, IT improvement plans designed to produce tangible productivity
and efficiency gains, or IT business cases that document how particular projects
improve efficiency.
4.3: Does the program demonstrate improved efficiencies or cost effective-
ness in achieving program goals each year?
Purpose: To determine whether management practices have resulted in ef-
ficiency gains over the past year.
Elements of Yes: A Yes answer needs to clearly explain and provide evi-
dence of each of the following [see Box I-2]:
• The program demonstrated improved efficiency or cost effectiveness
over the prior year. When possible, the explanation should include specific in-
formation about the program’s annual savings over the prior year as well as
what the program did to achieve the savings.
2
http://www.omb.gov/part/.

OCR for page 128
131
Appendix I
BOX I-2 Question Linkages
If a program received a No in Question 3.4, the program must receive
a No answer to this question.
Efficiency improvements should generally be measured in terms of dollars
or time. For example, programs that complete an A-76 competition—an indica-
tor of cost-efficient processes—would contribute to a Yes answer, provided that
the competition resulted in savings.
Not Applicable: Not Applicable is not an option for this question.
Evidence/Data: Evidence can include meeting performance targets to re-
duce per unit costs or time, meeting production and schedule targets; or meeting
other targets that result in tangible productivity or efficiency gains. Efficiency
measures may also be considered in Questions 4.1 and 4.2.
REFERENCES
OMB (Office of Management and Budget). 2006. Program Assessment Rating
Tool Guidance 2006-02. Office of Management and Budget. March 2006
[online]. Available: http://www.whitehouse.gov/omb/part/fy2006/2006_
guidance_final.pdf [accessed Dec. 17, 2007].