Results in Brief

Performance measures used in the state’s budgetary
process were not always reliable.About one-half of the measures we examined lacked sufficient
documentation, were based on inappropriate methodologies, or were calculated
incorrectly.In addition, the
description of the measure frequently did not reflect what was reported.As a result, managers and oversight
bodies used unreliable information when evaluating programs and making budget
decisions.

All of the agencies included in our audit lacked sufficient controls to
help ensure performance measures were reliable.Control weaknesses included inadequate
written procedures on how to collect and calculate performance measurement
data.In addition, we noted
insufficient review of the data collection process.Performance measurement systems should
have controls to ensure information is properly collected and reported.Reliability can be improved by the
Department of Administration providing agencies with additional guidance and
oversight concerning controls over the collection and reporting of performance
measures.

Principal Findings

·Some agencies could not provide underlying records to
support their performance measures reported in the Executive Budget.Of the 35 measures we examined, 15 did
not have sufficient documentation.Although these measures may be accurate, the lack of underlying records
prevents the information from being verified.(page 8)

·Thirteen of the measures were based on flawed
procedures.For instance, the
Northern Nevada Child and Adolescent Services’ method for tracking the average
number of clients on the Early Childhood Services waiting list overstated the
measure.The agency included
individuals that inquired about services but never followed up for an
appointment.These individuals
accounted for approximately 40% of the waiting list.(page 9)

·Calculation errors resulted in agencies reporting
inaccurate data in four of the measures.For example, the Department of Motor Vehicles’ measure of the number of
transactions processed by employee per month was calculated incorrectly.The reported number was the average for
each of the 18 field offices, not the average of all transactions
statewide.As a result, the number
reported by DMV was overstated by 21%.(page 9)

·The descriptions of 18 performance measures did not
reflect the reported information.For example, the Division of Health Care Financing and Policy described
one of its measures as the increase in enrollment of children in Medicaid.However, the Division actually reported
the number of children enrolled in August 2000, not the change in enrollment
during the year.(page 10)

·Many agencies have not developed written procedures
describing how to collect and calculate measures.The effect of not having written
procedures was greater because of turnover in positions that collected the data
reported in the Executive Budget.This made it especially difficult for some agencies to determine how the
performance measures reported in the Executive Budget were derived.(page 11)

·At most agencies, one person collected and calculated
performance measurement data with little or no review by anyone else.Performance measures are often the
result of numerous calculations.As
the number and complexity of calculations increases, the risk of errors
increases substantially. (page 12)

·State agencies
have been required to include performance measures in the Executive Budget since
the 1993 Legislative Session.Although the Department of Administration has provided training to assist
in the development of performance measures, the training has not addressed
procedures to help ensure reliability.In addition, the Department’s oversight of agencies’ internal controls
can be improved by including guidance regarding the reliability of performance
measures.(page 12)

Reliability of
Performance Measures

Used in the State’s Budget Process

Agency Response

to Audit
Recommendations

Recommendation

Number

Accepted

Rejected

1

Provide state agencies with guidance on controls
over the collection of performance measurement data, including developing
written procedures, retaining underlying records, and providing sufficient
review.