DOJ Workforce Planning: Grant-Making Components Should Enhance the Utility of Their Staffing Models

Below is a raw (and likely hideous) rendition of the
original report.
(PDF)

United States Government Accountability Office
GAO Report to Congressional Committees
DOJ WORKFORCE
December 2012
PLANNING
Grant-Making
Components Should
Enhance the Utility of
Their Staffing Models
GAO-13-92
December 2012
DOJ WORKFORCE PLANNING
Grant-Making Components Should Enhance the
Utility of Their Staffing Models
Highlights of GAO-13-92, a report to
congressional committees
Why GAO Did This Study What GAO Found
DOJ annually awards billions of dollars The Department of Justice’s (DOJ) three grant-making components—the Office
to thousands of grant recipients, which of Justice Programs (OJP), the Office of Community Oriented Policing Services
presents administrative and workload (COPS), and the Office on Violence Against Women (OVW)—have partially met
challenges for DOJ’s workforce. In May five of six leading practices, and fully met another, to ensure that the staffing
2011, DOJ’s grant-making components models the contractor prepared for each of them to assess their workloads and
began deploying new staffing models workforce capacities are sound and reliable. These six leading practices are
to analyze their workloads and better broad areas that represent a summary of the activities that should take place
ensure that staffing levels and during the three stages of model design, development, and deployment. For
personnel assignments reflect their
example, the components partially met the leading practice for ensuring the
responsibilities. In November 2011, the
credibility of data used in the models. This practice includes both verifying with
conference report accompanying
DOJ’s FY 2012 appropriation directed
staff that the models accurately depict the component’s operations—an activity in
GAO to evaluate each component’s which they fully engaged—and testing the models for logic and accuracy—an
staffing model. In response, this report activity in which they partially engaged. Specifically, the components did not
addresses (1) the extent to which the comprehensively check all of the models’ formulas, and GAO found errors with
grant-making components incorporated all three components’ models’ retirement calculations. When left uncorrected,
leading practices to ensure that their such errors have implications for workforce-related decisions and, ultimately, can
staffing models are sound and reliable affect the efficiency and effectiveness of component operations. In contrast, the
and (2) the components’ use of the components fully met one leading practice and its associated activities for
models to inform their workforce leveraging stakeholders. For example, all three components engaged grant
planning efforts, and the extent to managers and senior program officials when working with the contractor to apply
which they used the models for budget their knowledge and experience to design and develop the models. Moving
development. GAO reviewed the forward, fully incorporating leading practices when updating the models for future
staffing models the components use will better position the grant-making components to capitalize on the models’
procured under contract in 2011, and capacity and facilitate effective workforce planning and budgeting.
the updated, 2012 versions; identified
six leading practices based on our prior Due to the deficiencies GAO identified in the design, development, and
work for ensuring the models’ deployment of the staffing models, grant-making components have not used the
soundness and reliability; and models to identify workforce gaps or to inform the preparation of annual budget
assessed the components’ actions for requests. In particular, the three grant-making components have not used the
meeting each practice. GAO also models to determine critical skills and competencies of staff and identify related
interviewed component officials about gaps. Further, officials from all three components stated that they would like to,
their use of the models. or plan to, use the models to inform the upcoming fiscal year 2015 budget
requests, but they do not yet have a strategy for doing so. For example, officials
What GAO Recommends from all three grant-making components recognize that their human capital staff
will need to share and interpret the data in the models with their budget staff, but
GAO recommends that DOJ
as of late September 2012, no time frames, coordination principles, or guidance
components fully incorporate leading
were in place to ensure this exchange of information. Standard practices for
practices to help ensure their staffing
models’ soundness and reliability, and project management, including budget development, involve the establishment of
develop and implement a strategy for timelines, coordination principles, and guidance. Developing, documenting, and
using the models to inform workforce implementing a strategy that includes these elements could help the grant-
planning and budget development. making components better use the models to inform budget development. In
DOJ agreed with these particular, such a strategy should account for the full incorporation of leading
recommendations and reported it practices to ensure the models are sound and reliable and that the quantifiable
would take actions to address them. data from the models provide the proper context and justification to the Congress
for the funding levels the components request.
View GAO-13-92. For more information,
contact David C. Maurer at (202) 512-9627 or
maurerd@gao.gov.
United States Government Accountability Office
Contents
Letter 1
Background 6
Grant-Making Components Have At Least Partially Met Leading
Practices to Ensure that the Staffing Models Are Sound and
Reliable, But Can Strengthen Their Efforts 15
Grant-Making Components Have Not Used the Staffing Models to
Determine Workforce Gaps and Do Not Have a Strategy for
Using the Models to Inform Budget Requests 29
Conclusions 35
Recommendations for Executive Action 36
Agency Comments 36
Appendix I Objectives, Scope, and Methodology 38
Appendix II Comments from the Department of Justice 41
Appendix III GAO Contact and Staff Acknowledgments 45
Tables
Table 1: Overview of Active Grants, Authorized Positions, and
Onboard Positions, by Grant-Making Component from
Fiscal Years 2008 through 2011 9
Table 2: Four Phases of the Federal Grant Life Cycle 10
Table 3: Key Contractor Deliverables 14
Table 4: Leading Practices for Ensuring Model Soundness and
Reliability 16
Table 5: Extent to Which Grant-Making Components Took Actions
to Account for Risk Associated with Contracting Out
Model Development 16
Table 6: Extent to Which Grant-Making Components Took Actions
to Ensure the Credibility of Data Used in the Models 18
Table 7: Extent to Which Grant-Making Components Took Actions
to Preserve the Integrity of Data Maintained in the Models 21
Table 8: Extent to Which Grant-Making Components Took Actions
to Establish Roles and Responsibilities for Staff Assigned
to Update and Operate the Models 23
Page i GAO-13-92 DOJ Workforce Planning
Table 9: Timeliness of Components’ Designation of Responsible
Staff to Initiate Updates to the Models 24
Table 10: Extent to Which Grant-Making Components Took
Actions to Ensure Adequate Training for Staff Assigned to
Update and Operate the Models 26
Table 11: Extent to Which Grant-Making Components Took
Actions to Leverage Stakeholders’ Knowledge and
Experience in Designing and Developing the Models 27
Figures
Figure 1: Department of Justice Grant Funding Appropriated from
Fiscal Years 2008 through 2011 8
Figure 2: Timeline of Events That Affected the Grant-Making
Components’ Plans to Expand the Size of their
Workforces 13
Abbreviations
ACOR Assistant Contracting Officer Representative
COPS Community Oriented Policing Services
COR Contracting Officer Representative
DOJ Department of Justice
OJP Office of Justice Programs
OVW Office on Violence Against Women
This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.
Page ii GAO-13-92 DOJ Workforce Planning
United States Government Accountability Office
Washington, DC 20548
December 14, 2012
The Honorable Barbara Mikulski
Chairwoman
The Honorable Kay Bailey Hutchison
Ranking Member
Subcommittee on Commerce, Justice,
Science, and Related Agencies
Committee on Appropriations
United States Senate
The Honorable Frank Wolf
Chairman
The Honorable Chaka Fattah
Ranking Member
Subcommittee on Commerce, Justice,
Science, and Related Agencies
Committee on Appropriations
House of Representatives
At the close of fiscal year 2012, the Department of Justice’s (DOJ) three
grant-making components—the Office of Justice Programs (OJP), the
Office of Community Oriented Policing Services (COPS Office), and the
Office on Violence Against Women (OVW)—were responsible for
overseeing just over $13 billion in federal financial assistance to support
crime prevention, law enforcement, and crime victim services to about
31,000 grantees operating at the state and local levels. 1 The fiscal year
2012 enacted budgets of these three components comprise about 9
percent of DOJ’s total budget authority, and to manage their workload,
the three grant-making components are authorized for about 1 percent of
1
DOJ provides federal financial assistance pursuant to statutory authorization. The
assistance typically takes the form of formula grants; discretionary grants; and cooperative
agreements and all are generally referred to as grants. Formula grant programs are
noncompetitive awards based on a predetermined formula, typically established in statute.
Discretionary grants are usually awarded on the basis of a competitive selection process.
A cooperative agreement is a type of grant wherein the federal government is more
substantially involved than usual in carrying out associated activities. For more
information, see GAO, Justice Grant Programs: DOJ Should Do More to Reduce the Risk
of Unnecessary Duplication and Enhance Program Assessment, GAO-12-517
(Washington, D.C., Jul. 12, 2012).
Page 1 GAO-13-92 DOJ Workforce Planning
DOJ’s total staffing level—a total of 960 positions. Staff members within
these offices are involved in all aspects of grant awarding, including
preparing funding announcements to describe the allowable uses of new
grant funds, disbursing the awards, and routinely monitoring grantees’
compliance with financial, administrative, and programmatic
requirements. 2 In addition to this workload, the grant-making component
workforce has other responsibilities as well, including but not limited to
providing technical assistance to grantees, conducting research or
preparing publications, and responding to external audits. To ensure that
the grant-making components’ respective staffing levels and personnel
assignments accurately reflect the responsibilities and requirements of
each office, DOJ’s Justice Management Division (JMD) awarded a $2
million contract to a private sector consulting company to develop a
workload and workforce planning model (i.e., “a staffing model”) for each
of the three grant-making components. 3 Performance on the contract
began in December 2010, and in May 2011, the contractor delivered a
staffing model to each granting-making component based on common
principles and assumptions, yet unique to their respective business
processes. In addition, the contractor delivered a number of related
planning documents, such as suggested strategies for developing
workforce skills.
We have previously reported on the need for strategic workforce planning
to enhance performance and ensure accountability in federally funded
programs. 4 Further, we have designated strategic human capital
management as a high-risk area and reported that the government’s
management of its people—its human capital—was the critical missing
link in reforming and modernizing the federal government’s management
2
GAO-12-517.
3
Among other responsibilities, JMD helps ensure strategic and human capital planning
and oversees budget development for all of the agencies within DOJ.
4
GAO, Human Capital: Key Principles for Effective Strategic Workforce Planning,
GAO-04-39 (Washington, D.C.: Dec. 11, 2003).
Page 2 GAO-13-92 DOJ Workforce Planning
practices. 5 As we have also previously reported, the number of grant
programs and recipients, and the billions of dollars in funds DOJ awards
annually, present administrative and workload challenges for DOJ’s
workforce. 6
In November 2011, the conference report accompanying DOJ’s
appropriations act for fiscal year 2012 directed DOJ to detail actual and
estimated costs of grant-making components’ management and
administrative expenses, by grant program, in future budget
submissions. 7 The costs are to include categories of expenses other than
grants, such as providing technical assistance and conducting research
and statistics, among others. The conferees specifically encouraged the
grant-making components to minimize administrative spending in order to
maximize the amount of funding that can be used for grants. In addition,
the conference report acknowledged the grant-making components’
efforts to analyze their workloads, and directed us to evaluate each grant-
making component’s staffing model and make recommendations, as
warranted, on how each model could be further improved. This report
addresses the following questions: (1) to what extent have the grant-
making components incorporated leading practices to ensure that their
staffing models are sound and reliable?, and (2) how have the
components used the models to inform their workforce planning efforts
and to what extent have the components begun to use the models for
budget development?
To address the first question, we assessed how the staffing models are
structured and the steps the grant-making components have taken to
ensure that the models are sound and reliable. We primarily reviewed our
5
GAO, High Risk Series: An Update, GAO-01-263 (Washington, D.C.: Jan.1, 2001). Since
that time, the federal government has made progress in addressing its human capital
challenges and we have therefore narrowed the scope of this high-risk area to focus on
the most significant challenges that remain to close current and emerging critical skills
gaps in vital areas such as acquisition, foreign language capabilities, and oil and gas
management. See GAO, High Risk Series: An Update, GAO-11-278 (Washington, D.C.:
Feb. 16, 2011).
6
GAO-12-517.
7
H.R. Rep. No. 112-284, at 242-43 (2011) (Conf. Rep.)
Page 3 GAO-13-92 DOJ Workforce Planning
prior work 8 to identify six leading practices for ensuring the staffing
models’ soundness (i.e., their validity and reliability): (1) accounting for
risk associated with contracting out model development, (2) ensuring the
credibility of data used in the models, (3) preserving the integrity of data
maintained in the models, (4) establishing roles and responsibilities for
staff assigned to update and operate the models, (5) ensuring adequate
training for staff assigned to update and operate the models, and (6)
leveraging stakeholders’ knowledge and experience in designing and
developing the models. We then assessed each individual grant-making
component’s reported actions against the actions associated with each
leading practice and determined the extent to which the practices were
met. To do this, we used a scale of “fully met,” “partially met,” and “not
met.” A determination of “fully met” means that the grant-making
component provided evidence that it had completed all actions associated
with the leading practice. A determination of “partially met” means that the
grant-making component provided evidence that it had completed some
actions associated with the leading practice. A determination of “not met”
means that the grant-making component provided no evidence that it had
completed any actions associated with the leading practice.
We also interviewed grant-making components’ senior officials with
responsibility for grant administration as well as for human capital
management and budget planning to understand how they worked with
the contractor to design and develop the staffing models. We also
interviewed those in each grant-making component responsible for
updating and executing their respective models to get perspectives on the
training they received, their prior workforce planning backgrounds, and
8
We primarily used GAO, Internal Control Management and Evaluation Tool,
GAO-01-1008G (Washington, D.C.: Aug. 1, 2001) to develop the six leading practices.
These leading practices are also informed and supported by GAO, Designing Evaluations:
2012 Revision, GAO-12-208G (Washington, D.C.: Jan. 31, 2012); Federal Information
System Controls Audit Manual, GAO-09-232G (Washington, D.C.: Feb. 2009); and
Assessing the Reliability of Computer-Processed Data, GAO-09-680G (Washington, D.C.:
Jul. 2009). GAO has also examined the concepts of soundness and reliability in the
following reports: See GAO, Tax Administration: IRS Needs to Strengthen Its Approach for
Evaluating the SRFMI Data-Sharing Pilot Program, GAO-09-45 (Washington, D.C.: Nov.
7, 2008); Catastrophic Planning: States Participating in FEMA’s Pilot Program Made
Progress, but Better Guidance Could Enhance Future Pilot Programs, GAO-11-383
(Washington, D.C.: Apr. 8, 2011); Telecommunications: FCC’s Performance Management
Weaknesses Could Jeopardize Proposed Reforms of the Rural Health Care
Program,GAO-11-27 (Washington, D.C.: Nov. 17, 2010); and Equal Employment
Opportunity: Pilot Projects Could Help Test Solutions to Long-standing Concerns with the
EEO Complaint Process, GAO-09-712 (Washington, D.C.: Aug. 12, 2009).
Page 4 GAO-13-92 DOJ Workforce Planning
the time they spent working with the models. In addition, we analyzed
documentation about the grant-making components’ practices related to
updating the models, as well as the contractor’s guidelines for doing so.
We then compared the grant-making components’ practices for
designing, developing, and deploying the models for use to the leading
practices we identified and made judgments about the extent to which
they aligned. We also assessed each grant-making component’s model
independently, reviewing all of the contractor’s deliverables and
understanding the formulas and inputs constituting the models’
framework. To evaluate each model, we used a discovery method
approach, wherein we examined a small number of strategically drawn
formulas and calculations. While we were not attempting to develop a
specific error rate per se, this discovery method approach permitted
conclusions about the risk for (a) additional errors in other parts of the
model we did not examine, and (b) potential errors in the model’s outputs.
To address the second question, we interviewed grant-making
component and JMD officials on their workforce planning efforts prior to
having the staffing models and discussed how the components are
conducting workforce planning since obtaining the models and the ways
in which they are using the models to inform their activities. We also
discussed the grant-making components’ efforts and future plans to
integrate the models into their budget submissions and the role JMD
plays in facilitating these efforts. In addition, we reviewed related
documents to support the officials’ testimonial evidence and identified
relevant practices from our prior work for effectively linking budgets and
workforce plans and compared DOJ’s activities and plans with these
practices. More details on our scope and methods appear in Appendix I.
We conducted this performance audit from May 2012 to December 2012
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Page 5 GAO-13-92 DOJ Workforce Planning
Background
Workforce Planning Workforce planning is a process for getting the right number of people,
Concepts with the right skills, in the right job, at the right time to accomplish an
organization’s mission and goals. As we have reported previously,
strategic workforce planning addresses two critical needs: (1) aligning an
organization’s human capital management efforts with its current and
emerging mission and programmatic goals, and (2) developing long-term
strategies for acquiring, developing, and retaining staff to achieve
programmatic goals. 9
At its heart, workforce planning involves assessing workload;
understanding the skills and limitations (i.e., capacity) of the existing
workforce; and addressing the gaps needed to optimize performance.
While organizations’ approaches to workforce planning will vary, our
earlier work established five key principles that strategic workforce
planning should address, irrespective of the context in which the planning
is done. These include the following:
• involving top management, employees, and other stakeholders in
developing, communicating, and implementing the strategic workforce
plan;
• setting a strategic vision to determine the critical skills and
competencies that will be needed to achieve current and future
programmatic results;
• identifying and developing strategies that are tailored to address gaps
in number, deployment, and alignment of human capital approaches
for enabling and sustaining the contributions of all critical skills and
competencies;
• building the capability needed to address administrative, educational,
and other requirements important to support workforce planning
strategies; and
9
GAO-04-39.
Page 6 GAO-13-92 DOJ Workforce Planning
• monitoring, evaluating, and revising the organization’s progress
toward its human capital goals and the contribution that human capital
results have made toward achieving programmatic results.
In addition, we have previously reported on the importance of ensuring
that data used to make key decisions, such as for workforce planning, are
both sound and reliable—meaning that the data are accurate, credible,
and appropriate for analysis. 10
Grant-Making DOJ administers its grant programs through three grant-making
Components’ Roles, components—OJP, the COPS Office, and OVW.
Responsibilities, and
• OJP is comprised of a number of bureaus and offices and its mission
Funding is to develop the nation’s capacity to prevent and control crime,
administer justice, and assist crime victims. As such, it administers
federal financial assistance to state and local governments, for-profit
and nonprofit organizations, tribal jurisdictions, and educational
institutions to help address victim assistance, technology and
forensics, and juvenile justice, among other things.
• The COPS Office advances the practice of community policing, which
supports the systematic use of partnerships and problem-solving
techniques to address public safety issues such as crime, social
disorder, and fear of crime. The COPS Office makes its grants
available to police departments, including tribal law enforcement
components, as well as researchers. Across its programs, the COPS
Office has also funded schools and other organizations promoting
public safety.
• OVW seeks to provide federal leadership in developing the nation’s
capacity to reduce violence against women and strengthen services to
victims of domestic violence, dating violence, sexual assault, and
stalking. Like OJP and the COPS Office, its grant funding is available
to an array of recipients, including those providing legal assistance,
tribal and local law enforcement components, schools and campuses,
and other advocacy organizations.
10
GAO-09-45 and GAO-11-383.
Page 7 GAO-13-92 DOJ Workforce Planning
The levels of appropriations across all three grant-making components
have varied over time, as figure 1 illustrates, but OJP has consistently
received the greatest amount. In fiscal year 2011, OJP had approximately
$2.5 billion in funding for grants; the COPS Office had approximately
$528 million; and OVW had approximately $431 million.
Figure 1: Department of Justice Grant Funding Appropriated from Fiscal Years 2008
through 2011
Notes: (1) Appropriations figures include American Recovery and Reinvestment Act (ARRA), Pub. L.
No. 111-5, 123 Stat. 115, funds but do not include carryover from prior years or payment programs,
which are managed differently from typical grants in that they generally fund grantees on a
reimbursable rather than direct basis. OJP is the only grant-making component to operate payment
programs. (2) COPS Office appropriations reflect budget authority that has been provided to the
COPS Office by DOJ’s annual appropriation acts, such as the DNA Backlog Reduction Program,
even though these appropriations are shifted to OJP through an internal transfer that gives OJP
program administration responsibilities. The amounts were $243,084,000 in 2008; $272,000,000 in
2009; $203,000,000 in 2010; and, $168,153,020 in 2011.
Similarly, the number of grants in active status, number of authorized
positions, and the number of onboard positions across the grant-making
Page 8 GAO-13-92 DOJ Workforce Planning
components have varied over time, as shown in table 1, but OJP has
consistently been the largest component. 11 Specifically, OJP managed
more than 19,000 grants, and had 702 authorized positions and 668
onboard positions in fiscal year 2011. 12 In contrast, OVW, the smallest of
the three, managed more than 2,800 grants and had 70 authorized
positions with 61 onboard positions.
Table 1: Overview of Active Grants, Authorized Positions, and Onboard Positions,
by Grant-Making Component from Fiscal Years 2008 through 2011
OJP 2008 2009 2010 2011
Active grants 14,912 18,899 19,438 19,404
Authorized positions 697 697 702 702
Onboard positions 637 690 696 668
COPS Office 2008 2009 2010 2011
Active grants 4,614 4,505 4,844 4,401
Authorized positions 166 166 188 188
Onboard positions 108 116 134 136
OVW 2008 2009 2010 2011
Active grants 2,533 2,578 2,685 2,871
Authorized positions 65 65 65 70
Onboard positions 48 58 66 61
Source: OJP, COPS Office, and OVW.
Notes: (1) Active grants are those that were active for more than 1 day during the fiscal year, and do
not reflect the agency’s entire workload. (2) Authorized positions include full- and part-time federal
employees but do not include additional contractor staff. There are some authorized positions whose
responsibilities are not directly related to grants management yet whose duties are critical to the
operation of the component. (3) The level of actual, onboard personnel may be lower at any given
time than the total positions authorized, and may also be slightly higher so long as DOJ is within the
allowable, authorized position cap departmentwide.
The federal financial assistance the grant-making components administer
typically takes the form of discretionary grants, formula grants, and
11
Authorized positions include full- and part-time federal employees but do not include
additional contractor staff. There are some authorized positions whose responsibilities are
not directly related to grants management yet whose duties are critical to the operation of
the component.
12
Onboard positions refer to personnel that are actually in place at any given time.
Page 9 GAO-13-92 DOJ Workforce Planning
cooperative agreements, and all are generally referred to as grants. 13 The
grants that OJP, the COPS Office, and OVW award generally follow a
similar life cycle, including pre-award, award, implementation, and closeout
stages, as table 2 illustrates. OJP and OVW use a web-based system—the
Grants Management System (GMS)—to manage this process, while the
COPS Office uses a separate COPS Management System (CMS), which is
a desktop, integrated set of programmatic, financial, and related
applications that communicates with a centralized database.
Table 2: Four Phases of the Federal Grant Life Cycle
Grant phase Common activities
Pre-award • Announce the grant opportunity with a grant solicitation, which
notifies potential applicants of the grant’s purpose, the terms of the
award, the amount available, and the eligibility criteria, among
other information
• Receive and review applications
• Make award decisions based on requirements set by statute and
the grant-making component, as appropriate
Award • Notify the grantee
• Publicly announce the awarding of funds
Implementation • Disburse payment
• Collect and review grantee progress reports to check that grants
are being implemented as intended and that grantees are
compliant with statutory or regulatory requirements as well as any
applicable policy guidelines
• Conduct site visits to monitor grantees’ activities at their locations
• Review expenditures of grantees compared with their approved
budgets and allowable grant expenditures, and review compliance
with grant requirements such as the submission of required grant
reportsa
Close-out • Review and reconcile final data and other reports
• Ensure that each recipient has met all of the grant’s programmatic
and financial requirements, returned any unused funds, and
provided final reports
Source: GAO
a
For more on the grant monitoring and assessment functions of the three grant-making components
see GAO-12-517.
13
In a competitive program, applicants generally compete for funding that agencies award
at their discretion. In contrast, in a formula program, recipients and award amounts are
generally based on statutorily defined calculations that may incorporate a state’s
population and violent crime rate, for example. For more details on these programs, as
well as cooperative agreements, see GAO-12-517.
Page 10 GAO-13-92 DOJ Workforce Planning
DOJ and the Grant-Making Office of Personnel Management (OPM) regulations require organizations
Components’ Workforce to conduct workforce analyses as part of their human capital plans that
Planning Efforts describe their current states, project human resources needed to achieve
organizational goals, and identify potential shortfalls. 14 In response, DOJ’s
JMD human resources staff has taken the lead in developing human
resource strategies and establishing a workforce planning framework for
the Department. In October 2007, JMD convened a workforce planning
council, comprised of representatives from the major DOJ components,
including OJP, to ensure that workforce planning and human resource
initiatives would be integrated consistently and cost-effectively throughout
DOJ. In June 2010, the council issued the Department’s first workforce
plan, covering the major components from 2010 through 2013. The plan
is updated each year using annual addenda. In addition to the workforce
plan, JMD issued a strategic leadership succession plan in 2007 and a
departmentwide human capital strategic plan covering 2007 through
2012, which it has recently begun to update. JMD also works with all the
grant-making components and other department components to refine
their budgets for submission in the President’s annual budget request.
Because OVW and the COPS Office are not considered to be major
components, and instead are referred to as “Offices, Boards and
Divisions,” JMD’s current workforce plans do not specifically address
these two grant-making components’ workload and workforce capacities
and gaps. In recognition of this, and in part because of congressional
interest in seeing more detail in the grant-making components’ budget
submissions, JMD took the lead in awarding a contract for a consolidated
workforce planning effort that would result in a staffing model for
assessing workload demands and workforce capacity—as well as gaps—
and strategies for bridging the gaps for each of the three grant-making
components. DOJ sent a request for proposals to select vendors in
September 2010, describing its requirement for a comprehensive
workload and workforce analysis to ensure that its grant-making
components are properly structured, staffed, and managed to meet
current and projected responsibilities. The request for proposals stated
that the review should identify best practices, problems, and solutions
relating to the grant-making components’ existing and projected workload
and workforce requirements, and should examine and document the
processes, procedures, and data used by each grant-making component
14
5 C.F.R. § 250.203.
Page 11 GAO-13-92 DOJ Workforce Planning
in making workforce planning decisions. DOJ awarded the contract in
early December 2010.
During the contract’s term, as well as during the months preceding the
final award, a number of events transpired that affected the grant-making
components’ plans for expanding the size of their workforces. As the
timeline in figure 2 illustrates, these events included a governmentwide
continuing resolution as well as a departmentwide hiring freeze. 15
15
A continuing resolution is an appropriation act that provides budget authority for federal
agencies, specific activities, or both to continue in operation when Congress and the
President have not completed action on the regular appropriation acts by the beginning of
the fiscal year.
Page 12 GAO-13-92 DOJ Workforce Planning
Figure 2: Timeline of Events That Affected the Grant-Making Components’ Plans to Expand the Size of their Workforces
To meet DOJ’s specifications, the contractor provided regular progress
reports, slide sets to explain the methods used, and a set of five key
planning documents, or “deliverables,” as table 3 shows. In conducting
this work, the contractor assigned staff to each of the grant-making
components, and in May 2011, provided each with a unique set of these
five deliverables, all tailored to the individual grant-making component’s
circumstances and needs.
Page 13 GAO-13-92 DOJ Workforce Planning
Table 3: Key Contractor Deliverables
Deliverable Purpose/scope
Existing Workforce An initial profile of current and future workload (demand)
Environment/Future and current and future workforce (supply).
Workforce Environment
Workforce Capacity Gap A report highlighting current and potential future talent
Analysis gaps at each skill and grade level within an occupational
series.
A Description of the Staffing A description of the methods to develop the staffing
Model and Related Talent model and strategies that each grant-making component
Strategies can execute to develop the workforce capacity and
competencies it needs for the future.
Staffing Model An electronic spreadsheet that reflects the work required
for key activities (e.g., grant administration) and matches
resources required with forecasted needs (e.g., newly
authorized grant programs). It was designed to be
flexible enough to react to future changes, to be used
and updated independently by grant-making component
staff, and to provide input into each grant-making
component’s workforce plans and resource/budget
requests.
User Guide A technical manual that documents the model’s
methodology and provides a tactical “how to” for using
the model. It was intended for audiences already familiar
with the work done at each grant-making component
and who will be actively using the model and its outputs
(e.g., senior leadership and other decision-makers,
model users, and model administrators).
Source: GAO analysis of contractor deliverables.
The staffing model is the key contractor deliverable that was designed to
provide DOJ’s grant-making components with:
1. a repeatable methodology for better estimating future resource needs
and making fact-based cases for resources in their budget requests
and
2. a tool for better (a) understanding the factors that drive their
workloads, and (b) identifying the skill gaps they need to address to
be successful in the future.
According to DOJ officials, the model was specifically designed as one
tool among many for workforce planning and resource allocations rather
than a single solution, and has the flexibility to evaluate the impact of
potential changes and scenarios.
Page 14 GAO-13-92 DOJ Workforce Planning
According to grant-making component officials, to inform the models’
development, they provided the contractor with access to subject matter
experts to inform the process through interviews, focus groups, and
surveys. A JMD official served as the lead Contracting Officer
Representative (COR) and each granting-making component designated
an assistant, or ACOR, to review the deliverables from the component’s
perspective. 16 The lead COR also chaired a technical evaluation panel,
comprised of a representative from each of the grant-making components,
to evaluate the contractor’s proposal prior to awarding the contract.
The grant-making components have partially met five of six leading
Grant-Making practices, and fully met another, to ensure that their staffing models are
Components Have At sound and reliable. We identified these six leading practices—broad
areas that represent a summary of the activities that should take place
Least Partially Met during the three stages of model design, development, and deployment—
Leading Practices to from prior GAO work. 17 They are: (1) accounting for risk associated with
contracting out model development, (2) ensuring the credibility of data
Ensure that the used in the models, (3) preserving the integrity of data maintained in the
Staffing Models Are models, (4) establishing roles and responsibilities for staff assigned to
Sound and Reliable, update and operate the models, (5) ensuring adequate training for staff
assigned to update and operate the models, and (6) leveraging
But Can Strengthen stakeholders’ knowledge and experience in designing and developing the
Their Efforts models. We believe that fully adhering to these leading practices would
better position DOJ’s grant-making components to have reasonable
assurance that their staffing models are sound and reliable. Table 4
illustrates each of the six leading practices we included in our review,
their implications for soundness and reliability, and the extent to which the
grant-making components collectively met them.
16
The COR is responsible for ensuring that contractors abide by the contract. The COR
provides critical acquisition and technical functions to ensure that contractual agreements
are properly followed.
17
We primarily developed this framework using GAO-01-1008G. The leading practices are
also supported by other publications such as GAO-12-208G; GAO-09-232G; and
GAO-09-680G.
Page 15 GAO-13-92 DOJ Workforce Planning
Table 4: Leading Practices for Ensuring Model Soundness and Reliability
Extent to which
components met the
Leading practice Implications for soundness and reliability practice
Account for risk associated with contracting out Weigh the benefits and hazards of contracting out for Partially
model development project management versus managing the project in-
house
Ensure the credibility of data used in the models Provide assurance that data supporting a model, Partially
project, or activity are appropriate, complete, and
correct (e.g., check formulas in the model)
Preserve the integrity of data maintained in the Provide assurance that changes to the model are Partially
models authorized, complete and accurate and are preserved
for ongoing use (e.g. password-protected data)
Establish roles and responsibilities for staff Promote accountability and establish how resource Partially
assigned to update and operate the models needs will be met
Ensure adequate training for staff assigned to Provide assurance that a model, activity, or project can Partially
update and operate the models be used for its intended purpose
Leverage stakeholders’ knowledge and experience Provide assurance that a model, project, or activity Fully
in designing and developing the models meets user needs and specifications
Source: GAO analysis.
Note: A designation of “fully met” indicates that a component provided evidence that satisfied the
entire criteria for the respective leading practice; “partially met” indicates that a component provided
evidence that satisfied some portion of the criteria; and “did not meet” indicates that a component
provided no evidence that satisfied any portion of the criteria.
Account for Risk Associated The grant-making components partially met this leading practice, as table
with Contracting Out Model 5 illustrates. Accounting for risk associated with contracting out model
Development development includes verifying the contractor’s ability to complete the
work; outlining clear, relevant, and measurable objectives for the
contractor; and ensuring adequate technical assistance post-delivery of
the work product.
Table 5: Extent to Which Grant-Making Components Took Actions to Account for
Risk Associated with Contracting Out Model Development
Leading practice: account for risk associated with contracting out
model development ◑
• Verify contractor's ability to complete work ●
• Outline clear, relevant, and measurable objectives for the contractor ●
• Ensure adequate technical assistance post-delivery of work product 
 Not met
◑ Partially met
● Fully met
Source: GAO.
Page 16 GAO-13-92 DOJ Workforce Planning
To verify the contractor’s ability to complete the work, JMD and the grant-
making components confirmed the contractor had experience conducting
workforce and workload assessments. Specifically, the technical evaluation
panel requested information about the qualifications and experience of the
contractor staff who would be assigned to work on the models. Moreover,
to outline clear, relevant, and measurable objectives for the contractor, the
contract required the delivery of monthly contractor progress reports. JMD
appointed as lead COR a qualified individual with prior experience in
workforce analysis and ensured this individual received training in COR
roles and responsibilities. Together, the lead COR and the ACORs from
OJP, the COPS Office, and OVW reviewed these monthly progress reports
to check for completeness and timeliness of the work.
However, when deciding to contract out the work—rather than relying on
in-house staff—OJP, the COPS Office, and OVW stakeholders did not
design and execute the contract to ensure adequate technical assistance
post-delivery of the final work product. In particular, the components did
not address the provision of post-delivery technical assistance at any of
the following points during the contracting process:
• when the grant-making components drafted an initial statement of
work for JMD’s consideration;
• when JMD prepared the final request for proposals to outsource the
work;
• when the technical evaluation panel reviewed the prospective
contractor’s proposal—even though the proposal showed the work
period ending with delivery of the models; and
• when JMD approved the final contract.
As a result of not ensuring that the contract included a specific provision
for post-delivery technical assistance, all three grant-making components
have had to independently reach out to the contractor for help once they
started familiarizing themselves with the models and began to update
their data. As such, they have had to rely on the contractor’s goodwill to
answer any questions that have arisen since the contract period has
ended. For example, OVW officials stated that they were unable to
manipulate a formula in the staffing model to properly account for part-
time contractor staff. To work around this limitation, they had to enter
negative values elsewhere in the staffing model spreadsheet. OVW
officials reported reaching out to the contractor since they were not
Page 17 GAO-13-92 DOJ Workforce Planning
knowledgeable about how to correct this issue on their own, and they
ultimately received assistance in how to correct the error in the model.
More recently, in August 2012, a COPS Office official reached out to the
contractor for additional assistance updating their workforce data.
Specifically, the COPS Office found that the model was not saving
updates they had made. The contractor determined that there was a
“bug” in the model, and proceeded to correct the issue. According to
officials from all three grant-making components, at this time, they do not
see a need for additional technical assistance; however, should it be
needed, future assistance is not guaranteed and would require a new
task order for future contracted services. 18
Ensure the Credibility of Data The grant-making components partially met this leading practice, as table
Used in the Models 6 illustrates. Ensuring the credibility of data used in the models includes
verifying with staff that the models accurately depict their system and
testing the models for logic and accuracy.
Table 6: Extent to Which Grant-Making Components Took Actions to Ensure the
Credibility of Data Used in the Models
Leading practice: ensure data credibility
• Verify with staff that model accurately depicts their system
◑
●
• Test the model for logic and accuracy
◑
 Not met
◑ Partially met
● Fully met
Source: GAO.
To verify with staff that the models accurately depict their respective
systems, officials from the grant-making components told us that subject
matter experts reviewed the accuracy of the staffing models’ workforce
inputs, such as staffing levels, and workload assumptions, such as the
length of time certain tasks take. For example, OJP officials stated that they
reviewed the workforce data the contractor used from OPM to ensure that
the full-time equivalent (FTE) staffing counts appeared reasonable.
Moreover, OVW and COPS Office officials reported reviewing the
contractor’s summaries of interviews with grants management staff to
18
A task order is an order for services placed against an established contract or with
government sources. FAR § 2.101.
Page 18 GAO-13-92 DOJ Workforce Planning
determine the accuracy of the contractor’s estimates for completion of
certain grant management tasks, such as developing grant announcements
and notifying applicants of award decisions. Since these components do
not have systems to track the actual amount of time grant managers spend
per activity, they had no other way to verify the credibility of the workload
assumptions the contractor used to estimate resource needs.
With respect to testing the model for logic and accuracy, the components
reported taking steps to test the logic, but did not follow a systematic
process to test for accuracy in a manner consistent with leading practices.
To test the logic of the models’ design and development, the components
reported involving the ACORs, managers, and subject matter experts in
reviewing, evaluating, and approving the contractor’s assessments of
each component’s “existing environment.” OJP officials shared with the
contractor that they organize their workload around individual grant
solicitations rather than overarching grant programs. In contrast, officials
from the COPS Office and OVW ensured the contractor understood that
they organize their workload around the broader grant programs, rolling
up the individual solicitations. Thus, the contractor built the models in
recognition of these distinctions and the logic of each model reflects the
variation.
However, in testing the accuracy of the data and formulas in the model, the
grant-making components did not follow a systematic process to check the
completeness and accuracy of staffing model data. Specifically, in checking
the data, they did not use a random sample that was large enough to
conclude that all the data the contractor used to support the model were
sound and reliable. 19 Instead, all three grant-making components made
limited attempts to check the data and the formulas in the models they
received. For example, OJP officials stated they reviewed some of the data
but could not provide evidence of a sampling methodology that would allow
them to make conclusions about the overall accuracy of the data. Similarly,
officials from the COPS Office stated that they did not develop a systematic
verification plan to check that the data and formulas were free of flaws and
programming errors, but that they reviewed the model output and worked
backwards through the model to determine if the output seemed
questionable. OVW officials stated that they tested a sample of the data,
but did not use a scientific process to select that sample. They were unable
19
GAO-01-1008G.
Page 19 GAO-13-92 DOJ Workforce Planning
to check all data because only one employee was available to perform this
task. According to component officials, the contractor’s experience with
similar projects in the private and public sectors provided them the
assurance that the contractor’s data gathering and modeling
methodologies were sound. Nevertheless, even with their limited review,
the grant-making components reported uncovering several contractor
errors. These ranged from the contractor’s formulas not carrying over
beyond a certain row in the spreadsheet—and thus not producing
calculations—to formulas not accounting for activities associated with grant
closeout activities—and thus underestimating total workload
responsibilities.
Our review of the models also identified several issues indicating that the
components did not test the accuracy of the models in a manner consistent
with leading practices for ensuring soundness and reliability. 20 For
example, the OJP, COPS Office, and OVW models each revealed a
programming error related to retirement calculations that none of the offices
was aware of before we identified it during our audit work. Specifically, the
OJP and COPS Office models were 1 year off in their retirement
projections, miscalculating the annual impact of potential retirement on the
existing workforce and, in so doing, miscalculating the potential capacity of
the workforce. Additionally, another methodological error in OJP’s model
caused OJP to initially underestimate retirements by 86 percent (or
approximately 54 retirements) from 2013 through 2016. OJP has since
corrected the errors in its model and the COPS Office is working to address
its error. Further, the methodological error in the OVW model causes any
staff eligible to retire in the next 12 months to be counted as eligible in the
current calendar year, irrespective of the actual year of eligibility. Though
OVW is a small office and the current error impacts just one employee, if
left uncorrected, this formula error could have greater impact in the future
as more employees approach retirement eligibility. For example, this error
could cause a model user to believe that the workforce would be impacted
on an accelerated timeframe and provide inaccurate information to inform
workforce planning. During the course of our audit work, OVW officials
could not identify the steps they planned to take to address the incorrect
formula. Additionally, the COPS Office’s model, unlike the other models,
omits eligible Executive Management staff from its retirement calculation,
resulting in an underestimation and suggesting that the overall rigor of the
20
See appendix I for elaboration on this approach.
Page 20 GAO-13-92 DOJ Workforce Planning
COPS Office review of the model may not have been adequate to ensure
soundness. Upon acknowledgement of our finding, COPS Office officials
said they planned to correct the error.
When we asked the grant-making components about the extent of their
initial tests of the models’ logic and accuracy, OJP officials stated they
were satisfied with the retirement estimates that the model produced based
on the contractor’s use of an assumed retirement rate of 15 percent per
year. COPS Office officials stated that limited staff capacity—one person
was assigned to the model—and the added challenge of reorganizing their
Grants Administration Division while updating the model delayed more
comprehensive checking of the contractor’s formulas. OVW officials also
pointed to resource constraints, stating that they had sufficient resources to
check only a sample and found this sampling approach to be reasonable in
light of their limited staff capacity. Nevertheless, the errors we found raise
questions about whether the components systematically confirmed the
credibility of the staffing model data or verified the results. As a result, the
reliability of the models could be compromised. Moreover, using the
models without validating their results for decision-making could expose
components to additional risk—for example, by miscalculating the size of
the workforce needed to fulfill workload demands—ultimately impacting the
efficiency and effectiveness of their operations.
Preserve the Integrity of Data The grant-making components partially met this practice, as table 7
Maintained in the Models illustrates. Preserving the integrity of the data maintained in the models
includes four controls related to model use: authorization control;
completeness control; accuracy control; and integrity control.
Table 7: Extent to Which Grant-Making Components Took Actions to Preserve the
Integrity of Data Maintained in the Models
Leading practice: preserve data integrity
• Authorization control: authorize use with passwords, supervisory
◑
approval, and access rights ●
• Completeness control: make changes electronically and review
entries to ensure completeness ●
•
•
Accuracy control: check accuracy of data before and after entry
Integrity control: protect model integrity by documenting changes
◑
and maintaining records ◑
 Not met
◑ Partially met
● Fully met
Source: GAO.
Page 21 GAO-13-92 DOJ Workforce Planning
With respect to authorization control, leading practices indicate that only
authorized users should be granted the ability to access and update the
staffing model. All three grant-making components have restricted access
to the models or plan to restrict access as a means of protecting their
results. OJP officials stated that access to the model is password-
protected to limit access to certain key individuals, and OVW officials
stated that they have provided copies only to those staff members
assigned to update and operate the model. Similarly, officials from the
COPS Office stated that access is restricted to the staff member assigned
to work on the model and the quality reviewer using their individual data
drives.
Regarding accuracy control, leading practices also suggest that users
should systematically review information for accuracy prior to entering
data and that others should check to identify and correct any errors
associated with data entry after data have been entered. Two of the three
grant-making components reported checking data prior to entry, and all
three grant-making components reported reviewing new data entries
against source documentation. For example, OJP officials stated that
data are checked post-entry, with two employees reviewing the data that
another employee has entered using the source documentation. COPS
Office officials reported a similar process in which data are first verified
with managers and then entered by one employee. Then, post-entry, all
data are reviewed by that employee’s supervisor using the same source
documentation. OVW officials reported checking personnel data from
JMD and contractor data from managers prior to entry. After data are
entered, a supervisor reviews the updates against the source documents,
and program-related data are double-checked by a program expert.
However, when asked to describe their processes for ensuring accuracy,
none of the components reported reconciling updated versions of the
model against prior versions—an action important for ensuring accuracy
after data are entered. Because each model is contained within a large
spreadsheet consisting of multiple worksheet tabs, reconciliation against
prior versions helps identify any changes made in error and ensure that
information from prior versions which should remain in the model is in fact
carried forward into any updated versions. Moreover, these stored
versions could allow the grant-making components to evaluate how
accurately the model predicted certain outcomes, such as attrition, and
ensure a record of workload and workforce changes over time.
With respect to integrity control, leading practices state that users should
document and maintain records of changes made to the model. Officials
Page 22 GAO-13-92 DOJ Workforce Planning
from all three grant-making components reported storing prior versions on
shared access servers or drives. Specifically, officials from OJP described
saving versions using a shared access server, which captures the most
recent version and prevents multiple users from updating the model
concurrently. Officials from the COPS Office and OVW reported saving
updated versions using the date of the update as a guide, but they did not
report using a process to ensure that the most recent version is used
each time data are updated and that multiple users may not concurrently
update the models. At the time of our review, only one employee from the
COPS Office and one employee from OVW were assigned primary
responsibility for updating the data, resulting in a low risk of updating the
wrong version or allowing concurrent updates. However, officials from
both the COPS Office and OVW reported that they intend to increase the
number of model users in the future, and implementing integrity controls
to identify the most current model version and prevent concurrent updates
would help protect model integrity.
As a result of not fully establishing comprehensive policies to review the
model for accuracy and documenting and maintaining records of changes
to protect model integrity, components increase the risk that the models
will not produce reliable information to inform management decisions and
budget requests. Both current decisions and future uses of the models
could be compromised if components do not act to implement protections
before using the models on a regular basis.
Establish Roles and The grant-making components partially met this leading practice, as table
Responsibilities for Staff 8 illustrates. Establishing roles and responsibilities for staff assigned to
Assigned to Update and update and operate the models includes assigning staff to all necessary
Operate the Models tasks and defining their duties and responsibilities.
Table 8: Extent to Which Grant-Making Components Took Actions to Establish
Roles and Responsibilities for Staff Assigned to Update and Operate the Models
Leading practice: establish roles and responsibilities
◑
• Assign staff to all necessary tasks
Define the duties of responsible staff
◑
• 
 Not met
◑ Partially met
● Fully met
Source: GAO.
Page 23 GAO-13-92 DOJ Workforce Planning
Before work began on model design, development, and deployment, all
three grant-making components assigned staff to some, but not all,
necessary tasks. Specifically, all three components identified and assigned
staff to work on the design and development of the models. Nevertheless,
delays in being able to use the model resulted in part because the grant-
making components did not identify staff to begin updating the models with
more recent data until many months after the contractor finalized the fiscal
year 2010 baseline data, as table 9 illustrates.
Table 9: Timeliness of Components’ Designation of Responsible Staff to Initiate
Updates to the Models
Lapse in time
Designation of responsible staff to initiate from model
Date updates to the models receipt
January 2012 OVW reassigned an individual working under an 8 months
existing contract to work with the model and update
its data.
January 2012 OJP assigned a staff member it had hired in May 8 months
2011 specifically to begin collecting and assessing
updated data.
February 2012 COPS Office assigned a management analyst to 9 months
start working on updates.
Source: GAO analysis of OVW, OJP and COPS Office documents and interviews with OVW, OJP and COPS Office officials.
OJP and OVW officials stated that delays in identifying staff responsible
for review and updates to the data in the models were due to staff
availability and timing of updates. OJP officials stated that they started
updates in January 2012 because that is when the individual assigned to
the model became available and when they had sufficient fiscal year 2012
data to include in the model. Similarly, OVW officials also stated that they
began the updates when 2012 data were available. Officials from the
COPS Office explained that competing human capital priorities and DOJ’s
hiring freeze contributed to delays in hiring and assimilating qualified staff
with the skills needed to work on the model. All three components have
faced or currently face human resource constraints and challenges in
hiring and staffing qualified staff with the skills needed to update data in
the staffing models. For example, OJP was authorized for more human
capital staff than employed until October 2012, which is when it expected
to fill its three open human capital positions, and one of the individuals
filling these three slots would be assigned to work on the model. OVW
officials stated that they are authorized for one human capital position, but
the slot remains vacant due to budget concerns and the associated
departmentwide hiring freeze that remains in place. According to OVW
Page 24 GAO-13-92 DOJ Workforce Planning
officials, while JMD fulfills most of their human capital needs, having this
position vacant means they have no one to focus on strategic human
capital planning within their organization. Though OVW officials said they
could seek a hiring exemption to fill this slot, they have chosen instead to
request exemptions to fill more critical vacancies associated with grants
management. 21 Officials from the COPS Office stated they have only one
human capital position authorized and filled, and have had to “borrow”
staff from other units in order to complete their model updates. Because
the three grant-making components already faced constrained staffing
and limited personnel resources prior to accepting the additional
responsibility of operating the models, it was even more imperative that in
preparation for model deployment, they identified the human capital
capacity needed to carry out the related staffing model responsibilities.
Moreover, the grant-making components did not define the duties of staff
responsible for working with the model before staff members were
brought onboard. Officials from all three grant-making components stated
that newly assigned staff were not part of the discussions with the
contractor on the model’s design and development, and required
additional time to familiarize themselves with the model’s data before they
could begin making updates. For example, the individual OVW tasked
with model updates said that, initially, nearly 75 percent of her time was
spent familiarizing herself with the model in order to get up to speed on
the model’s capabilities. Although all three grant-making components
currently have staff onboard to continue updates to the data in the
models, it is unclear if these assignments can be sustained and thus, if
the roles and responsibilities can be institutionalized. For example, COPS
Office officials stated that the one COPS Office staff member originally
assigned responsibility for managing the model left the Department. Her
successor, an accountant within the COPS Office, has been assigned to
work with the model as time permits, given her other officially assigned
workload and responsibilities. Similarly, the staff member OVW assigned
with responsibility for data updates is a contractor and OVW is not certain
about renewal of the company’s contract. Going forward, clearly
21
Each grant-making component decides how many hiring exemptions to seek and then
sends these requests to JMD for review and recommendation. JMD reviews each
exemption for a number of elements, which, among others, includes the position
description, an explanation for why current staff are unable to fulfill the duties and
responsibilities of the position, and the availability of funding to support the position in the
future. If JMD recommends approval, JMD forwards the hiring exemption request to the
Deputy Attorney General for a final decision.
Page 25 GAO-13-92 DOJ Workforce Planning
delineating roles and responsibilities would help DOJ institutionalize the
processes required to manage the staffing models and would enhance
DOJ’s grant-making components’ ability to sustain, and, more importantly,
facilitate workforce planning and budget development decision-making.
Ensure Adequate Training for The grant-making components partially met this practice, as table 10
Staff Assigned to Update and illustrates. Ensuring adequate training for staff assigned to update and
Operate the Models operate the models includes ensuring a mechanism exists for providing
training to staff and ensuring that training is comprehensive and sufficient
to enable successful operation of the model.
Table 10: Extent to Which Grant-Making Components Took Actions to Ensure
Adequate Training for Staff Assigned to Update and Operate the Models
Leading practice: ensure adequate training
• Ensure mechanism exists for providing training to staff
◑
●
• Ensure training is comprehensive and sufficient to enable successful
operation of the model 
 Not met
◑ Partially met
● Fully met
Source: GAO.
JMD ensured a mechanism existed for providing staff training by stating
in the contract that the contractor would be required to provide training
sufficient to allow grant-making component staff to operate the staffing
model and update it to enable ongoing use. To meet this requirement and
with JMD’s concurrence, the contractor delivered User Guides to each
grant-making component, which were designed to serve as a training
mechanism for staff who would be updating and using the models.
However, the components did not ensure that the guides were
comprehensive and sufficient to enable successful operation of the
models. Specifically, they did not thoroughly examine the content of the
guides upon receipt, but rather waited to review them carefully until they
began to update the models. Moreover, component officials told us of
their varying levels of satisfaction with the User Guides—one component
indicated that the guide was helpful; a second component reported that it
was useful, but noted several functions that the Guide should have
included or better explained; and the third component stated that the
guide did not meet its needs. When we asked the grant-making
component officials about training beyond the guides, OJP and COPS
Page 26 GAO-13-92 DOJ Workforce Planning
Office officials said the contractor did not provide any, while the OVW
official who currently reviews the model’s updates said she recalled some
hands-on training from the contractor to facilitate her use. Since none of
the grant-making components’ primary model users were in their current
positions when the contractor was competing its work, all reported having
to develop familiarity with the models via hands-on, on-the-job exposure.
As a result of not receiving training, components have experienced delays
in updating the staffing model to enable its use for budget formulation and
workforce planning. Model users from OJP, the COPS Office, and OVW
reported that the delays in updating the models resulted, in part, from
needing additional time to familiarize themselves with the model. As users
have gained more familiarity with the models, some of the components
have made efforts to train other users, which is consistent with the
contractor’s notes in one of its planning documents, which states that in a
hiring freeze environment, capacity gaps will have to be addressed
through other means, such as cross training. For example, OVW officials
stated that the individual currently assigned primary responsibility for
updating the model began to cross-train another OVW human resources
staff member. As stated previously, the COPS Office recently lost its
original model user due to attrition, and as of August 2012, has utilized a
staff accountant whose workload contained additional capacity in order to
proceed with model updates. Like the COPS Office, OJP officials have
just one person assigned to the model and no back-fill available.
Leverage Stakeholders’ The grant-making components fully met this leading practice, as table 11
Knowledge and Experience in illustrates. Leveraging stakeholders’ knowledge and experience in
Designing and Developing the designing and developing the models includes involving stakeholders in
Models establishing contractor terms of work and involving stakeholders in design
and development.
Table 11: Extent to Which Grant-Making Components Took Actions to Leverage
Stakeholders’ Knowledge and Experience in Designing and Developing the Models
Leading practice: leverage stakeholders ●
 Involve stakeholders in establishing contractor terms of work ●
 Involve stakeholders in design and development ●
 Not met
◑ Partially met
● Fully met
Source: GAO.
Page 27 GAO-13-92 DOJ Workforce Planning
Stakeholder involvement was evident in establishing the contractor terms
of work. According to OJP officials, OJP produced the initial draft
statement of work to outline the scope and terms of the prospective
contractor’s work, and stakeholders from the COPS Office and OVW
provided their input by commenting on the draft. Further, OJP, the COPS
Office, and OVW each appointed an ACOR to serve as a stakeholder on
the technical evaluation panel, which was tasked with evaluating the
contractor’s proposal. As part of its proposal selection process, this
technical evaluation panel submitted two sets of follow-up questions to
the prospective contractor seeking additional information and clarifying
aspects of the proposal. For example, the panel desired clarification,
which they eventually received, on how the contractor would compare the
grant-making components’ operations against other public and private
organizations. The panel then evaluated the prospective contractor’s
responses in conjunction with the overall proposal before DOJ finalized
the contract. From the COR’s perspective, leveraging the perspectives of
stakeholders from the grant-making components—via this panel—helped
ensure that the stakeholders were engaged in the contractor selection
process and ultimately in agreement regarding the services the contractor
aimed to provide.
Stakeholder involvement also was evident during the models’ design and
development. As noted, DOJ’s initial request for proposal required the
contractor to submit monthly progress reports, and the lead COR
reviewed these to check for completeness and timeliness of the work.
According to officials from OJP, the COPS Office, and OVW, the lead
COR reached out to the ACORs to confirm with these stakeholders that
the contractor’s work met their needs. For example, officials from the
COPS Office and OJP stated that when the contractor concluded its
assessments of the conditions at the COPS Office and OJP at the start of
the work period, the COR asked each component’s ACOR to accept or
decline the assessment. Similarly, OVW officials stated that the COR met
with OVW’s ACOR to discuss the accuracy of the contractor’s
assessment of the conditions at OVW at the start of the work period. All
three of the grant-making components reported that their respective
senior management—yet another set of stakeholders—was involved in
confirming the accuracy of the contractor’s assessments of each
component’s existing conditions.
Moving forward, fully incorporating leading practices when updating the
models for future use will better position the grant-making components to
capitalize on the models’ capacity and facilitate effective workforce
planning and budgeting. Specifically, implementing a more rigorous data
Page 28 GAO-13-92 DOJ Workforce Planning
review process with clear methods for confirming the credibility of data
and verifying the results the staffing models generate; reviewing changes
to the data in the models to ensure accuracy; defining and determining
the respective roles and responsibilities of staff involved in reviewing,
operating, and assisting with the staffing models; and identifying training
needs that address staff needs to fulfill responsibilities associated with the
staffing models would better position the grant-making components to
facilitate effective workforce planning and make sound budget
development decisions.
Due to the deficiencies we identified in the design, development, and
Grant-Making deployment of the staffing models, the grant-making components have
Components Have not used the staffing models to determine workforce gaps, such as critical
skills and competencies of staff, or to develop long-term human capital
Not Used the Staffing plans that address these gaps. Additionally, the three grant-making
Models to Determine components have not yet begun to use the data in the models to inform
the preparation of annual budget requests. However, officials from all
Workforce Gaps and three grant-making components stated that they would like to or plan to
Do Not Have a use the models to inform the budget submissions for fiscal year 2015
funding, but they do not yet have a strategy in place for doing so.
Strategy for Using the
Models to Inform
Budget Requests
The Grant-Making Our prior work examining issues in strategic human capital planning
Components Have Not highlighted the use of workforce planning tools, such as staffing models,
Used the Staffing Models to assist organizations in addressing two critical needs in strategic
workforce planning: (1) aligning an organization’s human capital program
to Identify Capacity Gaps, with its current and emerging mission and programmatic goals, and (2)
but Some Have Used the developing long-term strategies for acquiring, developing, and retaining
Models to Support staff to achieve programmatic goals. The three grant-making components
Ongoing or Operational have not yet begun to use the models in either of these two areas, in part,
because of some of the staffing challenges noted earlier. However,
Planning according to officials from all three grant-making components, they are
pursuing various staffing options, such as exemptions to the
departmentwide hiring freeze to dedicate staff to work on the models,
reorganizing existing staff and shifting their responsibilities, or using
contractor personnel, so that they can expand their use of the model
accordingly.
Page 29 GAO-13-92 DOJ Workforce Planning
As of September 2012, OJP received approval for three hiring
exemptions intended to enhance the component’s human capital
capacity. According to OJP officials, once they fill these positions—
expected by October 2012—the current, sole human resources specialist
assigned to update the model will have support from one additional staff
member. Officials from the COPS Office and OVW stated that they are
not seeking hiring exemptions to add to their human capital capacity.
However, according to officials from the COPS Office, they are discussing
the possibility of moving internal staff from other offices to address their
human capital needs. OVW officials stated that they are continuing to
utilize a contracted staff person to work on updates to the model and that
any hiring exemptions OVW submits will concentrate on positions that
directly relate to its grant-making responsibilities.
Nevertheless, OJP officials stated that they used the staffing model’s data
to help determine who among their staff would be eligible for 2012
Voluntary Early Retirement Authority (VERA) and Voluntary Separation
Incentives Payment (VSIP)—two OPM initiatives that help components
resize their workforces. In addition, OJP officials explained that when two
OJP program offices separately considered whether to seek exemption
authority to hire more staff, OJP’s Human Resources Division relied on
the model to provide data on the number and type of positions each office
already had, and served as one tool to show that consolidation across the
two offices was plausible. As a result, OJP is currently considering
consolidation of these offices rather than expanding one or both of them.
At a more conceptual level, OVW officials said that OVW used the
model’s data to confirm a previously held workforce planning assumption
that the workload was increasing as the workforce was decreasing (due
to attrition and lag time in hiring replacement staff). As such, officials
reported that they envision the model being useful to them as they
embark on decisions about whether to reorganize OVW. In particular,
they said they plan to use the model to show the areas in the grant life
cycle in which staff members spend most of their time and what
processes OVW can improve to achieve greater efficiency. In addition,
the COPS Office’s restructuring is underway, and now that the model
accurately accounts for their new workload and assignment of
responsibilities, officials plan to analyze the model’s results for ongoing
strategic workforce allocations.
Page 30 GAO-13-92 DOJ Workforce Planning
The Components Do Not The grant-making components reported that they would like to or plan to
Yet Have a Documented use the models to inform the fiscal year 2015 budget request; however,
Strategy to Ensure the as of September 2012, they did not yet have documented plans in place
to do this, even though the budget planning process will begin in early
Staffing Models Inform the 2013 and according to OJP officials can start as early as late fall 2012.
Fiscal Year 2015 Budget
Request As noted, the contractor delivered the staffing model in May 2011 based
on fiscal year 2010 workload and workforce data, which was the most
recently available at the time. According to grant-making component
officials, they were unable to use the models to inform their budget
development decisions in the fall of 2011 (to inform the request for
funding in fiscal year 2013) because they considered the staffing models’
“snapshot” data already too old—that is, not reflective of current
conditions—for reasonable inclusion. Thus, the grant-making components
spent the first year after the contractor delivered the model learning its
functionality and ramping up staff and expertise to update it. The grant-
making components have completed these updates as of September
2012, and officials from the three grant-making components reported that
they would like to or plan to use the staffing models to inform the
upcoming budget request for fiscal year 2015. Consequently, the grant-
making components could better position themselves to do so by
beginning the planning process early. In this regard, standard practices
for project management state that managing a project involves, among
other things:
• developing a timeline with milestone dates, including specific points
throughout the project to reassess efforts underway and determine
whether changes are necessary;
• establishing coordination to understand the appropriate roles of those
involved with a project and how they engage the various stakeholder
groups; and
• providing guidance regarding the direction of a project and changes
affecting the outcome of projects. 22
22
Project Management Institute, The Standard for Program Management® (Newtown
Square, PA: 2006). The Project Management Institute is an organization that provides
guidelines, rules, and characteristics for project management.
Page 31 GAO-13-92 DOJ Workforce Planning
Implementing a documented project management strategy to include
timelines, coordination efforts, and guidance for using the data in the
staffing models to inform the fiscal year 2015 budget submission would
help the grant-making components leverage the quantifiable workforce
and workload data that the models generate and provide context and
justification to the Congress for the funding levels they request.
Timelines According to grant-making component officials, working with the data in
the staffing models necessitates time to validate and verify data going into
the models, update the models, and review updates to the models for
completeness and accuracy. However, officials from the grant-making
components provided no specific timeframes to address a comprehensive
analysis of the data, which could ensure the full incorporation of leading
practices, as we have discussed. Rather, officials from the COPS Office
and OVW stated that they each anticipate twice-a-year updates to the
data in the staffing models: (1) when the President’s Budget is prepared,
and (2) when each component receives its actual appropriation. Officials
from OJP stated that they also anticipate twice-a-year updates to the data
in the staffing model, but OJP officials are planning to update the OJP
model in January of each year and then again at the end of each fiscal
year. Nevertheless, during the course of our audit, the updates each
grant-making component had been making to the model have been
limited to ensuring the data are most current instead of comprehensively
analyzing the data and determining what the analysis means for
component operations. Prospectively, officials from all three grant-making
components stated that it is their goal to conduct a comprehensive
analysis of data in future updates that will help them improve budget
planning.
Grant-making component officials also explained that time will be
necessary for those working with the model to coordinate with those
developing the budget; however, they also have not established timelines
for such interaction. According to OJP officials, they expect that the
model’s 2012 data will inform the fiscal year 2015 budget request and
acknowledge that OJP’s human capital staff will need to share and
interpret the data in the models with OJP’s budget staff. To this end, OJP
officials reported in September 2012 that both human capital and budget
staff have recently held planning meetings but that more time is
necessary to finalize their plans. Officials from the COPS Office and OVW
stated that they have not yet set aside time to conduct similar planning
meetings. According to officials in the COPS office, they brought on a
new staff member in August 2012 to work on updates, and she will need
to rely on additional data from other COPS program offices to update the
Page 32 GAO-13-92 DOJ Workforce Planning
model with the most current data before informing the fiscal year 2015
budget request. OVW officials stated that staff constraints have limited
their ability to start planning meetings between human capital and budget
staff. The officials emphasized that OVW is a small office, and
specifically, within the budget and grants management units, staff
members have been focused on closing out grants for the 2012 fiscal
year rather than thinking ahead to the 2015 budget cycle. They hope to
start holding initial meetings by November or December 2012 but these
plans are tentative. While the offices have begun to discuss the
importance of timing, without establishing specific timeframes that take
into account the full incorporation of leading practices, as well as the
exchange of information between those running the models and those
developing the budgets, components risk missing an opportunity to
ensure that the models are sound and reliable and that what they are
learning is supporting what they are requesting in their budgets.
Coordination Staff members from the grant-making components stated they did not yet
know how they would use the results that the staffing models generated
to develop and support budget submissions. For example, at this time,
human capital and budget offices from all three grant-making components
have demonstrated limited efforts to coordinate their roles and identify
what information human capital staff will provide to budget staff and what
the budget office staff responsibilities will be to validate any of the
analysis the models’ users generate. Moreover, coordination between
human capital and budget office staff needs to take place not only to
inform the budget requests, but also to understand the staffing model
data and the analyses model users are conducting. OJP officials stated
that human capital and budget staff members have met to discuss
coordination efforts to prepare OJP’s budget request for fiscal year 2015
and that they continue to outline roles and responsibilities. Additionally,
OJP’s Acting Assistant Attorney General issued a memorandum to OJP
leadership in October 2012 that directed OJP offices to take the staffing
model into consideration when developing budget requests. The memo
did not provide specific instructions for how to coordinate, but did
emphasize the importance of integrating workforce, program, and budget
planning. Officials from the COPS Office agreed that coordination
between human capital and budget offices should occur. They reported
general discussions between the COPS Office Chief of Staff and the
COPS Office budget unit on how to integrate the model into budget
development, but acknowledged that they have not discussed specifics
yet and said they planned to begin formal coordination activities in the
spring of 2013, which is when DOJ components have typically begun
preparing for an upcoming budget cycle. OVW officials stated that initial
Page 33 GAO-13-92 DOJ Workforce Planning
coordination steps are necessary for budget staff to understand the
intricacies of the model in order to eventually decide how and what data
in the model will be most useful to justifying the budget. However, OVW
officials reported that limited staff and growing demands on their time
have limited their ability to begin coordination efforts between human
capital and budget staff.
As sound project management approaches suggest, involving planning
and operational personnel early is meant to avoid making decisions in
isolation that may waste resources and time. The future coordination
efforts between human capital and budget office staff should help
determine how the staffing models can best be used to justify any
requests for new funds. Mechanisms to facilitate coordination and
collaboration could enhance DOJ grant-making components’ budget
submissions by making them more consistent, accurate, and robust.
Further, because development of the fiscal year 2015 budget submission
will reflect the components’ first attempt to integrate analyses from the
staffing models, it is increasingly important that conversations between
human capital and budget staff begin early and be well-coordinated.
Guidance Staff members at each grant-making component have become more
familiar with the staffing models and their capabilities, but to date, there
has been no guidance about what analyses may be needed or useful
when developing budget requests. According to JMD officials, they do not
intend to dictate how and to what extent the components incorporate the
models into their budget submissions. JMD officials told us that during
their routine review of the budget justifications the grant-making
components submit, they may question the level of detail included or
omitted, but they do not envision their role as ensuring inclusion or even
securing consistency across the three grant-making components’
requests. From the perspective of JMD’s budget staff, JMD has never
inserted itself into DOJ components’ budget development processes and
has no directive from the Attorney General to do so. Further, JMD does
not plan to modify its approach given the presence of the staffing models.
While guidance may not be necessary from JMD, guidance from each
grant-making component outlining the process for developing budget
submissions through incorporation of the model’s data could help ensure
that on a routine basis, updates to the model incorporate leading
practices and that budget development discussions are based on sound
and reliable data and associated workforce and workload analyses.
Developing, documenting, and implementing a strategy with timelines,
coordination mechanisms between budget and human capital staff, and
Page 34 GAO-13-92 DOJ Workforce Planning
guidance on how to incorporate the models’ into budget development
could better position grant-making components to use the quantifiable
data the models provide to support the funds they seek in their budget
requests. Having such a strategy could also provide managers at the
grant-making components and other decision makers insights into the
grant-making components’ overall progress in using the models and
provide a basis for determining what, if any, additional resources are
needed to best manage their operations.
During fiscal year 2011, the workforce at OJP, the COPS Office, and
Conclusions OVW collectively administered nearly 27,000 active grants involving $3.5
billion in appropriated funds. Given the volume and overall size of these
grants, it is critical that OJP, the COPS Office, and OVW use their
resources in the most efficient and effective way possible. Since 2010,
DOJ has embarked on an effort to quantify the grant-making components’
workloads, assess their workforces, and identify and address any gaps by
funding the development of staffing models. As a result, DOJ’s grant-
making components now have increased capacity to engage in
systematic and strategic workforce planning.
The grant-making components have partially met leading practices to
ensure the models are sound and reliable, but can enhance their efforts
to more fully meet leading practices. For example, not fully checking for
data accuracy upon the model’s receipt and attempting to correct for
errors after the contractor’s term had ended challenge the components’
abilities to put the models to their intended use—to determine workforce
gaps, such as critical skills and competencies of staff, and to develop
long-term human capital plans that address these gaps. Fully meeting
leading practices would provide DOJ reasonable assurance that, among
other things, the models’ data are accurate, the results are valid, and the
staffing models provide a sound and reliable basis for components to
update their models, analyze the data in the future, and inform workforce
and budget planning. Further, concentrating for so many months on
updating the models’ baseline data has left the components without a
strategy for analyzing the models’ results and applying what they learn to
the development of their budget requests. By developing, documenting,
and implementing a strategy that includes timelines, coordination
principles, and guidance for incorporating the staffing models’ analyses
into their budget development processes, the components may be better
positioned to leverage the quantifiable workforce and workload data the
models generate, inform the Congress of their needs, and more fully
justify any requests for additional funds in the future.
Page 35 GAO-13-92 DOJ Workforce Planning
We recommend that the Attorney General direct the leadership of OJP,
Recommendations for the COPS Office, and OVW take the following two actions.
Executive Action
• fully incorporate leading practices to ensure that their respective
staffing models are sound and reliable, by:
• implementing a more rigorous data review process with clear
methods for confirming the credibility of data and verifying the
results the staffing models generate;
• reviewing changes to the data in the models to ensure accuracy
and documenting and maintaining records of changes to protect
model integrity;
• defining and determining the respective roles and responsibilities
of staff involved in reviewing, operating, and assisting with the
staffing models; and
• identifying training needs that address staff needs to fulfill
responsibilities associated with the staffing models.
• develop, document, and implement a strategy that includes timelines,
coordination efforts, and guidance for building budget submissions
that takes into account the full incorporation of leading practices to
ensure the staffing models are sound and reliable.
We provided a draft of this report to DOJ for comment and received
Agency Comments written comments on the draft report, which are reproduced in full in
appendix II. DOJ concurred with both recommendations and described
actions it has underway or plans to take to address the recommendations.
DOJ agreed with the first recommendation that the leadership of OJP, the
COPS Office and OVW fully incorporate leading practices to ensure that
their respective staffing models are sound and reliable, noting that it
appreciated our evaluation of the staffing models. DOJ stated that with
available resources, the grant-making components plan to continue to
integrate workforce planning into their operational, programmatic, and
budget planning processes, and expressed their commitment to a long-
term, systematic, and data-driven approach to doing so. Additionally, DOJ
reported that the grant-making components are currently benchmarking
against federal agencies and the private sector to ensure that their
Page 36 GAO-13-92 DOJ Workforce Planning
workforce and talent management efforts are developed and/or enhanced
based on best practices and industry standards.
DOJ agreed with the second recommendation to develop, document, and
implement a strategy that includes timelines, coordination efforts, and
guidance for building budget submissions that takes into account the full
incorporation of leading practices to ensure the staffing models are sound
and reliable. DOJ noted that staff in the human capital, program, and
budget offices of the grant-making components will continue to
collaborate on a strategic approach to the development of future budget
submissions and work to establish policies and procedures to govern this
coordinated effort.
We are sending copies of this report to the Attorney General, selected
congressional committees, and other interested parties. In addition, the
report is available at no charge on the GAO website at
http://www.gao.gov.
If you or your staff have any questions about this report, please contact
me at (202) 512-9627 or maurerd@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. Key contributors to this report are listed in
appendix III.
David C. Maurer
Director
Homeland Security and Justice Issues
Page 37 GAO-13-92 DOJ Workforce Planning
Appendix I: Objectives, Scope, and
Appendix I: Objectives, Scope, and
Methodology
Methodology
This report addresses the following questions: (1) to what extent have the
grant-making components met leading practices to ensure that their
staffing models are sound and reliable?, and (2) how have the
components used the models to inform their workforce planning efforts
and to what extent have the components begun to use the models for
budget development?
To determine the extent to which the three grant-making components
incorporated leading practices to ensure that their staffing models are
sound and reliable, we developed a detailed framework to define six
leading practices and 15 specific activities, or criteria, they encompass.
The framework covers model design, model development, and model
deployment—and the six leading practices reflect the specific activities in
which organizations should engage to ensure the models’ soundness
(i.e., their validity and reliability). These practices are: (1) accounting for
risk associated with contracting out model development, (2) ensuring the
credibility of data used in the models, (3) preserving the integrity of data
maintained in the models, (4) establishing roles and responsibilities for
staff assigned to update and operate the models, (5) ensuring adequate
training for staff assigned to update and operate the models, and (6)
leveraging stakeholders’ knowledge and experience in designing and
developing the models.
We developed this framework primarily using GAO’s Internal Control
Management and Evaluation Tool. 1 The leading practices are also
supported by GAO’s Designing Evaluations: 2012 Revision; Federal
Information System Controls Audit Manual; and Assessing the Reliability
of Computer-Processed Data. 2 Before applying the framework, and to
understand how the staffing models are structured, we participated in a
demonstration of the models in which each grant-making component
presented its respective version of the model. During this demonstration,
officials responsible for operating the model at their respective
components walked us through the various model functions, identified
their underlying data sources, and explained the model outputs they
produced. We also independently reviewed the models and the
associated deliverables the contractor provided. Finally, we interviewed
grant-making component officials assigned to operate the model to
1
GAO-01-1008G.
2
GAO-12-208G; GAO-09-232G; GAO-09-680G.
Page 38 GAO-13-92 DOJ Workforce Planning
Appendix I: Objectives, Scope, and
Methodology
confirm our understanding of the models’ structure. Then, to apply the
framework, we assessed the steps the grant-making components told us
they took to ensure the models were sound and reliable and
benchmarked those steps against the leading practices and criteria we
identified. A designation of “fully met” indicates that a component
provided evidence that satisfied the entire criteria for the respective
leading practice; “partially met” indicates that a component provided
evidence that satisfied some portion of the criteria; and “did not meet”
indicates that a component provided no evidence that satisfied any
portion of the criteria.
For example, to understand how the Department’s Justice Management
Division (JMD) and its three grant-making components worked with the
contractor to design and develop the model, we interviewed the lead
Contracting Officer Representative (COR)—a management analyst within
JMD—as well as senior officials from the three grant-making components
responsible for grant administration, human capital management, and
budget planning. Additionally, we interviewed these officials to understand
how the grant-making components have used the models for workforce
planning and budget development efforts since receiving the models from
the contractor. We also interviewed the staff in each grant-making
component tasked with updating and executing the model to get their
perspectives on the training they received, their prior workforce planning
backgrounds, and the time they spent working with the model. In addition,
we analyzed documentation about the grant-making components’
practices related to updating the models, as well as the contractor’s
guidelines for doing so. We then compared the grant-making
components’ practices for designing, developing, and deploying the
models for eventual use to the leading practices we identified and then
assessed whether the grant-making components’ actions fully met,
partially met, or did not meet the criteria. We also assessed each grant-
making component’s model independently, using a discovery method
approach, wherein we examined a small number of formulas and
calculations. While we were not attempting to develop a specific error rate
per se, this discovery method approach permitted us to make conclusions
about the risk for (a) additional errors in other parts of the model we did
not examine, and (b) potential errors in the model’s outputs.
To describe how the three grant-making components have used the
models to inform their workforce planning efforts and to determine to what
extent to which they have begun to use the models for budget
development, we interviewed officials from the three grant-making
components and JMD officials on their workforce planning efforts prior to
Page 39 GAO-13-92 DOJ Workforce Planning
Appendix I: Objectives, Scope, and
Methodology
having the staffing models. In particular, we interviewed staff members
from each of the three grant-making components who were working in a
human capital capacity and had direct oversight or responsibility for the
staffing models. In these interviews, we discussed the extent, if any, to
which each grant-making component was using the model to inform its
activities, future plans to integrate the model into its budget development
process, and the role JMD plays in facilitating these efforts. In addition,
we reviewed related documents including each grant-making
component’s initial staffing models provided by the contractor in May
2011 and the updated models completed by each grant-making
component as of September 2012. In reviewing the three grant-making
components’ plans to integrate the models into their budget development
process, we relied on standard practices for project management, The
Standard for Project Management, developed by the Project Management
Institute. 3 Through interviews with component officials at the three grant-
making components and review of documentation including meeting
notes, we reviewed the grant-making components’ use of the standard
practices for project management including the development of timelines
with milestone dates, coordination efforts between human capital and
budget staff at each grant-making component, and guidance regarding
the direction and use of the models.
3
Project Management Institute, The Standard for Program Management® (Newtown
Square, PA: 2006). The Project Management Institute is an organization that provides
guidelines, rules, and characteristics for project management.
Page 40 GAO-13-92 DOJ Workforce Planning
Appendix II: Comments from the Department
Appendix II: Comments from the Department
of Justice
of Justice
Page 41 GAO-13-92 DOJ Workforce Planning
Appendix II: Comments from the Department
of Justice
Page 42 GAO-13-92 DOJ Workforce Planning
Appendix II: Comments from the Department
of Justice
Page 43 GAO-13-92 DOJ Workforce Planning
Appendix II: Comments from the Department
of Justice
Page 44 GAO-13-92 DOJ Workforce Planning
Appendix III: GAO Contact and Staff
Appendix III: GAO Contact and Staff
Acknowledgments
Acknowledgments
David C. Maurer, (202) 512-9627 or maurerd@gao.gov
GAO Contact
In addition to the contact named above, key contributors to this report
Staff were Joy Booth (Assistant Director), Lydia Araya, Sylvia Bascopé,
Acknowledgments Christine Hanson, William Mowbray, Daniel Rodriguez, Rebecca Shea,
and Janet Temko.
(441079)
Page 45 GAO-13-92 DOJ Workforce Planning
The Government Accountability Office, the audit, evaluation, and
GAO’s Mission investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding decisions.
GAO’s commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and afternoon, GAO posts on its website newly released reports, testimony,
and correspondence. To have GAO e-mail you a list of newly posted
Testimony products, go to http://www.gao.gov and select “E-mail Updates.”
Order by Phone The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s website,
http://www.gao.gov/ordering.htm.
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional information.
Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
Visit GAO on the web at www.gao.gov.
Contact:
To Report Fraud,
Website: http://www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: fraudnet@gao.gov
Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470
Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional 4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations 7125, Washington, DC 20548
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548
Please Print on Recycled Paper.