TREASURY
INSPECTOR GENERAL FOR TAX ADMINISTRATION

The Data Center Consolidation
Initiative Has Made Significant Progress, but Program Management Should Be
Improved to Ensure That Goals Are Achieved

June 10, 2013

Reference Number:† 2013-20-013

This report has cleared the Treasury Inspector
General for Tax Administration disclosure review process and information
determined to be restricted from public release has been redacted from this
document.

THE DATA CENTER CONSOLIDATION INITIATIVE
HAS MADE SIGNIFICANT PROGRESS, BUT PROGRAM MANAGEMENT SHOULD BE IMPROVED TO
ENSURE THAT GOALS ARE ACHIEVED

Highlights

Final
Report issued on June 10, 2013

Highlights of Reference Number:† 2013-20-013 to the Internal Revenue Service Chief
Technology Officer.

IMPACT ON TAXPAYERS

In
February 2010, the Office of Management and Budget established the Federal Data
Center Consolidation Initiative mandating that Federal agencies reduce costs
and save energy by reducing the number of Federal data centers.† Decreasing
redundant or unnecessary data centers, as well as reducing operational costs
and energy consumption, is a significant part of Federal efforts to reduce
costs and make more efficient use of taxpayer funds.

WHY TIGTA DID THE AUDIT

In
response to the Office of Management and Budget mandate, the IRS established a
Data Center Consolidation Initiative (DCCI) Program Management Office (PMO) in
2010 and set a goal to reduce its data center space by 50 percent by the end of
Fiscal Year 2015.† TIGTA initiated this audit to determine whether the IRSís DCCI
would effectively reduce data center space and increase energy efficiency.

WHAT TIGTA FOUND

While the
IRS has exceeded its first two yearsí goals for reducing data center space and
improving the energy efficiency of its data centers, management of the DCCI
needs to be improved to ensure that the IRS meets its future DCCI goals.† Two
years of the IRSís five-year DCCI have elapsed without a clear plan for how the
data center space reduction goals will be accomplished.† The IRSís DCCI PMO has
not formalized a project management plan that addresses challenges and details
decisions, milestones, and time frames related to how the IRS will meet its
data center consolidation goals.† In addition, the IRS has not identified the optimal
footprint for its data centers.† TIGTA observed significant empty space at the
IRSís Enterprise Computing Center in Detroit, Michigan.† TIGTA was informed by
IRS management that the IRSís lease for this space will expire in April 2015
and that the systems at this location could be moved to other IRS data
centers.† During our audit, the IRS announced plans to close the Enterprise
Computing Center in Detroit, estimating the IRS will save approximately $15
million annually.

Further,
the IRSís DCCI does not include all of its data centers that meet the standard
Federal definition of a data center within its baseline inventory.† The IRS has
an additional 115,343 square feet of data center space in 61 buildings.

WHAT TIGTA RECOMMENDED

TIGTA
recommended that the Chief Technology Officer provide, as appropriate, additional
resources to the DCCI PMO, require the PMO to develop a project management plan
that addresses challenges to accomplishing project goals and solutions, and
ensure that the Enterprise Computing Center in Detroit is consolidated into the
Martinsburg and Memphis Enterprise Computing Centers.

TIGTA
also recommended that the Chief Technology Officer work with the Real Estate
and Facilities Management organization to correct the inventory, update the
inventory submitted to the Department of the Treasury, formalize plans to develop
and apply a baseline footprint to the IRSís small data centers, and coordinate
with the Chief, Agency-Wide Shared Services, to shut down computer room air conditioners
no longer needed and adjust the set point temperature, as appropriate, on the
remaining computer room air conditioners in accordance with industry best
practices.

In their response to the
report, IRS management agreed with seven of the report recommendations and
partially agreed with one recommendation. †The IRS has taken or plans to take
appropriate corrective actions in response to all of the report recommendations.

††††††††††††††††††††††††††††††††† ActingDeputy Inspector General for
Audit

SUBJECT:††††††††††††††††† Final
Audit Report Ė The Data Center Consolidation Initiative Has Made Significant
Progress, but Program Management Should Be Improved to Ensure That Goals Are
Achieved (Audit # 201220021)

This report presents the results of our review of the
Internal Revenue Serviceís (IRS) Data Center Consolidation Project.† The
overall objective of this review was to determine whether the IRSís Data Center
Consolidation Initiative would effectively reduce data center space and
increase energy efficiency.† This review is included in our Fiscal Year 2013
Annual Audit Plan and addresses the major management challenge of Achieving
Program Efficiencies and Cost Savings.

Managementís complete response to the draft report is
included as Appendix VI.

Copies of
this report are also being sent to the IRS managers affected by the report
recommendations.† If you have any questions, please contact me or Alan R.
Duncan, Assistant Inspector General for Audit (Security and Information
Technology Services).

In February 2010, the Office of Management and Budget (OMB)
established the Federal Data Center Consolidation Initiative (FDCCI) as a
Governmentwide initiative designed to reduce the energy and real estate
footprint of Federal data centers while increasing efficiency, strengthening
the overall Government security posture, and promoting Green Information
Technology[1]by reducing the total number
of Federal data centers.† The FDCCI guidance required agencies to inventory
their data center assets, develop consolidation plans, and integrate those
plans into agency Fiscal Year (FY) 2012 budget submissions.† The FDCCI would
reduce the number of data centers across the Government and assist agencies in
applying best practices from the public and private sector, with goals to:

∑Promote the use of Green Information Technology by reducing the
overall energy and real estate footprint of Government data centers.

∑Reduce the cost of data center hardware, software, and
operations.

∑Increase the overall information technology security posture of
the Government.

The OMB provided updated guidance in July 2011.† This
guidance required agencies and all subordinate organizations to complete all
missing elements of their respective consolidation plans and submit them to the
General Services Administration.† The plans were required to include a full
master plan schedule which identifies, by quarter, the data centers to be
closed through FY 2015.† The plans were also to reflect challenges experienced
to date and integrate lessons learned.

In response to the OMB directive to consolidate Federal data
centers, the Department of the Treasury and its bureaus, including the Internal
Revenue Service (IRS), developed data center consolidation plans.† In October
2010, the IRS created the Data Center Consolidation Initiative (DCCI) Program
Management Office (PMO).† The IRSís DCCI goal is to reduce its data center
space by 50 percent, 10 percent per year, from FY 2011 through FY 2015.† The
IRS also has a goal to reduce the number of data centers from 15 primary data
centers to eight by the end of FY 2015.† The IRSís DCCI plan states that the
approach is to centralize, virtualize, and decommission server resources to
meet the following goals:† consolidate the governance of all servers, reduce
the IRSís data center footprint, enhance the IRSís disaster recovery
capabilities, and streamline the IRSís processes to reach a more mature level
of the Information Technology Infrastructure Libraryģ.

There are currently three bills in Congress with potential
legislation that may affect the FDCCI.† Two of the three bills would
specifically require agencies to submit complete data center consolidation
plans to the OMB.[2]

This review was performed at the IRS Enterprise Computing
Centers (ECC) in Detroit, Michigan; Memphis, Tennessee; and Martinsburg, West
Virginia, and the IRS data centers in Atlanta, Georgia; St. Louis, Missouri;
New York, New York; Oklahoma City, Oklahoma; and Dallas, Texas, during the
period June through November 2012.† We conducted this performance audit in
accordance with generally accepted government auditing standards.† Those
standards require that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and
conclusions based on our audit objective.† We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions based on our
audit objective.† Detailed information on our audit objective, scope, and
methodology is presented in Appendix I.† Major contributors to the report are
listed in Appendix II.

The IRSís Chief Technology Officer (CTO) set a DCCI goal for
the IRS to reduce its baseline data center space (reported at approximately
392,000 square feet) by 50 percent (or approximately 196,000 square feet) by
the end of FY 2015.† The PMO plans to meet the goal by reducing the footprint
by 10 percent, or approximately 39,000 square feet, annually during FYs 2011
through 2015.† The OMB did not set data center space savings goals or
requirements for Federal agencies.† Similarly, the Department of the Treasury
did not set specific DCCI goals for its bureaus.

Project management best practices have established that a
project management plan is a key project planning document.† The project
management plan documents planning assumptions and decisions for the project
and provides a framework for managing project activities and for completing the
project successfully.† The objective of a project management plan is to define
the scope of work to be accomplished and the approach to be used by the project
team to deliver the solution.† In September 2010, the IRS submitted its DCCI
plan to the Department of the Treasury.† The plan stated that a project
management discipline will be applied to the effort, including development of a
comprehensive master program schedule, detailed implementation schedules, and
spending plans.

While the IRS has exceeded its goals in the first two years
for reducing data center space and improving the energy efficiency of its data
centers, management of the project needs to be improved to ensure that the IRS
meets its DCCI goals by the end of FY 2015.† Two years of the IRSís five-year
DCCI have elapsed without a clear plan for how the data center space reduction
goals will be accomplished.† In addition, challenges remain that, if not
addressed in a formal plan, could result in the IRS not meeting its future DCCI
goals.

Significant progress has been made in the first two
years of the DCCI

The IRSís DCCI PMO, in partnership with the IRSís Real
Estate and Facilities Management organization, has exceeded its first two
yearsí goals in reducing the IRSís data center footprint and improving data
center energy efficiency.† As a result of consolidation efforts, by the end of
FY 2012 (the second year of the five-year project) the IRS had closed two data
centers and successfully reduced 76,560 square feet of the baseline data center
square footage at five other data centers.† In doing so, the IRS has exceeded
its plan of reducing 10 percent per year of the baseline square footage for the
first two years of the five-year project period.† The IRS has worked with the
Department of the Treasury and made progress in relocating Department of the
Treasury systems into the IRSís data center space.

The IRS has also taken steps to improve the energy
efficiency of its data centers.† These steps, taken in response to a prior
Treasury Inspector General for Tax Administration (TIGTA) audit report,[3]include implementing ďhotĒ
and ďcoldĒ aisles for servers, installing motion sensors to control overhead
lighting, and shutting down redundant computer room air conditioners.† The IRS
is in the process of installing power metering in two data centers located in
Kansas City, Missouri, and Memphis, Tennessee.† This will enable the IRS to
determine the exact amount of electricity used in these data centers.†
Currently, the IRS is only able to estimate the power usage of its data
centers.† Following implementation of metering at these locations, the IRS will
assess the costs, benefits, and efforts required to install metering and
determine whether metering will be recommended for other IRS data centers.

A project management plan has not been
developed

The DCCI PMO prepared an operating charter that established
the PMO and set forth its objectives, governance responsibilities, membership
composition, and high-level operating framework.† While the DCCI PMO was formed
in October 2010, the operating charter was not developed until July 2012,
nearly two years after the start of the DCCI PMO

A project management plan is needed to address challenges to
accomplishing DCCI goals.† The IRSís DCCI PMO has not formalized a project
management plan that addresses challenges and details decisions, milestones,
and time frames related to how the IRS will meet its goal of reducing data
center space by approximately 196,000 square feet by the end of FY 2015.† The
IRS uses quarterly CTO briefing document presentations for managing the DCCI.†
Specific plans are developed for each local site as the IRS begins to address
specific campuses, computing centers, or other site locations.

In addition, while the IRS can perform DCCI planning and
analysis work during the filing season, it cannot make any major information
technology production environment changes or relocate hardware.[4]†This further shortens the
time frame and provides additional challenges which the IRS has to address in
the next three fiscal years to accomplish its DCCI goals.

The IRS lacks a strategy
for the ECCs

In a prior report, we
recommended that current and future IRS data center space needs be identified
and plans developed to consolidate or reduce excess data center space.† IRS
management agreed with this recommendation and stated that the Enterprise
Operations organization would take the lead in the process of assessing the
current state of data center space and developing a plan of action for space
consolidation, where appropriate.

Two years after the start of the DCCI, the IRS has not
identified the optimal footprint or configuration for its ECCs.† In August
2012, nearly two years after the start of the DCCI, the IRS prepared a
statement of work to engage a contractor to study the three ECCs and make
recommendations for how the IRS should create world-class data center
facilities to meet its current and future information technology needs.

We observed significant empty data center space at the
ECC-Detroit.† We were informed by IRS management that the IRSís lease for this
space will expire in April 2015 and that the systems that remain at this
location could be moved to other IRS data centers.† Based on this, it was our
opinion that the IRS should close the data center at the ECC-Detroit.† In July
2012, during our audit, the CTO announced the closure of the ECC-Detroit.† The
basis for this decision has not been documented as part of a project management
plan.† In September 2011, a draft business case was developed for the
ECC-Detroit that assumed the data center would remain open.† The business case
approach was then changed and replaced with a document, ďStrategic Overview of
the ECCs and Their Role in Supporting the IRS Data Center Consolidation
Initiative,Ē dated November 2011.† The strategic overview document also
includes future plans for the ECC-Detroit.† The document states three computing
centers are necessary to meet the IRSís information technology needs.† In 2012,
the DCCI PMO began to prepare a statement of work for a contractor to study the
ECCs and make recommendations on how the IRS can implement a world-class data
center organization.† In August 2012, the draft statement of work was revised
to reflect the IRSís plans to close the ECC-Detroit.† The CTO estimates the IRS
will save approximately $15 million annually by closing the ECC-Detroit.† We agree
with this decision as it is consistent with a prior TIGTA recommendation, and
we have also raised this issue during this audit. †The IRS expects to receive
the results of the contractorís world-class data center study in August 2013.

Closing the ECC-Detroit presents another issue not addressed
in a project management plan.† The IRSís telecommunications gateway
infrastructure, BlackBerry enterprise servers, and e-mail archiving are
configured to be located in three sites.† The systems are currently located at
the three ECC locations.† With the closure of the ECC-Detroit, the IRS needs to
identify and prepare a third location to support these systems.

A decision has not been made on how the Annex building at
the ECC-Martinsburg, a large space of approximately 36,000 square feet, will
contribute to the 195,000 square foot reduction in data center space.† This
building contains the command center for the ECC-Martinsburg, the Department of
the Treasuryís continuity of operations equipment, and the IRSís Research,
Analysis, and Statistics organization equipment.† The DCCI managers informed us
that there may be adequate unused space within either of the two
ECC-Martinsburg computer rooms to accommodate these items and there may be a
need for additional office space at the ECC‑Martinsburg.† To complicate
this decision, the Annex building is a privately owned building leased by the
IRS, whereas the ECC-Martinsburg facility is a Government-owned building.

The DCCI PMOís decision to reduce space at the
ECC-Martinsburg and the ECC-Memphis facilities was also not documented in a
project management plan detailing how the space reduction will be accomplished.†
The IRS briefing documents currently show that data center space at the two
computing centers will be reduced by 50,000 square feet.† In addition, the
decision has been made without knowing the results of the data center study, which
has not yet been completed.† One of the key focuses of the study is to project
the future state of the IRSís data centers; it would be beneficial to await the
results of the study prior to making a decision to reduce space at the
remaining computing centers.† The DCCI PMO has indicated that the results of
the data center study will improve decisions at the Memphis and Martinsburg ECCs
regarding potential space reductions, space to be made available for other Department
of the Treasury bureaus, or conversion to office space.

The DCCI PMO program manager plans to turn over
responsibility for implementing the World Class Data Center Study
recommendations to the ECC executives.† The DCCI PMO responsibilities related
to the data center study should be included in a project management plan and
should require a continued role to ensure that the recommendations are
implemented.† Without such a role, the ECC improvements may not be realized.

At the beginning of the project, the IRSís Information
Technology (IT) organization management did not provide sufficient resources to
the DCCI PMO for project planning and to ensure that the team could efficiently
identify the best way to accomplish the DCCI goals.† The program manager
estimated that he is only able to devote 25 percent of his time to the DCCI.†
In March 2012, 18 months after the start of the DCCI, the DCCI PMO did not have
a full-time senior project manager.† The DCCI PMO informed the CTO that due to
all the interconnected activities affecting consolidation, virtualization,
post-of-duty consolidation, and coordination with the IT organization and local
Real Estate and Facility Management organization staffs as well as the business
units, an additional full-time employee with project management expertise was
needed.† During our fieldwork, the DCCI PMO received a full-time senior project
manager and an additional staff member to assist the senior project manager.

Significant challenges remain, including a strategy for the
ECCs that have not been sufficiently addressed in a project management plan.†
If not addressed, the IRS may not fully achieve its DCCI goals by the end of FY
2015 as planned to reduce the real estate footprint and energy consumption of
its data centers.

Recommendations

To ensure that the IRS accomplishes
its DCCI goals in the final three years, the CTO should:

Recommendation 1:
†Provide, as appropriate, additional resources to support the DCCI PMO to
ensure that the future project goals are accomplished and ensure that the DCCI
PMO can maintain a role in guiding the implementation of the World Class Data
Center Study recommendations.

Managementís
Response:† The IRS agreed with this recommendation.† In November
2012, the Associate Chief Information Officer (CIO), Enterprise Operations,
added appropriate resources to the DCCI.† The IRS took this action to ensure
that project goals would be accomplished and to ensure that the DCCI PMO
maintained a role in guiding the implementation of the World Class Data Center
Study recommendations in the computing centers.

Recommendation 2:
†Require the DCCI PMO to develop a project management plan following project management
best practices.† The plan should include the development of a comprehensive
master program schedule, detailed implementation schedules, and spending plans
and should address challenges to accomplishing project goals and solutions.

Managementís
Response:† The IRS agreed with this recommendation.† As a result of
discussions during this audit, the IRS DCCI PMO consolidated the existing
strategy and planning documents and began developing a master project plan that
will address challenges, detail decisions, milestones, and time frames related
to how the IRS will meet its remaining data center consolidation goals.

Recommendation 3:† Ensure that the ECC-Detroit
is consolidated into the Martinsburg and Memphis ECCs.

††††††††††† Managementís Response:†
The IRS partially agreed with this recommendation.† The IRS responded that
citing specific locations in the recommendation limits the IRSís ability to
make the best decision as to where to locate the ECC-Detroit computing assets. †IRS
management stated they will ensure that the ECC-Detroit computing assets are
consolidated into an appropriate facility.

At the start of the FDCCI in February 2010, the agenciesí
data center inventories included a diverse collection of facilities.† As a
result, in October 2010, the Federal CIO announced a common baseline definition
for a Federal Government data center, developed by the FDCCI Task Force.† The
Task Force defined a data center as any room greater than 500 square feet in
area, devoted to data processing, and meeting a tier (I, II, III, or IV)
classification defined by the Uptime Institute.[5]

After more than a year of work under the FDCCI, the Task
Force updated its standard data center definition.† In March 2012, the Federal
CIO issued a memorandum formalizing an updated definition of a data center.† The
memorandum states, ďÖa data center is now defined as a closet, room, floor or
building for the storage, management, and dissemination of data and informationĒ
and ďUnder this revised definition, neither square footage nor the Uptime
Institute tier classifications are required to define a facility as a data
center.Ē† The memorandum also states, ďThe Task Force has made it clear that
facilities below 500 square feet in area that do not meet an Uptime Institute
classification also consume significant amounts of resources.Ē† The Federal CIO
explained that expanding the scope of the FDCCI effort to include even the
smallest spaces will better assist agencies in locating inefficient information
technology assets and consolidation opportunities.† Consolidation encompasses
not just large facilities but also small server clusters and wiring closets.

The IRSís detailed DCCI plan, submitted to the Department of
the Treasury in September 2010 and updated in June 2011, states that the IRSís
five-year plan for data center consolidation will focus on locations hosting
more than six computing assets, or roughly 124 IRS locations.† The plan also
states that the IRS is committed to policies and practices that reduce the
environmental impact of IRS operations.† The IRS has committed to such best
practices as implementing computer room air-conditioner temperature and
humidity set points that are consistent with industry best practices, and
shutting down unnecessary computer room air conditioners, as appropriate.

The IRSís DCCI does not include all data centers that meet
the standard Federal definition of a data center within its baseline
inventory.† The IRSís five-year DCCI is focused on achieving data center space reduction
goals at 17 locations, rather than 124 locations, as stated in the IRSís
detailed DCCI plan.† In its DCCI plan, submitted to and approved by the
Department of the Treasury in September 2010, the IRS identified 15 data
centers as primary data center locations in the data center baseline inventory
(three ECCs, one Annex building at the ECC-Martinsburg, nine campuses, and two
Washington, D.C., area data centers).† In June 2012, the IRS submitted to the
Department of the Treasury an updated baseline data center inventory of 17 data
centers (including the 15 data centers reported in September 2010 plus one
Criminal Investigation data center and one additional Washington, D.C., area
data center).† Given limited resources and time to accomplish data center
consolidation goals, the IRSís CTO decided to limit the IRSís focus to the 17
selected data centers.

The IRSís annual inventory updates submitted to the
Department of the Treasury in FYs 2011 and 2012 did not include the data
centers meeting the standard larger than 500-square-feet definition of 2011 or
the expanded 2012 definition that eliminated the square footage requirement.†
In the February 2010 memorandum launching the start of the FDCCI, the OMB
required agencies to update their asset inventories annually by the end of the
third quarter of each fiscal year.† The March 2012 memorandum reminded agencies
of this requirement.

In addition to the 17 data centers included in the DCCI, the
IRS has 115,343 square feet of data center space in 61 buildings in 31 states,
Washington, D.C., and Puerto Rico.† These additional data centers meet the
OMBís initial definition of any room greater than 500 square feet in area.† The
IRS also has an additional 25,244 square feet of space in 122 rooms that meet
the OMBís expanded definition that eliminates area as a requirement.

We visited eight data centers greater than 500 square feet
and not included in the IRSís DCCI, which were located in five cities.† The
data centers ranged from 1,437 square feet to 8,399 square feet and accounted
for 33,825 (29 percent) of the additional 115,343 square feet not included in
the DCCI.† Our review of the sites found that opportunities exist for space
reduction and energy savings at all sites visited.† Specifically, we found:

∑Computer room air conditioners were powered on and cooling rooms
with little or no computer equipment.

∑Set point temperatures on computer room air conditioners were not
consistently set to save energy and lower energy costs, as the IRS recently did
at its ECCs and campuses.† There were a total of 28 computer room air
conditioners at the eight data centers we visited.† Temperatures for 21 of the
28 units were set lower than 72 degrees, ranging from 68 to 71 degrees.† Only
seven units at one location were set at 72 degrees.†

∑The IRS data center inventory is not accurate.† For example, a
room at one location was not in the inventory, while the inventory at another
location included a room that no longer exists.† One large computer room was
listed as three separate smaller rooms in the inventory.

During our audit, the DCCI PMO informed us that it had been
asked by senior IT organization management to study such smaller data centers
to identify consolidation and energy savings opportunities and that they plan
to develop standard footprints for small data centers.

The IRSís DCCI PMO also informed us that data center
consolidation for the smaller data centers not included in the DCCI are being
addressed as building leases expire and also based on the Real Estate and
Facilities Management organization move or renovation projects currently taking
place.† Building leases for 38 (62 percent) of the 61 buildings that house
79,380 square feet of data center space will expire before the FY 2015 DCCI end
date.† Building leases for 23 (38 percent) of the 61 buildings housing 35,963
square feet will expire after the DCCI end date.

When the IRSís data center inventory does not include all of
its data centers, the Department of the Treasury cannot provide an accurate
data center inventory to the OMB.† In turn, the OMB then does not have an
accurate count of data centers within the scope of the FDCCI.† In addition, if
there are too many data centers (no matter their size) supporting the
organization, they lead to power inefficiencies and add unnecessary cost,
taking resources that could be used for other IT organization purposes.† The
IRS continues to incur energy costs at those locations where leases will be
renewed or where many years remain before lease expiration.† We could not
determine the additional cost the IRS is incurring to continue operating the
data centers not included in the IRSís DCCI.† The IRS does not have metering
technology in place to measure electric use at its data centers.† As a result,
the IRS could not provide data center energy usage and cost information.

Recommendations

To ensure the most energy and space-efficient data center
configurations, the CTO should:

Recommendation 4:
†Work with the Real Estate and Facilities Management organization to correct
the inventory when the DCCI PMO identifies discrepancies during local site
visits.

Managementís
Response: †The IRS agreed with this recommendation.† The Enterprise
Operations organization will work with the Real Estate and Facilities Management
organization within Agency-Wide Shared Services to develop a formal process to
correct the data center inventory.

Recommendation 5:
†Formalize plans to develop and apply a baseline footprint to the IRSís small
data centers.

Managementís
Response: †The IRS agreed with this recommendation.† The Enterprise
Operations organization, in coordination with the Real Estate and Facilities
Management organization, will develop a baseline footprint for the IRSís small
data centers.† This footprint will be applied to future build out and/or space
reduction projects in small data centers, where cost effective.

Recommendation 6:† Update
the annual inventory submitted to the Department of the Treasury to include all
data centers, regardless of size.

Managementís
Response: †The IRS agreed with this recommendation.† The IRS will
update the annual inventory submitted to the Department of the Treasury to
include all data centers.

To ensure the most efficient use of computer room air
conditioners at the IRSís data centers, the CTO, in conjunction with the Chief,
Agency-Wide Shared Services, should:

Recommendation 7:† Identify
and shut down computer room air conditioners no longer needed.

Managementís
Response: †The IRS agreed with this recommendation.† The Enterprise
Operations organization, in coordination with the Real Estate and Facilities Management
organization, will formalize a process for identifying and shutting down
computer room air conditioners that are no longer needed.

Recommendation 8:† Adjust the set point
temperatures as appropriate on the remaining computer room air conditioners in
accordance with industry best practices.

Managementís
Response: †The IRS agreed with this recommendation.† The Enterprise
Operations organization, in coordination with the Real Estate and Facilities
Management organization, will formalize a process for ensuring that computer
room air conditioners are adjusted to industry standard air conditioner set
point temperatures.

A.Evaluated the
governance and oversight of the IRSís DCCI to determine if executives are
sufficiently informed of the results.

B.Determined if
IRS data center consolidation plans are consistent with Department of the Treasury
and OMB guidelines as well as industry best practices.†

C.Determined if
goals have been established for the DCCI.

D.Determined if the
IRS has established metrics to monitor progress toward accomplishing DCCI
goals.

E.Determined
if the IRSís DCCI plans include all of its data centers, regardless of size,
and assessed any related consolidation plans.† We selected a judgmental[6]sample of eight of 107 data
centers, greater than 500 square feet, not included in the scope of the IRSís
DCCI.† The sample selection was based on data center square footage and type of
systems located at each data center.† We chose a judgmental sample to include
the largest of the data centers not included in the scope of the IRSís DCCI as
well as two data centers requested by DCCI PMO management.

F.Determined
if the IRS has considered multiple consolidation alternatives at the ECCs.

III.Determined if the IRSís DCCI plans include all costs, savings, funding,
and return on investment considerations to effectively measure and accomplish DCCI
goals.

A.Determined if
the IRS captures and tracks the costs of operating its data centers.

B.Determined if
the IRS can identify all costs associated with its DCCI plans and activities.

C.Determined if
the IRS has computed expected savings to be realized from its DCCI efforts.

D.Determined if
the IRS has calculated the return on investment for its DCCI.

E.Determined
if the DCCI plans address how the efforts will be funded.

IV.Determined if the IRSís DCCI plans are being effectively implemented to
reduce data center space and reduce energy costs.

A.Determined if
the IRS has been using metrics to demonstrate its progress towards meeting DCCI
goals.

B.Determined if
the IRS is using actual incurred cost data for tracking the effectiveness of
its DCCI efforts.

C.Identified DCCI
savings achieved.

D.Determined if
the IRS is meeting its DCCI goals to reduce energy costs and consumption.

Internal
controls methodology

Internal controls relate to managementís plans, methods, and
procedures used to meet their mission, goals, and objectives.† Internal
controls include the processes and procedures for planning, organizing,
directing, and controlling program operations.† They include the systems for
measuring, reporting, and monitoring program performance.† We determined the
following internal controls were relevant to our audit objective: †the
Enterprise Operations and Real Estate and Facilities Management organizationsí
plans, policies, and processes to manage, monitor, and report on the status and
progress of the IRSís DCCI.† We evaluated these controls by interviewing the
DCCI management, analyzing the DCCI documentation, and visiting selected data
centers.

This appendix presents detailed information on the
measurable impact that our recommended corrective actions will have on tax
administration.† This benefit will be incorporated into our Semiannual Report
to Congress

Type and Value of Outcome Measure:

∑Cost Savings Ė Funds Put to Better Use Ė Potential; $15 million
per year; $60 million over a four-year period (see page 3).

Methodology Used to Measure the Reported Benefit:

In a prior audit report[7]dated May 7, 2010, TIGTA
recommended that current and future IRS data center space needs be identified
and plans developed to consolidate or reduce excess data center space.† IRS
management agreed with this recommendation and stated that the Enterprise
Operations organization would take the lead in the process of assessing the
current state of data center space and developing a plan of action for space
consolidation, where appropriate.

In meetings with IRS managers in March 2012 during the
planning of this audit, we inquired about the status and plans for the
ECC-Detroit and were informed that no decision had been made.† During this
audit, we observed significant empty data center space at the ECC-Detroit.† We
were informed by IRS management that the IRSís lease for this space will expire
in April 2015 and that the systems that remain at this location could be moved
to other IRS data centers.† Therefore, it is our opinion that the IRS should
close the data center at the ECC-Detroit. †In July 2012, during this audit, the
IRSís CTO announced the closure of the ECC-Detroit. †Prior to the announcement,
all of the DCCI documentation indicated that the IRS would maintain a data
center at the ECC-Detroit.† We believe the decision to close the ECC-Detroit
was due, in part, to this audit as well as the TIGTAís prior report
recommendation.†

The IRSís CTO estimates that the IRS will save approximately
$15 million annually by closing the ECC-Detroit.† The $15 million estimated
annual savings for the closing of the ECC-Detroit was calculated in the
following manner.† Currently, the IRS occupies 484,000 square feet of space in
the Detroit facility, at a fully loaded rental rate of $53.23 per square foot.†
This equates to approximately $25 million in annual costs.† When the IRS ceases
data center operations in Detroit, the IRS will no longer require the facility
to have Level 5 security, but rather a standard level, and will only require
traditional office space.† As a result, the IRS will seek to obtain a rental
agreement that is more commensurate with the current Detroit market rate of
approximately $29 per square foot for (nonĖdata center) office space.† In
addition, after the data center function leaves the facility, the IRS can
eliminate the square footage currently needed for this function and can
right-size the office space as well; the estimated space required for the IRS
office function will be approximately 354,000 square feet.† If the IRS is able
to obtain a rental rate approximating the $29 per square foot market rate and
reduce the amount of space leased to approximately 354,000 square feet, the new
estimated annual costs would be approximately $10 million, representing an
estimated annual rent and operating cost savings of approximately $15 million
($25 million in current costs minus $10 million in new estimated costs). †While
this benefit will extend beyond one year for the IRS, we are conservatively
estimating the savings over four years.

The data processing arm of the IRS.† The campuses
process paper and electronic submissions, correct errors, and forward data to
the Computing Centers for analysis and posting to taxpayer accounts.

Consolidation

An approach to reducing data center space that
involves moving servers to a few selected data centers or moving small data
centers to larger centers.

Decommission

An approach to accomplishing data center consolidation
that involves turning off servers that are not being used or are used
infrequently.

Governance

A set of processes, guidelines, and policies that
guide and affect the direction of an organizationís behavior or assets.

Green Information Technology

Optimal use of information and communication
technology for managing the environmental sustainability of enterprise
operations, as well as that of their products, services, and resources,
throughout their life cycles.

Information Technology Infrastructure Libraryģ†

A set of practices for information technology service
management that focuses on aligning information technology services with the
needs of business.† The Information Technology Infrastructure Libraryģ
describes procedures, tasks, and checklists used by an organization for
establishing a minimum level of competency.† It allows the organization to
establish a baseline from which it can plan, implement, and measure.† It is
used to demonstrate compliance and to measure improvement.†

Virtualization

An approach that helps to accomplish data center
consolidation.† It involves moving applications and data on several physical
servers onto a single virtual server.

SUBJECT: ††††††††††††††††††††† Draft Audit Report -
The Data Center Consolidation Initiative Has Made Significant Progress, but
Program Management Should Be Improved to Ensure Goals Are Achieved (Audit#
201220021) (e-trak 2013-38864)

Thank you for the opportunity to review
and respond to the subject audit report.† We appreciate your flexibility in
working with the IRS team in revising the draft and noting the positive
accomplishments achieved by the Data Center Consolidation Initiative (DCCI).

The IRS is proud of the fact that in the
first two years of the project, we have exceeded our goals of reducing data
center space and are on track to meet our goals for the third year.† In 2011,
we released 13 percent of space; in 2012, we released 12 percent of space; and
this year, we are expecting to release another 13 percent of space.

We agree with seven of the recommendations
in your report, as well as with the potential savings outlined in the Outcome
Measure, and we partially agree with another recommendation.† The attachment
contains our planned corrective actions to address each finding.

We value your continued support and the
guidance your team provides.† If you have any questions, please contact me at
(202) 622-6800 or a member of your staff may contact Lisa Starr, Senior Manager
of Program Oversight, at (202) 283-3607.

Attachment

RECOMMENDATION #1:† To ensure
that the IRS accomplishes its DCCI goals in the final three years, the CTO
should provide, as appropriate, additional resources to support the DCCI PMO to
ensure that the future project goals are accomplished and ensure that the DCCI
PMO can maintain a role in guiding the implementation of the World-Class Data
Center Study recommendations.

CORRECTIVE ACTION #1:† We agree
with the recommendation.† In November 2012, the Associate Chief Information
Officer, Enterprise Operations added appropriate resources to the DCCI
project.† This action was taken to ensure that project goals would be
accomplished and to ensure that the DCCI PMO maintained a role in guiding the
implementation of the World Class Data Center Study recommendations in the
computing centers.

RECOMMENDATION #2:† To ensure
that the IRS accomplishes its DCCI goals in the final three years, the CTO
should require the DCCI PMO to develop a project management plan following
project management best practices.† The plan should include the development of
a comprehensive master program schedule, detailed implementation schedules, and
spending plans and should address challenges to accomplishing project goals and
solutions.

CORRECTIVE ACTION #2:† We agree
with the recommendation.† As a result of discussions during this audit, the IRS
DCCI PMO consolidated the existing strategy and planning documents and began
developing a master project management plan.† The plan will address challenges,
detail decisions, milestones and timeframes related to how the IRS will meet
its remaining data center consolidation goals.

CORRECTIVE ACTION MONITORING PLAN:† We enter
accepted Corrective Actions into the Joint Audit Management Enterprise System
(JAMES) and monitor them on a monthly basis until completion.

RECOMMENDATION #3:† To
ensure that the IRS accomplishes its DCCI goals in the final three years, the
CTO should ensure that the ECC-Detroit is consolidated into the Martinsburg and
Memphis ECCs.

CORRECTIVE ACTION #3:† We partially agree with the recommendation.† By
citing specific locations in the recommendation, the IRS's ability to make the
best decision as to where the ECC-Detroit computing assets should be located is
limited.† We will ensure the ECC-Detroit computing assets are consolidated into
an appropriate facility.

CORRECTIVE ACTION MONITORING PLAN:† We enter accepted Corrective Actions into the Joint
Audit Management Enterprise System (JAMES) and monitor them on a monthly basis
until completion.

RECOMMENDATION #4:† To ensure the most energy and space-efficient data
center configurations, the CTO should work with the Agency-Wide Shared
Services, Real Estate and Facilities Management (REFM) organization to correct
the inventory when the DCCI PMO identifies discrepancies during local site
visits.

CORRECTIVE ACTION #4:† We agree with the recommendation.† To ensure the
IRS accomplishes its DCCI goals in the final three years, Enterprise Operations
will work with the Real Estate and Facilities Management (REFM) organization
within Agency≠ Wide Shared Services (AWSS) to develop a formal process to
correct the data center inventory.

CORRECTIVE ACTION MONITORING PLAN:† We enter accepted Corrective Actions into the Joint
Audit Management Enterprise System (JAMES) and monitor them on a monthly basis
until completion.

RECOMMENDATION# 5:† To ensure the most energy and space-efficient data
center configurations, the CTO should formalize plans to develop and apply a
baseline footprint to the IRS's small data centers.

CORRECTIVE ACTION #5:† We agree with the recommendation.† Enterprise
Operations, in coordination with Agency-Wide Shared Services, Real Estate and
Facilities Management (REFM), will develop a baseline footprint for IRS's small
data centers.† This footprint will be applied to future build out and/or space
reduction projects in our small data centers, where cost effective.

CORRECTIVE ACTION MONITORING PLAN:† We enter
accepted Corrective Actions into the Joint Audit Management Enterprise System
(JAMES) and monitor them on a monthly basis until completion.

RECOMMENDATION #6:† To ensure
the most energy and space-efficient data center configurations, the CTO should
update the annual inventory submitted to the Department of the Treasury to
include all data centers, regardless of size.

CORRECTIVE ACTION #6:† We agree
with the recommendation.† The IRS will update the annual inventory submitted to
the Department of the Treasury to include all data centers.

CORRECTIVE ACTION MONITORING PLAN:† We enter
accepted Corrective Actions into the Joint Audit Management Enterprise System
(JAMES) and monitor them on a monthly basis until completion

RECOMMENDATION 7:† To
ensure the most efficient use of computer room air conditioners at the IRS's
data centers, the CTO, in coordination with the Chief, Agency≠Wide Shared
Services, should identify and shut down computer room air conditioners no
longer needed.

CORRECTIVE ACTION #7:† We
agree with the recommendation.† To ensure the most efficient use of computer
room air conditioners at the IRS's data centers, Enterprise Operations in
coordination with Agency-Wide Shared Services, Real Estate and Facilities Management
(REFM), will formalize a process for identifying and shutting down computer
room air conditioners that are no longer needed.

CORRECTIVE ACTION MONITORING PLAN:† We enter
accepted Corrective Actions into the Joint Audit Management Enterprise System
(JAMES) and monitor them on a monthly basis until completion.

RECOMMENDATION #8:† To ensure
the most efficient use of computer room air conditioners at the IRS's data
centers, the CTO, in coordination with the Chief, Agency≠Wide Shared Services,
should adjust the set point temperatures as appropriate on the remaining
computer room air conditioners in accordance with industry best practices.

CORRECTIVE ACTION #8:† We agree
with the recommendation.† To ensure the most efficient use of computer room air
conditioners at the IRS's data centers, Enterprise Operations, in coordination
with Agency-Wide Shared Services Real Estate and Facilities Management (REFM),
will formalize a process for ensuring computer room air conditioners are
adjusted to industry standard air conditioner set point temperatures.

[4]Every year the Production Environment Control Process
(formerly known as the Filing Season Moratorium) is put into place at the IRS
to stabilize its production environments during peak processing times.† During
this time, no changes to the production environment are allowed to be
implemented without executive approval.