Is the cloud overhyped? Predicted savings hard to verify

Oct. 8, 2012 - 07:01AM
|

Many IT managers favor cloud computing because it reduces their costs of ownership of servers and other IT infrastructure. But calculating those savings can be a challenge. (Library of Congress)

ADVERTISEMENT

One of the most attractive benefits of cloud computing can also be one of the hardest to prove: cost savings.

Agencies often lack the details needed to compare their pre-cloud costs —for hardware, software, laborand the like— with cloud-computing fees.

“It’s kind of a shaky premise to say we are moving to the cloud and going to save all this money, and they [organizations] don’t have a sense of what they’re spending today and how much they will save,” said Ed Anderson, a research director at Gartner whose focus is the cloud computing market. “I see a big danger in that because cost savings is the No. 1 justification for moving to cloud.”

The General Services Administration, Agriculture Department, U.S. Agency for International Development and others have projected millions of dollars in annual savings by moving to the cloud. While proponents and skeptics of cloud computing agree that savings are likely, how much is uncertain.

A recent inspector general audit looking into the cost savings of GSA’s June 2011 transition from a Lotus Notes email system to Google’s email system illustrates the challenge. The audit found that the agency could neither verify those savings nor clearly determine if the cloud migration is meeting agency expectations.

GSA has estimated its new cloud email system will save $15 million over five years, but the chief information officer has not maintained documentation on the costs of the old email system and the current Google email system, according to the Sept. 28 report. And GSA’s performance measures for the email migration are unclear, lack targets or were not updated.

GSA spokeswoman Mafara Hobson said GSA stands by its estimate that it will save at least $15 million over five years and said the agency already saved almost $3 million so far.

GSA did not have cost-savings documentation in a detailed format at the time the IG requested it, but the agency will submit a detailed cost-savings analysis to the IG within the next 30 days, Hobson said.

However, the IG report said the GSA has failed to meet its goal to reduce the number of people who maintain the new email system. GSA did not provide details on projected staff reductions.

The IG report noted that the Office of Management and Budget requires agencies to be explicit about the underlying assumptions they use to estimate benefits and costs of federal programs, including the rationale behind those assumptions and a review of their strengths and weaknesses.

“I never report savings to OMB or the IG that I can’t back up with data,” said Skip Kemerer, who heads the Nuclear Regulatory Commission’s capital planning and applications support branch.

NRC projected it would save $50,000 annually by moving to a cloud-based budget management and reporting tool provided by GSA. Licensing fees for the previous tool cost $250,000, and now NRC spends $200,000 a year.

“It’s a piece of data they can’t question,” Kemerer said.

But by using only verifiable figures to report cost savings, Kemerer is likely underestimating his savings. For example, there are savings on electricity — cooling costs in agency data centers that are eliminated or reduced — with moves to the cloud. But calculating those savings is no small task, Kemerer said.

Agencies have to decide how much time and money they are willing to invest in drilling down to savings that in some cases are hard to quantify, he said.

Calculating labor costs and estimating how gained productivity from a new system may translate into an agency needing fewer employee positions can be challenging.

“People aren’t sliceable,” he said.

Calculating labor and partial labor costs for any large technology project is difficult, and cloud is no different, said Tim Andrews, a vice president of technology at Booz Allen Hamilton. If agencies can’t move money or people across programs, it isn’t clear what money is being recouped.

Andrews said the company has an OMB-approved methodology for calculating benefits, such as increased employee collaboration on work projects, that may or may not translate into dollar savings.

“You’re not going to see agencies return that money into coffers,” but instead, they will use it to provide better citizen services and address other needs, said Andras Szakal, vice president and chief technology officer for IBM’s federal division.

Not knowing the amount of potential costs or savings, or misstating savings, could affect future funding decisions, GSA’s IG said.

USAID has projected $20 million in annual savings by moving to a cloud email system and reducing hardware, technical support, licenses and other operational costs. When asked about those savings, the agency directed questions to OMB.

Other agencies are holding off on announcing estimated savings.

The National Oceanic and Atmospheric Administration expects savings by consolidating dozens of help desks across the country into a cloud service. But it has not yet verified any projected savings.

“It’s difficult to capture detailed cost data at this point in the project because of the multiple service desk contracts that are currently in place across the organization,” Dan McCrae, director of NOAA’s Service Delivery Division, said in an email. “One of the objectives of this standardized service desk contract is to centralize cost and gain visibility into the total cost of ownership.”

Tracking the costs avoided by moving to Google’s cloud email system last year — as opposed to building and hosting an email system in a NOAA data center — was more straightforward.

NOAA CIO Joe Klimavicz said it would have cost NOAA about $23 million to fund the latter option. By moving to Google’s cloud email, the agency is spending $11 million over the next three years, a cost avoidance of about $12 million in total costs.

“I’m very confident those are the right numbers,” Klimavicz said.

Denise Stephens, CIO at the Energy Department’s Idaho National Laboratory, estimates that migrating 5,000 people to Google cloud email and collaboration tools will save her lab about $300,000 annually.

For now, the Energy lab is basing its savings projections on expected reductions in licensing costs and the costs of owning and maintaining hardware and software.

The challenge is tracking and quantifying how the new email system enhances employee performance, Stephens said. It’s easier to measure the system’s performance based on potential disruptions in service, the number of reported user issues and how quickly they were resolved.

Unisys migrated the Energy Department’s Idaho National Lab and GSA to Google’s cloud email and also had a hand in NOAA’s email migration. The company said although it provides cost-savings estimates, it did not work with Energy or GSA to project cost savings.

Although the lab has projected savings, those numbers could change. This year is critical to understanding where the lab has made headway in reducing costs and how that money has been reallocated, said spokeswoman Misty Benjamin. “Until those play out, we don’t know our true savings.”

When it comes to labor cost savings, it can be challenging if positions aren’t eliminated but employees are redirected to other projects. There are also new labor costs tied to employee training on the new system and help-desk support.

“We need to understand rather than guess what cost savings are,” Benjamin said.