Search in:

Computing the cost of clouds

Geoff Maslen

Mark Twain once said it was OK to keep all your eggs in one basket — as long you kept a close eye on that basket. His 19th-century advice appears particularly apt for a 21st-century technological marvel: cloud computing.

This is the means by which companies such as Amazon, Google, IBM and Microsoft now offer data storage and virtual servers their customers can access on demand without having to store masses of information on their own computers.

But moving the electronic data held by a university, company or government department to an outsider has generated widespread concern. Victoria's Privacy Commissioner, Helen Versey, warned last month that the cost of dealing with privacy and security issues could outweigh any capital and operational savings.

Ms Versey told state government organisations they should only use cloud service providers that agreed to comply with Victoria's information privacy laws and preferably had locally based data centres. She said where a provider was located offshore, or even outside Victoria, protecting personal information from misuse, loss, unauthorised access, modification or disclosure might be difficult or even impossible.

Advertisement

Despite such concerns, international IT research and advisory company Gartner Inc rates cloud computing among the top 10 strategic technologies for 2011, predicting that within two years the market for cloud will exceed $150 billion.

Gartner could be right: big business, government departments and universities across the globe are adopting cloud computing as a way of solving their ever-expanding data storage and processing problems.

In Australia, a review of the federal government's use of IT estimated savings of $1 billion could be achieved over the next 15 years by developing a data centre strategy based on cloud computing*. But while this technology offers almost unlimited capacity for data storage and processing, the costs to users are rising at almost the same rate as the technology is being adopted.

Now researchers at Swinburne University believe they have devised a means of cutting those costs. With a $360,000, three-year Australian Research Council Discovery Project Grant, the researchers from Swinburne's Centre for Computing and Engineering Software Systems, along with others at the University of Technology, Sydney, are developing cost-effective strategies to assist large users of cloud computing.

Swinburne's Professor Yun Yang says the trade-off is between how much data should be stored in the cloud and how much should be deleted, remembering that whatever is stored incurs a cost.

"Our strategy is to try to work out whether to store all data or separate out data you don't want and delete it. Or if you use the data only occasionally, delete it and if you do need it again later, you can regenerate the data," Professor Yang says. "The aim is to find the minimum cost depending on how the data is used, how much is stored and how much computing time will be needed."

With Professor John Grundy and Associate Professor Jinjun Chen, now at the UTS, Professor Yang's team has developed a mathematical model to find the best deals for cloud storage that takes account of the amount of initial data, the rates charged by a service provider and the intermediate data that could be deleted.

The researchers have also devised an "intermediate data-dependency graph" for the strategies to help users decide whether they would be better off spending money on storage and computation costs for intermediate datasets.

In a novel way of testing the mathematical formula, the researchers simulated data compiled by colleagues in Swinburne's astrophysics centre in their search for pulsars utilising the Parkes radio telescope in New South Wales. Pulsars are rapidly spinning neutron stars that emit beams of electromagnetic radiation.

"The search for pulsars processes vast amounts of data — the data stream from the telescope is typically at one gigabyte per second — and that data will be processed and may be reanalysed by astronomers all over the world for years to come," Professor Yang says. "From one set of raw beam data collected by the telescope, our formula examined six milestone intermediate datasets and three different cost scenarios. We used the prices offered by the Amazon cloud's cost model: 15¢ per gigabyte per month for storage and 10¢ per hour for computation."

The minimum cost for storing the intermediate data resulting from one hour of observation from the telescope for 30 days was calculated to be $200 while not storing the intermediate data but regenerating it when needed came to $1000. The researchers then had a set of options on which data to keep and which to delete.

"We could delete the intermediate datasets that were large in size but with lower generation expenses and save the ones that were costly to generate, even though small in size," Professor Yang says.

"The work we have done so far can give a good estimation of the costs involved in switching to cloud computing. If a company decides to move all its data to cloud, it must pay for that, although in many cases it would only be necessary to store just the core data on cloud and regenerate other data that is not essential when it is needed."

He says he is a strong believer in cloud computing, while accepting that many people have serious concerns about its security: "But 10 years ago, people were concerned about using credit cards online and feared being defrauded, whereas now everyone accepts it. So any new technology has a curve to climb before it becomes widely accepted. There are issues about security and privacy with cloud computing, but I am not concerned given past experience; in any case, cloud computing has ways of solving these problems, for example with a private cloud that can be made solely available to a company or institution."

* Some federal government agencies have already begun pilot trials and proofs of concept to evaluate the potential of cloud computing. Examples include the Tax Office, the Bureau of Statistics and the Immigration Department.