How to make the most of your virtual environment

See also

There is no such thing as an IT department with too much money. That's always been true, but budgets right now are universally tight. For many organisations I deal with, the question is not which "nice-to-have" optimisation to leave off the list this year, but which essential project to delay.

This is not the only thing driving more and more organisations to virtualise their IT environments - there are many other factors like speed, reliability and flexibility - but making the best possible use of scarce resources is a big factor.

The savings of virtualisation come from the fact that the more virtual servers you can run on a single physical machine, the more energy you will save and the less often you will have to buy new hardware. In most organisations, the business case for virtualisation is built on the assumption of a "consolidation ratio" of between 10 and 20 to one - in other words, that every physical machine will host 10-20 virtual machines (VMs).

Once virtualisation has been implemented, though, users quickly notice many other advantages. One of the key benefits is the sheer convenience of creating new virtual machines. Instead of going through a lengthy procurement process to buy a new server, a development or project team can commission as many new VMs as they need in minutes.

This is great - but if you don't do your housekeeping properly, the proliferation of VMs can slow your environment to a crawl and fatally undermine the business case for going the virtual route in the first place. It's such a common problem it even has a name: VM sprawl.

Part of the reason VM sprawl happens is the ancient human instinct not to throw anything useful away "just in case". So even though the project is wrapped up, all the files are archived and there are full backups of all the VMs which can be restored at a moments notice - people still hesitate to delete the VMs themselves. Often the machine is un-registered so it's easy to forget about, but it's still there in a kind of undead zombie state.

Superficially it seems less risky to let your old VMs hang around in case you need them one day - after all, hardly anybody ever gets in trouble for NOT deleting something. But in reality, it's VM sprawl that poses the real risks.

The first risk is the one that really kills your business case: Every zombie VM is still using up valuable system resources, especially expensive storage. Let's say it's been allocated 500GB of data - even if that storage space is empty, it still can't be used because it's reserved for that VM. It's like putting a traffic cone in an empty parking space: It instantly turns free space into wasted space.

Combine zombie VMs with other junk data like old ISO files and defunct system snapshots and you can push an entire storage array to breaking point suprisingly quickly. The larger the organisation, the more people there are contributing to the sprawl and the faster trouble will happen, no matter how well-resourced you are.

There's also a compliance risk: How many operating system licences do you have? Every zombie VM is using up one of them. When the vendor comes to do an audit and finds you over the limit, "whoops we seem to have overlooked that one" is not going to be a good enough excuse.

Business processes that prevent wanton creation of new VMs are one way to solve the problem - but a process that overcomplicates things also undermines the flexibility benefits of virtualisation in the first place.

You also need to implement thin provisioning, particularly for SAN storage space which is notoriously expensive. Thin provisioning means you can configure your VM with all the storage you think is required - but actual storage space will only be released when it's needed, up to the maximum you have configured for. With this you can ensure the most efficient use of your entire available storage pool.

Another solution is to be rigourous about clearing out old and unused files - but to do that, you first need to find them. This is not a job that can be done manually - it needs specialised tools that can not only identify junk files but also enable you to delete them safely.

Another problem that undermines the business case for virtualising is misallocation and over-allocation of system resources.

The easiest way to create a new VM is to do it from a template - every new machine gets, say, two CPUs, 8GB of RAM and 100GB of storage space. That's a good, average spec - but unfortunately, not every VM has average needs. Some will need more resources, and some will need much less - and it's very seldom possible to know in advance which is which.

Assigning too many resources is wasteful and inefficient - it needlessly ties up resources you could be using somewhere else. But assigning too resources few hurts performance.

Finding the right balance between efficiency and performance needs the right tools - by which I mean tools specifically designed for managing virtual environments.

Virtualisation works because it allows us to over-commit resources, knowing that most processes don't need as much as they are allocated. It's like the way airlines overbook flights, knowing that there's a fairly predictable proportion of people who won't turn up. Because of this, and the multiple layers of abstraction created in the process of virtualising, it is almost impossible to understand the true resource usage of any VM using the same tools and approaches we use in physical environments.

You need the right monitoring and reporting tools to ensure that virtual overcommitment doesn't become real. These tools will also allow you to manage your VMs dynamically, allocating and taking away resources according to their needs.

Because the rules of virtual environments are so different, it's important to choose monitoring tools that give recommendations, not just reports. What you need is not just one more report, but an expert system that contains a lot of the specialist knowledge any organisation running a virtual environment needs.

You also need a system that can identify problems affecting specific servers, departments or applications. Throwing more RAM at a slow machine, for example isn't always the right solution. What if the real problem is disk related, or an application with a memory leak? Monitoring tools need to supply that information - in the specific context of a virtualised environment with overcommitted resources.

Human decision making is always paramount, of course, because humans know things about context and the future that machines can't. If the monitoring tool notices that between January and June a particular server was under-utilised, it may recommend switching resources away from that machine. The human who knows that the company's financial year-end is coming up in July will also know not to implement that particular recommendation.

Between an IT manager's knowledge of the business context, and a good monitoring tool's knowledge of the virtual environment, it is possible to run an virtualised IT shop that delivers both better performance and lower cost, meeting the ROI targets set by the business.

There is no doubt that organisations will increasingly continue to virtualise their IT infrastructure. Traditional IT policies, legacy management and reporting tools are fast becoming inefficient in meeting the demands of an environment delivering increased performance, scale and availability. A rethink of IT management to meet these demands and ensure you reap the most from your virtualised infrastructure is a no brainer. This will eliminate the potential pitfalls that could seriously undermine the business case for virtualising.

Actions

Share

Comments (2)

Boris :

19 Apr 2014 6:29:35pm

Does anybody do any physical work nowadays? Not gym, or mowing and washing dishes, I mean does anybody earn a living doing physical work?

All I know is my backside gets bigger and I have less time to do essential work, and I run a small business. I don't have the resources to worry about this,but my pc gets slower and I think oooh Linux, Unix.....

Lise Summers :

17 Apr 2014 10:40:15pm

Thanks for opening the discussion on this important topic Charles. While I agree with your central thesis - monitor your VM environment and the data on it properly - you forgot to mention the need to liaise with records and knowledge management staff to ensure that data is kept and retained for no longer than it needs to be. This goes beyond identifying the upcoming annual audit, and moves into the business and legislative environment in which organisations operate.

How Does this Site Work?

This site is where you will find ABC stories, interviews and videos on the subject of Technology and Games. As you browse through the site, the links you follow will take you to stories as they appeared in their original context, whether from ABC News, a TV program or a radio interview. Please enjoy.