How Will the Virtual World Create Storage Problems for Web Hosts?

Virtual computing has created a tremendous opportunity for businesses to offload their IT requirements to cloud providers. Today, businesses with very large storage requirements, such as distributors of video media, can use the cloud as a way to implement management-free storage. They no longer have to worry about matching capacity to the changing market, of providing proper physical environments for storage servers, or deal with security issues.

However, in a sense a virtual storage system is the exact opposite of a KISS (keep it simple system). With increased complexity, the ability to control input, throughput, and output patterns is severely limited. Virtual clients do not have to deal with this problem, but web hosts must be able to efficiently handle all the performance issues associated with a virtual environment.

The Cloud Environment: Perception versus Reality

Virtual systems providers have done a terrific job of marketing the cloud. The average person knows their data is “somewhere out there” just like rain in a cloud. However, even an advanced architect does not know exactly where any particular set of data is stored. Storage is performed algorithmically – even cloud technicians cannot map specific data any more than the Westworld employees knew exactly what was happening with their virtual people.

Watch Your Defragmenter

To get an idea of the difference between managing traditional data as opposed to controlling and optimizing virtual data, watch a computer’s disk defragmenter in action. The defragmenter shows a very different (and realistic) picture of how files are stored on media. The average computer user uses a file cabinet paradigm, and often imagines files are stored in computer memory in a way somehow similar to how they are stored in their desk drawers. However, the defragmenter shows how files are split and spread all over the hard drive. Only the disk manager knows the rhyme and reason behind the way the files are stored.

Although the files may look like they’ve been dropped in a blender and spread over the landscape of the hard drive, in reality, they are stored in a very precise and efficient manner which allows the computer to access data files in a smooth speedy fashion. However, the data manager is essentially blind to the virtual system running on top of it. It cannot manage the resources in the virtual environment in the same efficient way it does the underlying media. The ability to manage virtualized workloads is the challenge for the modern hosting provider.

In the Heat of Battle

It would be advantageous for hosting providers to be able to deal with the increased storage requirements for virtual applications in the future, but unfortunately, as far as virtualization is concerned, the future is now. The IDC states the currently 72% of workloads are now virtualized. In addition, they estimate that this value will grow to 86% by the year 2018. Physical workloads already comprise only a small part of all workloads. Hosting providers must hit the ground running with a VM management solution.

Input/output Management Challenges

Improper I/O management can lead to several potential problems, including:

Insufficient control cache – cache provides a buffer for the short-term dynamics of the I/O flow. However, the unpredictability of virtual data flow can overwhelm this buffer.

Read/write array scheduling – read/write scheduling has a major influence on performance. The increased number of read/write operations can cause a significant degradation.

Providers plan virtual performance capacity through a parameter called IOPS. This allows system planners to effectively measure virtual throughput. This was an easy task when data streams originated from a single computer. However, VM environments break large data blocks into small read/write data packets. For this type of transmission, a bits/sec measurement is not meaningful. Instead, VM planners use Input/output Operations per Second (IOPS).

How Hosting Providers Maximize IOPS

VM administrators calculate their required IOPS by monitoring performance over some period of time, commonly 30 days. This gives a good idea of how many IOPS are being used by the application server.

In order to boost IOPS capacity, designers often implement platforms with an extra 40 to 60 percent excess storage over what they need. Many providers upgrade spinning disk systems with flash storage technology. The improved performance of flash gives an equivalent boost in IOPS. This is an attractive strategy due to the plummeting prices of flash storage. Administrators can increase IOPS further by using data compressions and duplication to optimize their flash drives.

Managing the Virtual Environment Challenges

Match VMs to workloads – similar workloads make planning and scheduling more predictable. Designers can modify SAN configurations in order to group like-workloads instead of simply using a single array of disks.

Take advantage of storage offloading functions – modern hypervisor systems are designed to coordinate with the underlying storage functions in order to offload as many storage functions as possible. This increases the efficiency of storage operations, eases the burden on the hypervisor, and helps reduce congestion on the storage connections.

Moving the Focus to the Virtual Level

Once the platform hardware is deployed and configured, cloud administrators completely move their management focus to the virtual level. This means that performance measurements such as snapshots, replication, and statistics gathering must be done at the virtual machine level. By employing the proper management systems, this strategy allows the system to monitor and fine tune itself.

In this way, cloud providers can tame the I/O management problem. Technicians view a dashboard with the following specifications:

CPU usage

RAM usage

Network saturation

IOPS usage

By monitoring these simple outputs, technicians can see in a glance that their virtual system is healthy and robust.

WHG StaffWriter, Blogger & Social Media Ne'er-Do-Well. A former member of the board for the Graphic Artists Guild and co-chair of the GAG Professional Practices Committee. I write for global blogs on business practices, ethics, technology, social media, QR codes and Augmented Reality.Other posts by WHG Staff

Data Centers in China to Increase in 2015

With nearly 1.4 billion inhabitants, China leads the world in Internet traffic. It is also the second largest market for San Francisco-based CloudFlare. The online security and performance company has…

ABOUT US
No biased "expert" reviews, forged speed/uptime ratings, deceptive "actual testing" analyses etc. Just plain, real customer reviews, both positive and negative. We've been collecting them since 2004. So far we have 14647 reviews of 1734 hosting companies. Please note that some content may include sponsored links.