3 Introduction Private cloud infrastructures are gaining wide acceptance among enterprises and service providers. Consolidation of multiple servers and storage devices into private clouds based on Vblock (Cisco UCS + VMware vsphere + EMC vplex) and FlexPod (Cisco UCS + VMware vsphere + NetApp FAS) are examples of such adoption. The particular behavior of networked storage components in such virtualized contexts must be investigated so customers can establish the performance limits of their private clouds, and optimize their configuration. This white paper is based on actual Load DynamiX customer experiences. Users have demonstrated that a pre-production testing approach using Load DynamiX provides critical configuration information and reveals important infrastructure behavior issues before deployment. These insights save customers valuable time and help them design more robust solutions for greater performance and uptime. This white paper describes how Load DynamiX is used to investigate a boot storm scenario as well as the run-time behavior of a virtualized application. The process generates insight used into the configuration of not only networked storage components, but also their related network and compute components. Background on Load DynamiX Load DynamiX provides an all-in-one test solution for network storage (File, Block and Object storage) designed for storage engineers and architects in business critical data center environments. Load DynamiX allows users to investigate new storage configurations and test the limits of their infrastructures. Users leverage the Load DynamiX test development environment (TDE) and tools to emulate their production workloads; they can then generate realistic traffic using the Load DynamiX load generation appliances. Users can thus continually evaluate new configurations or solutions. Load DynamiX empowers customers with the insight they need to make better design and configuration decisions, so they can control their cost, performance, and risk. Model of the Private Cloud Infrastructure Leaving Firewall and Load Balancer out of scope, a simplified representation of the Private Cloud can be shown as follows: v (NFS/CIFS/iSCSI/FCoE) /TCP/IP/Ethernet (NFS/CIFS/iSCSI)/TC P/IP/Ethernet UCS chassis Switch Fabric FC Unified Network Storage Whitepaper: Private Cloud Migration October

4 The UCS chassis can host multiple blades; each of them may host a vsphere ESXi hypervisor and multiple VMs. The hypervisors and VMs can use various network storage protocols, such as NFS, iscsi, CIFS, and FC/FCoE. The switch connects the UCS to Network Storage over multiple VLANs and FC interfaces. The Network Storage is often a cluster with multiple physical ports, IPs, FC interfaces on the network side providing access to the storage arrays through multiple layers of protocols and software. Performance thresholds are usually set up in the monitoring system, which is a part of the Private Cloud management. Configuration of a complex system like this is a no small task. Thus, the probability of a mistake is high. Testing and verification of the private cloud, even partial, prior to moving it into production will save lots of efforts going forward. The Load DynamiX approach to pre-deployment testing is to emulate the computational resources (effectively replacing the UCS resources) with the Load DynamiX appliance and run synthetic network storage workloads resembling the production workloads against the Network Storage and Switch Fabric. v (NFS/CIFS/iSCSI) /TCP/IP/Ethernet (NFS/CIFS/iSCSI) /TCP/IP/Ethernet Load DynamiX FC Switch Fabric FC Unified Network Storage The Load DynamiX Appliance is capable of generating feature-rich scalable complex network storage workloads emulating large amount of the network storage clients over a set of networks and interfaces. Load DynamiX Pre-Deployment Testing Strategy The Load DynamiX solution is provided with a suite of pre-built test workloads designed for testing the Private Cloud. Whitepaper: Private Cloud Migration October

5 Pre-Test Workloads Prior to running any tests, Load DynamiX creates: 1. An emulation of the VM images. Depending on the storage type, the VM images are represented as sets of directories and files or block ranges on the LUNs 2. Directories and files on the volumes/shares representing the objects used by VMs 3. Ranges of the blocks on the LUNs representing logical disks used by VMs Boot Storm Workload Test Boot storm is an industry term for a simultaneous boot up process of multiple VMs from a single storage device. A boot storm creates an extraordinarily high load for read requests on the network storage. This load is many times greater than normal production loads. Load DynamiX models the boot storm as a set of concurrent clients reading a significant amount of data (1GB) from the designated parts of the Network Storage. In the case of NFS these are previously created files representing the VM images, in the case of iscsi or FC these are the sub areas of the LUNs filled with the VM image data. Whitepaper: Private Cloud Migration October

6 Load DynamiX Appliance An example of the Boot Storm Workload Test. The VM images are stored on the NFS volumes. The test provides the following insights: Reveals hidden bottlenecks in the switch/storage configuration Allows for evaluation of the optimal cache and boot concurrency settings in order to optimize the boot process There is a special kind of the boot storm known as a VDI boot storm. The specifics of the VDI boot storm are that the images of the VM are very similar to each other and, therefore, deduplicable and that the boot is often accompanied by the authentication process. Load DynamiX accommodates these specifics very easily in its workloads. Whitepaper: Private Cloud Migration October

8 The ratio of the IOPs, their sizes, and the content are configurable in order to emulate the production workloads. The number of concurrently active VMs can be scaled up to thousands. This test reveals: Performance limits Potential misconfigurations of the network and/or the network storage Potential interference of the multiple interfaces Acceptable number of simultaneously running VMs Functioning of monitoring system alarms Private Cloud Analysis The prebuilt workloads outlined above provide a number of open parameters that allows for customization for the specific Private Cloud under test. These parameters and settings are used throughout the test suite and can be broken down into three major groups: Computational resources parameters and settings Network resources parameters and settings Configuration parameters and settings Computational resources Number of UCS blades Number of VMs Distribution of VM size Type of storage used for VM images (FC, NFS, iscsi) Types of external storage used by VMs - FC - iscsi drives - NFS mounts - CIFS shares An example of the VM size distribution is shown in the figure below. The horizontal axis represents the file size in logarithmic scale. Whitepaper: Private Cloud Migration October

9 70 VM File Size Distribution 60 Typical VM sizes 50 Number of files Large size outliers VM size (GB) Network resources and configuration Network configuration and IP addresses allocated to the Hypervisors IP addresses of the guest OSs running on the VMs. These IP addressed will be used by the Load DynamiX clients representing VMs and hypervisors Storage resources and configuration IP addresses of the Network Storage servers, volume, shares, initiators and target names are open parameters of the Load DynamiX workload tests. Additional information on the specifics of the production workloads can be used as an input to the Load DynamiX workload: Whitepaper: Private Cloud Migration October

10 Topology of the directory structure Number of files and their size distribution Ratio of the Read/Write/Meta Data operations Degree of compressibility and de-duplicability of the file contents Authentication In order to test the authentication infrastructure as a part of the Private Cloud, user names and passwords should be created for the testing purposes and configured in Load DynamiX. Examples of Test Results Execution of the Load DynamiX workloads in the Private Cloud environment reveals many issues otherwise overlooked, usually to be uncovered at a later point in production. The first two charts present an example of optimization of the VM boot process and the discovery of a frame size misconfiguration found during the testing process. VM Boot Process Fig. R1 shows the number of active VMs (VMs that have completed the boot up process) as a function of time for three different values of Startup Delay. The Startup Delay is a pause between starts of two consecutive boot processes. Time count starts at the start of the boot of the first VM. Whitepaper: Private Cloud Migration October

12 Fig. R2. VM Boot Process for different frame sizes 2000 MTU = 1500 B Number of active VMs MTU = 8000 B Time (min) Configuring frame size: Fig. R2 shows the boot up process over NFS of a large number of VMs using regular Ethernet frames (green line) and jumbo frames (red line). At around 43 min in the process, traffic slows down to a crawl in the jumbo frame case. This test revealed a misconfiguration of the switch which was caught prior to release to production. Run Time Scenarios In these scenarios, Load DynamiX is being used to test and configure the responsiveness of the virtualized application. Load DynamiX workloads emulate VMs updating business data over networked storage. Load DynamiX measures the server response times to commands initiated by the VMs. These response times are directly related to the user experience and better characterize performance of the network storage than would a simple measurement of IOPs on the storage device. The response time for Write/Read operations over NFS shows dependence on the number of simultaneously working VMs and the size of the VM image file. Fig. R3 shows that the larger the size of the VM image the more significant the slowdown experienced by a VM as the number of the VM grows. Whitepaper: Private Cloud Migration October

14 Conclusions The Load DynamiX testing platform is applied to a pre-deployment testing of a Private Cloud environment. Load DynamiX workloads can be customized to capture specifics of a particular Private Cloud and emulate a variety of production loads (nominal, peak, limits). These unique insights into the performance bottlenecks of the Private Cloud obtained using Load DynamiX allow customers to make important design and configuration decisions pre-deployment, saving valuable time and resulting in greater performance and uptime. Whitepaper: Private Cloud Migration October

Technology Concepts and Business Considerations Abstract A virtual information infrastructure allows organizations to make the most of their data center environment by sharing computing, network, and storage

DR-to-the- Cloud Best Practices HOW TO EFFECTIVELY CONFIGURE YOUR OWN SELF-MANAGED RECOVERY PLANS AND THE REPLICATION OF CRITICAL VMWARE VIRTUAL MACHINES FROM ON-PREMISES TO A CLOUD SERVICE PROVIDER CONTENTS

Dell EqualLogic Best Practices Series Sizing and Best Practices for Deploying Citrix XenDesktop on VMware vsphere with Dell EqualLogic Storage A Dell Technical Whitepaper Storage Infrastructure and Solutions

Optimizing Virtual Infrastructure Storage Systems with Xangati Virtualized infrastructures are comprised of servers, switches, storage systems and client devices. Of the four, storage systems are the most

This document shows you how to use a Drobo iscsi SAN Storage array with Veeam Backup & Replication version 5 in a VMware environment. Veeam provides fast disk-based backup and recovery of virtual machines

Learn the Essentials of Virtualization Security by Dave Shackleford by Dave Shackleford This paper is the first in a series about the essential security issues arising from virtualization and the adoption

V3 Appliance FAQ Why is the V3 appliance so effective as a physical desktop replacement? The V3 appliance leverages local solid-state storage in the appliance. This design allows V3 to dramatically reduce

Empowering Private Cloud with Next Generation Infrastructure Martin Ip, Head of Advanced Solutions and Services Macroview Telecom People Media Healthcare Govt Business Private Hybrid Public & more Connecting

WhitePaper March 2012 Best practices for backing up virtual servers with Veeam software and StarWind storage While virtualizing your applications increase their efficiency, you still need a backup solution

Cloud Optimize Your IT Windows Server 2012 The information contained in this presentation relates to a pre-release product which may be substantially modified before it is commercially released. This pre-release

Using VMWare VAAI for storage integration with Infortrend EonStor DS G7i Application Note Abstract: This document describes how VMware s vsphere Storage APIs (VAAI) can be integrated and used for accelerating

What s New in Version 3.8 Product release highlights Load DynamiX unveils major updates in LDX-E version 2.5 and TDE version 3.8. The October 2014 release is built on two primary objectives: 1. Boost test

W h i t e p a p e r Top Ten reasons to use Emulex OneConnect iscsi adapters Internet Small Computer System Interface (iscsi) storage has typically been viewed as a good option for small and medium sized

Nutanix Tech Note Configuration Best Practices for Nutanix Storage with VMware vsphere Nutanix Virtual Computing Platform is engineered from the ground up to provide enterprise-grade availability for critical

Across the globe, many companies choose a Cisco switching architecture to service their physical and virtual networks for enterprise and data center operations. When implementing a large-scale Cisco network,

REDCENTRIC INFRASTRUCTURE AS A SERVICE SERVICE DEFINITION SD021 V2.2 Issue Date 01 July 2014 1) OVERVIEW Redcentric s Infrastructure as a Service (IaaS) enables the to consume server, storage and network