Things that need to be changed on vSphere for Xtreme IO

Xtreme IO is the newest and fastest (well EMC say so) All Flash Array in the market. I have been running this in my “lab” running a POC which is quickly turning into a major VDI refresh for one of the clients. Having run throug the basics of creating storage and monitoring alerting etc in my previous posts., I am going to concentrate on what parameters we need to change in the vSphere world to ensure we get the best performance from Xtreme IO.

The parameters also depend on what version of ESXi you’re using, as Xtreme IO supports ESXi 4.1 + .

Without further delay, lets start.

Adjusting the HBA Queue Depth

We are going to sending a lot more IO through to the Xtreme IO array than you would to the traditional hybrid array. So we need to ensure that the HBA queue depth is allowing a lot more IO requests through.
You can find out the module by using the command

Step 1: esxcli system module list | grep ql (or lpfc for emulex)

Once you find out the module that is being used. The command below can be used to change the HBA queue depth on the server.

Qlogic – esxcli system module parameters set -p ql2xmaxdepth=256 -m qla2xxx (or whatever is the module from the command in Step 1.)

Emulex – esxcli system module parameters set -p lpfc0_lun_queue_depth=256 -m lpfc820 ( or whatever is the module from the command in Step 1)

Multi Pathing

If you are not going to use Powerpath, since its an active active X number of controllers array (yeah, i know its got 2 controllers per disk shelf so as of today you can scale upto 6 disk shelves per cluster so 12 controllers), we will be using Round Robin if using NMP.

The engineers who work with Xtreme IO recommend that the default number of iops be changed from 1000 to 1, yes “ONE”. So essentially you are sending an IO request to each controller in the cluster. I haven’t really seen any improvement in the performance by doing so but it is only a recommendation at the end of the day. If you see that you are not going to achieve any significant performance by doing so, the onus is on you to make that decision.

First, lets get all the volumes that’ve been configured on Xtreme IO.

esxcli storage nmp path list | grep XtremeIO

this will give you the naa.id of all the volumes that are running on XtremeIO.

You should also change the Disk.DiskMaxIOSize from the default of 32767 to 4096. This is because XtremeIO reads and writes by default in 4k chunks and thats how it gets the awesome deduplication ratio.

In ESXi 5.0/5.1 you can set the SchedNumReqOutstanding by using

esxcli storage core device set -d <naa.id> -O 256

In vSphere 5.5 you can set this paramter on each volume individually instead of configuring on per host.

vCenter Server Parameters

Depending on the number of xBricks that are configured per cluster, the vCenter server parameterconfig.vpxd.ResourceManager.maxCostPerHost needs to be changed. This adjusts the maximum number of full cloning operations.

Thats the end of this post. Please feel free to correct me if I’ve got any commands wrong.

Recommendation (as per AusFestivus’ comment): EMC recommend that PP be used for best performance. But it always comes down to the cost constraints and how much the client wants to spend. In my opinion, PP is more like “nice to have for best performance without tinkering”. But if you can keep tinkering and changing things to get the best performance out, you can do without PP.