One of the major features that came out of the vSphere release 16 months ago was the rebuild of the IP stack and iSCSI initiators. With this change was the introduction of the concept of multiple vmkernel ports on the same network, allowing for multipathing your iSCSI connections. Shortly after this change, Dell released a tech paper describing how to create a dedicated iSCSI vSwitch, multiple vmkernel ports, assigning IPs to those ports, setting all of them to use Jumbo Frames, and finally enabling the iSCSI initiator to use the ports concurrently. This became the standard for multipath I/O with iSCSI in general and EqualLogic specifically. Within your vSphere infrastructure, you could now set your datastore connections to use Round Robin pathing and increase both your bandwidth and redundancy on SAN connections. Fast-Forward another year, and VMware releases their latest version, vSphere 4.1 on us last summer. Again, there is major changes to the way that vSphere handles storage. With the release of the vStorage APIs, VMware allowed storage vendors to write hooks directly between their products and vSphere. Some new features introduced to vSphere include offloading storage transactions, faster backups, direct cloning and copying support. However, to get these new features you need three things: Enterprise Plus licensing on your ESX hosts, vSphere 4.1, and the software from your 3rd party vendor. With EqualLogic, this is not necessarily the case. EqualLogic released their vSphere API software, the “MEM”, in September 2010 for use by their customers in virtual infrastructure. If you are currently at vSphere 4.1 and at Firmware v4.3.5 (?), preferably 5.0.2, you can install and use the EqualLogic MEM today regardless of your ESX licensing level. You will see some performance increase, automatically configure all EqualLogic datastores for MultiPathing, and set the Datastore Pathing to a new “Dell_PSP_EQL_ROUTED” option. This pathing option is optimized for EqualLogic storage, and some of our customers have seen up to a 15% increase in performance over the previous manual multipathing and Round Robin connections. Additionally, when installing and configuring the MEM, the setup script automatically builds, configures, and connects your vNICs and IPs, vSwitches, and vmkernel ports. What is important to know is that the MEM works at ANY vSphere licensing level. What doesn’t work is the higher level vStorage API hooks into EqualLogic. If you are at the Enterprise Plus licensing level, you can now leverage those APIs within vSphere for your day to day operations. In a traditional sense, every time you copy files from one datastore to another, such as moving VMs, cloning, snapshots, and such were resource intensive on the ESX hosts. Data would need to move from the Datastore to the ESX host and then back to the new Datastore location even if it was on the same datastore. With the MEM in place and Enterprise Plus, the vStorage APIs allow for those transactions to happen directly on the array without traversing up the line to the ESX host. This change can significantly decrease the back end load on your ESX hosts, increasing availability for supporting your VMs and overall efficiency. What is needed to install the MEM? First of all, install the CLI tools on a management host or your vCenter server or download and install the VIM appliance to allow CLI access to your vSphere infrastructure. You need to have a current support contract with Dell and download the MEM to your vCenter server, or CLI/VIM host. For simplicity, we will go with CLI from here on. Download and unzip the MEM and open up the vSphere CLI interface. Set your ESX host into Maintenance Mode, browse to the MEM folder and run ‘setup.pl –install –server=”192.168.xxx.xxx”‘ and watch the magic happen. Actually, there isn’t much magic. The script will ask for a username and password, and after a few minutes it completes. Reboot your ESX host, and once it comes online again, open up the CLI, browse to the folder, and run ‘setup.pl –configure –server=”192.168.xxx.xxx”‘ This time, there is actual magic happening. The script asks several questions as part of the configuration, such as ‘vSwitch name’, ‘vmNICs to use’, ‘IPs for the vmNICs’, ‘Frame Length’, and ‘Group IP’. At the end of the conversation, it will show the actual configuration script being run, and once you hit ‘YES’, it will build the vSwitch as indicated, rescan the iSCSI bus, and set the connections to the new “Dell_PSP_EQL_ROUTED” option. With additional software available for free (as long as you are on maintenance) from Dell|EqualLogic, why wouldn’t you use this to increase performance and available resources within your existing infrastructure?

5 Responses to EqualLogic MEM and vStorage APIs

I know this blog entry is a couple of years old, but I’m wondering if you can clarify your statement above of “If you are currently at vSphere 4.1 and at Firmware v4.3.5 (?), preferably 5.0.2, you can install and use the EqualLogic MEM today regardless of your ESX licensing level.”? The reason I ask is that I installed the latest MEM on a new ESXi 5.1 host and while it configured and appears to run perfectly, it only allows TWO paths to the EQL (when I have three)… actually, when the host is initially booted I DO see all three paths briefly, but then one of them “logs out” of the EQL and I’m left with two. If I switch back to Round Robin and reboot, all three paths stay active and logged into the EQL indefinitely. No one so far has been able to explain this behavior to me.

In my subsequent research, Dell claims MEM has NEVER been supported on any version of vSphere Standard and only Enteprise/E+ have been able to use the MEM… but your statement above seems to contradict that. Have things changed with 5.0/5.1 and the ability to use MEM on standard since you wrote this blog? What’s confusing/curious to me is that everyone at Dell claims the MEM *DOES NOT WORK* on vSphere Standard, yet I have it running perfectly here… albeit with only two of the three paths working, for reasons unknown. Just curious if you have any insights.

Jim,
Good question. The EqualLogic MEM functions in several ways. First, the installer will help you configure a dedicated vSwitch for iSCSI storage traffic and configure the NICS correctly for multipathing. Second, it leverages the custom Dell multipathing construct, PSP_EQL_Routed, as the preferred connection method for all volumes served from EqualLogic arrays. Third, and most important, it allows for VAAI functionality to occur between the vSphere host and the EqualLogic Group.
When you install and configure the MEM on your host, the first feature can occur regardless of licensing level on your vSphere host. This process occurs due to the software and installer from Dell. There is no additional features that is being leveraged by the MEM at this point.
However, the second and third features, 3rd Party Multipathing and VAAI integration, are regulated completely by the vSphere host licensing level. At this time, both features are available for Enterprise and Enterprise-Plus licensing levels.
Therefore, you can still use the EqualLogic MEM to configure your iSCSI vSwitch, and it will set up the NICs correctly for you as well. You just can’t use all of the features available without reaching for those higher license levels.
Additionally, as part of our standard builds, we usually don’t use more than 2 NICs dedicated to the iSCSI switch for multipathing. We’ve found that additional NICS don’t offer significant benefits and are not worth managing the additional connection counts for volumes and members.
Hopefully this helps.
-Tim

Tim-
Thanks for the additional info… that really helps! I think I finally understand the different pieces of MEM and the availability of some at only the higher license levels, but one nagging thing in my mind remains, and that’s the fact that I can change the pathing (on my vSphere 5.1 standard host) to PSP_EQL_ROUTED and it in fact appears to work… it’s just limited to two connections to a volume. If I switch back to Round Robin, I get all three. Do you think this is by design, or is some sort of bug because I installed the MEM when the host was in demo mode and only applied my Standard license AFTER that? Or maybe the two-path-only MEM will stop working after 60 days?🙂 Just curious.

Maybe I would be fine with two iSCSI NICs, as well… that’s what I have to my old (non-EQL) SAN and it’s always worked fine… and I guess if I’m somehow getting two paths of MEM on standard that would be better than 3 paths of Round Robin…? Hmm.

Jim, I’m in the same boat as I have installed MEM on vSphere 5.1 during the 60 day Trial period. I’m still on trial and will soon be forced to enter my Acceleration Standard license.
What did you come up to?
Thanks!