How to Enable Support for Nested 64bit & Hyper-V VMs in vSphere 5

With the release of vSphere 5, one of the most sought out feature from VMware is the ability to run nested 64bit and Hyper-V guest virtual machines in a virtual ESXi instance. Previous to this, only 32bit virtual machines were supported as the VT-x/AMD-V Hardware Virtualization CPU instructions could not be virtualized and presented to the virtual ESX(i) guest. This feature is quite useful for home and lab setups in testing new features or studying for VMware certifications and running multiple vESX(i) instances.

You will still be required to have a 64bit capable system and CPU and you will need to be running ESXi 5.0, this will not work for ESX(i) 4.x or older.

The above diagram depicts the various levels of inception where pESXi is your physical ESXi 5.0 hosts. We then create a vESXi 5.0 host which will contain the necessary Hardware Virtualization CPU instructions to support a 64bit nested guest OS which I've created as another ESXi host called vvESXi.

Note: You will not be able to run a 4th level nested 64bit VM (I have tried by further passing the HV instructions in the nested guest) and it will just boot up and spin your CPUs for hours.

This feature by default is disabled in ESXi 5.0, to enable this virtualized HV (Hardware Virtualization) you will need to add the following string vhv.allow = "TRUE" to /etc/vmware/config of your Physical ESXi 5.0 host

Once the configuration change has been made, the feature goes into effect right away. A reboot of the system is not necessary. To verify, you should now be able to power on a 64bit guest OS and see that the HV instructions bits are being passed into the guestOS which will then allow you to run a nested 64bit guestOS. You can also verify by looking in the vmware.log file of the virtual machine and grep for the string "monitorControl.vhv" and if you see the following message, then Virtualized HV is not enabled.

In the past to run a virtual ESX(i) instance, a few advanced .vmx configuration entries were needed as documented here. With ESXi 5.0, if you are using virtual hardware version 8, then you do not need to make any additional changes. If you are using hardware version 4 or 7, then you will need to add a few changes to the VM's configuration file.

Creating vESXi 5.0 Instance using Hardware Version 8:

1. To create a virtual ESXi 5.0 instance, start off by just creating a standard RHEL5/6 64bit VM using the vSphere Client

2. Once the VM has been created, edit the settings of the VM and change over to the "Options" and now have the ability to select a new guestOS type: VMware ESXi 5.x or VMware ESXi 4.x under the "Other" section.

Note: I'm not sure why these two additional guestOS type is not available from the default creation menu, but are available after the initial VM shell is created.

3. You are now ready to install ESXi 5.0 in this new vESXi host and then you can create and power on nested 64bit guestOS within that vESXi instance as denoted from the picture below

Creating vESXi 5.0 Instance using Hardware Version 4/7:1. To create a virtual ESXi 5.0 instance, start off by just creating a standard RHEL5/6 64bit VM using the vSphere Client

2. Now you will need to add the following advanced .vmx parameter: monitor.virtual_exec = "hardware" which can be done through the vSphere Client and/or editing the .vmx parameter manually.

3. Next you will need to add some cpuid bits, depending if you are running an Intel or AMD CPU, the respective entries are required:

4. You are now ready to install ESXi 5.0 in this new vESXi host and then you can create and power on nested 64bit guestOS within that vESXi instance

By using a VM that is hardware version 8, you can easily automate the creation of vESXi 5.0 instance by changing the guestOS string in the .vmx parameter to "vmkernel" and the above configurations other than "vhv" string needed for either an Intel or AMD system are automatically configured.

For proper networking connectivity, also ensure that either your standard vSwitch or Distributed Virtual Switch has both promiscuous mode and forged transmit enabled either globally on the portgroup or distributed portgroup your nested ESXi hosts are connected to.

Creating a vHyper-VInstance on physical ESXi 5.0:1. To create a virtual Hyper-V instance, start off by creating a Windows 2008 Server R2 64bit VM using the vSphere Client

2. If you are using Hardware Version 7, you will need to follow the instructions in "Creating vESXi 5.0 Instance using Hardware Version 4/7" to add the additional parameters to the VM. If you are using Hardware Version 8, you just need to change the guestOS type to VMware ESXi 5.0

3. You need to add one additional .vmx parameter which tells the underlying guestOS (Hyper-V) that it is not running as a virtual guest which in fact it really is. The parameter is hypervisor.cpuid.v0 = FALSE

4. You are now ready to install Hyper-V in a virtual machine and you can also spin up nested 64bit guestOSes in this virtual Hyper-V instance.

As you can see, now you can even run Hyper-Crap, err I mean Hyper-V as a virtualized guest under ESXi 5.0. I did not get a chance to try out Xen, but I'm sure with the ability to virtualize the Hardware Virtualization instructions, you should be able to run other types of hypervisors for testing purposes.

This is a really awesome feature but note that this is not officially supported by VMware, use at your own risk.

I am unable to run 2008 r2 nested. I have 2 vesxi5 running. I have made necessary changes on the physical host for vhv.allow. Created virtual cluster and added 2 vesxi5 host. Was able to create virtual machine and installed 2008 r2. After finishing the install, I restarted the machine. The OS boot but BLACK screen. Why, I am unable to run nested virtual machine on vesxi5 host. No clue. Still searching for an answer. Hope somebody here can help me.

I have followed all the steps for VHyper-V on a physical ESXi 5.0 and receive a black screen when booting up the 2008 server as well. Has anyone found a solution to this yet? I need to run hyper-V for a program that requires it but don’t have another physical server to install it on. Any help would be great.

Hi, Can anyone help me here. Im able to create Nested 64 bit Windows 2008 servers, but those manys are not reachable from outside. I understand this is a problem with Esx 5.0, as 32 bit windows 2008 when created from Esx 4.1 works fine.

I am able to create vESXi 5.0 v8 guest hosts in my existing ESXi 5.0 cluster with no issue, and can create 32bit v8 guests on the nested host. However, I do not have the option to choose ESXi as the OS after creation, and I cannot create 64 bit nested guests. I’ve set vhv.allow = “TRUE” and hardware virtualization is enabled on my pyhsical hosts.Any suggestions as to what my issue may be?

Not sure I follow what your statments, you mention you can create vESXi 5.0 v8 host but then later say you can not select ESXi as guestOS. This is contradicting? If you running ESXi 5.0 on your physical host and follow the steps above, you can run nested ESXi hosts and power on 64bit nested VM within that nested ESXi host.

Recommend you re-read the article and perform some troubleshooting such as checking “monitorControl.vhv” in your vmware.log file

Thank you for your reply and my apologies for the confusion. I have added the vhv.allow = “TRUE” line to etc/vmware/config on my physical hostsI have confirmed that hardware virtualization is enabled on the physicsal hosts.I have created guests running ESXi 5.0 by choosing “Red Hat Enterprise Linux 5″ as the guest OS. On these vESXi hosts I have successfully created 32-bit guests. (Hardware ESXi > vESXi > 32-bit VM)

However…I do not have the option to choose “VMware ESXi 5.x” in the edit settings dialog of my vESXi VM.I cannot make 64-bit nested guests off of the vESXi

AllI’m running Esx5 on top of Vmware fusion and it’s working well , but when i try to run 64 bit VM on ESX5 it doesn;t boot and say that 64bit support is not available. Let me know what can be done, Appreciate the help.

I landed at this article searching “installing vm tools on a vESXi host. (one of the comments in this thread, the question was asked….. still searching for that answer…..)

I just got all this working right now..(kind of impressed with myself……so I thought I’d brag to the masses……):Physical box: HPDV8T Laptop(!) (all this running on 1 5400 RPM Laptop HDD! its slow as hell, everything works though (mostly……see lower).Host OS: Windows 7 Ult x64, with VMWorkstation 8.0.(I did try to actually install esx on this laptop but it wouldn’t work…(long story of failure…..)VMW8 VM Guests:2x ESXi5 servers (2cpu’s, 2gb of ram each (just enough for one guest) 1x Open filer 2.99.1 x64 (1 cpu, 1gb of ram) Hosting 2 100GB Iscsi luns (so I can test storage vmotion as well as vmotion).The vc was installed on another machine (8GB is not enough to work with, and as it is, windows, VMW8, three nested guests, are pushing that 1! 5400rpm! HDD to its breaking point……

Wierd things begin to happen in vsphere client connected to vc during migrations:

Hosts start alerting,or disconnecting from VC (greying out in the host list etc). (but not really abending or rebooting etc……

HA fires off alerts that guests should be moved, due to host loosing connectivity to VC.

This all due to the HDD LED on my laptop now solidly lit.

Windows starts to bug out just slightly (can’t move things around on desktop. mouse button starts to stop working on and off as resources are pegged).

I’m exteremely impressed that I got this all working on one laptop running windows (minus vc). An SSD and maybe 12/16gb of ram to play with, and they’d all run (barely…to get one working nested guest…..maybe two).

I’m extrememly impressed that one 5400rpm laptop HDD can push all this.I’m extremely impressed that ESX finally has virtualized Intel VT so that this all works now…….

Now I just need to save up enough cash to do this with 3 physical boxes, so it will actually perform beyond being able to post an article in a board saying “It works! (barely)”.

yeah..tried this on an IBM x3590M2 with no love. HyperV never installs, says it requires a VT enabled processor etc. Did everything, same issue with KVM..installs but can never create the vms. So I think it really is what type of processor you have in the box…will try it on another v5 system…or just reload v5 with HyperV

I just figured out the problem for when a guest gets stuck, and won’t boot. You must use the VMXNET 3 Adapter for nested hosts to boot correctly. This will also require VMware Tools to be installed. I was having a problem nesting Windows 7 64bit as the first layer, then switching it over to ESXi as the Guest OS.

Hey William,As mentioned above I am also getting a black screen when I try to boot the VM.Can you help me here resolving this.I have a x3550 M3 on which I have installed ESXI 5.0.Have added property as mentioned in the link to host.Then I created a new VM with Win2k8R2 and installed the same operating system.100GB Disk, 2 core CPU,8 GB memory for the VM.OS was installed correctly then I shutdown the VM and make the changes as mentinoed for hyperviousr.cupid and VMWare ESXI 5.0 selection.Once I power on the VM and then there is black screen nothing happens after this

HyperV 2008 2008R2 2012 – PROBLEM RESOLVEDSimply just add few options to following1. Shutdown machine and upgrade it to version 9 by right click on Machine name2. Add/change few MachineName.vmx optionsfeatMask.vm.hv.capable = “Min:1″ (add at the end)guestOS = “winhyperv” (change from previous) virtualHW.productCompatibility = “esx” (change from hosted)Power On Machine and use

The times they are a changing…This might help someone trying to do this using ESXi 5.5…I am using ESXi 5.5 and following your instructions did not work. After doing more research, I had to add an additional string to ‘/etc/vmware/config’ to make it work. So in addition to the string you provided: vhv.allow = “true”

, I also had to add the following enable string: vhv.enable = “true”

Also, after you create the vESXi VM, you can go back to edit settings, and change the OS to “VMware ESXi 5.x”.