Hyper-V 2008/2008 R2: Gotchas

Hyper-V 2008/2008 R2: Gotchas

Note: This article is based on Hyper-V 2.0 and might not apply to Hyper-V 3.0 (Server 2012)

TechEd Demo Edit
These are the top "gotchas" or "known issues" that folks seem to have trouble with in Hyper-V. "Gotchas" are sometimes also called "lessons learned" and "best practices." It is a best practice to avoid a known "gotcha". Please add any you have encountered.Applies to:

All versions of Hyper-V

TIP: Subscribe to the RSS feed for this wiki page to get auto-notification when it is updated! To see when this topic was last updated, and with what information, click the "History" tab.

Live Migration Fails if Disk Space Low

When running a live migration using SCVMM 2008, live migration failed with Error (10698) Virtual machine [guest] could not be live migrated to virtual machine host [host] using this cluster configuration. (Unspecified error (0x80004005))

When trying the migration from the Failover Cluster Manager, the job fails with the following;

Event 1205 The Cluster service failed to bring clustered service or application '%1' completely online or offline. One or more resources
may be in a failed state. This may impact the availability of the clustered service or application.

I found that the clustered disk holding the server VHD and resource files was very low on space (2gb free in my case). Also note that the server I'm migrating has a dynamically expandable drive which could be relevant

I then tried to live migrate the storage using SCVMM of one of the machines and the job failed with Error (12700) VMM cannot complete the Hyper-V operation on the [host] server because of the error: [guest] failed to save RAM contents during a snapshot operation:
The I/O operation has been aborted because of either a thread exit or an application request. (0x800703E3). (Virtual machine ID xxx)

Fortunately, I managed to free up disk space by deleting an old test guest VM after which I could freely live migrate once again.

Alternatives would have been to add more disk space or manually move the disk after powering down the box.

Not enough disk space

When setting up your virtual machines, be aware that the virtual machine may try to cache a running copy of memory (using .bin files) to the HDD its configuration info is hosted on. Leave yourself enough HDD space for as much RAM as you wish to assign. Consider
putting the main VM folder on a dedicated drive; apart from your other VHDs, if not using a RAID 5/6 array.

Snapshots fail to merge with error 0x80070070 - This occurs because of low disk space. The resolution is to free up disk space to allow the merge to complete or move the .VHD and .AVHD file to
a volume with sufficient disk space and manually merge the snapshots.

One method to work around this is with Hyper-V R2 and newer is to take a snapshot and export that snapshot to a new virtual machine. This will result in a new VM with a single VHD. Import this Exported VM, verify it works and then delete the original and
its virtual disks.

Virtual machines show a state of "Paused-Critical" - This is cause by a lack of free disk space on the volume hosting the .vhd or .avhd files. The resolution is to free up disk space on the
volume hosting the .vhd or .avhd files, as mentioned in the Hyper-V Snapshot FAQ.

This commonly happens because snapshots quickly increase the amount of storage that is consumed.

Snapshots are lost when the parent VHD is expanded - There is a warning about this in the Edit Disk Wizard, but some folks miss it. This is also discussed in the

Backing up and restoring virtual machines

Hyper-V does not have a built-in method for backing up and restoring individual virtual machines. This requires the use of Windows Server Backup or an external product such as Data Protection Manager, or many of the third party backup products.

Snapshots are not backups. Please see the snapshot topic.

Avoid simultaneous back up operations from the guest and from the host. This usually happens by accident through scheduled backups. The result is heavy disk IO that affects both backup jobs.

This issue is caused by a race condition in the hypervisor. For Windows Server 2008 R2 systems that have Service Pack 1 installed, install hotfix
KB2263829. For Windows Server 2008 R2 systems that do not have Service Pack 1 installed, install hotfix
KB974909.

Other Hyper-V networking related hotfixes to be aware of for non-SP1 systems:

This issue is actually caused by a bug (or in Intel terms “erratum”) that causes these processors to generate spurious interrupts. You can read about this in the latest Intel documentation for this process series:
http://www.intel.com/assets/pdf/specupdate/321324.pdf. To fix this: Install hotfix KB975530 or install Service Pack 1 for Windows Server
2008 R2 (KB976932).

On a Windows Server 2008 or 2008 R2 system that has the
Hyper-V role installed, the startup time increases after backing up Hyper-V virtual machines

VM unable to access the network after it has been configured to use a VLAN ID

This is generally caused by a mismatch between the VLAN ID used by the virtual machine and the VLAD ID configured on the physical network switch. The resolution is to configure the virtual machine to use the VLAN as covered in the Hyper-V Planning and Deployment
guide.

BIOS errors enabling Hyper-V

Some computer BIOS changes require that the machine is physically powered off, and then back on before the change will take effect. Just "restarting" the server is not sufficient. For more information see:

Errors enabling Hyper-V

Most instructions for enabling Hyper-V assume you have not done anything else to the machine beforehand. If you are re-purposing an existing machine, you must take special steps if you are using BitLocker Encryption or Antivirus on that
machine, since both will generate errors when you enable Hyper-V. It is best to turn off both BitLocker and AV before enabling the Hyper-V role. See also:

VM Export/Import permissions issues

Some customers have run into problems when trying to export a VM to a file share and get an access denied error. This is because both the machine account and the user account must have access to the share (often times only the user account
has access). This is explained and demonstrated starting at timecode 2:43 into
this video.

NIC listed as an unknown device in device manager

The cause of this is that the correct Integration Services (formerly called Integration Components, or ICs) are not installed. To fix this, install the correct integration services on the VM according to
these instructions in the TechNet library.

Trying to run Hyper-V on a laptop

Trying to run Hyper-V on a AMD E-350 motherboard

AMD-E350 motherboards provide a very budget cost-efficient way to get a test lab with Hyper-V installed. Ensure that the SATA controller is in AHCI mode; otherwise the OS will hang on bootup once the Hyper-V role is installed.

Trying to PXE-boot a VM using legacy network adapter

Hyper-V VM's are only able to PXE-boot if no bootable disk has been detected. If a disk has been detected that is marked "active", the legacy network adapter will not even attempt to load the PXE-boot ROM and skip through to boot from CD/IDE disk. The priority
of the boot-order does not matter in this case.
If you try to use the same VM for each time you need to test out an OS installation (creating reference images, testing final image deployment and such), it's best to create a snapshot the moment the VM has been created. This allows you to revert back to a
clean disk state very quickly allowing you to PXE-boot again, without needing to delete&create a new vhd.

Disk I/O trouble

Be sure to plan proactively for the performance you require. One guideline is to be generous with RAM - more memory means less disk access. In general, a fixed virtual hard drive is faster than a dynamic virtual drive. Ensure that correct
integration services are installed in the VM.

Running un-needed software, such as anti-virus on the parent partition (also called virtualization
server and management partition)

If the only thing you are running on the virtualization server is virtual machines, why do you need to run anti-virus or other software that you normally run on a Windows Server 2008 R2 server?
Review this short video showing what happens when you enable the Hyper-V role, and change your software installation so that all the VMs get the correct software installed as required by your IT policy. Review
the best practice recommendations at
Planning for Hyper-V Security and evaluate if they fit your environment. If your IT policy requires AV in the management operations system, see the list of Hyper-V AV exclusions in the primary AV exclusion KB for Enterprise customers (KB822158).

When you attempt to create or start a virtual machine, you receive one of the following errors:

The requested operation cannot be performed on a file with a user-mapped section open. ( 0x800704C8 )

Failed to Power On with Error 'The specified network resource or device is no longer available.' (0x80070037).

The I/O operation has been aborted because of either a thread exit or an application request. (0x800703E3)

Slow system logon, sluggish keyboard and mouse

Task Manager shows all CPU cores with high utilization. This problem seems to be related to the version of the network card driver included in Windows 7 or Windows Server 2008 R2 and affects only certain systems, as described in this
article from Intel, which also describes the workaround: http://www.intel.com/support/motherboards/server/sb/CS-031086.htm. The workarounds are to:

Misconfiguration

In fact, OCSetup.exe (as well as ServerManagerCmd.exe) is considered
deprecated in Windows Server 2008 R2 and Hyper-V Server 2008 R2. You're supposed to use new
Server Manager PowerShell cmd-lets. The good thing about these cmdlets is that they are not case sensitive (as everything in PowerShell).

But there may be special considerations when Server Manager PowerShell cmdlets are not enough. Consider the following scenario. You install Microsoft System Center Data Protection Manager (SCDPM) in Virtual Machine. You need to install
Hyper-V Server Role to enable Item-level recovery (ILR) from backed up virtual machines. But installation of Hyper-V role normally requires the presence of Hardware-Assisted Virtualization (HAV). This feature is not available in Virtual Machines. So you need
to bypass this check. In order to do it you have to use another command-line tool that is also new to Windows 7 and Windows Server 2008 R2 and called “Deployment Image Servicing and Management (DISM)”. And this tool is a case-sensitive one. So you should type
the following command exactly as shown.

dism.exe /online /enable-feature /featurename:Microsoft-Hyper-V

Other common misconfiguration issues include:

Not planning for the lack of UI in the free version of Microsoft Hyper-V Server. There is not much difference in resources required between Enterprise/Datacenter with Hyper-V and the free version - Microsoft Hyper-V server. However, if you do not already
have experience managing Windows Server Core installations, the lack of UI may cause some confusion at first.

Not planning for high availability for VMs, and not changing configurations for Live Migration. For example, when a VM in a cluster in hyper-v switches to another node, you may see a configuration error message indicating "virtual network missing". To resolve
these issues you must select the Refresh Virtual Machine Configuration option in the Failover Cluster Management MMS when a VM configuration changes.

For more information and best practices for the design and operation of highly available VMs, see the following documentation:

Active Directory Domain Controllers in VMs

Never save state or pause a DC. Always shut down virtual Domain Controllers properly to avoid replication errors. Pausing a Domain Controller for a long period of time before resuming it can interfere with timely replication.
If you do pause the domain controller for a long time, replication may stop and cause lingering objects.

Backup and restore Domain Controllers the right way

Only use the standard way to backup and restore Active Directory Domain Controller, since the default checks of Active Directory consistency will only be performed when the Domain Controller is aware of a restore.

Using imaging solutions like
Handy Backup, Symantec Ghost, Acronis TrueImage seem like
smarter backup and restore solutions, but they are imaging solutions (in contrast to proper backup and restore solutions.) Restoring an image of a Domain Controller may cause an Update Sequence Numbers (USN) rollback situation to occur, which may lead to replication
problems and other undesired effects. More information
here.

Like shutting down a virtual Domain Controller and start up a version of the VHD file from a while back, you want to avoid improperly using imaging solutions.

To avoid USN Rollback and Invocation ID problems but still using imaging solutions or even copying back old versions of a VHD file (big No-no), be sure to check out this post on
backing up and restoring Active Directory in this unsupported manner.

Beware of pending snapshot merges

When you delete a snapshot on a running virtual machine it disappears from the GUI immediately, but the snapshot is not merged into the parent VHD until the VM is powered off and remains powered off until the merge is complete. This
can cause issues if the volume runs out of space as the AVHD grows and grows. Eventually the VM will pause and you will not have sufficient space to merge the snapshot even if the machine is powered off.

The solution is to monitor for pending deleted merges. This can be done by monitoring this entry in the config file of each VM:

configuration/global_settings/disk_merge_pending = true

Importing a VM into a Hyper-V Failover Cluster

When importing a VM into a Hyper-V Failover Cluster you need to make sure that the import is taking place on the machine/node that is the owner of the Cluster Shared Storage. If you try to do the import on any other machine/node in the cluster, you will
get a general permission denied error.