yadr – A vdisk reporter

I know there are already numerous scripts to report on virtual hard disks and most of them without a doubt much better then what I came up with for this post.

The reason I started with this script was a question in the PowerCLI Community from Alan in his Thin Provisioned Disks post. He wanted to know if you could get the provisioned and the allocated disk size for a thin provisioned virtual disk.

From past experience I knew that the answer was to be found in the layoutEx property of the VirtualMachine object.

But while I was writing the script I saw opportunities to expand the script to make the produced report the “mother of all virtual disk reports”. Or at least an attempt to include much more information in the produced report 😉

The layoutEx property

The layoutEx property, that came with API 4.0, contains a wealth of information concerning the virtual disks connected to your guests.

Since you can find back the mapping to all individual files that constitute a virtual disk, it is the ideal property to show the actual storage used by each virtual disk and the snapshots.

The following graphic shows how the three major properties in the VirtualMachineFileLayoutEx object are organised. The values in the graphic are from a virtual machine, with one virtual hard disk, and two snapshots.

Handling of snapshots

I assume that most of you know what happens in the background with a virtual disk when you take a snapshot of a virtual machine. The following graphic is small aide-mémoire for understanding the basics.

If you look closely at the Snapshot table (first graphic) you will notice that an entry points to the files, or chain of files, before the snapshot was taken.

For example, Snapshot 2 results in the FileKey sequence 0, 1, 2 and 3. While the actual new files that are created for Snapshot 2, are those with FileKey 4 and 5.

Since I wanted to know how much datastore space Snapshot 2 was taking, I decided to let the script look for the files linked to Snapshot 2. In this case that would be the files with FileKey 4 and 5.

The script

There were a few problems to be solved:

Not all virtual machines have the same number of snapshots. If the data needs to be exported to a CSV file this poses a problem with the Export-Csv cmdlet. The solution I took:

For all guests determine the number of snapshots, if any, they have and take the maximum of these numbers.

Define a new object, I called it DiskInfo in the script, and define a sufficient number of snapshot-related properties (Name and Size) so that the guest with the highest number of snapshots can be represented.

Define the new object in C# and use the Add-Type cmdlet to create the new type.

Like I mentioned earlier, I wanted to display the actual datastore space a specific snapshot takes. For that I had to find the FileKey values that are not in the Snapshot table. My reasoning for this was that I wanted to know how much datastore space would become free when a specific snapshot is removed.

Annotations

Line 1-11: The “static” part of the C# code that defines the DiskInfo type.

Line 13: Get all the virtual machines. I used the Get-View cmdlet for speed and also because the script only needs properties from the VirtualMachine object.

Line 14: Determine the highest number of snapshot present on any guest.

Line 15-18: The “dynamic” part of the C# code. For each snapshot this adds two properties, SnapiName and SnapiMB,

Line 19: The “newline” character is not really. Just gives a better view when you dump the content of the $DiskInfoDef variable.

Line 21: The new type is created. Note that when you change anything in the definition, you will most probably have to restart your PS environment, otherwise you will get a message that the type already exists. There is, afaik, no way to get rid of a type definition in the .Net environment.

Line 23-30: This filter will recursively run through the snapshot tree and will create a hash table. The table will later be used to find the snapshot name.

Line 32-98: The main loop.

Line 35-37: Create a hash table from the snapshot tree.

Line 39: Loop through all the virtual disks connected to this virtual machine.

Line 41-43: Get all the FileKeys for this specific virtual disk

Line 46: Get the name of the first disk file.

Line 48-50: Summarise the size of each file that comprises the virtual hard disk.

Line 52-55: Check if the virtual hard disk is Thin Provisioned. If it is, calculate the percentage of storage that is actually allocated on the datastore.

Line 57: Create a new object based on the DiskInfo type.

Line 58-65: Start filling in the values for the properties of the DiskInfo object.

Line 66-94: Handle the snapshots (if present) for this virtual hard disk.

Line 67: Start by checking if there is at least one snapshot for this virtual hard disk.

Line 68: Loop through all the snapshots.

Line 69-71: Get the FileKey for all the files for this virtual hard disk before the snapshot was taken. Store these FileKey values in the $prevFiles array.

Line 72-78: Find the first pair of FileKeys in the chain that is not in the $prevFiles array. These are the files that constitute the snapshot.

Line 79: Store the pair of FileKeys in the array $snapFiles.

Line 82-89: Calculate the total size of the snapshot files and get the name of the snapshot from the hash table created in the beginning of the main loop.

Line 90-91: Store the values in the correct properties in the DiskInfo object.

Line 97: Export the array to a CSV file.

Result

The following screenshot shows a sample run of the script.

There are some interesting results in this sample run.

The maximum snapshots any guest has is two.

Server1 (row 2-5) has three virtual hard disks. The first one is thin provisioned and uses currently 47.7% of it’s capacity.

The other two hard disks of Server1 are RDM disks. Notice that the AllocatedMB column says “zero” because there is not really a point in talking about allocated space for RDM disks.

The guest in row 8 has two snapshots.

And last but not least, I suspect there is a bug in the VirtualMachineFileLayoutEx object, more specifically in the Size property of some entries in the Files array.

Have a look at row 20, this disk is not a thin provisioned disk but the AllocatedMB value is less then the CapacityMB value. I would have expected the same value in those two columns.

When I investigate from the console with a simple “ls -l” I see the correct value.

For one reason or another, there are some (very few) virtual disks that have an incorrect size values in the Files table. I couldn’t find out in which cases this seems to happen.

@SAL, that is a problem with the object types you add with the Add-Type cmdlet. These types can not be removed nor renamed.
There are 2 solutions, you stop/start your PowerShell session, or you add a random number to the typename.
That last option is the one I took in LUN report – datastore, RDM and node visibility.
In this script you can change these lines
$DiskInfoDef = @"
public struct DiskInfo {
public string VMname;
public string Label;
public string Path;
public bool Thin;
public bool RDM;
public string Used;
public string CapacityMB;
public string AllocatedMB;
"@

Martin

March 16, 2012 at 16:21

Hi LucD, Many thanks for a great site!
I’m new to PowerCLI but is possible to also report on the space used even if the disk is thick provisioned (rather than na)? This would show the potential disk savings you couls get by converting to thin provisioned.
(I realise that this script is a couple of years old now but assume it hasn’t been superseded by something better)?

Amit Patange

February 14, 2012 at 13:28

Hi Luc, thanks for the script, it fantastic, I was wondering – is there a way to get the output in one line per vm like:
VMname, vmhost, HardDisk1, Path, all other details, HardDisk2, path, all other details

txolson

February 3, 2012 at 17:51

Hello — Ran script yesterday no problem. Today, it only shows max. of one snapshot on any VM although I have several with 2. I added 2 snapshots to another VM and it reports it properly, but still only reports one on the others.

Any ideas?

I am testing by using lines 13 & 14 (have also run entire script with same results)

Robert

January 25, 2012 at 13:57

Thank you for this great script
Replacing line 13 with:
$vms = Get-Datacenter $datacenter| Get-VM | Get-View | sort name
makes more sense as it will not get templates. If you have any template in the collection you get an error as mentioned in some comments here.

@MikeP, I’m afraid not. The script uses the Snapshot property in the VirtualMachine object to find the snapshots.
If the chain of snapshots is not correct, the script will not show these broken snapshots.
I suspect you will have to have a look at the headers of some of the .vmdk files to retrieve “broken” snapshots.

steve

February 23, 2011 at 20:03

Luc,

I am trying to use your script to return the guest.disk.diskpath and it keeps coming up empty. I defined “public string Drive” in the Diskinfo and added $diskinfo.Drive = $vm.Guest.Disk.DiskInfo and it did not work. I should note that the machine I am testing on has multiple drives.

@Steve. The yadr script looks at the virtual disks as they are seen by the ESX(i) host for a specific guest.
The DiskPath you mention is the path to the partition defined inside the OS running in the guest. This information is provided through the VMware Tools but there is no link to the virtual disks. To make that link the script would need to find a method to link these OS partitions to virtual disks. That is currently not possible with the PowerCLI cmdlets nor by any of the SDK methods.
Arnim did an excellent write-up on this in his PowerCLI: Match VM and Windows harddisks – Part 2 post.

Paulo Aguiar

Paulo Aguiar

October 14, 2010 at 19:43

Hello LucD:
After see this amazing script I immediatly try it and worked pretty fine but I am in a need to only use the part where you do the RDM analysis and merge it with my script I am trying to do it but I have been failing could you give me a hand with this buddy?
This is the script I am actually using:

Hi Luc, I´ve encountered an error if the vCenter Installation is localized (i. e. German), so the Hard disk gets back as Festplatte.
There is an easy change to run the script in any localized VMware environment, I wanna share.

Vuong Pham

May 12, 2010 at 23:24

This is a fantastic YADR!!
calculations:
Trying to decipher the data to see how much disk is actually saved via thin disk ..
I sort out the thin= false
…
or figure out what the over-allocated value is.
—
Do a rollup report to management.. “with thin disk prov.. we’ve now saved X disk space and this month.. etc” and “translates to $$ saving”

— more interesting would be to trend this as a rate to calculate how much disk usage is happening even under thin provisioning..

Andrea

April 8, 2010 at 02:42

This is great LucD!
Interestingly I got the following error, looped 7 times in my environment the first time I tried your script:
“Cannot index into a null array.
At C:\yadr.ps1:46 char:33
+ $vmdkName = $vm.LayoutEx.File[ <<<< $diskFiles[0]].Name
+ CategoryInfo : InvalidOperation: (2:Int32) [], RuntimeException
+ FullyQualifiedErrorId : NullArray
"
I had some templates that had not been labeled as VMs since moving to vSphere. Their "provisioned" and "used" values in VIC were n/a. converting to VMs filled in these values and switching them back to templates kept the values, no more script errors.

I'm sure there's a quick error check you could do for null values coming in. I just thought I would pass back this info.