ESXi

If you plan to update your vCenter Server to Windows 2016 then you might want to make sure you do your homework. Recently after reviewing the following KB its apparent that vCenter Server for Windows 2016 is only supported with vCenter Server 6.5. This might be a great time to consider moving to the vCenter Server Appliance (aka VCSA).

I love the Correlating build numbers and versions of VMware products (1014508). This one KB has made my job, and I’m sure yours too, so much easier. Before this KB was released it was a bit difficult to correlate build, patch, and update levels to vSphere Environments. Now with just a few clicks one can find out all this information and more. However, I really need the ability to correlate multiple core products. Typically, I work with — ESXi, vCenter Server, and vSAN. So, today I took the time today to align all this information.

It took me about 5 mins to build the chart below but it will save me loads of time. Can’t tell you how many times I’ve been asked which version of ESXi was related to which version of vSAN and Oh, what version of vCenter Server was released with it? Well with this cart below you can answer those questions and more.

~ Enjoy!

vSAN version

ESXi version

Release Date

Build Number

vCenter Server

Version

Release Date

Build Number

vSAN 6.6.1

ESXi 6.5 Update 1

7/27/2017

5969303

vCenter Server 6.5 Update 1

7/27/2017

5973321

vCenter Server 6.5 0e Express Patch 3

6/15/2017

5705665

vSAN 6.6

ESXi 6.5.0d

4/18/2017

5310538

vCenter Server 6.5 0d Express Patch 2

4/18/2017

5318154

vSAN 6.5 Express Patch 1a

ESXi 6.5 Express Patch 1a

3/28/2017

5224529

vCenter Server 6.5 0c Express Patch 1b

4/13/2017

5318112

vSAN 6.5 Patch 01

ESXi 6.5 Patch 01

3/9/2017

5146846

vCenter Server 6.5 0b Patch 1

2017-03-14

5178943

vSAN 6.5.0a

ESXi 6.5.0a

2/2/2017

4887370

vCenter Server 6.5 0a Express Patch 1

2/2/2017

4944578

vSAN 6.5

ESXi 6.5 GA

11/15/2016

4564106

vCenter Server 6.5 GA

11/15/2016

4602587

vSAN 6.2 Patch 5

ESXi 6.0 Patch 5

7/11/2017

5572656

vSAN 6.2 Express Patch 7c

ESXi 6.0 Express Patch 7c

3/28/2017

5251623

vCenter Server 6.0 Update 3b

4/13/2017

5318200/5318203

vSAN 6.2 Express Patch 7a

ESXi 6.0 Express Patch 7a

3/28/2017

5224934

vCenter Server 6.0 Update 3a

3/21/2017

5183549

vSAN 6.2 Update 3

ESXi 6.0 Update 3

2/24/2017

5050593

vCenter Server 6.0 Update 3

2/24/2017

5112527

vSAN 6.2 Patch 4

ESXi 6.0 Patch 4

11/22/2016

4600944

vCenter Server 6.0 Update 2a

11/22/2016

4541947

vSAN 6.2 Express Patch 7

ESXi 6.0 Express Patch 7

10/17/2016

4510822

vSAN 6.2 Patch 3

ESXi 6.0 Patch 3

8/4/2016

4192238

vSAN 6.2 Express Patch 6

ESXi 6.0 Express Patch 6

5/12/2016

3825889

vSAN 6.2

ESXi 6.0 Update 2

3/16/2016

3620759

vCenter Server 6.0 Update 2

3/16/2016

3634793

vSAN 6.1 Express Patch 5

ESXi 6.0 Express Patch 5

2/23/2016

3568940

vSAN 6.1 Update 1b

ESXi 6.0 Update 1b

1/7/2016

3380124

vCenter Server 6.0 Update 1b

1/7/2016

3339083

vSAN 6.1 Express Patch 4

ESXi 6.0 Express Patch 4

11/25/2015

3247720

vSAN 6.1 U1a (Express Patch 3)

ESXi 6.0 U1a (Express Patch 3)

10/6/2015

3073146

vSAN 6.1

ESXi 6.0 U1

9/10/2015

3029758

vCenter Server 6.0 Update 1

9/10/2015

3018524

vSAN 6.0.0b

ESXi 6.0.0b

7/7/2015

2809209

vCenter Server 6.0.0b

7/7/2015

2776511

vSAN 6.0 Express Patch 2

ESXi 6.0 Express Patch 2

5/14/2015

2715440

vSAN 6.0 Express Patch 1

ESXi 6.0 Express Patch 1

4/9/2015

2615704

vCenter Server 6.0.0a

4/16/2015

2656760

vSAN 6.0

ESXi 6.0 GA

3/12/2015

2494585

vCenter Server 6.0 GA

3/12/2015

2559268

If you like my ‘no-nonsense’ blog articles that get straight to the point… then post a comment or let me know… Else, I’ll start writing boring blog content.

These are pretty common asks amongst operators of ESXi – ‘Which VMtools version came with my ESXi Host’ and ‘Where can I view and download all the VMTools directly?’ The answers are below and the outputs aren’t pretty but they sure are useful!

1st – Check out the URL below to see all the ESXi Host build to released versions.

One of the joys of working with a home lab is doing something that no one has done before. Sure, your configuration might be similar to others, but in a way your home lab is unique. However, with this uniqueness comes its share of installation challenges. My new lab was no exception, there were a few challenges and one major issue I uncovered while setting up this new environment. In this blog post I am going review the environment I am working on, break down some of the hardware layout placement challenges, fun using the MAC PowerBook to complete the installation, and finally overcoming ESXi installation challenges.

Here is my new environment:

Mac Powerbook with macOS Sierra (Used for remote connection into my environment, normally I use a PC)

Gigabyte MX31-BS0

Intel Xeon E3-1230 v5

32GB DDR4 RAM

1 x Mellanox Connectx InfiniBand HCA

4 x 200GB SSD, 1 x 64GB USB (Boot)

1 x IBM M5210 JBOD SAS Controller

1 x Mini SAS SFF-8643 to (4) 29pin SFF-8482

1 x 64GB USB Boot Stick:

Hardware layout/placement challanges:

32GB of RAM: Pay attention to the placement of the RAM. As Channel 1 for the RAM are the two closest slots to the CPU, channel 2 being the two farthest away. Normally you would place the RAM pairs in like colors however this Mobo is a bit different

Mellanox Connectx InfiniBand HCA: Placed it in the 16x slot right next to the CPU. The HCA requires an 8x slot so this slot should not slow it down. No BIOS changes were required and I could see this HCA in the BIOS.

IBM M5210 JBOD SAS Controller: Placed it in the 8x slot which goes through the C232 chipset on the motherboard. Next, I needed to update the firmware but this proved to be a challenge. Keep in mind the M5210 with NO cache will not allow you to enter its BIOS management page (aka MegaRAID webbios). This means you’ll need to use the command line or other software to update and view its information. Initially, I tried several command line options (UEFI Shell, DOS CLI, etc.) with the MegaRAID CLI but I just could not find the right combination to get it to work. My solution — I simply used an older SSD drive, installed Windows Server 2012 on it, and used the Windows exe to update the firmware. It worked perfectly with no issues.

After the update, I had some issues decoding the M5210 running firmware version vs. the vSAN HCL. As you know when running vSAN in a home lab the closer you are to the HCL and vSAN HCL the better. (NOTE: as I’m sure you know production environments MUST match the HCLs). The published firmware version on the vSAN HCL is 4.660.00-8218. However, when the M5210 boots it shows 24.16.0-0104.

Solution: When you are looking at the boot screen you are seeing the FW Package number not the Firmware of your controller. Simply look at the release notes for the ‘FW Package’ and you’ll find the correct MR FW versions that match the vSAN HCL.

Boot Screen

Release notes

200GB SSD: The Sonata cases I am using are a bit dated but they fully meet my needs so there is no need to replace them. There are 4 x 3.5″ bottom mount disk trays in each case. Bottom mount means you insert your 3.5″ drive into the tray and bolt it to the tray from the bottom. I bought several 3.5″ to 2.5″ converters which will allow me to mount my 2.5″ SSDs. However, the converters didn’t have bottom mount holes that lined up with the standard 3.5″ holes. Fix — I used a hole in the existing tray to secure the converter to the tray. I also made sure I mounted the converter as far back as I could to ensure the SAS cables would not be on the side of the case. This mount position moved the drives back about 1.5″(38mm). The red line in the PIC show where the original mount point was.

Mini SAS SFF-8643 to (4) 29pin SFF-8482: From the PIC above you can see the disk end of the SAS cables. What is nice about them is each one has a disk number labeled and has integrated power and all 4 drives go back to a single connector. The only downside to the cable I bought was they seemed a bit frail, so I’d recommend if you plan to mod your environment frequently look into a better-quality cable. If you interested more in SAS and the associated cables I would recommend this wiki page – https://en.wikipedia.org/wiki/Serial_Attached_SCSI

64GB USB Boot Stick: I decided to use the internal USB port freeing up the rear ports for other items. The USB stick I am using is the SanDisk Ultra Fit 64GB USB 3.0 Flash Drive. ESXi will only take up ~10GB of this stick, so is 64GB overkill? Keep in mind I plan to run vSAN 6.6.x and one of the benefits is the log files now write to RAM and in case of a system failure, they can write these logs to the USB stick. However, the default partition sizes (2.5GB for diags) might not be large enough. The vSAN team as released a nifty script that will estimate and resize you USB partitions. I’ll cover this topic in later posts and show you how to “auto-resize” your USB storage after you have installed vSAN.

Fun with the MAC:

Function Keys: One of the challenges was MAC keyboard mapping into the remote KVM. For some reason, the function keys on a MAC always assume you want to their special function vs. the F# key you are pushing. This proves to be a challenge when you are trying to pass standard function keys. Simple fix: System Preferences > Keyboard > Ensure ‘Use F1, F2, etc. as standard function keys’ is checked.

Another option for F# keys is to create a macro inside of the vKVM Viewer to pass the key. The screenshot below shows where you can setup user defined Macros and in the background is the MeregPoint console for one of my ESXi hosts.

Java: One of the joys of this motherboard is the use of vKVM viewer and VM Media. However, these functions need JAVA installed and up to date to function properly. If your JAVA is behind, trust me just update it’ll save you hours of pain. Here is the remaining gotcha. In the Mergpoint web page, you simply click on the ‘Launch Java vKVM Viewer’ button to start your host remote session. The webpage will download a .jnlp file. If you just click on this file you are presented with an error stating it can’t be opened because it is from an unidentified developer. Solution – After the java app downloads, click on the down arrow next to the file and choose ‘Show in finder’. When finder launches select that file by holding down the control key and right-clicking on it. A pop-up window will appear, release the control key and finally choose open. This allows you to override the ‘unidentified developer’ error and launch the viewer.

ESXi Installation:

Setting up the ESXi hosts had one big challenge – after the install of ESXi I could not see my SAS disks. I am using the ESXi 6.5U1 Rollup.iso to do my installs and my main goal was to install and boot ESXi from the 64GB USB stick and be able to access the 4 x 200GB SSD attached to the IBM M5210

Problem – During the install of ESXi, I booted the host using the ESXi6.5 ISO via virtual media console. The installer program would recognize the IBM M5210 controller, the attached 4 x SAS disks, and the 64GB USB stick. The installation would complete without issue. However, after ESXi booted the SAS disks and the controller would not appear but I could see the 64GB USB stick.

Other observations –

First, in the ESXi Log files I noticed the megasas was having firmware issues:

2017-09-21T10:26:31.310Z cpu5:66065)<6>megasas: Waiting for FW to come to ready state 2017-09-21T10:26:31.310Z cpu5:66065)<7>megasas: FW in FAULT state!!

Second, I use Partition Wizard bootable ISO to remove all partitions prior to installing ESXi. I noted that sometimes after I booted to it as virtual media it would see the 4 x SAS disks and other times it would not.

Third, Installation of ESXI onto SAS or SATA SSD as the boot disk worked perfectly. After booting I could see the M5210 and SAS disks but my goal of using the 64GB USB stick for the boot device was not achieved.

Fourth, occasionally when I booted the ESXi host to the USB stick it would work okay, but upon reboot would not

Final Solution – The core reason why I could not see the SAS disks with ESXi or Partition Wizard was the boot type was UEFI and not legacy. During boot time the boot order would sometimes change if I had virtual media connected, meaning sometimes it would boot the 64GB USB stick or Partition Wizard as UEFI and other times as legacy. Apparently, UEFI boot was giving the M5210 firmware issues not allowing the SAS disk to come online.

FIX – I went into the BIOS of the motherboard > Advanced > CSM Configuration > changed ‘Boot option filter’ to ‘Legacy Only’ and all my issues went away.

Summary – I spent a lot of after-hours and weekends working out all various installation tweaks but what can I say, this is the joy of setting up a home lab! My hopes are in some way this post helps you move your home lab forward too. In my next post, I’ll be going over how to enable the InfiniBand HCA in ESXi 6.5.

If you like my ‘no-nonsense’ blog articles that get straight to the point… then post a comment or let me know… Else, I’ll start writing boring blog content.

I have decided to move my Home Lab away from Gen III into Gen IV. In doing this I am going to follow my best practices laid out in my ‘Home Lab Generations’ and ‘VMware Home Labs: a Definitive guide’. As you read through the “Home Lab Generations page” you should notice a theme around planning each generation and documenting its outcomes and unplanned items. In this blog post, I am going to start laying out Design Considerations which include the ‘Initial use case/goals and needed Resources as they relate to GEN IV.

Design Considerations:

Initial Use case / goals:

Support vSphere 6.5 and vSAN 6.6 All Flash Environment

Move networking vSAN and FT to high-speed InfiniBand

Support headless ESXi Environment

Resources needed:

To meet the initial use case/goals I’m will be investing quite a bit into this total refresh.

Some initial GEN IV resource choices (Still in the works and not all proven)

Mellanox IS5022 – I liked the footprint of this device over it not having built-in IB subnet manager. An IB subnet manager is required to manage the “network paths” on an IB network. Without an active IB subnet manager available your IB HCAs will not connect. Since the IS5022 I chose does not have an IB Subnet manager I will need a place to supply this service. I can choose an ESXi Host or a Windows Host. Since my ESXi hosts will be going up and down I plan to use my Windows PC as my subnet manager as it is always on and available.

Mellanox ConnectX HCA rebranded as HP INFINIBAND 4X DDR PCI-E HCA CARD 452372-001 – I initially choose these HCAs based on some other blog posts. They are at an attractive price point but they are much older and no longer have driver support. I was able to get them operational with ESXi 6.0 and will be soon working with them on ESXi 6.5 My advice is Mellanox has great products and support, however, I would recommend if you can afford it to go with a newer card that supports ESXi and save yourself the trouble of modifying ESXi software vibs. I’ll post more on this topic as I start deploying them with ESXi 6.5

Windows PC – I repurposed my Gigabyte Z68XP-UD3, i7 CPU, 90GB SSD, 16GB of DDR3 and then bought a cheap BitFenix case to build out my Windows PC. This PC will serve as my Plex Home Media Server and IB Subnet Manager. I also plan to run Workstation 10 and set up various service VM’s (AD, vCenter Server / VUM, DNS, etc). So far it’s working pretty well but this Mobo has been known to give me issues.

ESXi Hosts – I have 3 Antec Sonata cases, one that I have had since 2003 that I will reuse in this environment. I choose the following parts to make up my new ESXi Hosts: Mobo: Gigabyte MX31-BSO, 32GB DDR4, E3-1230v5 Xeon, and 4 x 200GB SSDs per host. This mobo is a bit limited on the ports but so far it seems to be working out well. For the boot disk, I plan to use the onboard USB port and a 64GB USB Stick. However, the question should be – What am I going to so with those 6 x 1TB SATA disks from GEN III — I put them into my old IX4

Here are a few PICs of the current build:

Next Post I’ll be going over the Gigabyte Motherboard.

If you like my ‘no-nonsense’ blog articles that get straight to the point… then post a comment or let me know… Else, I’ll start writing boring blog content.

ESXi no longer supports Realtek Network drivers. To be able to install ESXi with the Realtek drivers users will have to create a custom ISO. Keep in mind these are unsupported drivers by VMware, so use at your own risk. Normally I would use my trusty ESXi-Customizer GUI program but it is no longer supported for ESXi 6. However, their CLI is still supported but I’d rather rely on the VMware supported tools vs. 3rd party as you never know when 3rd party tools will go away. VMware enables users to create custom ISOs via PowerCLI and in this blog I’ll explain how I used PowerCLI create my ISO. Keep in mind these are the steps that worked for me, your environment may vary.

To get started you will need two files and PowerCLI Installed on a Windows PC.

If you have ever had trouble identifying specific hardware devices (NIC’s, Storage Adapters, Disks, etc.) on the vSphere compatibly guide the following command is for you! As I’m sure you know, there can be multiple device descriptions listed on the VCG and it can be confusing to know which device matches the device you have. When this happens, it is necessary to locate the following ID’s:

VID = Vendor Id

DID = Device Id

SVID = Sub-Vendor Id

SSID = Sub-Device Id (aka Max SSID)

These IDs enable users to pinpoint the device without question and help to ensure you have the correct hardware guidance. VMware’s guidance to pinpoint devices is outlined in KB1021534 This KB is very helpful and its where I found my new favorite command. With this one command it will produce nearly all the IDs in one shot vs. multiple commands.

vmkchdev -l << that is -l as in lama

In this example, I am looking to identify vmhba1.

I simply input the ID numbers into the VGC and it pinpoints the product, NO more guessing.

Summary – By using this simple command and the vendor ID you’ll be sure you are identifying your hardware devices properly.

If you like my ‘no-nonsense’ blog articles that get straight to the point… then post a comment or let me know… Else, I’ll start writing boring blog content.

One question I get a lot as a TAM is – “How can users of VMware products, such as vSAN, stay current with various service advisories?” One very simple way is to enable your my.vmware.com profile for product subscriptions. Product subscriptions allow you to choose products of interest and then you will receive emails around the various service advisories. It’s a very simple way to keep up to date and it only takes a few minutes to enable.

I currently have an IOMega ix4-200d with 4 x 500GB Hard Disk Drives (HDD). I am in the process of rebuilding my vSAN Home lab to all flash. This means I’ll have plenty of spare 2TB HDDs. So why not repurpose them to upgrade my IOMega. Updating the HDDs in an IOMega is a pretty simple process. However, documenting and waiting are most of this battle.

There are 2 different ways you can update your IOMega: 1 via Command Line and 2 via the Web client. From what I understand the command line version is far faster. However, I wanted to document the non-command line version as most of the blogs around this process were a bit sparse on the details. I started off by reading a few blog posts on the non-command line version of this upgrade. From there I came up with the basic steps and filled in the blanks as I went along. Below are the steps I took to update mine, your steps might vary. After documenting this process I can now see why most of the blogs were sparse on the details, there are a lot of steps and details to complete this task. So, be prepared as this process can be quite lengthy.

NOTES:

YOU WILL LOSE YOUR DATA, SO BACK IT UP

You will lose the IOMega configuration (documenting it might be helpful)

Here are the steps I took:

Ensure you can logon to the website of your IOMega Device (lost the password – follow these steps)

Backup the IOMega Configuration

If needed screen shot the configuration or document how it is setup

Backup the data (YOU WILL LOSE YOUR DATA)

For me, I have an external 3TB USB disk and I used Syncback via my Windows PC to back up the data

Remove the non-boot 500GB disks from the IOMega and label them (Disks 2-4), Do not remove Disk

Special Notes:

From what I read usually Disk 1 is the “boot” disk for the IOMega

In my case, it was Disk 1

For some of you, it may not be. One way to find this out is to remove disks 2-4 and see if the IOMega Boots, if so you found it, if not power off try with only disk 2 and so on till you find this right disk

Replace Disks 2-4 with the 2TB HDDs

Power on system (Don’t forget to plug it back in)

The IOMega display may note there are new disks added, just push the down arrow till you see the main screen

Also at this point, you won’t see the correct size as we need to adjust for the new disks

Go into web client

Settings > Disks Storage

Choose “Click here for steps…”

Check box to authorize overwrite

About a minute or two later my IOMega Auto Restarted

Note: Yours may not, give it some time and if not go to the Dashboard and choose restart

After reboot, I noted my configuration was gone but the Parity was reconstructing with 500GB disks

This is expected, as the system is replicating the parity to the new disks

This step took about 12 hours to complete

After the reconstruction, I went into the Web client, type in the device name, time zone, email, and then it auto Rebooted

After Reboot note all the disks are now healthy and part of the current 1.4TB parity set. This is expected.

Now that the Iomega has accepted the 3 x 2TB disks we need to break parity group and add the final 2TB HDD

First, you have to delete the shares before you can change the parity type.

Part of the joy of building out a home lab or virtualization workstation is finding those one-off items that enable you to build something great, cheap, and unique. Below is a list of some those niche items and distributors I’ve found along the way. I’ll continue to update this post as we go along and I encourage you post up some of your findings too!

This place is full of all types of unique adapters and gadgets to make your home lab or workstation PC better. What I like about their product line focus is the support of cards with a PCIe x1 slot. Various server based add on cards (example 2/4 port NIC cards) typically require a PCIe x4 or x8 port. However, most home labs typically have plenty of x1 slots and very little to no support for x4 and x8. Syba seems to make a “plethora” of add on cards that support x1. There only downside poor documentation / support.

Some products I like from them —

2 Port Gbe PCIe x1 card (SY-PEX24028): I own and use several of these, they seem to work quite well. Dislikes – No Jumbo frames and it uses a Realtek 8111e chips set which means you must add these drivers to support ESXi

Another cool item they make is a M.2 to 4-port SATA III Adapter. This little RAID controller allows you to plug directly into a M.2 port and allow for 4 mort SATA devices. I think this would be handy for smaller systems (ie. NUC builds)

StarTech is really becoming a great company with a very diverse and well supported / documented product line. I think they are really starting to give Blackbox a run for their money. I really like their cable and adapter card lines.

Winyao is a “boutique” distributor specializing in NICs, Fibre adapters, and Transceivers. One item I find of value is their PCIe x1 Dual NIC with Intel or Broadcom chipset. Personally, I don’t know much about this company or own any of their products, but at $40-$60 per brand new adapter I wished I had found them before buying the Syba adapters.

If you are looking for your next server, workstation, media, or top of the line PC case then take a peek at Fractal Design. Founded in 2007 and based out of Sweden they have really started to dominate the custom case design market. Their innovative designs blend elegance with flexibility, which I might add is a hard combination to find. I like their Arc Midi and Arc Mini R2 line of cases for home lab build outs. However, when or if my trusty Antec Sonata from 2003 lets me down, then Fractal will be next on my list. Here is a great blog post from Erik Bussink around his use of Fractal Design for his 2014 Home Lab.

Came across this interesting case / mod company that builds all kinds of custom cases, cables, etc to mod your PC’s. I like the Prodigy Mini-ITX case, with 2 PCI Slots and a spare slot for Disk or other mods it could be a good fit for a NAS project. However I’m not fond of the excessive top and bottom ornaments.

ASUS

ASUS came out with a great M.2 to U.2option allowing users to interface with SAS disks. They claim this option will help users to interface with SAS SSD and get extreme performance. There are some contrains around this (cables, disks, chipsets, etc) so read up on this before you buy.

If you like my ‘no-nonsense’ blog articles that get straight to the point… then post a comment or let me know… Else, I’ll start writing boring blog content.