Primary Menu

Breadcrumbs

Fixing ESXi 6.5 On HPE G7 Servers3 min read

HP, you motherfuckers.

So there I was, moving VMs off of my main host (DL380 G7, 2x x5690’s, 192GB RDIMM), getting ready to replace the 8x 300GB RAID10 array I’ve been using for a while now with some 1TB disks and SSDs, awesome, right?

So the host was powered down, ready for an upgrade, and being the logical guy I am I decided to do some software upgrades.

I used the latest SPP and ran that through, for some reason I was using a BIOS from 2010? Once that was done I had updated firmware, so I moved onto ESXi. I’ve been running 6.0.0U2 for the longest time and thought this would be the ideal time to upgrade.

I got the HPE official ISO, ran through the upgrade, waited for it to boot annnddd….

PSOD.

After some digging, it would appear this is actually a known issue with most G7 servers (unsure about G6).

The issue.

It would appear that one driver is causing this in the 6.5 image, “hpe-smx-provider” (650.03.11.00.17-4240417). Installing the standard ESXI 6.5 ISO does allow the server to boot, but is missing a lot of drivers and does not give the pretty all-inclusive system stats that the HPE ISO does.

So what now? I didn’t want to settle for the standard image, so I did the following to bundle the U2 hpe-smx-provider into the 6.5 image.

For the lazy: There is a link at the end of this post with a download to the compiled ISO.

Using PowerCLI Image Builder.

Once that is all installed, grab both of the following files and put it somewhere easy, I just shoved them in C:\HPE.VMware-ESXi-6.5.0-OS-Release-4564106-HPE-650.9.6.0.28-Nov2016-depot.zipVMware-ESXi-6.0.0-Update2-3620759-HPE-600.9.6.5.7-Dec2016-depot.zip

I also renamed them to make it easier for myself, “ESXIHP65” “ESXIHP6U2”.

Like this:

Post navigation

114 thoughts on “Fixing ESXi 6.5 On HPE G7 Servers3 min read”

Hello, Great post btw! I am having the same issue with my dl380’s. So I have 6U2 and 6.5 HPE images. Did you just extract the iso and save as zip? I’m stuck on step one. I extracted the iso saved contents as zipped ran powercli with same commands and the index.xml file cannot be found. Any help is appreciated!

So the files your looking for are actually the offline bundles for running on the host, not the ISO. These can be acquired on the internet if you search for the packages. They are also on the /r/homeab FTP, you can find that on the Homelab wiki…

I think the main problem here is that even everything is working correct this server isn’t supported for 6.5 by VMware neither by HP. So for homelab or labs this is totally fine but when running this server in production and you have further problems, you would be out of luck when opening a support ticket.

Indeed, however even though it’s officially unsupported it kind of sucks that HP either didn’t bother checking before release or checked and didn’t care. I still see plenty of G7’s in the wild and considering you have to pay for even software updates one would think they would repay that with at least checking their updates…

HP have now released a patch for this now, only a week or so after I posted this so all is well again :). I am getting a lot of questions about this and I can’t seem to find it now… If I do I will make sure to update this post.

So I did a complete reinstall to SD card as I had to replace it. Everything went well with the custom ISO but VC is not showing sensor data. I have removed host from VC and reset entire network to reconfigure it again then added it back. Any thoughts?

Hi.I have faced the same trouble too.I have tried your guide step by step,but it still does no work. (I used upgrade way,not reinstall way) So I’m interested about your firmware version.I used HP SPP released at 2016.10 to updated my DL380 G7 server.My BIOS is using ver 2015,P410i firmware is ver 6.64,and ILO ver 1.88.How about yours?Thanks.

It was apparently the version of PowerCLI I was using. That being said after rebuilding the image I am still getting the PSOD. I am going to try a full install instead of an upgrade and see hwo that goes. THanks again.

Still no go. I had tried adding some other drivers from newer HPEUTils release so one of these may still be causing it. This is really frustrating that HPE won’t release a fixed image. Next round of servers we buy I will vote with my pocket book.

Well I got it now. Initially I was trying to slip in some of the latest HP Updates and do this at the same time. I think my script was overwriting something when I did it. Here is a quick script to do it all.

So, there isn’t a real issue with running the base image however you will be missing some system stats, monitoring and drivers that are baked in.

I pretty much need these things, and whilst you can add them into the stock image with VIBs it is a lot easier just to use the HP image with everything baked in, things like iLO management and RAID management from the ESX CLI can all be done with the HP image. If you also look at the hardware view of the host you will get a *much* better view with the HP image.

So, I tried it, followed the guide and ran the ISO, started the upgrade and got a random directory is not empty error at about 19% and then it stopped… Rebooted the box and ESXi failed to boot, ended up having to restore from a backup before the upgrade. I think I’ll stick with dodgy sensors I think!

Great work! As i am totaly noob to VMs and VMware in general but recived an outsorted ML110 G7 to begin learn this stuff at home, i like to ask: is there a posibility that you share your custom made image with us? Kind regards, Adam

I try the new ISO with the described workaround and it works on a DL385 G7. But under the monitoring of the storage thjere seems to be an issue. There are no metrics available and I see just this text: “The Small Footprint CIM Broker Daemon (SFCBD) is running, but no data has been reported. You may need to install a CIM provider for your storage adapter.”

I too am working on this issue. Does anyone know if this is caused by the smx driver (hpe-smx-provider–600.03.11.00.9-2768847–HPE–VMwareAccepted–2016-01-22) replacing hpe-smx-provider–650.03.11.00.17-4240417–HPE–VMwareAccepted–2017-11-15? I’m working on a comparison of all VIBs but its hard to tell what VIB provides what functionality.

I´ve tried it with a DL360 G7 – new install. It fixed the purble screen issue – but I wasn´t able to get it work. I had problems with the esxi web GUI – I wasn´t able to configure a network to my vm´s … And I´ve got many display problems and it was extremly slow so I decided to install the 6.5 from vm ware again. I really hope, that HP will get it work soon. It makes no fun …

I was unable to boot with any USB device connected, PSODs on VMKUSB — I have read that 6.5 added/replaced the USB drivers with this new module (however removing it doesnt help). If I boot the installer fully, with all USB removed (so yea, have to iLo, PXE, or burn a CD..yuck) then plug in my target USB stick I’m able to install no problem

I would try the generic VMware update and not the HPE one as the VMware ISO wont have any HP specific drivers. VUM should also run a check on compatibiliy issues first. But no guarantees on the outcome 😉

I have 6.5 installed on a ML350 G6 succesfully using the generic VMware ISO, so no HP drivers installed. Now I’m obviously missing out on hardware monitoring. What would happen if I install th hpe-smx-provider-600.03.11.00.9-2768847.vib on my ML? I’m a bit hessitant because I’m not a great fan of PSOD’s 😉

Do you have updated .iso with release 7388607?
This is the last ESXi issued from VMware which I have.
Unfortunately suffers same illness as the previous image but I do not have those vib file “hpe-smx-provider”.
Also do you have by any chance 1.89 BIOS for DL360G7?

I used this on a fresh install and it worked great. I am curious. Can I use this to upgrade from 5.5 to 6.5 or would I need to perform a clean install? I have a few G7s. Here we are nearly a year later and the latest custom ISO doesn’t seem to work.

Hello!
I want to make a fresh build based on the latest releases
VMware-ESXi-6.0.0-Update3-6921384-HPE-600.10.2.0.23-Feb2018-Depot
and
VMware-ESXi-6.5.0-Update2-8294253-HPE-Gen9plus-650.U2.10.2.0.14-May2018-Depot

Hi- This has been working well. I just updated a few G7s to 6.5 U2. It seems that this may have broken the health sensors. Whenever vCenter updates the hosts hardware information (seems to run a few times a day), I see an invalid fault in the logs for the hosts I’ve updated. I have to assume it’s something with a VIB the G7s are using. Any insight?

Got the DL580 G7 running on esxi 6.7 from VMware-ESXi-6.7.0-8169922-HPE-Gen9plus-670.10.3.0.30-Jun2018-depot.zip with hpe-smx-provider injected from VMware-ESXi-6.0.0-Update3-7967664-HPE-preGen9-600.9.8.0.7-Jun2018-depot.zip. So far works OK.

This was great! Thank you for the excellent instructions and to all the contributors in the comments. I too had to remove smx-provider from 6.5U2 (not hpe-smx-provider as stated in article). Other than that driver name, all seems to be okay. YEAH! My G7 with 192GB ram is going to sail along nicely…in lab environment.

Thanks Muffin. We upgraded our SQL Servers to a DL380 G10 and I wanted to repurpose the DL380 G7 as an Application Server Host. Some of the Application Virtual Machines are used for production applications, but I find once you figure out how to work around HP’s failures – in this case, the wonky driver – and VMWare issues, everything usually runs tickety-boo from then on. Right now many of these Application Virtual Machines are running on a DL365 G1 with ESXi 5.1, but it’s getting a little long in the tooth now (just like me), and performance can be a bit slooowww.

I arrived at this site after I tried installing the standard ESXi HP OEM PreGen9 ISO on our G7, and fairly quickly got a PSOD on the console although I could still communicate with ESXi and operated it, but PSODs usually suggest some instability.

I tried the Lazy technique of using Muffin’s ISO next and it seemed stable but I couldn’t get a networking connection. I’d connected both NICs of the G7 using Link Aggregation to two ports on the local switch and figured that something in Muffin’s ISO had a networking issue, perhaps because it was an older version of 6.5.0 than is currently available to me.

Seeing as how networking had worked on my original 6.5.0 ISO, and Muffin’s ISO was otherwise stable on my G7, my third try was to follow Muffin’s incredibly well documented instructions with the latest 6.5.0 HP OEM PreGen9 ISO and the SMX-Provider from the latest 6.0.0 HP OEM ISO and build a custom ISO that I loaded onto a thumb drive, installed freshly onto the SD card of the G7 and now it’s been running great for a few days.

The G7 is still under warranty with HP so it’s very annoying that they don’t directly support this when it’s so simple to make it work, and judging by the number of people who have made it work with Muffin’s and others’ instructions it’s viable software.

On my home server DL ProLiant DL380 G7 (2 x Intel(R) Xeon(R) CPU E5649 / 120 GB RAM) I’ve just upgraded the HPE ESXi 6.0 U3 with VMware-ESXi-6.7.0-8169922-HPE-Gen9plus-670.10.3.0.30-Jun2018 (and applied also the latest ESXi patch ESXi670-201808001 – build 9484548).
After the upgrade, the system was stable from the beginning, after patching, the same.
The mention is that the ESXi 6.7 was automatically upgraded with smx-limited version 600.03.14.00.3-2768847.
I’ve tried to install the esxi6.7uX-mgmt-bundle-3.3.0-10.zip but this over-written the smx-limited version with smx-provider-670.03.14.00.2-7535516, which hanged the OS after reboot.
I’ve Rolled-Back the ESXi and installed only the ams-esxi6.7-bundle-11.3.0-5, which updated the smx-limited to 670.03.14.00.3-7535516, but with the same result, ESXi server is unavailable for management
After roll-back, the system is working properly only with smx-limited version 600.03.14.00.3-2768847.