Saturday, February 27, 2016

As users of new Skylake (LGA1151) systems are discovering, Intel has completely removed EHCI support from the new architecture. XHCI (USB 3.0) is supposed to be completely backwards compatible to USB 2.0, but the lack of EHCI support has some less than pleasant effects on trying to boot from USB using any OS that is expecting USB 2.0 support. Specifically, this means that GRUB 2 cannot currently boot an OS on XHCI-only systems, which makes installing FreeNAS a bit of a pain.

The symptom of this problem is that on XHCI systems the boot process will proceed up to the point where it tries to mount the root filesystem, and then it will die with an "error 19".

This is actually a problem that affects all XHCI systems, but if your system supports both EHCI and XHCI, you can disable XHCI in the BIOS to make USB booting work. Skylake systems, however, have no EHCI support at all, not even on the USB 2.0 motherboard headers, so this workaround isn't available.

Some people have found success with PCI cards that add EHCI USB ports, but you have to use caution with this approach since many (most?) PCI USB cards don't provide bootable USB ports. I didn't want to have to go pick up extra hardware just to install the OS, so I've opted for another approach: load the installer over the network via PXE.

The FreeNAS developers use PXE booting when testing new builds, and there is a guide for doing this with FreeNAS 9.2. However, the guide is two years old and I found it to be missing several steps when trying to apply it to a current version of FreeNAS. It's even worse when trying to use current versions of the FreeNAS developers' tools, as they're completely missing large sections of setup instruction (they're clearly not intended for use outside the project).

So, I'm publishing an update to the guide here. Eventually this will be out of date too, but hopefully it will save someone time down the road.

If you want to follow this guide you will need:

a FreeBSD server which will be your PXE and DHCP server

a machine you want to install FreeNAS on (presumably you already have this, since you're reading this guide)

Set up the BIOS

You'll want to modify your system BIOS boot order on the NAS host to make sure that PXE (or Network) boot is enabled, and will be attempted before any other valid boot option (e.g. if there's an OS on any disk in your system, that disk should be ordered after the PXE boot). Exactly how you do this is going to be specific to your BIOS.

Setting up the DHCP Server

Install the isc-dhcp43-server package, and use a config file that looks mostly like the following. Update it for the subnet you use on your network: "next-server" should be the IP address of your PXE server.

Boot!

That's it. You should now be able to boot the installer over the network, and install FreeNAS on a disk installed in your NAS server. Don't forget to consult the FreeBSD handbook section on diskless booting if you need help troubleshooting anything. After installing, you may need to alter the boot order again to ensure that your freshly installed OS is booted before PXE.

During the day, and often at night, Matt works as a DNS infrastructure operator. He is the former DNS Operations Manager for the Canadian Internet Registration Authority (the organization that manages the .ca Internet domain), has worked DNS operations at Afilias (.info, .org, and several others), has served on the board of directors of DNS-OARC, and currently works as the Sr. DNS Engineer for Rightside (.ninja, .rocks, and many others) ... though he doesn't speak for any of the above here.

He enjoys live music, good beer, and speaking about himself in the third person.