Exadata Security: the joy of PXE and UEFI: Secure boot violation

Today I had the honour to join my colleague Freek D’Hooge (find him here: @dhoogfr )in the datacenter. Task for today, reimage an 1/8th X7 Exadata using PXE. Andy (Colvin) warned me already that, when trying to reimage them, they are now using UEFI – boot. Knowing this in advance, this would be an #exatastic day.

As usual, Andy was right. Trying to boot one of the cells on the pxe “the old method” resulted in:

Step 1, find your sole source of truth … my oracle support. This time, close but no cigar. The MOS-note “How to setup a PXE Boot Server to Re-Image an Exadata Compute Node (Doc ID 1577323.1)” was not complete. It is lacking the information on how to do a PXE boot with an UEFI system.

Because it’s a bit long, I’ll highlight the steps we had to take to convert our legacy PXE server, which we normally use and is built using the mos-note guidelines, into one who can support the UEFI boot.

Packages

First of all, you need some more required packages: BOOTX64.efi, grubx64.efi, shim.efi
and also, there are some optional ones as well: MokManager.efi

These rpms can be found in the online yum repo from oracle, or in the ISO files from oracle linux.

1

2

3

# cd /tempdir

# rpm2cpio grub2-efi-version.rpm | cpio -idmv

# rpm2cpio shim-version.rpm | cpio -idmv

Of course, in the datacenter you don’t have internet access. No problem for that, these files are in the PXE – images you have downloaded from edelivery as well.

The grubx64, BOOTX64.efi and MokManager.efi are located in the nfsimg-18.xxxx.tar file.
You can get them out like this:

1

2

3

# tar xf nfsimg-18.<fill in your version>-cell.tar EFI

# cd EFI/BOOT

# cp BOOTX64.efi grubx64.efi MokManager.efi grub.cfg /tftpboot/efi

the shim is a little trickier, you get it from the cellbits:

1

2

3

4

5

# tar xf nfsimg-<fill in your version>-cell.tar cellbits/commonos.tbz

# cd cellbits

# tar xf commonos.tbz shim-0.9-2.0.6.2.el6.x86_64.rpm

# rpm2cpio shim-0.9-2.0.6.2.el6.x86_64.rpm | cpio -idmv

# cp ./boot/efi/EFI/redhat/shim.efi /tftpboot/efi/

that’s all for the extra package requirements.

dhcpd.conf

Yes, this file needs some attention as well.

In the general section on top following info must be added:

1

2

3

set vendorclass=option vendor-class-identifier;

option pxe-system-type code93=unsignedinteger16;

set pxetype=option pxe-system-type;

And in the subnet section you need an if-clause:

1

2

3

4

5

6

7

# Enable UEFI Netboot

ifsubstring(vendorclass,0,9)="PXEClient"{

ifpxetype=00:06orpxetype=00:07{

filename"shim.efi";

}else{

filename"pxelinux.0";

}

take into account, that this is the way on my system. This is a relative path. Just keep that in mind it can be different for you.

Grub-files

In a normal pxe linux boot you need to create the files in pxelinux.cfg/01-<mac address lower case and – instead of : >

Well, this is a little different here as well. The name of the file it’s grub.cfg-01-“MAC ADDRESS with – separator” and in my case, it was also expecting an – at the end.

So we have two components, components and storage cells.

Compute nodes

They get this as content for the 01-<mac address> file for the compute node:

Software location

As this is a first version, I will definitely change it to clean it up, the system expects the files in /tftboot. So move the files vmlinux-nfs* and initrd-nfs-* to /tftpboot/ and make sure to respect the proper permissions.

Restart services

As the configuration from the dhcp server has been changed, this service must be restarted. This can be done using

1

# service dhcpd restart

And basically, that’s it. No other changes are needed. Set the bootdevice from the cell/compute to PXE and restart the components. They will be happy to boot from your PXE server.

Disclaimer: this works for me, and this is not intended as a copy/paste procedure. Also, make sure that you are comfortable with knowing what you are doing or changing. If in doubt about anything, seek some help and if necessary ask Oracle or an installation partner to assist you in this.