This is a write-up from the rough notes I made when I made my build environment. Hopefully it'll help you to get going too. These instructions assume you are building either a physical or virtual Linux system dedicated to this task. If you are sufficiently experienced with installing, compiling and configuring software for Linux systems then these notes should also give you what you need to adapt an existing Linux system for the same purpose.

What is the best system to build with? If you download the ESXI open source bundle, the embedded documentation will advise you to build using a CentOS 5.3 system. If you're just looking to build new drivers to add hardware compatibility to ESXI, then that is not in fact necessary. The source bundle contains the toolchain which cleanly builds the drivers, so therefore you can build the toolchain (and subsequently the drivers) using any modern linux system.

For my build environment, despite being a CentOS person, I selected Ubuntu 12.10 simply because it had a fairly small .iso file and I wasn't patient enough to wait for the latest CentOS iso.

The first step is to install Ubuntu Server. Conducting an Ubuntu Server installation is well documented on the Ubuntu website, and beyond the scope of this post, but here are relevant details from my installation which later parts of these instructions depends upon. During installation:

1) Create your user account with the username set as "build". Password to your liking.2) During server type, make sure "OpenSSH Server" is selected a role.3) Make sure there's at least 10GB of space for the system.

Easy huh? Once you have an installed system, the next step is to prepare the system for building software. Boot it up and login as the "build" user. All of these instructions assume you are operating as the build user ID; do not do any of this as root. Confirm network connectivity is working, and then complete the following sequence of commands:

This installs most of the packages that will be needed during the build of the toolchain with the exception of the texinfo software. For that you need a specific version to be installed, and simply doing apt-get install texinfo is going to get you the wrong version. Instead we have to download the version we need and compile it ourselves:

Why do we need a special texinfo? There is a problem with one of the build components for the toolchain. It looked like the automake environment wasn't considering the makeinfo binary as being available because the reply to --version had 4.xx in it rather than 4.x. So the quick and dirty solution was to use a version that didn't break the build.

With this done we need to do a little more house-keeping and then get a quick reboot out of the way.

If you're tempted to look at the BUILD.txt you'll see it suggests you "must" be root in order to do most of this, but that is not my experience. The build environment and instructions contained within, for the sake or clarity I guess, contains paths such as "/build" which would require root, but a simple bit of tweaking to the build scripts and they work just fine without root. So that's the direction I went in. You shouldn't really be running anything as root unless it's crucial.

Now we get to building the toolchain:

Code:

cd glibc-2.3.2-95.44vi BUILD.txt

Change "TROOT=/build/toolchain" to "TROOT=/home/build/toolchain" and save BUILD.txt. You may need to use :w! and force the save since the file is read only.

Code:

bash BUILD.txtcd ..

glibc should not be built and you should be back in ~/vsphere/vmkdrivers-gpl.

This was a little different. Without these CFLAGS set, binutils fails to build because somehow a compile setting is activated which makes a warning level error turn in to a full error that halts the compilation. Similarly, any warning for format errors also causes a failure in this setup. I couldn't quite figure out how and where those compile options were getting set, so in the interest of getting on with things, I chose to set CFLAGS at the shell with the options to override the previous settings. This works because those flags are appended to the compilation string and they negate the previous settings. Next it's on to gcc, and you should be back in ~/vsphere/vmkdrivers-gpl:

Code:

cd gcc-4.1.2-9vi BUILD.txt

Again, change "TROOT=/build/toolchain" to "TROOT=/home/build/toolchain" and save BUILD.txt. Yes you've done this 3 times now because I don't want you doing this as root. Sorry

Code:

export CFLAGS=""bash BUILD.txtcd ..

Sit back and relax while gcc builds, it will take a while. We cleared the CFLAGS setting before commencing the gcc build because it wasn't necessary.

Once gcc is done, the toolchain is now built and ready for use. The next step is to validate that your toolchain and build environment is capable of producing modules that will load in to ESXI. You should be back in the ~/vsphere/vmkdrivers-gpl directory.

First we'll unpack the driver source and then we'll modify the driver build script:

Save the file, and then let's try building all of the drivers that VMWare supplies:

Code:

./build-vmkdrivers.sh

A successful output will look like this:

Code:

build@esx-build:~/vsphere/vmkdrivers-gpl$ ./build-vmkdrivers.sh In file included from vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_init.c:12:vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i.h: In function 'pci_free_consistent_esx':vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i.h:1413: warning: assignment makes integer from pointer without a castvmkdrivers/src_9/drivers/char/random/random.c: In function 'current_fs_time':vmkdrivers/src_9/drivers/char/random/random.c:311: warning: 'notime.tv_nsec' is used uninitialized in this functionvmkdrivers/src_9/drivers/char/random/random.c:311: warning: 'notime.tv_sec' is used uninitialized in this functionIn file included from vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_hwi.c:12:vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i.h: In function 'pci_free_consistent_esx':vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i.h:1413: warning: assignment makes integer from pointer without a castvmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_hwi.c: In function 'bnx2i_cm_remote_close':vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_hwi.c:2578: warning: format '%d' expects type 'int', but argument 3 has type 'char *'vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_hwi.c:2578: warning: too many arguments for formatvmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_hwi.c: In function 'bnx2i_cm_remote_abort':vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_hwi.c:2616: warning: format '%d' expects type 'int', but argument 3 has type 'char *'vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_hwi.c:2616: warning: too many arguments for formatIn file included from vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_sysfs.c:12:vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i.h: In function 'pci_free_consistent_esx':vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i.h:1413: warning: assignment makes integer from pointer without a castvmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_sysfs.c: At top level:vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_sysfs.c:230: warning: initialization makes integer from pointer without a castIn file included from vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_iscsi.c:12:vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i.h: In function 'pci_free_consistent_esx':vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i.h:1413: warning: assignment makes integer from pointer without a castvmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_iscsi.c: In function 'bnx2i_process_scsi_resp':vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_iscsi.c:2854: warning: 'return' with a value, in function returning voidvmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_iscsi.c:2859: warning: 'return' with a value, in function returning voidvmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_iscsi.c:2864: warning: 'return' with a value, in function returning voidvmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_iscsi.c: At top level:vmkdrivers/src_9/drivers/scsi/bnx2i/bnx2i_iscsi.c:2294: warning: 'bnx2i_session_recovery_start' defined but not usedvmkdrivers/src_9/drivers/char/openipmi/ipmi_si_drv/ipmi_si_intf.c: In function 'init_ipmi_si':vmkdrivers/src_9/drivers/char/openipmi/ipmi_si_drv/ipmi_si_intf.c:3133: warning: implicit declaration of function 'register_reboot_notifier'vmkdrivers/src_9/drivers/char/openipmi/ipmi_si_drv/ipmi_si_intf.c: At top level:vmkdrivers/src_9/drivers/char/openipmi/ipmi_si_drv/ipmi_si_intf.c:1652: warning: 'hotmod_handler' defined but not usedvmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c: In function 'megaraid_reset':vmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c:2222: warning: unused variable 'mc'vmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c: In function 'megadev_ioctl':vmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c:3934: warning: cast to pointer from integer of different sizevmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c:3986: warning: cast to pointer from integer of different sizevmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c:4012: warning: cast to pointer from integer of different sizevmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c:4065: warning: cast to pointer from integer of different sizevmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c:4105: warning: cast to pointer from integer of different sizevmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c: At top level:vmkdrivers/src_9/drivers/scsi/megaraid2/megaraid.c:4675: warning: 'mega_support_cluster' defined but not usedvmkdrivers/src_9/drivers/net/forcedeth/forcedeth.c: In function 'nv_probe':vmkdrivers/src_9/drivers/net/forcedeth/forcedeth.c:6306: warning: assignment discards qualifiers from pointer target typevmkdrivers/src_9/drivers/net/forcedeth/forcedeth.c: At top level:vmkdrivers/src_9/drivers/net/forcedeth/forcedeth.c:4781: warning: 'nv_poll_controller' defined but not usedvmkdrivers/src_9/drivers/net/sky2/sky2.c: In function 'sky2_init_netdev':vmkdrivers/src_9/drivers/net/sky2/sky2.c:3976: warning: assignment discards qualifiers from pointer target typevmkdrivers/src_9/drivers/net/sky2/sky2.c: In function 'sky2_mhz':vmkdrivers/src_9/drivers/net/sky2/sky2.c:2685: warning: control reaches end of non-void functionvmkdrivers/src_9/drivers/net/tg3/tg3_vmware.h:28: warning: 'tg3_netq_index' defined but not usedvmkdrivers/src_9/drivers/net/igb/igb_main.c:3541: warning: 'igb_update_itr' defined but not usedbuild@esx-build:~/vsphere/vmkdrivers-gpl$

It does not compile cleanly, but there were no build errors and just warnings.

So now the build host is running, the source is installed, the toolchain is prepared, and the provided source is compiled. All that remains is to check that your compiled modules are actually usable. To do that we need to try and load a module you just produced in to a running kernel.

First, pick a module you think you probably do not have hardware for, because in that case the module isn't likely to already be loaded.

Code:

ls -l BLD/build

You'll see all the provided drivers. I chose the sky2 NIC driver. Now we need to get the newly compiled module over to the target ESXI system. If you have an operating target ESXI server, you can simply scp the file over, otherwise if you're working with a system that you can't even get installed because of the lack of the driver you are hoping to make, you'll need to copy the module over to the system using the USB flash drive technique I describe in another post. For the sake of simplicity I will assume you have a running system you can simply scp the module to:

Great series of posts.... U must be a esxi wizard.... I tried ur steps and was successfully able to create a module and insert in esxi...

Now, i have this problem, in my real module, i deal with pages and the flags inside that.like lock_page, unlock_page etc.How can i port that to esxi.....Because, from the headers i can see that esxi has defined page asstruct page {}

There are no member variables inside it......

Can u help me out with ur knowledge as to how i can get access to the page flags .....

Wed Sep 11, 2013 6:28 am

ehidle

Joined: Thu Dec 30, 2010 7:55 amPosts: 104

Re: ESXI 5.x Drivers Part 1: Making a Build Environment

SO I just tried this, and it seems some things have changed since you wrote this awesome tutorial.

First, it looks like the glibc folder is now glibc-2.5, and the BUILD.txt in that folder does not have a TROOT declaration in it.

Since I'm just blinding following orders, maybe you'd be willing to give us an update?

Who is online

Users browsing this forum: No registered users and 1 guest

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum