Windows, Open source and beyond

Windows driver debugging with WinDbg and VMWare

Virtualization Software such as VMware Workstation enables driver and kernel-mode code developers to speed up development, debugging and testing resulting in faster time to deployment. Snapshots provide a fast and easy way to revert the virtual machine to a previous state. This feature greatly simplifies recreation of specific states or conditions to troubleshoot problems and system crashes.

WinDbg is a debugging tool from Microsoft for user and kernel mode debugging. WinDbg is a GUI interface and a console interface along with some debugging extensions. Using virtual machines, WinDbg can be used to debug kernel code without the need for two physical computers.

This tutorial shows how to debug a simple Windows driver running inside a VMware virtual machine with WinDbg using a single physical machine. The following typical debugging scenarios are covered:

Prerequisites

I assume that your host and guests root system partition is C:\. The kernel debugging setup will use a virtual serial port on COM2 at 115200 bps. The custom named pipe is called BriolidzDbgPipe.

These parameters can be changed to fit your own setup.

VMware machine configuration

In the first debugging scenario, you will debug a virtual machine directly over a named pipe from the host operating system. In the second scenario, you will connect two virtual machines to the same named pipe by creating a virtual null-modem cable. The following configuration steps can be easily adapted for other virtualization software such as VirtualBox and Virtual PC.

In both scenarios, you need to setup the guest virtual machine to debug as follows:

1. Powered off the virtual machine.
2. Open VM then select Settings in VMware Workstation menu.
3. In the VMware Machine Settings dialog box, click Add.
4. In the Add Hardware Wizard dialog box, select Serial Port and click Next.
5. On next page, select Output to named pipe and click Next.
6. Set Name pipe to \\.\pipe\BriolidzDbgPipe. Make sure you select This end is the server and The other end is virtual machine. Check Connect at power on then click Finish.
7. After clicking Finish, select the newly created serial port and check Yield CPU on poll.

You should have something similar to the following screenshot. Note that new serial port has number 2 which corresponds to COM2. If you get assigned another number then make sure to replace COM2 with COM<your assigned number> in the Virtual machine configuration section.

Skip this step, if you are going to use WinDbg on your host (physical computer).

So you decided to use a second virtual machine as a development and debugging machine to replace the host physical machine. In this case, do:

1. Powered off the virtual machine.
2. Open VM then select Settings in VMware Workstation menu.
3. In the VMware Machine Settings dialog box, click Add.
4. In the Add Hardware Wizard dialog box, select Serial Port and click Next.
5. On next page, select Output to named pipe and click Next.
6. Set Name pipe to \\.\pipe\BriolidzDbgPipe. Make sure you select This end is the client and The other end is virtual machine. Check Connect at power on then click Finish.
7. After clicking Finish, select the newly created serial port and check Yield CPU on poll.

You should have something similar to the following screenshot. Note that the new serial port has number 2 which corresponds to COM2. As stated earlier, if you get assigned another number then replace COM2 with COM<your number> in the Attaching WinDBG Debugger section.

Virtual machine configuration

To enable kernel debugging, you need to change Windows boot parameters. Your virtual machine will also be configured to use serial debugging on COM2 at 115200 bps. Power on your virtual machine, log in then apply the following modifications according to your operating system version.

1. Windows XP

Windows XP uses a configuration file called boot.ini on the root of the system partition (generally the C:\ drive) to control how the operating system is booted and any startup options.

The boot.ini file has the Hidden, System, and Read-Only attributes set by default. Open a command prompt, and change them:

attrib -s -h -r C:\boot.ini

The boot.ini file is a standard ASCII text editor. Double click it to edit and duplicate the matching default entry defined in boot loader section, change its title by adding [DEBUG] for instance, add /Debug, /debugport=com2 and /baudrate=115200 switches. Below, a modified sample for Windows XP professional:

Set the operating system selection menu to be displayed when booting and loading the operating system.

bcdedit /set {bootmgr} displaybootmenu yes

You can reduce boot timeout as follows:

bcdedit /timeout 10

To view the current configuration, run:

bcdedit

Sample driver code

Depending on your debugging scenario, log in to your development machine: the host (the physical computer) or the virtual machine. You will write the necessary files to create a do-nothing sample driver called BriolidzSampDrv.sys. This driver has minimal DriverEntry and DriverUnload routines and will write some debug message.

In the working folder, create a file called BriolidzSampDrv.c containing the following source code:

Building the sample driver

Download the Windows Driver Kit. Run the installer and make sure to select at least Build Environments and Debugging Tools for Windows. This will install WinDbg and the necessary environments to build drivers.

When writing this tutorial, I used WDK version is 7.1.0 which include WinDbg version 6.12.2.633. Under Windows 7 professional 64-bit, the default installation folder is C:\WinDDK\7600.16385.1 and the WDk’s start menu group is called Windows Driver Kits. WinDbg is installed in C:\WinDDK\7600.16385.1\Debuggers folder. In case you are using a different WDK version, you will just have to update paths in the following sections.

From the Windows Driver Kits start menu group, launch the command-line build environment for the desired target platform and architecture. Since The BriolidzSampDrv sample driver does not use platform specific code, you can safely use the x86 Checked Build Environment and x64 Checked Build Environment under Winows 7 submenu.

For both target platforms, open the command-line build environment, change to the working directory containing the sample driver source code and enter the command:

C:\Briolidz\Dev\BriolidzSampDrv> build

The 64-bit driver is built in C:\Briolidz\Dev\BriolidzSampDrv\objchk_win7_amd64\amd64\BriolidzSampDrv.sys and the 32-bit driver is built in C:\Briolidz\Dev\BriolidzSampDrv\objchk_win7_x86\i386\BriolidzSampDrv.sys.

Sample driver installation

If you are planning to debug the x64 driver version on Windows Vista/7 (necessarily x64), then press F8 key and select Disable Driver Signature Enforcement from the menu. This operation must be done at every boot because 64-bit Windows systems will not allow drivers to be loaded unless they have a valid digital signature.

If you cannot digitally sign your driver, you can test-sign it and enable the TESTSIGNING mode using the bcdedit command to avoid manually disabling driver signature procedure at every boot time.

After logging in a session with administrator privileges, copy the BriolidzSampDrv.sys (from your working folder) to C:\Windows\System32\drivers. Also copy the InstallSampDrv.reg file to a location of your choice then double-click it to install the driver.

Changes will take effect after the next reboot.

Attaching WinDBG Debugger

At this point your debugging platform is ready to be used. These are the main steps to follow:

1. The WinDbg command line gives you the ability to use environment variables in order to create workspaces that contain your custom debugging settings.

In the following, if you are debugging the 32-bit driver then replace C:\Briolidz\Dev\BriolidzSampDrv\objchk_win7_amd64\amd64 with C:\Briolidz\Dev\BriolidzSampDrv\objchk_win7_x86\i386.

Inside the development machine (host or virtual machine), open a command prompt and type:

6 thoughts on “Windows driver debugging with WinDbg and VMWare”

“Windows driver debugging with WinDbg and VMWare | Kamel Messaoudi” was indeed a good
blog post. If only there were a lot more sites like this excellent one on the world wide web.
Anyhow, thanks a lot for ur time, Stefanie

Great article. Filled in a lot of details that other articles on similar subject left out. With that being said I still can’t seem to connect my two virtuals. I get the error message, “Could not start kernel debugging using com:port-com2…”. I notice that in device manager that the communications port is set to COM1. Should this be changed to COM2? When i changed it to COM2 i got another message, “Opened \\.\com2 Waiting to reconnect…” I am using VMware Workstation 10.

Thanks so much for the article. I finally can debug the windows xp platform by following your steps.
But unfortunately, My windbg of x64 just can NOT connect to the VM(windows 7) after i did all the steps your said.

Below information is the waiting status when I reboot the VM(windows 7). I am pretty sure I have setup the configuration on the windows 7. And I am using the physical windbg to debug VM.

Thank You!!! very much…. Its a really very nice article…
Actually I was fed up by using VS2012 debugging method…
Its require ‘attach to process’ then ‘provisioning’ and takes long time… but now you have given me much relief.

I’m Currently using Physical Host Machine and Target as VM on the same Host via VMware.
I’m very much thankful to you….

Kindly please keep going on and give some more simple ‘n’ simple drivers based on WDF but should include both UM and KM drivers, i.e. passing data from UM to KM and interaction between them or any signal inter-exchange.