Tuesday, April 17, 2007

Debugging Linux kernels with Workstation 6.0

We just quietly added an exciting feature to Workstation 6.0. I believe it will make WS6 a great tool for Linux kernel development. You can now use gdb on your host to debug the Linux kernel running inside the VM. No kdb, no recompiling and no need for second machine. All you need is a single line in VM's configuration file.

To use the new feature, grab the latest build of Workstation here, or free 30-day evaluation here. Put this line into configuration file of your Linux VM:

debugStub.listen.guest32=1

Now whenever you run the virtual machine, you'll see the following in the vmware.log file (debug builds will also print this message to Host console):

VMware Workstation is listening for debug connection on port 8832.

Run gdb on the Host, reference it to the kernel with symbols and attach to the virtual machine:

That's it. The VM is blocked now, so you can "continue" it and "^C" back to gdb. Breakpoints, single step, memory inspection - all this works as usual. If you have SMP VM, then each VCPU is mapped on a thread, so use "info threads" and "thread NN" to switch between them.

Debugging the 64-bit kernel works in the same way, except you need to use a different option:

debugStub.listen.guest64=1

and connect to port 8864. Since gdb starts in 32-bit mode by default, you may also need to switch it to i386:x64-64 before connecting:

(gdb) set architecture i386:x86-64(gdb) target remote localhost:8864

The kernels with symbols are sadly lacking on most distributions, but if you use RHEL then this website may help (look for kernel-debuginfo rpm):

My company has its own internal debugger that targets 32-bit Windows, 64-bit Windows (Itanium and x64), linux, IBM Z/OS, and several other platforms. Will you be publishing this interface at some point so we can add support to our debugger?

Thanks for noticing this! The release build will not print the message to the console, but you can find the message in the vmware.log file, and you will be able to attach to running VM with gdb running on the Host. I just double checked that with WS60 GA build 45731.

The VM is waiting for debugger asynchronously, so VM will be running as usual until debugger attaches. Let me know if this is inconvenient.

Once again, thank you for the comment. I will update the article.

P.S. The RHEL5 uses dynamically relocatable kernel. You may need to tell gdb where its sections are loaded to make symbols match.

I tried it already and managed to work with a running linux kernel. As I work a lot with loadable modules, I would like to be able to debug code in modules. However, I couldn't figure out how to convince gdb to load symbols for loadable modules. Is it possible, and if so how ?

To orenl: I haven't tried it with linux modules, but generally you need to tell gdb location of the sections of the module you are interested in. The gdb "add-symbol-file" command does this in addition to providing gdb with symbols. Depending on your kernel version, you can get section information from "insmod -m", /proc/ksyms, etc...

To Bradley: hardware breakpoints should work in WS6 RC2 or later. The gdb didn't know about the int3 that you put manually, so it passed it to the Guest OS and it couldn't handle it.

I suspect you have hit a bug. I would like to ask you a few questions, but blogspot comments are not really suitable for discussions. Could you please repost the question in Community Forums? I will follow up there. Thank you!

Replay debugging now has dedicated Community Forum secton. I would appreciate if additional comments are posted there, so that other engineers and customers can participate in discussion too. Thank you.

I downloaded a 30 day evaluation version for Linux viz. 6.0.3 build- 80004. I put the line as per your specification in .vmx file and then rebooted the OS but I do not see the vmware.log message which you have mentioned. I am running OpenSuse 10.1 with kernel 2.6.18.2-34. I have a question on the kernel symbol file which you have mentioned in the gdb file command. I could find a rpm for my kernel symbol version, and installed it but I am not able to see the file which you have mentioned in the invocation of gdb. Could you tell me more on that? like where kernel symbol table files get installed and what would be the file for my version as mentioned above. I am really stuck on an issue related to a kernel panic and need the debug environment as soon as possible.

Hi JS. I have a feeling I was asked the question about OSX in Forums already. The short answer is that Darwin gdb does not seem to like format of vmlinux. So the easiest solution is to run two Linux VMs - one target and one with gdb, and attach gdb form second VM to the first VM. This will be considered a remote gdb connection by first VM, so you'll have to add this line to the config file of the first VM assuming VM is 32-bit):

Hi Mrunal, and sorry for the late replay. You have asked several questions and I'd need more information about your environment to diagnose the issue. If it is still a problem, could you please repost in the Forums? Thanks a lot. the URL of the Forum is:

Hi Ben, thanks for your question and sorry for late reply. The Record/Replay debugging is perfect for your scenario. The way to use it is to enable recording mode in VM, say, while it is still in the BIOS and keep it recording why your PXE bootloader is working. Once it manifests a bug, you stop recording, and debug the recording like you'd debug a kernel issue. One caveat is that if you use 16-bits mode you'll have to adjust linear addresses in gdb.

That was a summary. If you have additional questions, please post in our Forum: