Debugger: Overview of New Features

In the past few weeks, I've managed to find a bit more free time to work on Haiku. As per usual, the lion's share of this time was spent on improving our integrated debugger. As such, I thought I'd give a brief overview of what's been added, and how it can be useful.

Return Values

A feature that's been on my todo list for quite some time is retrieving and displaying function return values. This can be quite handy when stepping over code where the return value is used inline without actually being separately stored anywhere. An example would be when a function is called within a loop condition to determine if that loop should continue to execute or not, as shown below:

It can also handle when multiple values are returned in a single statement, i.e. when adding together the values returned by several different calls.

Improved typecasts

As some of you may recall from my previous post on this topic, during the last Haiku code sprint, the debugger gained the ability to enact simple typecasts on variables. However, one thing that was not supported which is quite often needed is typecasting to an array type. A simple example would be the argument array that's passed into main(). In its default state, the variable is of somewhat limited use, but fortunately we know its size, and as such can coerce it into something more helpful:

This supports multiple dimensions as well, so it can be applied in a number of circumstances.

Array Ranges

Still on the subject of arrays, and container types in general, a limitation of the debugger up until now is, when confronted with such a type, it would restrict itself to displaying at most the first 10-20 elements of it. This was partly done for sanity reasons, as, if you hypothetically have a container with a few thousand elements in it, expanding that to all its elements immediately would be a) time/memory consuming, and b) likely not all that helpful.
However, until now there hasn't been a way to fine-tune the range that's displayed, since chances are the elements of interest are somewhere other than said initial elements. This has now been addressed, as shown below:

The BObjectList in question contains 1000 elements, of which only the first 20 are shown. We can now do the following to change that though:

As shown, the feature supports specifying an arbitrary list of subranges from the array/container, which can be quite handy if you need to be able to compare elements that are otherwise quite far apart.
Another set of improvements that have been made which won't be quite as user-visible are a number of improvements to our crash reports. They now also show area/semaphore information for the crashed team, and have had their format improved to be a bit easier to follow/analyze.
As always, I'll continue to work on further improvements as time permits. Hopefully the above are found helpful, and if any problems are found, please report them via our bug tracker.

The Haiku debugger blows my mind. It's amazing! I'm a software developer but I don't have much experience with debuggers aside from the occasional use of GDB. Does the Haiku debugger absolutely excel as far as debuggers go, especial for one that comes with the operating system for free?

To be honest, I would have to tell you that both gdb and most other debuggers out there have been capable of the things listed in my blog posts for quite some time and we're mostly playing catch up there. Either way though, glad you like it!

Something I miss in my usage of VS2008 C# is the ability to copy array values from the debugger. It just copies the array name which is quite usless. It would be nice to have formattng of certain types such as 0xFF 0x12.. ie

ushort array[3] = 0x00 0x01 0x02; should copy out of the variable in the debugger as array[3] = 0x00 0x01 0x02

Also the Haiku debugger doesn't seem to be well integrated with IDE type tools so a Haiku IDE might be in order? Perhaps as a GSoC project for the IDE itself, clang static analysis, cling for faster code/debug cycles.

The Debugger here has all the makings of a decent IDE except for the fact that it isn't in the right shape :P

I can tell you that VS2012 hasn't improved the situation :). In any case though, that's a good point as copying the values into the clipboard is one thing that isn't currently implemented. As far as IDEs go, the debugger is nicely separated out as far as subsystems go, so plugging it into an IDE later wouldn't really be all that difficult. So far I've mainly been focused on getting the actual debugging side of things in shape though, and in that respect I still have plenty to do, so I doubt I'd tackle the IDE idea any time in the near future I'm afraid.

There is the Paladin IDE. I've only briefly tried to use it.. I find it a little confusing in that as far as I can tell it uses it's own system to make a project.. I'd find it more useful if it could arbitrarily use different make systems (Makefiles/automake, jam, cmake etc). But I didn't try it for long so maybe it does... and I think it is open source, so it could be modified...

As of the moment there is not that I'm aware of. It's still a component that's under heavy development though, i.e. there's quite a significant difference between the version in Alpha 4.1 and what's there in the latest nightly. As far as how to use it, the best I can direct you to for the time being is the presentation on Haiku's debugging tools from the last Begeistert. Otherwise, it works pretty much the same as most graphical debuggers on other platforms.

You've made some awesome progress. I checked out Debugger just a couple days ago and noticed the new "f(x) returned y" feature, I really liked that. I also noticed that Debugger could use a few more tool-tips to guide first time users. For example when hovering the dots besides the source view, it could mention that clicking will install a breakpoint. In any case, great to follow the progress, you are doing good work!

That's a good idea, I have to admit I've mostly been focusing on actual features and lower level details, so most of that kind of UI polish hasn't really come to mind as much. If you have any other suggestions for places where tooltips might be handy, or generally UI improvements, then it'd be great to have them gathered in an enhancement ticket so I don't forget; of course, you're welcome to add them directly yourself if you're so inclined as well. In any event, glad you like the changes!

A friend and I were talking about debuggers the other day. How do you do the stack unwinding? This seems like by far the most painful part. It would be very interesting to know a bit more about what's involved.

There are actually two possible ways to do this. For binaries that are built with full debugging information (ergo gcc -g), some of the ELF sections that are available include .debug_frame and/or .eh_frame. Those actually contain instructions for how to unwind the stack (and furthermore, retrieve or compute the value of all the registers at that point). .eh_frame is actually also what's used by C++ exception handling to unwind the stack when throwing an exception.

However, in some cases, neither of those will be available, as in the case of a C binary with no symbolic debugging. In such a case, one must fall back to a more manual strategy that essentially depends mainly on the target CPU architecture and platform calling convention. In the case of x86 for instance, for a normal function call, the bounds of the current stack frame are set by the ESP register (current stack pointer), and EBP (frame pointer). The latter essentially delineates where the current function's stack frame began, and normally the return address to jump back to the previous function is stored there, along with the frame address of the previous function. As such, with those two pieces of information, ESP can then be set to EBP, EBP is set to the frame address retrieved off the stack, and the instruction pointer is set to the return address retrieved off the stack. The process then repeats itself until you make your way to the top.

Note that this is also why it's very difficult to unwind the stack for code compiled with -fomit-frame-pointer, since in that case the previous function's EBP is no longer dropped onto the stack. As such, in order to be able to unwind the stack for code of that nature you'd actually have to rely on disassembly + instruction analysis to do so, which is potentially error prone. If you're curious as to the details in our particular instance, see Stack trace creation, x86 architectural stack frame unwinding and DWARF-based stack frame unwinding. Furthermore, if you'd like to see the actual written specification for the format of .{debug,eh}_frame, see the latest DWARF specification, specifically section 6.4.1.

I tried to use the new debugger to debug a Object Pascal (*.pas) application compiled with the Free Pascal Compiler (v2.6.2). I just wanted to see how the debugger works.

1) I couldn't find a way to tell the debugger to start the test application (just a simple console app), and break at the first line in the main function.

2) I eventually setup a "press enter to continue..." prompt in my app, so I could attach the debugger. But now the debugger shows "Source file unavailable". The source file 'test.pas' is in the same directory as the executable 'test'. I did compile the executable with full DWARF debug info, but does the debugger only look for C/C++ source files?

Most of the development/testing of it has been done with C/C++ specifically, so it's possible we're simply missing some features that Object Pascal makes use of. I'd have to have such an executable + source to test with in order to validate that though, as I don't know Object Pascal myself.

Sorry about that. I recompiled my test app with DWARF2 and DWARF3 debug info. It made no difference to the Haiku Debugger - which still only showed ASM code while debugging, and not the *.pas source code file.

As a further update by the way, the reason the debugger immediately starts running your program rather than waiting for you to start stepping is that it currently assumes programs will always start in main(), as is the case for the C/C++ standard. Unfortunately, this appears to not be the case for Freepascal (the binary you submitted does contain a main(), but it's not executed).

As such, the only way to fix this would essentially involve analyzing the binary to determine the language/ABI, and derive the starting symbol appropriately from that. That is going to be a non-trivial chunk of work though, which I can't promise to have done any time soon. However, so it's not forgotten, I've filed http://dev.haiku-os.org/ticket/9760 to track the problem, in case you want to cc yourself on it to be notified with regards to progress.