During my first week at my new job, I had the opportunity to teach some of my new coworkers about gdb breakpoint commands and conditional breakpoints. I had a lot of fun teaching these techniques my friends here and thought others might find the story enjoyable as well.

Breakpoint commands

The first question I had was: where is our server doing reads? To answer this question, I used an often overlooked feature of gdb: breakpoint commands. At a high level, these allowed us to run an arbitrary set of gdb command automatically when a break point is hit. In my case, I wanted to see what stack trace was causing the reads:

I put a breakpoint on the libcread function call and automatically do two things: print the backtrace of the thread that hit the read and then continue execution. The overall effect of this is that gdb runs the program as normal but prints a backtrace every time the program reads:

Conditional breakpoints

I also noticed it was opening an interesting file, so I wanted break into a debugger to inspect the program to figure out why. Unfortunately, it opens a lot of files (logs, etc), so I needed a way to filter out only the interesting calls to open. To do this, I used gdb conditional breakpoints. The example below creates a breakpoint on open that only triggers if /home/alex is a substring in the filename: