<none>

Mechanizing Git bisect: Bug hunting for the lazy

Git bisect is a powerful automated tool for searching deep into a project's history. Instead of searching for relevant commit messages (git log) or patches (git log -S), bisect actually allows you to run a functional test on each revision until the first bad commit is identified. (Okay, it doesn't test every revision, it performs a binary search, which results in at most log2(N) tests. This allows a relatively large history to be searched quickly.)

The test can be done interactively, with the human performing each check, or mechanically if you can supply a testing script. Randy Fay has done a nice screencast on the interactive method; this post will instead focus on mechanizing the process.

For an example, let's look at a core Drupal bug that impacts this very site: #812990: Search page title changes to Home. For the moment, we'll pretend the cause of this bug isn't already known, and hunt it with git-bisect.

Before running bisect, I'll need to find a known good revision. I picked an arbitrary revision on April 1, and manually confirmed the title "Search" is correct.

Note - Make sure you test that the <good> commit you specify is genuinely good, or git will simply report it's next child as the first bad commit.

I think what Joaquin might have been asking is how the script tells git whether the commit is bad.

The answer of course is the exit code. 0 means success (or a good commit) and 1 (or any other number(?)) means failure. Bash has a nicely backwards interpretation of true (0) and false (1).

Most scripting languages have an exit function that allows you to pass an exit code.

Of course if you have unit tests the command you use to run these should return the appropriate exit code too without any extra fuss. What I'm not sure about is what if you only just added the unit test in the latest commit, presumably the command to run the test would fail everywhere else because the test file would disappear whenever you check out an earlier revision.

Ah, yes - git is looking for the exit status. Although it's not exactly fair to blame bash for having "backwards" truth values. The exit status isn't boolean (although it's commonly used that way in practice), and this has been a standard part of Unix at least since the 80's and is probably much older.

You can look at sysexits.h on your system to see some standard status codes. (It's in /usr/include/sysexits.h on OS X).

About the Author

Dylan Tack, Director of Technology

Dylan is a software engineer with more than a decade of experience working with a wide variety of clients including the Linux Foundation, PBS, Habitat for Humanity, TV.com and the Emmys. His background includes training as an electrical engineer, but he became passionate about open source through his work with a university genetics lab.

Dylan is a proud member of the Drupal community, a member of the Drupal security team, and has extensive experience with Perl and Java. His other interests include computer security, embedded design, climbing, and brewing.