Do you know the feeling when you discover a bug in a functionality that was working couple of weeks (or versions) ago? Too bad we didn’t have any automated tests and what used to be fine, now is broken. Let’s take this simple repository as an example:

Write test first

We noticed that some particular functionality was OK in version 1.0 but is broken in 1.1. What is the first thing we do? Of course write a test case to make sure this bug, once fixed, never comes back! Writing (failing) test for every bug you find has many advantages:

It documents bugs and proves they were fixed

Non-obvious workarounds and solutions will not be removed ("why was he checking for null here?! It’s impossible, let's simplify it") by accident

You gradually improve overall code coverage, even in legacy codebase.

So you have a failing test case. But even with isolated test you can't reliably figure out what is wrong. If only we could find a commit that broke that test - assuming commits are small and focused. But if we commit our test right now and check in one of the older versions to run it - it's not yet there. After all, test made its why to the codebase just now, if it was there from the first revision, we wouldn't have the problem altogether:

Interactive rebasing

Maybe instead of committing the test after version 1.1 (where we know it's broken) we should make a patch or stash this test? This way we could go through all revisions between 1.0 and 1.1, unstashing or applying patch with test and running it. Hope you agree this is far from perfect. The first trick is to use interactive rebase in order to shift commit with failing test case back in time. However we don't want to rebase masterbranch so we make a temporary copy and rebase it instead:

git bisect

Most importantly, our test case is now injected right after version 1.0 (known to be good). All we have to do is check in all revisions one after another and run this test. STOP! If you are smart (or lazy) you will start from commit right in the middle and if this one is broken you proceed with first half the same way - or take second half otherwise. It's sort of like binary search. However keeping track of which commit was last seen good and bad and also manually checking in revision in the middle is quite cumbersome. Luckily git can do this for us with git bisect command. In principal after starting bisecting we specify last known good and first known bad commit. Git will check in revision in between and ask us whether it's good or bad, continuing until we find exactly which commit broke code. In our case we simply run mvn test and proceed depending on its outcome:

See how we iteratively call git good/bad executing our test case in between? Also notice how quickly the number of commits to test shrinks. You might think this is neat and fast (logarithmic time!), but we can actually go much faster. git bisect has a hidden gem called run mode. Instead of relying on manual answer from the user after each iteration we can provide a script that tells whether given revision is good or bad. By convention if this script exits with code 0 it means success while any other exit code signals an error. Luckily mvn script follows this convention so we can simply execute git bisect mvn test -Dcom.nurkiewicz.BugTest, sit back and relax:

Program above is non-interactive and fully automated. git, after few iterations, points precisely which commit was the first one to break the test. We can run all tests, but there is no point since we know only this one fails. Of course you can use any other command rather than mvn. You can even write some simple script in any JVM language of your choice (use System.exit ()). git bisect, combined with interactive rebasing, are wonderful tools to look for regressions and bugs. Also they promote automated testing and automation in general.