Invoking Buildbot builds from CLI

So, recently I decided to try out Buildbot, a continuous integration tool with emphasis on programmability and flexibility.

After setting up basic building procedures, I wanted to integrate Bitbucket into the mix for triggering these builds when a commit has been pushed. From some reason, I couldn't get Bitbucket polling to work, while BuildBot's www hooks seemed to be using a legacy API. Being the practical person that I am, I decided to sidestep my problem instead of fixing it!

..But I wound up spending the better half of the day with that sidestep. I figured that I could put up a simple HTTP server, which listens to POST requests from Bitbucket commit change hooks and invokes Buildbot build procedures from the command line. Easy enough?

Well, since I am a stubborn idiot who rather continues following his intuition than reads actual documentation with any focus, it took a while to get it working.

I'm documenting my poor solution here just in the case that someone ever has a similar problem.

We have multiple repositories for a single project. There's overhead in doing it like this; In reality, building and running each repository should happen inside its own BuilderConfig, but for convenience's sake, I did it like this.

Here we have the usual ForceScheduler and a SingleBranchScheduler, which is triggered on any changes in the category "api".

Finally, our changesources:

####### CHANGESOURCES
# the 'change_source' setting tells the buildmaster how it should find out
# about source code changes. Here we point to the buildbot clone of pyflakes.
c['change_source'] = []
c['change_source'].append(changes.PBChangeSource(port=9999,
user='manual',
passwd='password'))

Here, we have a PBChangeSource, listening to any connections on the port 5024.

To trigger a change, I set up a simple web server. It listens to connections and upon receiving a POST message, triggers a build. It looks like this:

Now the URL http://my.doma.in:9000/build/api/my_secret_key can be given to services such as BitBucket or GitHub to call when ever there's a commit, thus triggering a build.

The important part here is the ChangeSources and PBChangeSource. With PBChangeSource set as it is, you can trigger a changeset to propagate through BuildBot via CLI by calling buildbot sendchange --master localhost:9999 --auth manual:password --who buildbot -C api. The last parameter - -C api - sets the category of the changeset. Then, Schedulers are configured to listen to changes in the category "api". When one such change is spotted, the builders are called.

So here it is, how I set up triggering builds in BuildBot from BitBucket or GitHub by using the CLI. If you happen to be new to BuildBot, instead of looking at this post, you ought to be checking out Buildbot Tutorial and after that, Buildbot in 5 minutes!