← Kartar.Net

• Sunday August 24th, 2008

So rather than doing the work I actually should be I’ve been playing with BuildBot. I had intended to get around to setting up BuildBot sometime in the next couple of months but I got hooked. The reason I wanted to have a look at BuildBot was that Puppet has reached a stage where we simply can’t test every platform it runs on. We are also starting to get patches from a wider variety of sources. Buildbot will allow us to execute our tests on a wider variety of platforms. Hopefully with the cooperation of the community we can gather a really big collection of build platforms to test on. Here’s the blurb for BuildBot:

The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve.

The overall goal is to reduce tree breakage and provide a platform to run tests or code-quality checks that are too annoying or pedantic for any human to waste their time with. Developers get immediate (and potentially public) feedback about their changes, encouraging them to be more careful about testing before checkin.

It’s a very easy tool to deploy. The hardest part has been the slightly broken Git source handling and the assumption that any Git repository is local. I need to have a local Git repository to allow BuildBot to submit the right commits references to the PBChangeSource function. But I designed a basic process for handling new commits:

1. Commit pushed to GitHub.
2. Commit bot at GitHub picks up commit and sends it to BuildBot Master.
3. BuildBot uses the git_buildbot.py script to calculate the before/after commit and branch references and tel BuildBot about them.
4. BuildBot executes the build and tells each slave to retrieve the commit and runs the tests. Currently we’re running: a. All the Unit tests b. All the RSpec tests
5. We then get the results of the tests on the website and in an email to the new Puppet Builds mailing list.

In addition I’ve also enabled BuildBot’s IRC bot and added a new bot, called pinocchio, to the #puppet channel that reports on build status. At this stage it’s all in test mode and when I’ve ironed out a few issues we should be in a position to do a production installation at ReductiveLabs and start canvassing for build slaves.

UPDATE After mucking around with Buildbot I just couldn’t get a whole bunch of issues with Git resolved so we changed to Hudson as our CI - which works much better.

The message overall is - CI and Git: still a young pair.