Navigation

The following are the release notes for Buildbot 0.9.0.
This version was released on October 6, 2016.

This is a concatenation of important changes done between 0.8.12 and 0.9.0.
This does not contain details of the bug fixes related to the nine beta and rc period.
This document was written during the very long period of nine development.
It might contain some incoherencies, please help us and report them on irc or trac.

This version represents a refactoring of Buildbot into a consistent, well-defined application composed of loosely coupled components.
The components are linked by a common database backend and a messaging system.
This allows components to be distributed across multiple build masters.
It also allows the rendering of complex web status views to be performed in the browser, rather than on the buildmasters.

The branch looks forward to committing to long-term API compatibility, but does not reach that goal.
The Buildbot-0.9.x series of releases will give the new APIs time to “settle in” before we commit to them.
Commitment will wait for Buildbot-1.0.0 (as per http://semver.org).
Once Buildbot reaches version 1.0.0, upgrades will become much easier for users.

To encourage contributions from a wider field of developers, the web application is designed to look like a normal AngularJS application.
Developers familiar with AngularJS, but not with Python, should be able to start hacking on the web application quickly.
The web application is “pluggable”, so users who develop their own status displays can package those separately from Buildbot itself.

Other goals:

An approachable HTTP REST API, with real time event features used by the web application but available for any other purpose.

A high degree of coverage by reliable, easily-modified tests.

“Interlocking” tests to guarantee compatibility.
For example, the real and fake DB implementations must both pass the same suite of tests.
Then no unseen difference between the fake and real implementations can mask errors that will occur in production.

No additional software or systems, aside from some minor Python packages, are required.

But the devil is in the details:

If you want to do web development, or build the buildbot-www package, you’ll need Node.
It’s an Angular app, and that’s how such apps are developed.
We’ve taken pains to not make either a requirement for users - you can simply ‘pip install’ buildbot-www and be on your way.
This is the case even if you’re hacking on the Python side of Buildbot.

Node has a very specific package manager named npm, which has the interesting property of allowing different version of package to co-exist in the same environment.
The node ecosystem also has the habit of creating packages for a few line of code.

Buildbot UI uses the node ecosystem to build its javascript UI.

The buildsystem that we use is called guanlecoja, which is just an integration of various javascript build tools.

Through npm dependency hell, guanlecoja is depending on 625 npm packages/versions.
We do not advise you to try and package all those npm build dependencies.
They are not required in order to run buildbot.

We do release pre-built packages in the form of the wheel format on pypi.
Those wheels contain the full python source code, and prebuilt javascript source code.

Depending on distro maintainers feedback, we could also release source tarballs with prebuilt javascript, but those would be pypi packages with different names, e.g. buildbot_www_prebuilt.0.9.0.tar.gz.

Another option would be to package a guanlecoja that would embed all its dependencies inside one package.

Buildbot-0.9.0 introduces the Data API, a consistent and scalable method for accessing and updating the state of the Buildbot system.
This API replaces the existing, ill-defined Status API, which has been removed.
Buildbot-0.9.0 introduces new WWW Server Interface using websocket for realtime updates.
Buildbot code that interacted with the Status API (a substantial portion!) has been rewritten to use the Data API.
Individual features and improvements to the Data API are not described on this page.

Buildbot now supports plugins.
They allow Buildbot to be extended by using components distributed independently from the main code.
They also provide for a unified way to access all components.
When previously the following construction was used:

Mercurial hook was updated and modernized.
It is no longer necessary to fork.
One can now extend PYTHONPATH via the hook configuration.
Among others, it permits to use a buildbot virtualenv instead of installing buildbot in all the system.
Added documentation inside the hook.
Misc. clean-up and reorganization in order to make the code a bit more readable.

UI templates can now be customizable.
You can provide html or jade overrides to the www plugins, to customize the UI

The irc command hello now returns ‘Hello’ in a random language if invoked more than once.

Fixed SVN master-side source step: if a SVN operation fails, the repository end up in a situation when a manual intervention is required.
Now if SVN reports such a situation during initial check, the checkout will be clobbered.

The build properties are now stored in the database in the build_properties table.

The list of changes in the build page now displays all the changes since the last successful build.

GitHub change hook now correctly responds to ping events.

GitHub change hook now correctly use the refs/pull/xx/merge branch for testing PRs.

buildbot.steps.http steps now correctly have url parameter renderable

When no arguments are used buildbotcheckconfig now uses buildbot.tac to locate the master config file.

buildbot.util.flatten now correctly flattens arbitrarily nested lists.
buildbot.util.flattened_iterator provides an iterable over the collection which may be more efficient for extremely large lists.

Seamless upgrading between buildbot 0.8.12 and buildbot 0.9.0 is not supported.
Users should start from a clean install but can reuse their config according to the Upgrading to Nine guide.

BonsaiPoller is removed.

buildbot.ec2buildslave is removed; use buildbot.buildslave.ec2 instead.

buildbot.libvirtbuildslave is removed; use buildbot.buildslave.libvirt instead.

buildbot.util.flatten flattens lists and tuples by default (previously only lists).
Additionally, flattening something that isn’t the type to flatten has different behaviour.
Previously, it would return the original value.
Instead, it now returns an array with the original value as the sole element.

Deprecated workdir property has been removed, builddir property should be used instead.

To support MySQL InnoDB, the size of six VARCHAR(256) columns changes.(author,branch,category,name);object_state.name;user.identifier was reduced to VARCHAR(255).

StatusPush has been removed from buildbot.

Please use the much simpler HttpStatusPush instead.

Worker changes described in below worker section will probably impact a buildbot developer who uses undocumented ‘slave’ API.
Undocumented APIs have been replaced without failover, so any custom code that uses it shall be updated with new undocumented API.

Web server does not provide /png and /redirect anymore (bug #3357).
This functionality is used to implement build status images.
This should be easy to implement if you need it.
One could port the old image generation code, or implement a redirection to http://shields.io/.

Support of worker-side usePTY was removed from buildbot-worker.
usePTY argument was removed from WorkerForBuilder and Worker classes.

html is no longer permitted in ‘label’ attributes of forcescheduler parameters.

public_html directory is not created anymore in buildbotcreate-master (it’s not used for some time already).
Documentation was updated with suggestions to use third party web server for serving static file.

usePTY default value has been changed from slave-config to None (use of slave-config will still work).

New-style steps are now the norm, and support for old-style steps is deprecated.
Upgrade your steps to new-style now, as support for old-style steps will be dropped after Buildbot-0.9.0.
See New-Style Build Steps for details.

Status strings for old-style steps could be supplied through a wide variety of conflicting means (describe, description, descriptionDone, descriptionSuffix, getText, and setText, to name just a few).
While all attempts have been made to maintain compatibility, you may find that the status strings for old-style steps have changed in this version.
To fix steps that call setText, try setting the descriptionDone attribute directly, instead – or just rewrite the step in the new style.

Old-style source steps (imported directly from buildbot.steps.source) are no longer supported on the master.

The monotone source step got an overhaul and can now better manage
its database (initialize and/or migrate it, if needed). In the
spirit of monotone, buildbot now always keeps the database around,
as it’s an append-only database.

The Periodic and Nightly schedulers can now consume changes and use onlyIfChanged and createAbsoluteTimestamps.

All “timed” schedulers now handle codebases the same way. Configuring codebases is strongly recommended.
Using the branch parameter is discouraged.

Logs are now stored as Unicode strings, and thus must be decoded properly from the bytestrings provided by shell commands.
By default this encoding is assumed to be UTF-8, but the logEncoding parameter can be used to select an alternative.
Steps and individual logfiles can also override the global default.

The PB status service uses classes which have now been removed, and anyway is redundant to the REST API, so it has been removed.
It has taken the following with it:

buildbotstatuslog

buildbotstatusgui (the GTK client)

buildbotdebugclient

The PBListener status listener is now deprecated and does nothing.
Accordingly, there is no external access to status objects via Perspective Broker, aside from some compatibility code for the try scheduler.

The debugPassword configuration option is no longer needed and is thus deprecated.

The undocumented and un-tested TinderboxMailNotifier, designed to send emails suitable for the abandoned and insecure Tinderbox tool, has been removed.

Buildslave info is no longer available via Interpolate and the SetSlaveInfo buildstep has been removed.

The undocumented path parameter of the MasterShellCommand buildstep has been renamed workdir for better consistency with the other steps.

Botmaster no longer service parent for workers. Service parent functionality has been transferred to WorkerManager.
It should be noted Botmaster no longer has a slaves field as it was moved to WorkerManager.

The sourcestamp DB connector now returns a patchid field.

Buildbot no longer polls the database for jobs.
The db_poll_interval configuration parameter and the db key of the same name are deprecated and will be ignored.

The interface for adding changes has changed.
The new method is master.data.updates.addChange (implemented by addChange), although the old interface (master.addChange) will remain in place for a few versions.
The new method:

returns a change ID, not a Change instance;

takes its when_timestamp argument as epoch time (UNIX time), not a datetime instance; and

does not accept the deprecated parameters who, isdir, is_dir, and when.

requires that all strings be unicode, not bytestrings.

Please adjust any custom change sources accordingly.

A new build status, CANCELLED, has been added.
It is used when a step or build is deliberately cancelled by a user.

This upgrade will delete all rows from the buildrequest_claims table.
If you are using this table for analytical purposes outside of Buildbot, please back up its contents before the upgrade, and restore it afterward, translating object IDs to scheduler IDs if necessary.
This translation would be very slow and is not required for most users, so it is not done automatically.

All of the schedulers DB API methods now accept a schedulerid, rather than an objectid.
If you have custom code using these methods, check your code and make the necessary adjustments.

The addBuildsetForSourceStamp method has become addBuildsetForSourceStamps, and its signature has changed.
The addBuildsetForSourceStampSetDetails method has become addBuildsetForSourceStampsWithDefaults, and its signature has changed.
The addBuildsetForSourceStampDetails method has been removed.
The addBuildsetForLatest method has been removed.
It is equivalent to addBuildsetForSourceStampDetails with sourcestamps=None.
These methods are not yet documented, and their interface is not stable.
Consult the source code for details on the changes.

The preStartConsumingChanges and startTimedSchedulerService hooks have been removed.

The triggerable schedulers trigger method now requires a list of sourcestamps, rather than a dictionary.

The SourceStamp class is no longer used.
It remains in the codebase to support loading data from pickles on upgrade, but should not be used in running code.

The BuildRequest class no longer has full source or sources attributes.
Use the data API to get this information (which is associated with the buildset, not the build request) instead.

The undocumented BuilderControl method submitBuildRequest has been removed.

The debug client no longer supports requesting builds (the requestBuild method has been removed).
If you have been using this method in production, consider instead creating a new change source, using the ForceScheduler, or using one of the try schedulers.

The BuilderConfignextSlave keyword argument takes a callable.
This callable now receives BuildRequest instance in its signature as 3rd parameter.
For retro-compatibility, all callable taking only 2 parameters will still work.

properties object is now directly present in build, and not in build_status.
This should not change much unless you try to access your properties via step.build.build_status.
Remember that with PropertiesMixin, you can access properties via getProperties on the steps, and on the builds objects.

The Buildbot worker now includes the number of CPUs in the information it supplies to the master on connection.
This value is autodetected, but can be overridden with the --numcpus argument to buildslavecreate-worker.

The DockerLatentWorker image attribute is now renderable (can take properties in account).

The DockerLatentWorker sets environment variables describing how to connect to the master.
Example dockerfiles can be found in master/contrib/docker.

DockerLatentWorker now has a hostconfig parameter that can be used to setup host configuration when creating a new container.

DockerLatentWorker now has a networking_config parameter that can be used to setup container networks.

On Windows, if a ShellCommand step in which command was specified as a list is executed, and a
list element is a string consisting of a single pipe character, it no longer creates a pipeline.
Instead, the pipe character is passed verbatim as an argument to the program, like any other string.
This makes command handling consistent between Windows and Unix-like systems.
To have a pipeline, specify command as a string.

Support for python 2.6 was dropped from the master.

public_html directory is not created anymore in buildbotcreate-master (it’s not used for some time already).
Documentation was updated with suggestions to use third party web server for serving static file.

usePTY default value has been changed from slave-config to None (use of slave-config will still work).