Hudson

Nightly builds job (cbi-modisco-nightly)

The build is started on a defined schedule (every 6 hours), but can also be started manually.

Hudson first exports the contents of the MoDisco SVN [2] to its Workspace (/opt/users/hudsonbuild/.hudson/jobs/cbi-modisco-nightly/workspace). If the build was started automatically and nothing changed on the SVN since the last build, then the build job stops there.

The Hudson job is configured to execute a shell script that:

Checks out the MoDisco plugins and features from the head of the trunk into /opt/users/hudsonbuild/.hudson/jobs/cbi-modisco-nightly/workspace/MoDiscoTree

Integration builds job (cbi-modisco-integration)

Hudson first exports the contents of the Releng project [4] to its Workspace (/opt/users/hudsonbuild/.hudson/jobs/cbi-modisco-integration/workspace). If the build was started automatically and nothing changed in the Releng project since the last build, then the build job stops there. This allows the integration job to start if the map files have changed.

touches the /opt/public/modeling/gmt/modisco/modiscoBuildPromoteSignalI file if the build succeeded

Athena CBI

The start.sh script does some more setup (creates and sets up the build directory, retrieves the releng project, installs support for SVN,...), and then starts an Eclipse instance with the org.eclipse.ant.core.antRunner application, on /opt/public/cbi/build/org.eclipse.dash.common.releng/buildAll.xml, which is the Athena Common Builder main build script.

The buildAll.xml script in turn invokes other Athena ant scripts, and makes use of PDE Build functionality to:

fetch sources from the repository according to map files in the Releng project (unless for nightly builds, where a local source checkout directory is used instead, to always pull the head of the trunk instead of the tags or branches from the maps)

build plugins, features and source plugins with a headless invocation of PDE Build

build tests and run them under a new Eclipse instance

create a p2 update site

compress the update site with pack200

sign the plugins with the Eclipse Foundation certificate, apart for nightly builds (signing takes a while)

package several zips : runtime, SDK, examples, tests, all in one update site

clean up the build directory at the end, unless there were errors

Athena makes use of these elements in the Releng project:

build.properties : the most important configuration file : specifies the steps to do, the feature to build, the build dependencies, the Java paths, plus a few other parameters, some being optional.

the map files : define where to get plugins and features from (CVS or SVN path) and which version to take for each one (head, tag, or branch). The maps are used by the integration job, but not the nightly job, which always builds from the trunk.

buildExtra.xml : optional file containing ant targets that will be called by Athena at pre-defined points during the build. This is useful to do custom packaging, fetch extra dependencies, compile javadoc, launch code generators, etc.

Promote

Once the Hudson job is finished, the build results are found in /opt/users/hudsonbuild/.hudson/jobs/cbi-modisco-nightly/lastSuccessful/ (or /opt/users/hudsonbuild/.hudson/jobs/cbi-modisco-integration/lastSuccessful/).

The following entry was added to a crontab (crontab -e):

*/5 * * * * /opt/public/modeling/gmt/modisco/cronPromote.sh

So that the cronPromote.sh executes every five minutes on the build server.

This script compares the date of the signal files (modiscoBuildPromoteSignalN and modiscoBuildPromoteSignalI) touched by the Hudson script at the end of each build with the date of a reference file touched at the end of each promote (/opt/public/modeling/gmt/modisco/lastPromoteRefN and lastPromoteRefI).

The promote is not done directly from Hudson because it does not have the necessary rights.

The promote-N.properties and promote-I.properties define where to get the files from, where to put the archives and update site, and whether to add to or replace the existing contents.

The cronPromote.sh script logs everything it does to /opt/public/modeling/gmt/modisco/log-cronPromote, and trims this log file regularly to keep about the last 10 days of log. It also logs the ant output in a file in /opt/public/modeling/gmt/modisco/promo_logs.

Additionally, it removes old nightly and integration builds on the download server, keeping the last 5.

Hudson important parameters

Files to archive = build/?20*/** : archive the results of the build

Excludes (below "Files to archive", click on Advanced) = build/?20*/eclipse, build/?20*/eclipse/**, build/?20*/testing, build/?20*/testing/**, build/?20*/*-AllFeaturesAndPlugins-*.zip*, build/?20*/*-Master-*.zip*, build/?20*/noclean : don't archive work directories, that are left over by Athena when the build failed.

Test report XMLs = build/?20*/testresults/xml/*.xml : the test results, that will be displayed in Hudson