WTP/Build/How to promote a build

To "promote a build", means to copy it from the build machine to the download machine. This happens, of course, after a build has been deemed promotable ... passed smoke tests or whatever other criteria there are for that build.

Contents

Background

Why doesn't the build script just put every build on the download server? There are a couple of reasons. First, the build machine's files are not mirrored, which turns out to be much more efficient for day-to-day continuous build. Since not many people download those, we would trigger a great deal of of unnecessary activity in the Eclipse.org infrastructure if we put every build on download server. The second reason is that this makes it a little more apparent which build consumers and early testers can pick up. We produce many builds throughout a given week, some of them have never even been downloaded or looked at by anyone so we don't want innocent users or testers downloading a very bad build.

Mechanics

There's basically two things that have to be done. One, copy the files. In our current setup, this is accomplished by copying a directory from the build machine somewhere in the /shared/webtools/committers directory to its download location, somewhere on ~/downloads/webtools/downloads/drops. The other thing that has to be done is that a few files have to have some URLs updated to reflect the new location (most URLs are relative, though, so are valid no matter what the top level directory is). Besides these main things, there's also cleanup to do. On the build machine, any builds during the week that are not promoted should be deleted. It is a good idea, though, to leave the build directory that is being in place, partially to give time for any downloads from users or testers to complete, if they happen to be getting the latest at the same time you promote. Plus, downloads from the build machine are faster or more available for many committers, since once copied to the download server, it can take 10 to 30 minutes (or more) to "show up" by the time it replicates. To actually start showing up on mirrors takes several hours (and usually overnight). There is another type of cleaning not really addressed here (that is, by the automation described below) and that is cleaning up old builds on the download server itself .... this happens when we "change types" of builds. For example, when we promote a Milestone build, then shortly after that we should remove the I-builds that led up to that milestone.

The Promote Automated Script

There is a script (a bash script) in the releng.control project called promote.sh that automates the tasks mentioned above: copies the most recent build in what ever project is named, fixes up the URLs, and removes all but the most recent build.

As of this writing, the script can be ran from an SSH console on the build.eclipse.org machine, from the /shared/webtools/releng.control directory.

Note that "most recent" is something to be careful of. While it does not happen often, occasionally we'll do a "final" build, only to discover it didn't go well, and decide to promote some previous build instead. In those cases, you would have to "manually" delete any build directories that were more recent than the one you wanted to promote before using the scripts.

The script can be ran with something like

./promote.sh -[vcd]p build-project

where build-project is one of the projects such as wtp-R3.0-M, wtp-R3.1-R, etc.

The options are all singe letters and technically optional (though not much is done if there are no options.

v verbose:
provides more output about what it's doing
c copy:
actually does copy the most recent version from the specified project.
If 'c' is not provided, no copy is done but the most recent one is listed to the console.
This is one way to 'sanity check' that the one that will be copied matches the one you think it is.
d delete:
deletes the previous, older builds from the same project that is being promoted.
p project:
the 3-part name associated with any CC projects, such as wtp-R3.0-M, wtp-R3.1-R, etc.

So, for example, the command to promote a weekly M build on the 3.0 maintenance branch would be as follows (ran from the /releng/control directory.

./promote.sh -dcp wtp-R3.0-M

The script will prompt you for your password twice, once at the beginning and once at the end. To setup the proper id for the password:

Make a copy of the /shared/webtools/releng.control/commonVariations.shsource file
in your home bin directory /home/data/users/<your id>/bin
edit the commonVariations.shsource file in your home bin dir and update the REMOTE_SSH_COMMAND="ssh yourid@build.eclipse.org"

What if things change?

The process (and script) makes a few assumptions about the structure of the build machines directory structure and the webtools downloads directory structure. Hence, if any of those things change the script would have to be redone. Similarly, if and when there are new projects added, say "xml-4.0-R" then some tweaks might be required.