Release Reviews / x.y.0 Final Builds

DO NOT PUBLISH any build using a vanity name ending in ".0" without a release review.

DO PUBLISH as many I or S builds as you want -- ideally, you should be publishing one build a week, but only if there are changes.

You can have unlimited weekly Integration (I) builds, then about once every 6 weeks and at the end of your development cycle, you should publish a Stable (S) build. These are usually vanity-named with "0.8.0M2" or "0.8.0RC" to describe the Milestone or Release Candidate status and number.

At the end of your development cycle, you should do an RC build at least one week before your final planned release date. When you pass your Release Review, you can use the /home/www-data/build/modeling/scripts/renameBuild.sh script to rename your "0.8.0RC" Stable (S) build to a "0.8.0" Release (R) build, and repromote it under the new name. This will ensure the correct metadata is properly updated (RSS feed, release notes).

Once your x.y.0 build is reviewed, passed, and released, you can continue in HEAD working on x.y.1 (or branch immediately). Publish as many (M)aintenance builds as you want, then again, as you get close to the end, a Stable 0.8.1RC can be done. Maintenance branch releases need not be reviewed prior to release.

At some point you'll want to branch HEAD (eg., 0.9.0) and R0_8_maintenance (eg., 0.8.1 or 0.8.2). When you start doing 0.9.0 builds, they will be I builds, interspersed with S milestone builds and RC build(s) towards the end of the cycle. Finally, your last RC will be reviewed and you'll once again rename and repromote.

It is HIGHLY ADVISABLE to not branch midway through a release, in order to preserve accurate release notes.

Please also be aware that your build server is SHARED -- clean up your old builds (especially any -noclean builds) when they are obsoleted by newer ones in order to save space for others.

Renaming a Build / Releasing Your GM Build

When releasing your final build, you may decide to rename an RC build if nothing bad has happened since that build was released. For example, you might plan to release your final build on Sept. 28, so you'd do an RC1 build two weeks before. If any stop-ship bugs were found, you'd spin an RC2 build the week after, on the 21st. Then, if no more bugs were found, you'd be able to promote that build as your GM (Golden Master) on Sept. 28 - however, you'd have to rename the zips, md5 files, dependencies listed in build.cfg, etc.

The good news is that there's an easy way to do this - renameBuild.sh. This script contains detailed instructions and examples of usage; just run it from the commandline w/o options to see. Additionally, here's an example showing a rename of JET from 0.7.1RC2 to 0.7.1:

When you're done renaming, you can simply promote the new build as you did the previous week (-buildID S200609210820), using the new ID (-buildID R200609210820), and you'll have your GM build published.

After releasing the final build of a given version, you need to remember to upgrade the version of the plugins you change. In other words, if a plugin is not changed then its version remains the same. After upgrading the version of a plugin, you need to ensure that the version of the appropriate features and branding plugins are also upgraded, and that the feature is referring to the new versions of the plugins. Also, if you change a plugin with source code, you may need to upgrade the appropriate documentation plugin (and documentation feature and branding plugin) if the build is adding Javadoc to it.

Build Naming Conventions

Because Modeling consists of many projects with an ever-expanding set of components, it's important that we all adhere to consistent naming rules for each build type. It's also important to be consistent within your component or your release notes will be sloppy or incomplete. Build Aliases must conform to one of these patterns in order to be automatically added to the release notes database:

/[IM]\d{12}/

or

/\d+\.\d+\.\d+.*/

When building, use the Build Alias field to name your S and R type builds.

Promoting A Build

The First Time

Next, ensure your promoteToEclipse.*.properties file is up to date. Once your file is correct in CVS, you can run this (sudo'd as the web user, www-data or apache) to check out the latest version of that file from CVS:

Review the log file generated by the promote process. Log files are stored in /home/www-data/promo_logs/. If you encounter any errors in the log, review Modeling Project Releng or contact User:Nickb.

Should a step fail, you can re-run the whole promote (but you may get duplicate entries in your RSS feed and update site). To avoid this, you can remove the new content before re-running the build, or you can run only the relevant parts that failed via the commandline promote script interface. For example, to re-run the javadoc generation step, use:

If you plan to contribute to a coordinated release such as Ganymede, make sure you've added your component to the list, and that you have write access to /cvsroot/callisto. If you don't, open a bug assigned to webmaster@eclipse.org and cc:'d to your PMC User:Nickb. You should ask:

Update Manager Sites & Build Types

If you go to http://www.eclipse.org/modeling/emft/updates/, you'll see a note about two different UM sites. These two sites are there to split final, release builds from interim builds, so that the two types of users can benefit from the two types of releases.

So, for example, when you publish an I build (eg., 0.7.5 I200610050000), it gets added to site-interim.xml. When you do your final build for that stream, (eg., 0.7.5 R200612020000), it will be added to site.xml.

Note also that the URL to give to Update Manager (or to put in your features) is NOT http://www.eclipse.org/modeling/emft/updates/, but download.eclipse.org. www.eclipse.org stores website content only, and provides a location for extracting, updating, and committing changes to the site.xml and site-interim.xml files, in order to provide CVS versioning & history. The actual site is http://download.eclipse.org/modeling/emft/updates/site*.xml, which is also benefitted by the fact that it's mirrored (unlike www.eclipse.org).

# pattern to use when searching for features to include in coordsite; defaults to ".*eclipse/features/org.eclipse..*(${subprojectName}|${subprojectName}.sdk)_.*\/$"
coordsiteFeaturePattern=".*eclipse/features/org.eclipse..*(emf|emf.sdk|xsd|xsd.sdk|emf.ecore.sdo|emf.ecore.sdo.sdk|emf.common|emf.common.ui|emf.ecore|emf.ecore.edit|emf.edit|emf.edit.ui|xsd.edit)_.*\/$"
# pattern to use when setting feature to include in coordsite's main feature ('Models and Model Development'); all other matching features will go in 'Enabling Features'
coordsiteMainFeaturePattern="org.eclipse.(emf|emf.sdk|xsd|xsd.sdk|emf.ecore.sdo|emf.ecore.sdo.sdk)$"

# pattern to use when searching for features to include in coordsite; defaults to ".*eclipse/features/org.eclipse..*(${subprojectName}|${subprojectName}.sdk)_.*\/$"
coordsiteFeaturePattern=".*eclipse/features/org.eclipse.(emf.ocl|ocl.all|ocl.all.sdk|ocl|ocl.uml)_.*\/$"
coordsiteMainFeaturePattern="org.eclipse.ocl.(all|all.sdk)"

# pattern to use when searching for features to include in coordsite; defaults to ".*eclipse/features/org.eclipse..*(${subprojectName}|${subprojectName}.sdk)_.*\/$"
#omit SDK because this is only to create a temp file which will be combined w/ rest of QTV components
coordsiteFeaturePattern=".*eclipse/features/org.eclipse.emf.(emfqtv.all|emfqtv.all.sdk|query|query.ocl|validation|validation.ocl|transaction|workspace)_.*\/$"
coordsiteMainFeaturePattern="org.eclipse.emf.emfqtv.(all|all.sdk)"

# pattern to use when searching for features to include in coordsite; defaults to ".*eclipse/features/org.eclipse..*(${subprojectName}|${subprojectName}.sdk)_.*\/$"
#omit SDK because this is only to create a temp file which will be combined w/ rest of QTV components
coordsiteFeaturePattern=".*eclipse/features/org.eclipse.emf.(validation|validation.ocl)_.*\/$"
coordsiteMainFeaturePattern="NONE"

Build Promotion Page

Generating Release Notes

Generating release notes is easy - promoteToEclipse.sh does it for you! However, there are a few simple steps to follow to ensure the generated content is accurate & non-duplicative. Here's the general process for closing a bug and using Bugzilla & CVS to provide the information required to generate release notes.

Working in Eclipse, fix your code. Test is as much as possible before committing it to CVS.

Commit your change(s) to CVS, using the bug's number in square brackets to begin your commit comment, eg:

[127509] javadocs now available

If you use Mylyn you can use this template, with a space at the end:

[{$task.id}]

Note that you can include multiple bugs (each number on its own line) and comments are not required. The information that will appear in the release notes comes from the bug's Summary field, not your CVS comments.

Browse to the bug's webpage, and assign it the status Assigned. Your bugzilla comment could be something like:

Troubleshooting

My new build isn't listed on the website!

My new build isn't listed in the Release Notes!

First, check your RSS feed file to ensure it's been updated. If it hasn't, check your promote log for failures in updating the feed. If it has, please be patient and wait for the next database update. Your RSS feed file controls the list of releases that are imported into the Releases table of the Search CVS database, and thus appear on the Release Notes page.

There are errors in my release notes! How do I edit?

If you want to remove or edit values in your RSS feed and flush old/incorrect values from the database, please edit the file /cvsroot/org.eclipse/www/modeling/projectName/feeds/builds-componentName.xml (eg., projectName = emft; componentName = teneo). When done, send an email to User:nickb requesting a database flush for your component's releases.

Bugs are missing from the Release Notes!

Database updates can take a while. If it's been 24hrs and you *still* are missing data, contact User:nickb.

RSS Update Errors When Promoting

Problem

Solution

addEntry:
[AddEntry] [Fatal Error] builds-teneo.xml:13:2:
The content of elements must consist of
well-formed character data or markup.
[AddEntry] org.xml.sax.SAXParseException:
The content of elements must consist of
well-formed character data or markup.

Check the file on the build server in /var/www/html/emft/feeds/. Is it corrupt? Is it full of invalid XML due to a CVS update/merge? If so, so the following:

Then rerun your promote using the -rssonly flag to redo that single step. After that, you can force an update of the database - including loading any new CVS commits - with the -searchcvsonly flag. If you don't need to refresh commits, just the Releases table, you can run this (on emft.eclipse.org only):

cd /shared/modeling/searchcvs; \
php relupdate.php -noheat

Permission denied

Check the file on the build server in /var/www/html/emft/feeds/. Is it writable by you or one of the groups of which you're a member? Try this:

Then rerun your promote using the -rssonly flag to redo that single step. After that, you can force an update of the database - including loading any new CVS commits - with the -searchcvsonly flag. If you don't need to refresh commits, just the Releases table, you can run this (on emft.eclipse.org only):

cd /shared/modeling/searchcvs; \
php relupdate.php -noheat

Automated 'Fix available in ...' Bugzilla Comments

As of 2008-01-21, you can use promo.php or the -bugzonly flag to automatically move your RESOLVED:FIXED bugs to VERIFIED:FIXED, and place a comment stating in which build & branch the bug was fixed. See also bug 206558, bug 191571.

If you prefer, you can enable this step to happen automatically with every promoted build using these parameters in your promoteToEclipse.*.properties file

You are now set up to post automatically to the newsgroup while promoting a build. If you want to post as a specific person, rather than the default name & email, there are two additional steps.

2. Edit org.eclipse.emft/releng/[subproject]/promoteToEclipse.[subproject].properties<example>, placing the correct name and email address for the person whose name will appear on newsgroup posts. This step - and the next one - is ONLY required if the name & email are not the same as the default value recorded here.

To announce a build that's already been built, you can use the -announceonly flag to skip all other steps and ONLY post to the newsgroup. Otherwise, adding -announce when promoting will add the extra step of posting a quick note to the newsgroup.

Newsgroup Post Threading

Should you want to thread your announcements so that they can be hidden or categorized, here's how you can do so:

create the parent thread into which subsequent posts will go (just post a message to the newsgroup), then

get the post's Message-ID (using Thunderbird, just turn on all headers to see this), and

If you put the Message-ID in the newsgroupThreadReferences property, announcements about builds will be nested under that existing message.

Note that to edit this file you must sudo to the web user:

On emf.toro: sudo -u www-data vi promoteToEclipse.jet.properties

On emft.eclipse: sudo -u apache vi promoteToEclipse.cdo.properties

If you'd like to store a copy of your changes in CVS, please contact your releng manager so that your changes can be committed into your releng module for safekeeping/versioning.

SSH Key Setup

Getting Access

You must have ssh access to your build server, download1.eclipse.org, and dev.eclipse.org. If you do not have access, please contact the administrators of these servers to gain access. You need a full login shell (eg., bash), not just a CVS login shell (cvssh). If you cannot ssh into dev.eclipse.org or download1.eclipse.org, please open a bug asking for a full login shell, eg., bug 206146.

To get access to your build server, you must forward your static IP address to the server's administrator. If you do not have a static IP, you can set up your router to broadcast your IP to a domain name registration service like dyndns.org and provide your domain name for access to the build server.

Once you have ssh access, you need to set up limited ssh key access between your build server and the eclipse.org download and cvs servers. This will allow you to run the promote script without being prompted for a password periodically.

Connecting out: authorizing remote hosts to let you in

To copy your local public key to the remote machine, you will need to scp (secure copy) the file. From the build server, copy the file to the remote server, then ssh to that server and combine the new key with the any existing keys in the authorized_keys file. In most cases, build_id == eclipse_id, eg., jlescot == jlescot:

Note that if you have not created yet the .ssh folder on the dev.eclipse.org server, you first need to achieve it using the following instructions :

(this time you should get in using your key instead of needing a password)

Once you have ~/.ssh/authorized_keys and ~/.ssh/known_hosts updated with your public key and server fingerprint(s) (respectively), you should be able to connect to these systems without being prompted, and the promote script should be good to go unassisted.

Connecting out: creating host keys

SSH to your build server (if not already there). Then, one by one, ssh from there to these target systems, accepting whatever yes/no questions are posed about host keys or IP addresses. Once in, exit back to your build server. In most cases, build_id == eclipse_id, eg., jlescot == jlescot:

All of those should work without being prompted for a password. If you are prompted, you need to set an ssh key in your ~/.ssh/authorized_keys file for each user & hostname pair to which you want to connect. The first time you try each of these connections, you should be be prompted to add the host's fingerprint to your ~/.ssh/known_hosts file. This is also required.

Connecting the web user to your account

In order to run builds from the web site on the build server, the web user must be able to ssh to your account on the build server. Put its public key in your authorized_keys file to make this work.

Additional

Promotion FAQ

Many things can go wrong when promoting a build (CVS or SCP errors, artifacts not created, java failures). Most are a result of network or permission problems, and can be fixed by trying again or correcting the permission problem.

If you decide to repromote completely using the web UI, make sure you've removed this build's site*.xml contribution because a full promote will add it again, slowing down the operation of the update site and presenting users with confusing duplicate entries.

Update Site

Please check your promote logs for errors or transient glitches, like CVS disconnects or SCP problems. Sometimes one problem can manifest as several problems, as in the case of this EMFT Validation build's UM jar creation, which threw four warnings that something had gone wrong, all starting from a transient CVS checkout problem (bandwidth overload or connection reset).

[umj-co] [1] Checking out org.eclipse.releng.basebuilder from dev
using HEAD
cvs checkout: failed to open /home/vramaswamy/.cvspass for reading:
No such file or directory
cvs [checkout aborted]: recv() from server dev.eclipse.org:
Connection reset by peer

The java class is not found: org/eclipse/core/launcher/Main
features/*.jar: No such file or directory
plugins/*.jar: No such file or directory