STEM Release Engineering

This page details how to operate the STEM builder to create binary, platform-specific builds of STEM. This information is for advanced users and committers only.

Background

The STEM build process is built on several foundational pieces of the Eclipse Platform. These pieces include:

Eclipse Products

Eclipse Features

PDE Build Process

P2 Provisioning

These fundamental aspects of the Eclipse Platform are orchestrated to build platform-specific binaries of STEM.

Adding New Code to the Builder

This section describes how to add new plug-ins, features, and third-party dependencies to the STEM Build process.

Important Note About Feature Cross-Dependencies

Plug-in dependencies across features are not as simple as they appear. For a feature (Feature A) containing plug-ins built from source (e.g. not binary), then any feature(s) (Feature B) it depends on must either be built completely first or the plug-ins in Feature B that Feature A depends on must also be included directly in Feature A.

Example:

STEM has Features org.eclipse.stem.core.feature and org.eclipse.stem.models.epidemiology.feature

Plug-in org.eclipse.stem.ui is in the core feature but depends on the org.eclipse.stem.diseasemodels plug-in, which is (logically) in the epidemiology feature

Plug-in org.eclipse.stem.diseasemodels depends on the org.eclipse.stem.core plug-in, which is in the core feature

The epidemiology feature cannot be built completely first because it depends on plug-ins in the core feature

The core feature cannot be built completely first because it depends on plug-ins in the epidemiology feature

Therefore, to solve this cross-dependency, the org.eclipse.stem.diseasemodels plug-in must be in both core and epidemiology features

Adding a new plug-in to an existing feature

STEM is built from Eclipse features and plug-ins. This section describes how to add one or more new plug-ins to the STEM build process to an existing feature. If you want to create a new feature, see the next section.

Determine which Feature(s) the plug-in(s) should be added to

See note above about feature cross-dependencies to help determine which features you should add the plug-in to

In Eclipse, expand the feature project the plug-in(s) should be added to and double-click feature.xml

In the Feature Editor, select the Plug-ins tab

On the Plug-ins and Fragments page, click the Add... button

In the Plug-in Selection dialog, type the name of the plug-in you wish to add. When it shows up in the list, select it and click OK

Repeat this step for all plug-ins you wish to add to this feature

Save feature.xml (File-Save or Control-S)

Commit feature.xml (Right-click Team-Commit)

Add the plug-in to Git

In Eclipse, select the plug-in project you wish to add.

Right click on the project, highlight Team and select Share Project

Follow the steps to add it to the repository.

Note: The STEM Source Repository has a specific layout. Please make sure to place your new plug-in in an appropriate location. Plug-ins are grouped in folders with other plug-ins of similar functionality (such as disease models ). Remember this location, as it's needed for the build.map step below.

[PLUGIN_NAME] should be replaced with the plug-in name (Example: org.eclipse.stem.core)

[REPOSITORY_PATH] should be replaced with the path in SVN where the plug-in was added, relative to trunk (Example: plug-ins added to /trunk/core would use core for the [REPOSITORY_PATH])

Important Note: Plug-ins are grouped in the build map for project set generation. To ensure that a plug-in gets checked out, make sure it's grouped appropriately with similar plug-ins

Save and commit build.map

Adding a new feature

If the new functionality you're adding is substantially different from existing functionality in STEM (such as a new type of modeling, like climate modeling ), then it may be appropriate to add it as a new feature. These steps will help you add a new Eclipse Feature to STEM's runtime and build process.

Important Note: These steps add a feature to the default build process (which, in turn, adds the feature to the main STEM distribution). If you do not wish for the feature to be part of the STEM distribution, only follow the first step below (Creating the Feature Project).

Create the Feature Project

In Eclipse, select the File menu and click New

In the New wizard, expand the Plug-in Development category and select Feature Project . Click Next

On the New Feature page, populate the fields

Project Name should be fully qualified and end with .feature (Example: org.eclipse.stem.new.feature)

Feature ID should be the same as Project Name

Feature Name should be a short description of what functionality the feature provides

Feature Provider should be your organization's name (Example: Eclipse Foundation)

Click Next

On the Referenced Plug-ins and Fragments page, select the plug-ins to be part of this feature.

Note: If the plug-ins aren't created yet, use the steps from the previous section to add them to the feature later

Click Finish

Add the new feature project to SVN

Note: In the STEM Source repository, feature projects are placed in the /trunk/features folder.

Add the Feature to the STEM Product Feature (optional)

In Eclipse, expand the org.eclipse.stem.product.feature project. Open feature.xml

In the Feature Editor , select the Included Features tab

On the Included Features page, click Add

In the Feature Selection dialog, type in your Feature Project's name. When found, select it and click OK

Save and commit feature.xml

Add the Feature to the build map

In Eclipse, expand the org.eclipse.stem.releng project

Expand the maps folder and open stem.map in the text editor

Add the feature to the build map at the end of the rest of the features. The syntax is as follows:

feature@[FEATURE_NAME]=COPY,${checkoutDir}/features,[FEATURE_NAME]

Where:

[FEATURE_NAME] should be replaced with the feature project name (Example: org.eclipse.stem.core.feature)

Save and commit build.map

Add a new dependency

Adding a dependency from another Eclipse project

Adding a third-party (non-Eclipse) dependency from Eclipse Orbit

The easiest way to include a third-party (non-Eclipse) dependency is to select one from the Eclipse Orbit project. Orbit is a formal process by which non-Eclipse, third-party libraries are vetted by Eclipse Legal and approved for general use and redistribution by Eclipse projects. This significantly reduces the amount of work required for both committers and Eclipse Legal to get libraries approved for redistribution.

Checking out the source from SVN

The STEM Builder looks for projects based on the SVN Repository layout, not in the flat structure of an Eclipse workspace. Therefore, you should checkout the repository AS IS using the SVN command line client.

This svn command checks out the STEM source from the SVN trunk into the current working directory.

Whether a build should be prepared for publishing to Eclipse. Only valid for STEM project release engineering.

publishDir

no

/opt/stem/builds

Path that builds should be published to. Only valid for STEM project release engineering.

cleanupFirst

no

true

Removes all files from build type's publishDir before copying new build. Only valid for STEM project release engineering.

Launching the Builder from Command Line

The builder scripts are located in the releng/org.eclipse.stem.releng folder.

The launcher script is bootstrapManual.sh. This file contains the environment variable definitions from above. Edit this file to so that the values are appropriate for your enironment.

From a command line or terminal, change into the org.eclipse.stem.releng directory

cd [STEM_CHECKOUT_PATH]/releng/org.eclipse.stem.releng

Edit the bootstrapManual.sh file to configure the builder for your environment (see above for help)

Invoke bootstrapManual.sh :

bash bootstrapManual.sh

Wait while STEM builder compiles and packages the binaries

Depending on your system speed and options specified, this step could take from 30 minutes to 1 hour to finish

After it's finished, the zip files containing the binaries are located in the build folder:

cd build/I.WeeklyBuild

Launching the Builder from Hudson/Jenkins

If you wish to setup STEM builds using Hudson or Jenkins, you can easily do so. In Hudson/Jenkins, you can configure a job to define the environment variables listed above and then invoke the releng/org.eclipse.stem.releng/bootstrapHudson.sh script. This will invoke the builder. See section on Continuous Integration below for more information.

Running the JUnit Tests

JUnit tests are run as part of the standard build process. Whether or not they are run is controlled by the RUNTESTS environment variable. To run JUnits as part of the build process, simply set runTests=true in build.properties before invoking ant.

Test results are aggregated and collected in the releng/org.eclipse.stem.releng/tests/ folder.

Important Note: Running JUnit tests requires a windowing environment to be available to the terminal invoking the build script (e.g. $DISPLAY must be defined). There are several options available for all-text environments, such as running Xvnc. Most users won't be affected by this.