Eclipse, Model Driven Software Development and whatever else is interesting to make software development more efficient

Main menu

Monthly Archives: July 2009

I just downloaded the new FitNesse release 20090709. From this version on FitNesse is released just as a single jar with ~5MB size. In previous versions FitNesse was delivered as zipped distribution. I wondered where the Zip distribution can be downloaded from and read the release notes for version 20090709. There was an interesting entry there:

“Self Install/Update. From now on FitNesse will be released as a jar file. When you first run a new version with java -jar fitnesse.jar it will install itself (or update itself) into the directory in which it is running. No more zip files. No more missing jar files. No more moving pages into new installations. FitNesse takes care of it’s own environment for you.”

So I just downloaded the jar file and launched it:

java -jar fitnesse.jar
Unpacking new version of FitNesse resources. Please be patient.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................
You must now reload FitNesse. Thank you for your patience........

After this was finished FitNesse created the well known directory structure.

Also the start scripts (run.sh/bat) have gone. The jar is executable, so I started it:

Normally Xtext projects are built within the Eclipse IDE. But what do you do if the requirement is to have server side builds for for Xtext based projects, including the Grammar project itself, and of course without checking in generated sources? Setting up a working build process for Eclipse plugin projects is often a hell. I often face this requirement, and this is where Maven is a suitable alternative. This article explains how you can build Xtext based projects with Maven.

At first this requires that you have Maven installed on your machine and that the mvn command is on your path. Type on your shell

The aim is to run the generator, which of course requires that at least the grammar project is built. We will also build the UI project, but it won’t result into a deployable plugin yet. I plan to investigate on this later. For the moment the projects are just compiled, packaged, and sources generated.

For this article I created projects using the Xtext project wizard, which creates by default the projects

org.xtext.example.mydsl

org.xtext.example.mydsl.ui

org.xtext.example.mydsl.generator

Additionaly a further project was added:

org.xtext.example.mydsl.parent.

All projects can be downloaded here as Zip. Your project structure will look like this:

Maven Repositories

The most important thing for Maven based builds is that every artifact and plugin required must be available in a Maven Repository.

This is where the story begins, because Eclipse itself has no Maven repository, and thus every artifact (i.e. plugin jar) required within the build must be made available in a Maven Repository. This is at the moment maintained manually. I just deployed all required artifacts to the Maven Repository at openArchitectureWare.org. It contains now the latest 0.7.1 build of TMF Xtext, M2T Xpand and EMFT MWE.

Next repository we need is the one where to get Maven plugins from. Maven itself has just a small kernel and all features are added by plugins. Most of them are quite common and central available (e.g. Compiler Plugin), others are available in project specific repositories. For running Xpand based code generators the Fornax oAW Maven plugin is used, which is available through the Fornax Maven Repository.

In this example we make us of a small plugin (Replacer Plugin) which is available at the JBoss Maven Repository.

Last but not least the central Maven repository will be used to fetch some common artifacts that are not available in both of them. So we need the following repositories:

POM Files

Maven builds are described by Project Object Models (POM). The POMs are stored in pom.xml files for each subproject. These files are basically the build scripts for the projects and this is where all the magic goes in. Mainly it is “just” adding the right POMs to the projects that enable your projects to be built by Maven. This is totally non invasive. For more information about POMs read the Maven POM Reference.

In this article I show just excerpts from the POMs, so make sure to get the sources. Gaps in the POM are marked with “…”.

Parent Project

The org.xtext.example.parent project is the so called “parent project”. The parent’s POM is parent to all POMs from the submodules. The parent pom.xml

Module aggregation

We want to build all submodules when running the build on the parent project. This is done by declaring modules by relative paths. Since the projects are organized within an Eclipse workspace all projects including the parent are on the same level.

Grammar Project

The Grammar project org.xtext.example.mydsl contains a workflow that runs the Xtext generator to create the Xtext implementation artifacts. By default the file is GenerateMyDsl.mwe in package org.xtext.example.

When running a build this generator workflow should be executed, generating the Xtext sources. Afterwards the sources should be compiled.

Project parent and basic descriptor tags

The project’s parent is declared by groupId, artifactId and version in the <parent> section. Since groupId and version should always be the same as the parent they are just inherited. This way the must only declared once.

Alternative Source/Resource paths

Xtext projects don’t follow Maven’s default paths for Java sources and resources. By default Maven will compile Java sources only from the src/main/java folder and will add src/main/resources. For Xtext projects there are two source folders, /src and /src-gen. To compile both an additional plugin from Codehaus, the Build Helper Maven Plugin, is needed.

Workflow execution

To execute the MWE workflow we need the Fornax oAW Maven Plugin. Maven builds have a lifecycle, where predefined lifecycles exists. The oAW Maven plugin must be executed in the generate-sources phase. Since version 3.0.0 the Fornax plugin supports both workflow engines, oAW 4 Workflow Engine and oAW 5 MWE. Current version is 3.0.1.

To enable execution of MWE workflows the configuration section must be configured with <workflowEngine>mwe</workflowEngine>. The <workflowDescriptor> parameter specifies the path to the workflow file.

UI Project

Configuring the POM for the UI project org.xtext.example.mydsl.ui is pretty similar to the Grammar project. In this project even no workflow execution is required, just source paths must be adjusted to /src and /src-gen (see above, Grammar project).

Generator Project

The POM for the Generator project again is similar to the Grammar project. This plugin of course executes a workflow during the build, so the Fornax oAW Maven plugin needs to be configured again. Again the source paths must be configured in the build section, and the Build Helper plugin used. See the Grammar project for this configuration.

Workflow execution

The configuration of the oAW Maven plugin is similar to the Grammar project. Of course the path of the workflow file differs. To avoid unneccessary generator execution the plugin can be configured to just run the workflow when specified resources change. This is done through the checkResources section. In this case the generator will only run when the model file src/model/MyModel.dsl changes.

These are the first log statements that the build prints out. You can see from here in which order Maven will build the projects. First the parent project will be built, followed by Grammar project, Generator project and UI project.

Downloading artifacts

When you start with a clean Maven installation Maven will download all plugins and artifacts required for the build.

This is already a lot! All of the downloaded artifacts will be stored in your local repository, which is usually located at ~/.m2/repository. Once the artifacts are downloaded you don’t need to access the internet again. You can than go to offline mode by adding the -o option:

mvn -o clean install

You may need to configure a proxy to access the public Maven repositories. Read “Configuring a proxy” to get more information.

Generating the Xtext artifacts

When running the build on the Grammar project the artifacts from the Xtext grammar file are generated. You will see that the Fornax oAW Maven plugin starts execution and the Xtext generator is run.

In one of my projects we make use of the Fornax oAW 4 cartridges Hibernate and JavaBasic. This week we migrated our generator projects to Eclipse Galileo (we use the EPP Modeling Package at the moment) with MWE, M2T Xpand and TMF Xtext. (Note: I’ll name this package here oAW 5 although it does not exist yet, but this is another topic).

Now these cartridges are not available for oAW 5 yet and will not in the near future. So we have these alternatives:

migrate the cartridges

replace the cartridges by a equivalent, project specific cartridge which takes the same input and produces the same output

adapt the cartridges and leave them as is

We decided to try adapting it. Now the problem is that the cartridges are oAW 4 specific and won’t work without changes. We faced some problems, and luckily we were able to solve them. Some are quite hacks ;-)

Dependencies

The required plugins have changed. With oAW 4.3.1 only the aggregator plugin org.openarchitectureware.dependencies and org.openarchitectureware.uml2.adapter were required dependencies. In our generator project we now have these dependencies (besides project specific ones):org.eclipse.xtend.typesystem.uml2;bundle-version="0.7.0",
org.eclipse.xtend.util.stdlib;bundle-version="0.7.0",
org.eclipse.xpand;bundle-version="0.7.0",
org.eclipse.xtend;bundle-version="0.7.0",
org.eclipse.jface.text;bundle-version="3.5.0",
org.eclipse.xtend.typesystem.emf;bundle-version="0.7.0",
org.eclipse.core.runtime;bundle-version="3.5.0",
org.eclipse.jdt.core;bundle-version="3.5.0"

Workflow

The standard workflow is packaged into the cartridge jars and cannot be changed. The namespaces of the used components has changed. Looking up the component class names with CTRL+SHIFT+T (Open Type) quickly revealed the package names the workflow components have now.
Configuration of the components did not change.

Tipp: if you want to know which workflow component are available then open the interface org.eclipse.emf.mwe.WorkflowComponent and press F4. This opens the type hierarchy and allows you to browse and inspect all known WorkflowComponent implementations. The most interesting ones are derived from AbstractWorkflowComponent2.

Loading profiles

Formerly profiles were just located by classpath. We had the Persistence.profile.uml in folder model, which was on classpath root. This profile could not be loaded anymore. The path was changed to a platform resource URI (platform:/resource/PROJECT/model/Persistence.profile.uml).

With the UML profile file from Fornax we had the problem that when instantiating the ProfileMetaModel instance with that file our generator process hang. It turned out that it was tried to open a HTTP connection to load “http://www.eclipse.org/uml2/2.1.0/UML“, which is declared as xmlns:uml in the model file. Eclipse did not have a internet connection nor should it require. The profile was loaded in MagicDraw 16 and reexported, and the file changed to xmlns:uml="http://www.eclipse.org/uml2/2.0.0/UML". After doing so the profile could be loaded with ProfileMetaModel. Since we use Galileo, the installed UML2 plugins have version 3.0.0.

Stdlib Extensions

The cartridges make use of the oAW’s stdlib extensions. The extensions exist unchanged, but now can be found in the plugin org.eclipse.xtend.util.stdlib with the same package name. So the namespace of those extensions changed and workflow execution failed as follows:

To solve this we added extension files with the same name as the old stdlib file and same package, and just reexport the new stdlib extensions. For example, for the IO extensions a file org/openarchitectureware/util/stdlib/io.ext was created with the following content:
extension org::eclipse::xtend::util::stdlib::io reexport;

Dieter Moroff had already the idea to provide a stdlib compatibility package that contain all oAW 4 stdlib extensions mapped this way. I think it would be great for all users to have that, it would make migrations more smooth. Maybe I find some time to realize it. Main question would be where to host this plugin. I think the sources should be kept at Sourceforge in the oAW 4 repository, and the plugin should be available through an update site on openArchitectureWare.org.

In our own templates that we have under development we just replaced the usage of org::openarchitectureware::util::stdlib... by org::eclipse::xtend::util::stdlib...

PLE feature

The cartridges make use of the org.openarchitectureware.util.featureconfig plugin, which has not been migrated yet. We had to replace the behavior. The cartridges use this plugin mainly to conditionally execute workflow components. Since we had to replace the workflow files anyway we changed all <feature isSelected="FEATURENAME">...</feature> occurances by <if cond="${FEATURENAME}">...</if>, which has the same behavior, but configuration of the features is done by workflow properties and <if> is a built-in feature of the workflow engine. The if-components were not documented in the MWE reference yet, so we filed a bug which already has been fixed now.
The cartridge makes also use of the features.ext extensions, which defines a function isFeatureSelected(). We replaced the behavior by using the stdlib properties extensions. Therefore we configured our features in a properties file and read them with the PropertiesReader component.<component>
<propertiesFile value="featureconfig.properties"/>
</component>

We created a file org/openarchitectureware/util/featureconfig/features.ext:

Changed namespace of internal types

oAW has a built-in typesystem (with types Type, Operation etc.) whose types were in oAW 4 in the namespace oaw. So if you have the expression uml::Class.metaType.name the result would be oaw::Type. With Eclipse M2T the namespace of those types changed to xpand2, so oaw::Type is now xpand2::Type. Normally you don’t work with the internal types directly until you do some reflection stuff, but extensions in the Fornax JavaBasic cartridge make use of those types. This leads to errors like this already when an extension is loaded and analyzed:

To work around this the extension files need to be replaced. Since they are packaged in a jar it is required to copy them and make them available through the classpath before the cartridge. We use the approach to have a compatibility plugin which contains this .ext file as well as the stdlib compatibility extensions.

Problems with UML2 typesystem

We experienced some severe problems with the org.eclipse.xtend.typesystem.uml2 plugin that lead to EvaluationExceptions through our generator process. Basically the plugin is just a migrated version of the org.openarchitectureware.uml2.adapter plugin, but the behavior of the underlying EMF typesystem changed slightly but significant. It was not possible to handle different typesystems at the same time it (e.g. for M2M transformation), which blocked us. We opened Bug#283469 and hunted it down. Some other issues with the typesystem were detected and fixed. A patched plugin you can download from here. We created a folder dropins/oaw5-patch/plugins and put it there until the official service release will be available.
As a positive side effect it is not necessary anymore to register an UML2MetaModel nor EmfRegistryMetaModel instance in the workflow when a ProfileMetaModel is used, since it will delegate to them internally.

Hybridlabs Beautifier

The cartridges make use of the Hybridlabs Beautifier 1.1.9, which is only compatible with oAW 4. The new version 2.0.0 is now able to work also with M2T Xpand.

Conclusion

Our projects including cartridge adaption and bugfixing were migrated in about 4 days. The scenario is a real-world case, and some issues (esp. the mentioned bugs) we faced required deep knowledge about the implementation. I think most projects can be migrated even easier now, but if you face problems it might be worth ordering expert help before spending too much time. Experienced users of oAW 4 will not have much problems migrating the Xpand based code generators.