since I know from my own experience now that it is a non trivial task to
build a working RCP application from a CI system like hudson, I'd like
to create a wiki page about that.
In this example I'd like to show the necessary steps to not only create
a p2 repository from the org.eclipse.buckminster.tutorial.mailapp but
actually build the product itself (utilizing the buckminster plugin for
hudson).
However, I ran into some problems when I tried to build the application.
Invoking to p2 director to create the product from the repository built
by buckminster produced the following error:

!MESSAGE Missing requirement: Equinox Provisioning Engine
1.0.100.v20090525 (org.eclipse.equinox.p2.engine 1.0.100.v20090525)
requires 'package org.eclipse.equinox.internal.p2.persistence 0.0.0' but
it could not be found

In order to make it run, I had to remove all plugins listed in
org.eclipse.buckminster.tutorial.rcpp2.feature and instead include the
org.eclipse.equinox.p2.user.ui feature and the
org.apache.commons.logging plugin.

On top of that I added a cspecx and an ant script that invokes the p2
director to the org.eclipse.buckminster.tutorial.mailapp.product.feature.

A patch with the modification and the 2 new files is attached.

Now while this was sufficient to build the product from eclipse using
Buckminster -> invoke action -> create.product.zip I'd like to get this
more "real world ready" by showing how I'd integrate that into Hudson CI
server.

For that demonstration I would like to use a new cquery and resource map
for two reasons:
1. on your company's build system you need complete control over your
artifacts so you probably don't want buckminster to fetch artifacts from
all over the world

2. In my opinion it is not the best approach to have buckminster
check-out stuff from your VCS if you are using a system like hudson. Not
only that it's not immediatly visible where your workspace materializes
from in the jobs configuration page, you're also losing the valuable
feature that hudson shows you the change logs since the last build.
In my build environment, I therefore let hudson check out the necessary
modules and do only a local resolution with buckminster. Spoken in terms
of rmaps, the only provider I'm using is this and it's always the same
for all my products:
<provider readerType="local"
componentTypes="osgi.bundle,eclipse.feature" mutable="true" source="true">
<uri format="file:///{0}/{1}/">
<bc:propertyRef key="workspace.root" />
<bc:propertyRef key="buckminster.component" />
</uri>
</provider>
All other resolutions are done with just the target platform provided by
the setPref command.

Now why I'm writing all this:
First of all, do you think a tutorial like that would make sense in the
wiki?
And if so, do you agree with the way I'm building things, or should I
change something for the wiki page?

To make the example really useful for people, the mail app would need to
be buildable and I'd need the extra files available in SVN (see attached
patch). Would you be willing to apply the patch and eventually host
the new cquery that points to the simplified local-resolution rmap (I'll
deliver both if you are ok with my general approach) so that they will
be available for public?

Johannes Utzig wrote:
> !MESSAGE Missing requirement: Equinox Provisioning Engine
> 1.0.100.v20090525 (org.eclipse.equinox.p2.engine 1.0.100.v20090525)
> requires 'package org.eclipse.equinox.internal.p2.persistence 0.0.0' but
> it could not be found
>
This sounds like a regression of some kind. It used to work but it's been a while since anyone tried it.

> First of all, do you think a tutorial like that would make sense in the
> wiki?
Yes, absolutely. I think it's great that you do this. We are in a bad shape when it comes to examples and tutorials so a
contribution like that is very valuable and appreciated.

> And if so, do you agree with the way I'm building things, or should I
> change something for the wiki page?
>
I agree that the sample should have an RMAP of its own but I'm not too keen about using an approach where Hudson
populates the workspace. By doing that, you bypass the whole resolution/materialization piece in Buckminster. Not so
good if the purpose is to use it all as an example. It would also impede future examples of more advanced features such
as CQUERY based branch/tag selection.

Would it be possible to make Hudson understand that it should use Buckminster as the check-out tool? Or perhaps we could
make Hudson use the information that it finds in a BOM file (the intermediate file that Buckminster creates when it runs
the resolver) to do the check-out?

> To make the example really useful for people, the mail app would need to
> be buildable and I'd need the extra files available in SVN (see attached
> patch). Would you be willing to apply the patch and eventually host the
> new cquery that points to the simplified local-resolution rmap (I'll
> deliver both if you are ok with my general approach) so that they will
> be available for public?
>
Yes, we'd be willing to do that. I think you should open a bugzilla where you can paste a copy of this discussion and
then attach patches. The reason for that is that Eclipse.org cannot accept patches unless they come as proper bugzilla
attachments. An accepted patch also means that your name is added to the contributor list of our IP-log.

Thomas Hallgren schrieb:
> Hi Johannes,
> Comments inline...
>
> Johannes Utzig wrote:
>> !MESSAGE Missing requirement: Equinox Provisioning Engine
>> 1.0.100.v20090525 (org.eclipse.equinox.p2.engine 1.0.100.v20090525)
>> requires 'package org.eclipse.equinox.internal.p2.persistence 0.0.0'
>> but it could not be found
>>
> This sounds like a regression of some kind. It used to work but it's
> been a while since anyone tried it.
>
I see :)

>> First of all, do you think a tutorial like that would make sense in
>> the wiki?
> Yes, absolutely. I think it's great that you do this. We are in a bad
> shape when it comes to examples and tutorials so a contribution like
> that is very valuable and appreciated.
>
I'm glad you like the idea
>> And if so, do you agree with the way I'm building things, or should I
>> change something for the wiki page?
>>
> I agree that the sample should have an RMAP of its own but I'm not too
> keen about using an approach where Hudson populates the workspace. By
> doing that, you bypass the whole resolution/materialization piece in
> Buckminster. Not so good if the purpose is to use it all as an example.
> It would also impede future examples of more advanced features such as
> CQUERY based branch/tag selection.

I don't think it's really bypassing. Materialization takes still place
(kinda :) )since there is no workspace, just checked out modules.
Resolution also takes place, just a simplified version with only two
possible location: filesystem or target platform (other would be
possible as well if you want for example some maven artifacts from your
company's repository).
The simple file-system resolution makes a lot of things easier for
Hudson in my opinion:
-you do not need to configure VCS credentials since this is done
globally (by the administrator) in the hudson config
-you get the change log on every build
-the logs can be used to do advanced automated stuff like blaming test
failures, updating bugzilla, report and changelog generation...
-workspaces do not usually get deleted in hudson before/after a build
and I'm not even sure if buckminster can do a cvs/svn update, or if you
always have to do a full checkout (which is way slower). If it can, the
only way to achieve this would be to maintain an additional mspec, or is
there another way?

But I definetly do get your point, for an example it would make sense to
demonstrate the buckminster capabilities, I'm just not so sure if it
really is the best, or rather, easiest approach in a CI server.

>
> Would it be possible to make Hudson understand that it should use
> Buckminster as the check-out tool? Or perhaps we could make Hudson use
> the information that it finds in a BOM file (the intermediate file that
> Buckminster creates when it runs the resolver) to do the check-out?
>
That actually sounds very interesting. I can't answer that just now, but
I will think about how that could be done.
However, if for example the buckminster hudson plugin would also
register itself as as SCM plugin there's still the question I asked
above: can buckminster currently do svn, cvs, perforce,... updates, or
only checkout? Can the commit comments be retrieved? Can it be done
without an additional mspec?
Without commit logs, updates in favor of full check-outs and globally
configured credientials, I doubt that many hudson users would do the
check out with buckminster.
But I definetly like the idea, and if you think that there is a way to
preserve those features, then I'd be in.

About the later (using the bom to check-out) I'm not completly sure, but
I don't think it will work because SCM plugins always run before build
plugins run and all the check-out/update stuff is not done by the hudson
core, but specific plugins depending on the SCM in use. I'll have a look
but I don't think it could currently be done without changing a lot of
hudson code for this very special use-case.

>> To make the example really useful for people, the mail app would need
>> to be buildable and I'd need the extra files available in SVN (see
>> attached patch). Would you be willing to apply the patch and
>> eventually host the new cquery that points to the simplified
>> local-resolution rmap (I'll deliver both if you are ok with my general
>> approach) so that they will be available for public?
>>
> Yes, we'd be willing to do that. I think you should open a bugzilla
> where you can paste a copy of this discussion and then attach patches.
> The reason for that is that Eclipse.org cannot accept patches unless
> they come as proper bugzilla attachments. An accepted patch also means
> that your name is added to the contributor list of our IP-log.
>
> Regards,
> Thomas Hallgren

I totally agree that the best strategy for integration builds is to have the
CI server checkout/update the source. Buckminster can populate the workspace
using a local resolver. This is much faster than having Buckminster perform
the checkout.

Having said that, I see the point in the example showing SVN checkout
instead of local resolution. Maybe we can have _both_ resolvers in the RMAP
and select the appropriate one using a resolutionFilter? It would default to
SVN checkout, but in the presence of a "integrationBuild" property, the RMAP
could then select the local resolver.

I'd like to bring up another point for a _really_ comprehensive example:
target platform. I am a strong proponent for defining a target platform up
front that everybody, i.e. all developers and the integration build, compile
against. So the first step in a build process would be to populate the
target platform. We're currently using Buckminster to do this, although I
really like PDE's new target platform story. At the moment, though, it can
only deal with exact bundle versions, which is not really what I want. A
combination of Buckminster and PDE would be great. Maybe we can hook
Buckminster into PDE as a target platform source. I'll investigate that...

Cheers,
Achim

Johannes Utzig wrote:

> Hi Thomas,
>
> comments inline as well
>
> Thomas Hallgren schrieb:
>> Hi Johannes,
>> Comments inline...
>>
>> Johannes Utzig wrote:
>>> !MESSAGE Missing requirement: Equinox Provisioning Engine
>>> 1.0.100.v20090525 (org.eclipse.equinox.p2.engine 1.0.100.v20090525)
>>> requires 'package org.eclipse.equinox.internal.p2.persistence 0.0.0'
>>> but it could not be found
>>>
>> This sounds like a regression of some kind. It used to work but it's
>> been a while since anyone tried it.
>>
> I see :)
>
>>> First of all, do you think a tutorial like that would make sense in
>>> the wiki?
>> Yes, absolutely. I think it's great that you do this. We are in a bad
>> shape when it comes to examples and tutorials so a contribution like
>> that is very valuable and appreciated.
>>
> I'm glad you like the idea
>>> And if so, do you agree with the way I'm building things, or should I
>>> change something for the wiki page?
>>>
>> I agree that the sample should have an RMAP of its own but I'm not too
>> keen about using an approach where Hudson populates the workspace. By
>> doing that, you bypass the whole resolution/materialization piece in
>> Buckminster. Not so good if the purpose is to use it all as an example.
>> It would also impede future examples of more advanced features such as
>> CQUERY based branch/tag selection.
>
> I don't think it's really bypassing. Materialization takes still place
> (kinda :) )since there is no workspace, just checked out modules.
> Resolution also takes place, just a simplified version with only two
> possible location: filesystem or target platform (other would be
> possible as well if you want for example some maven artifacts from your
> company's repository).
> The simple file-system resolution makes a lot of things easier for
> Hudson in my opinion:
> -you do not need to configure VCS credentials since this is done
> globally (by the administrator) in the hudson config
> -you get the change log on every build
> -the logs can be used to do advanced automated stuff like blaming test
> failures, updating bugzilla, report and changelog generation...
> -workspaces do not usually get deleted in hudson before/after a build
> and I'm not even sure if buckminster can do a cvs/svn update, or if you
> always have to do a full checkout (which is way slower). If it can, the
> only way to achieve this would be to maintain an additional mspec, or is
> there another way?
>
> But I definetly do get your point, for an example it would make sense to
> demonstrate the buckminster capabilities, I'm just not so sure if it
> really is the best, or rather, easiest approach in a CI server.
>
>>
>> Would it be possible to make Hudson understand that it should use
>> Buckminster as the check-out tool? Or perhaps we could make Hudson use
>> the information that it finds in a BOM file (the intermediate file that
>> Buckminster creates when it runs the resolver) to do the check-out?
>>
> That actually sounds very interesting. I can't answer that just now, but
> I will think about how that could be done.
> However, if for example the buckminster hudson plugin would also
> register itself as as SCM plugin there's still the question I asked
> above: can buckminster currently do svn, cvs, perforce,... updates, or
> only checkout? Can the commit comments be retrieved? Can it be done
> without an additional mspec?
> Without commit logs, updates in favor of full check-outs and globally
> configured credientials, I doubt that many hudson users would do the
> check out with buckminster.
> But I definetly like the idea, and if you think that there is a way to
> preserve those features, then I'd be in.
>
> About the later (using the bom to check-out) I'm not completly sure, but
> I don't think it will work because SCM plugins always run before build
> plugins run and all the check-out/update stuff is not done by the hudson
> core, but specific plugins depending on the SCM in use. I'll have a look
> but I don't think it could currently be done without changing a lot of
> hudson code for this very special use-case.
>
>
>>> To make the example really useful for people, the mail app would need
>>> to be buildable and I'd need the extra files available in SVN (see
>>> attached patch). Would you be willing to apply the patch and
>>> eventually host the new cquery that points to the simplified
>>> local-resolution rmap (I'll deliver both if you are ok with my general
>>> approach) so that they will be available for public?
>>>
>> Yes, we'd be willing to do that. I think you should open a bugzilla
>> where you can paste a copy of this discussion and then attach patches.
>> The reason for that is that Eclipse.org cannot accept patches unless
>> they come as proper bugzilla attachments. An accepted patch also means
>> that your name is added to the contributor list of our IP-log.
>>
>> Regards,
>> Thomas Hallgren
>
> Ok, done. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=281503
> I hope it is ok like that...
>
>
> Best regards,
> Johannes

Achim Demelt wrote:
> Johannes,
>
> I totally agree that the best strategy for integration builds is to have the
> CI server checkout/update the source. Buckminster can populate the workspace
> using a local resolver. This is much faster than having Buckminster perform
> the checkout.
>
While it might be faster and give some advantages for the Hudson user, it also has serious implications:
1. The Hudson build is no longer the same build as you would run locally (reproducibility is key).
2. You no longer maintain the data about the SCM's in one place (the build model is scattered).
3. You cannot control the resolution/materialization with a CQUERY.
4. The list of material to check-out will be static.

Using Hudson to check out is perhaps faster but remember that the reason it takes a bit longer when Buckminster does it
is that Buckminster doesn't maintain a list of things to check out. The list is computed dynamically. AFAIK, Hudson is
not capable of doing that. This feature is of some importance since the developer that adds a bundle to a feature might
have insufficient privileges to change the Hudson configuration.

> Having said that, I see the point in the example showing SVN checkout
> instead of local resolution. Maybe we can have _both_ resolvers in the RMAP
> and select the appropriate one using a resolutionFilter? It would default to
> SVN checkout, but in the presence of a "integrationBuild" property, the RMAP
> could then select the local resolver.
>
That's better, but you still need to maintain a static list and you still have issues with permissions.

I would much rather see a solution where Hudson can actually benefit from what Buckminster finds. Which makes me
curious, what does Hudson need in terms of functionality in order to maintain its workspace + changelog? Perhaps it's
not that hard to accomplish.

> I'd like to bring up another point for a _really_ comprehensive example:
> target platform. I am a strong proponent for defining a target platform up
> front that everybody, i.e. all developers and the integration build, compile
> against. So the first step in a build process would be to populate the
> target platform. We're currently using Buckminster to do this, although I
> really like PDE's new target platform story. At the moment, though, it can
> only deal with exact bundle versions, which is not really what I want. A
> combination of Buckminster and PDE would be great. Maybe we can hook
> Buckminster into PDE as a target platform source. I'll investigate that...
>
IMO, Buckminster now has an even better way to populate the target platform. Simply use the eclipse.import reader and
the p2 materializer. The advantage is that the provisioning that takes place when you do it that way can be platform
independent and very exact (based on the resolution). The target platform provision mechanism provided with PDE will use
the planner when installing from a remote repository. The planner is unfortunately always bound to a particular platform
configuration. The PDE provider must also be told explicitly what features to include from the repository.

Also, Buckminster will provision the target platform in parallel with the SCM checkouts. How's that for speed ;-)

I generally structure my rmaps to prioritize local providers and then if
they fail, fall back to SCM checkout providers. This makes for fast
local builds for me, but for CI builds, I always check out from the
SCM. I don't use a resolutionFilter to do this - I just have the providers
in series, i.e.

Am I missing some supercool Buckminster feature I could be using
instead? (most likely ;)

> IMO, Buckminster now has an even better way to populate the target platform.
> Simply use the eclipse.import reader and the p2 materializer.

The eclipse.import reader I know about, but not the p2 materializer
- is there an example of how to use this anywhere I could look at?
I usually create the platforms manually, and stick with them - changes
occur infrequently (except with release train :) - but I don't want to
keep doing that, automation being the goal.

turns out that we're doing the exact same thing, just the order is
differently. I also use an rmap that is capable of resolving everything
through SCM, but in CI build the local resolution kicks in, whereas the
developers can use the cquery + rmap to populate their workspace from SCM.
As for target platforms, I also maintain them manually so far (install
everything I want, and copy them to the build server). This is not
perfect, just simply faster than any other way I found so far (and
target platform changes only once or twice a year fo me).
So if there are better ways now, I'd also love to hear about them.

Oisin Hurley schrieb:
> I generally structure my rmaps to prioritize local providers and then if
> they fail, fall back to SCM checkout providers. This makes for fast
> local builds for me, but for CI builds, I always check out from the
> SCM. I don't use a resolutionFilter to do this - I just have the providers
> in series, i.e.
>
> <searchPath name="sca.features">
> <provider readerType="local"
> componentTypes="eclipse.feature,osgi.bundle" mutable="true"
> source="false">
> <uri format="{0}/org.eclipse.stp.sca/features/{1}">
> <bc:propertyRef key="project.root" />
> <bc:propertyRef key="buckminster.component" />
> </uri>
> </provider>
> <provider readerType="svn"
> componentTypes="osgi.bundle,eclipse.feature,buckminster"
> mutable="true" source="true">
> <uri format=" http://dev.eclipse.org/svnroot/stp/org.eclipse.stp.sca-tools /org.eclipse.stp.sca/trunk/features/{0}">
> <bc:propertyRef key="buckminster.component" />
> </uri>
> </provider>
> </searchPath>
>
> Am I missing some supercool Buckminster feature I could be using
> instead? (most likely ;)
>
>> IMO, Buckminster now has an even better way to populate the target platform.
>> Simply use the eclipse.import reader and the p2 materializer.
>
> The eclipse.import reader I know about, but not the p2 materializer
> - is there an example of how to use this anywhere I could look at?
> I usually create the platforms manually, and stick with them - changes
> occur infrequently (except with release train :) - but I don't want to
> keep doing that, automation being the goal.
>
> cheers
> --oh

Thomas Hallgren schrieb:
> Hi Achim and Johannes,
>
> Achim Demelt wrote:
>> Johannes,
>>
>> I totally agree that the best strategy for integration builds is to
>> have the CI server checkout/update the source. Buckminster can
>> populate the workspace using a local resolver. This is much faster
>> than having Buckminster perform the checkout.
>>
> While it might be faster and give some advantages for the Hudson user,
> it also has serious implications:
> 1. The Hudson build is no longer the same build as you would run locally
> (reproducibility is key).
Well, it is, as long as the target platform is the same. Doesn't matter
who checks something out, as long as it is the same thing that's checked
out.
> 2. You no longer maintain the data about the SCM's in one place (the
> build model is scattered).
My rmap contains both the local resolver and the SCM resolver actually.
A new developer resolves the query to populate the workspace from SCM
and once that's set up, just invokes the action which will act in
exactly the same way as it does in the CI environment.
> 3. You cannot control the resolution/materialization with a CQUERY.
A cquery is involved and maintains what's getting resolved. The only
thing that's controled by the CI system is the availablity of certain
artifacts
> 4. The list of material to check-out will be static.
Not necessarily static. I'd say it is a very common scenario that all
the bundles involved in a specific build job reside in one common module
in the SCM. In the hudson job you check out that parent module and let
buckminster resolve whatever other artifacts are necessary and allowed.
You add a new bundle to the scm -> the checkout job will get it, throw
it into the hudson workspace, buckminster will resolve try to resolve
the query again, notices a new dependency and tries to pick it up.

>
> Using Hudson to check out is perhaps faster but remember that the reason
> it takes a bit longer when Buckminster does it is that Buckminster
> doesn't maintain a list of things to check out. The list is computed
> dynamically. AFAIK, Hudson is not capable of doing that. This feature is
> of some importance since the developer that adds a bundle to a feature
> might have insufficient privileges to change the Hudson configuration.
>
See above, most of the time you'll just check out the parent module, and
even if that's more than you actually need, buckminster won't even see
the additional artifacts.

>> Having said that, I see the point in the example showing SVN checkout
>> instead of local resolution. Maybe we can have _both_ resolvers in the
>> RMAP and select the appropriate one using a resolutionFilter? It would
>> default to SVN checkout, but in the presence of a "integrationBuild"
>> property, the RMAP could then select the local resolver.
>>
> That's better, but you still need to maintain a static list and you
> still have issues with permissions.
>
> I would much rather see a solution where Hudson can actually benefit
> from what Buckminster finds. Which makes me curious, what does Hudson
> need in terms of functionality in order to maintain its workspace +
> changelog? Perhaps it's not that hard to accomplish.
>
I'd really be interested in that option.
It should be possible since AFAIK it is the SCM plugin's responsibility
to fill the hudson workspace with content and hudson shouldn't care at
all how that happens and where everything comes from.
The thing really is: how to make buckminster behave well as a 'meta scm'
for hudson? How to force an update on every build, how to retrieve the
commit logs? If you show me the way , I'll do my best to implement the
dirty little details.

>> I'd like to bring up another point for a _really_ comprehensive
>> example: target platform. I am a strong proponent for defining a
>> target platform up front that everybody, i.e. all developers and the
>> integration build, compile against. So the first step in a build
>> process would be to populate the target platform. We're currently
>> using Buckminster to do this, although I really like PDE's new target
>> platform story. At the moment, though, it can only deal with exact
>> bundle versions, which is not really what I want. A combination of
>> Buckminster and PDE would be great. Maybe we can hook Buckminster into
>> PDE as a target platform source. I'll investigate that...
>>
> IMO, Buckminster now has an even better way to populate the target
> platform. Simply use the eclipse.import reader and the p2 materializer.
> The advantage is that the provisioning that takes place when you do it
> that way can be platform independent and very exact (based on the
> resolution). The target platform provision mechanism provided with PDE
> will use the planner when installing from a remote repository. The
> planner is unfortunately always bound to a particular platform
> configuration. The PDE provider must also be told explicitly what
> features to include from the repository.
>
> Also, Buckminster will provision the target platform in parallel with
> the SCM checkouts. How's that for speed ;-)
>
> Regards,
> Thomas Hallgren

Johannes Utzig wrote:
> Hi Oisin,
>
>
> turns out that we're doing the exact same thing, just the order is
> differently. I also use an rmap that is capable of resolving everything
> through SCM, but in CI build the local resolution kicks in, whereas the
> developers can use the cquery + rmap to populate their workspace from SCM.
> As for target platforms, I also maintain them manually so far (install
> everything I want, and copy them to the build server). This is not
> perfect, just simply faster than any other way I found so far (and
> target platform changes only once or twice a year fo me).
> So if there are better ways now, I'd also love to hear about them.
>
I'm working on getting a Buckminster build of Buckminster itself running using Hudson at Eclipse.org. You can look at
the files here:

So what this does is essentially:
-download p2 director
-create a headless buckminster with the features necessary for the build
with p2
-import an mspec into a not-yet-existing target platform? Did I get this
part right? That sounds really nice, I didn't know that was possible...
-import the buckminster sources with the dev-query
-build the site

Well, that surely is an interesting example, thank you for sharing it.

For the tutorial things would be somewhat easier though, because
buckminster would already be up and running.

I'm not sure I'm completly understanding all of this just yet, so let me
try to put it together. For the tutorial this would mean:

1. buckminster uses an mspec to materialize a platform independant
target platform out of thin air

2. set the target platform to the output of 1

3. resolve a cquery for mailapp.product.feature (bundles and features
will be fetched from SVN, the rest comes from the target platform)

4. build site

5. create product with p2 director

Is that kinda what you meant?
If the RCP needs more than the bare platform, would you write several
mspecs to import or introduce another component (cspec, or feature?) to
describe the contents of the target platform?

Johannes Utzig wrote:
> -import an mspec into a not-yet-existing target platform? Did I get this
> part right? That sounds really nice, I didn't know that was possible...
>
Yes, that's right.

> I'm not sure I'm completly understanding all of this just yet, so let me
> try to put it together. For the tutorial this would mean:
>
> 1. buckminster uses an mspec to materialize a platform independant
> target platform out of thin air
>
> 2. set the target platform to the output of 1
>
> 3. resolve a cquery for mailapp.product.feature (bundles and features
> will be fetched from SVN, the rest comes from the target platform)
>
> 4. build site
>
> 5. create product with p2 director
>
> Is that kinda what you meant?
>
Yes.

> If the RCP needs more than the bare platform, would you write several
> mspecs to import or introduce another component (cspec, or feature?) to
> describe the contents of the target platform?
>
No, I would use one mspec and make the rmap richer. Or perhaps appoint the Galileo release instead of just the Eclipse
Platform (Galileo includes the platform). If need be, appoint Galileo+ http://download.cloudsmith.com/galileoplus/
(Galileo plus a lot of stuff that isn't hosted at Eclipse.org. Subversion connectors for instance).

> ... how to make buckminster behave well as a 'meta scm'
> for hudson? How to force an update on every build, how to retrieve the
> commit logs? If you show me the way , I'll do my best to implement the
> dirty little details.
>
OK, let's try and resolve this. My knowledge of Hudson is very limited so I think we need to meet half way here. Can you
give me some pointers to where and how an SCM would be hooked in?

I know you said that the check-out is made before the build, but perhaps this could be set up as two builds? The first
would just produce a list of things to check-out/update and Hudson would uses that the second. The way to accomplish
that would be to run a buckminster import that just produces a BOM (no materialization), then extract the relevant SCM
entries from the BOM and feed that list to Hudson.

Thomas Hallgren schrieb:
> Johannes Utzig wrote:
>
>> ... how to make buckminster behave well as a 'meta scm' for hudson?
>> How to force an update on every build, how to retrieve the commit
>> logs? If you show me the way , I'll do my best to implement the dirty
>> little details.
>>
> OK, let's try and resolve this. My knowledge of Hudson is very limited
> so I think we need to meet half way here. Can you give me some pointers
> to where and how an SCM would be hooked in?
>
> I know you said that the check-out is made before the build, but perhaps
> this could be set up as two builds? The first would just produce a list
> of things to check-out/update and Hudson would uses that the second. The
> way to accomplish that would be to run a buckminster import that just
> produces a BOM (no materialization), then extract the relevant SCM
> entries from the BOM and feed that list to Hudson.
>
> Regards,
> Thomas Hallgren

The Hudson philosophy (for a freestyle project) is fairly simple.
A job uses 0 to 1 SCM plugins that can be configured in the job
configuration to check out things into the job's workspace.

After that 0 to n build steps are performed on that workspace.

Once the build is done 0 to n publishers can be invoked that do stuff
like gathering test result, checking for TODOs in the source code, and
so on.

So hooking up a new SCM plugin is fairly simple from what I know. You
write a plugin, add some configuration settings and are free to execute
code. I could try doing that if I'd know how to force update behavior
and which part of buckminster I'd have to patch to be able to receive
the change logs.
For a proof of concept however, it might be sufficient to force a
replace and ignore the commit logs. That would result in the required
behavior and the rest could be added later.

The bom idea would IMO not work because the existing SCM plugins are
meant to be configured in the job configuration (i.e. which modules they
should check out for the job). I don't think it really fits into the
concept to have a pre-scm step that actually changes the job
configuration for the scm plugin that's invoked after. More so: I don't
think it's currently even possible to check out stuff from let's say
both CVS and SVN for the same job.
With a completely new job kind specific to buckminster projects it might
be possible to create a bom and feed SCM plugins from that, but that's
beyond my so far gathered hudson-knowledge and would be more of a long
term goal.

>
>> If the RCP needs more than the bare platform, would you write several
>> mspecs to import or introduce another component (cspec, or feature?) to
>> describe the contents of the target platform?
>>
> No, I would use one mspec and make the rmap richer. Or perhaps appoint the
> Galileo release instead of just the Eclipse Platform (Galileo includes the
> platform). If need be, appoint Galileo+
> http://download.cloudsmith.com/galileoplus/ (Galileo plus a lot of stuff
> that isn't hosted at Eclipse.org. Subversion connectors for instance).
>

How would making the RMAP richer help? There can be only one root request
per CQUERY. If _one_ feature doesn't describe your whole platform, you'd
either have to define a pseudo-feature or a Buckminster component with a
CSPEC. We went for the second approach. The only drawback is that the p2
materializer can only cope with eclipse.feature and osgi.bundle components.
We therefore have this additional line in our MSPEC:

Achim Demelt wrote:
>>> If the RCP needs more than the bare platform, would you write several
>>> mspecs to import or introduce another component (cspec, or feature?) to
>>> describe the contents of the target platform?
>>>
>> No, I would use one mspec and make the rmap richer. Or perhaps appoint the
>> Galileo release instead of just the Eclipse Platform (Galileo includes the
>> platform). If need be, appoint Galileo+
>> http://download.cloudsmith.com/galileoplus/ (Galileo plus a lot of stuff
>> that isn't hosted at Eclipse.org. Subversion connectors for instance).
>>
>
> How would making the RMAP richer help? There can be only one root request
> per CQUERY. If _one_ feature doesn't describe your whole platform, you'd
> either have to define a pseudo-feature or a Buckminster component with a
> CSPEC. We went for the second approach. The only drawback is that the p2
> materializer can only cope with eclipse.feature and osgi.bundle components.
> We therefore have this additional line in our MSPEC:
>
> <md:mspecNode namePattern=".*" componentType="buckminster" exclude="true"/>
>
> But maybe I'm missing something here...
>
If the requirement is to have one CQUERY that only describes the target platform, and if there is no feature present
that is suitable, then yes, you'd need to do something like that. But there are alternatives.

1. Do not use a special component for the TP. Simply default to saying that everything that isn't found in your source
defaults to be TP components. This requires a more advanced MSPEC of course since it needs to specify what goes where
(WS or TP).
2. The KISS apporach. Run several CQUERY's. One for each separate TP feature that you want to install. Buckminster will
not install the same thing twice so overlaps is not a problem.

I like your approach better then #2 but I wanted to mention it anyway.

> OK, let's try and resolve this. My knowledge of Hudson is very limited
> so I think we need to meet half way here. Can you give me some pointers
> to where and how an SCM would be hooked in?
>

For the buckminster case we'd have very few configuration options (no
urls or anything) which is good.
Hudson expects an SCM plugin to behave like that:

let hudson know if the SCM supports polling, if a workspace is necessary
for polling and if polling is supported, determine if there have been
changes since the last build that would require an update/check-out. In
easiest case we could simply not support polling.

If there are changes, or polling is unsupported, hudson will ask to SCM
plugin to populate the workspace and passes the configuration of the
form your plugin added to the job configuration.
In buckminster case this would be a list of cquerys or mspecs I'd say.

If an update can be implemented in favor of a 'replace', the plugin
should gather the changes (which files, comment, author, time,
action,...) and persist those into a file changelog.xml which will be
evaluated by hudson to present it to the user.

Optionally an scm plugin could also offer a repository viewer if the SCM
system supports that (like the web viewer for SVN + Apache). The user
would use that feature to click on entries in the change log. This would
be rather difficult for a buckminster meta-scm since there might be
different systems involved, but I don't think it's that important.

So yes, I'm pretty sure I could implement something like that in hudson
if only there are ways to have buckminster update things and tell me
about the changes.

In fact this approach would be quite appealing since there'd be almost
no configuration effort in hudson, just query what you want and have
buckminster worry about how to get that.

One thing I don't like too much though, I hate leaking SCM credentials
in a resource map. It should be possible to write a buckminster
extension that accepts credentials with the setpref commands, right? If
so (and if we can solve the other issues), I'd rather code this logic
into the hudson plugin. Credentials for repositories are entered into
the global hudson configuration and the plugin sets the credentials
silently with 'setpref' before executing the actual 'resolve' the user
wanted.

Johannes Utzig wrote:
> Hi Thomas,
>
> I was reading some more and here is what I found out so far:
>
>> OK, let's try and resolve this. My knowledge of Hudson is very limited
>> so I think we need to meet half way here. Can you give me some
>> pointers to where and how an SCM would be hooked in?
>>
>
> For the buckminster case we'd have very few configuration options (no
> urls or anything) which is good.
> Hudson expects an SCM plugin to behave like that:
>
> let hudson know if the SCM supports polling, if a workspace is necessary
> for polling and if polling is supported, determine if there have been
> changes since the last build that would require an update/check-out. In
> easiest case we could simply not support polling.
>
I think polling can be supported later. Let's start without.

> If there are changes, or polling is unsupported, hudson will ask to SCM
> plugin to populate the workspace and passes the configuration of the
> form your plugin added to the job configuration.
> In buckminster case this would be a list of cquerys or mspecs I'd say.
>
Yes, that sounds reasonable.

> If an update can be implemented in favor of a 'replace', the plugin
> should gather the changes (which files, comment, author, time,
> action,...) and persist those into a file changelog.xml which will be
> evaluated by hudson to present it to the user.
>
This should be added to the CVS and SVN providers in Buckminster. It cannot currently handle that.

> Optionally an scm plugin could also offer a repository viewer if the SCM
> system supports that (like the web viewer for SVN + Apache). The user
> would use that feature to click on entries in the change log. This would
> be rather difficult for a buckminster meta-scm since there might be
> different systems involved, but I don't think it's that important.
>
Perhaps we can use change-log entries that will tell Buckminster how to redirect to a proper viewer. The viewer as such
is of course beyond Buckminsters scope.

> So yes, I'm pretty sure I could implement something like that in hudson
> if only there are ways to have buckminster update things and tell me
> about the changes.
>
> In fact this approach would be quite appealing since there'd be almost
> no configuration effort in hudson, just query what you want and have
> buckminster worry about how to get that.
>
OK, so I read this to mean that if a buckminster query is re-run on a workspace, it should have the ability to update
that workspace and collect a change-log of the updated entries. That log must then be available (using some command) so
that Hudson can obtain it. Does that sound OK?

One question here is how to handle things that are not source (binaries imported to the project or added to the target
platform).

> One thing I don't like too much though, I hate leaking SCM credentials
> in a resource map. It should be possible to write a buckminster
> extension that accepts credentials with the setpref commands, right? If
> so (and if we can solve the other issues), I'd rather code this logic
> into the hudson plugin. Credentials for repositories are entered into
> the global hudson configuration and the plugin sets the credentials
> silently with 'setpref' before executing the actual 'resolve' the user
> wanted.
>
I agree. This is something that needs to be resolved in a much better way. A first cut could simply pass credentials on
the command line (using -Dsvn.user=xxx -Dsvn.pass=yyy). A corresponding RMAP entry can then look like this:

Thomas Hallgren schrieb:
> Hi Johannes,
> Answers inline.
>
> Johannes Utzig wrote:
>> Hi Thomas,
>>
>> I was reading some more and here is what I found out so far:
>>
>>> OK, let's try and resolve this. My knowledge of Hudson is very
>>> limited so I think we need to meet half way here. Can you give me
>>> some pointers to where and how an SCM would be hooked in?
>>>
>>
>> For the buckminster case we'd have very few configuration options (no
>> urls or anything) which is good.
>> Hudson expects an SCM plugin to behave like that:
>>
>> let hudson know if the SCM supports polling, if a workspace is
>> necessary for polling and if polling is supported, determine if there
>> have been changes since the last build that would require an
>> update/check-out. In easiest case we could simply not support polling.
>>
> I think polling can be supported later. Let's start without.

Yeah, polling really is no big deal at first. I'm not even sure if it
makes much sense for buckminster since you'd have to poll possibly many
different kinds of source repositories.

>
>> If there are changes, or polling is unsupported, hudson will ask to
>> SCM plugin to populate the workspace and passes the configuration of
>> the form your plugin added to the job configuration.
>> In buckminster case this would be a list of cquerys or mspecs I'd say.
>>
> Yes, that sounds reasonable.
>
>> If an update can be implemented in favor of a 'replace', the plugin
>> should gather the changes (which files, comment, author, time,
>> action,...) and persist those into a file changelog.xml which will be
>> evaluated by hudson to present it to the user.
>>
> This should be added to the CVS and SVN providers in Buckminster. It
> cannot currently handle that.

As long as you think it's possible. I'll have a look at the
implementation to get an impression on how big the changes would have to
be. Might take a few days though until I can get to that, though.

>
>> Optionally an scm plugin could also offer a repository viewer if the
>> SCM system supports that (like the web viewer for SVN + Apache). The
>> user would use that feature to click on entries in the change log.
>> This would be rather difficult for a buckminster meta-scm since there
>> might be different systems involved, but I don't think it's that
>> important.
>>
> Perhaps we can use change-log entries that will tell Buckminster how to
> redirect to a proper viewer. The viewer as such is of course beyond
> Buckminsters scope.

A viewer is a very optional thing, like the polling, I only mentioned it
to sum up how things currently work in hudson (to the best I know). I
mean, it could be kinda nice to have the svn repository url together
with the change logs, but it's really not that important I think.
Besides, the viewer is also out of scope for hudson itself, I think it's
meant to be just a link to for example the webfrontend of e.g SVN.

>
>> So yes, I'm pretty sure I could implement something like that in
>> hudson if only there are ways to have buckminster update things and
>> tell me about the changes.
>>
>> In fact this approach would be quite appealing since there'd be almost
>> no configuration effort in hudson, just query what you want and have
>> buckminster worry about how to get that.
>>
> OK, so I read this to mean that if a buckminster query is re-run on a
> workspace, it should have the ability to update that workspace and
> collect a change-log of the updated entries. That log must then be
> available (using some command) so that Hudson can obtain it. Does that
> sound OK?

Yes, that's exactly what I meant. Some way to tell buckminster "do a
cvs/svn update on the stuff that was previously resolved into this
workspace and tell me what happened meanwhile". I think that could also
make sense for regular buckminster usage from within eclipse. Re-run a
cquery/mspec and have the option of fetching updates from the SCM on the
way (not the default option though, but it would be a nice quick way to
get up to date with two clicks).

>
> One question here is how to handle things that are not source (binaries
> imported to the project or added to the target platform).
>

I don't think that matters at all. Let's say a maven artifact got
updated. I wouldn't expect to see an actual SCM change log for that. The
change log is interesting for CI because of the automated unit testing.
Out of a sudden one of your tests fails. You click on the build that
failed, click on the change log in hudson and see 'ah, developer XY
changed the class that failed in the test-runs since the last build' and
you know immediately what's wrong with your build. The other cool thing
about the change log is reporting. I think most companies have some kind
of naming scheme for commit comment on fixed bugs. You can parse the log
entries for something like [BUG#12345] and automatically generate change
logs from that to let people know what has been improved from 1.0.0 to
1.0.1.
Binary artifacts don't count for much there because
1. you usually don't add a comment to whatever artifact repository you
are using when you deploy a new version, so what information would you
expect to see there
2. binary artifacts are out of scope for unit testing (they'd only
matter for integration tests).

>> One thing I don't like too much though, I hate leaking SCM credentials
>> in a resource map. It should be possible to write a buckminster
>> extension that accepts credentials with the setpref commands, right?
>> If so (and if we can solve the other issues), I'd rather code this
>> logic into the hudson plugin. Credentials for repositories are entered
>> into the global hudson configuration and the plugin sets the
>> credentials silently with 'setpref' before executing the actual
>> 'resolve' the user wanted.
>>
> I agree. This is something that needs to be resolved in a much better
> way. A first cut could simply pass credentials on the command line
> (using -Dsvn.user=xxx -Dsvn.pass=yyy). A corresponding RMAP entry can
> then look like this:
>
> <uri format="svn+ssh://{0}:{1}@oursvn/path/trunk/{2}">
> <bc:propertyRef key="svn.user" />
> <bc:propertyRef key="svn.pass" />
> <bc:propertyRef key="buckminster.component" />
> </uri>
>
> - thomas

This sounds perfectly fine to me, simple properties would be the easiest
way to get the desired behaviour.

This could really be a very cool feature and with hudson getting the
most popular CI server on the market, I'm sure tight integration can be
a boost for buckminster's popularity (just think of the cool demos you
could do with that :) ).
However, it looks like there's still a bit to do before we got all the
pieces together, but an RCP build tutorial + CI system integration could
be done much earlier already, even though without the cool meta-scm feature.

So how would you like it to look like?
Since you'd like to see buckminster do the actual checkout, the
workspace needs to be cleaned up on every build. Therefore the first
hudson build step would be a shell invokation that wipes the hudson
workspace.
The next build step will invoke buckminster and place 4 commands:
1. passes an mspec to materialize the target platform for the MailApp.
similar to what you've shown me, right?
2. The 2nd will be setPref to set the target platform location to the
newly created one
3. The next command will be resolve mailapp.product.feature to do the
checkout from svn
4. the last command will be a mailapp.product.feature#create.product.zip
to build the actual RCP with the ant script from the patch submitted to
bugzilla.

Does that sound ok to you? If the patch is ok to you we'd only need the
rmap, cquery and mspec then. I'm not too familiar with the mspecs yet
because I could find virtually no documentation for their usage except
for the schema, but I'll I'll do my best to deliver those three pieces
within the next days.

I added a first draft of the tutorial in the wiki. Maybe you can read it
one of these days and let me know if the general approach is ok with you
and what should be changed.
Once it's good enough for a first version, I can create a bugzilla and
attach the MSPECs, CQUERYs and the reduced rmap so you can put it online
somewhere. Once the files are in place I'll update the tutorial to point
to those files and put it in the buckminster tutorial category if that's
ok with you.

Right now, I'm testing Buckminster in Hudson by having Hudson check out
the sources and pointing Buckminster at them as a local source for
materialization. Besides all the problems incumbent with a static
checkout list etc - and as has been pointed out elsewhere - Buckminster
doesn't recognize that the sources have come from SVN originally, so I
can't use the subversion latest revision qualifier replacement in the
plugin/feature versions.

Is there any movement on the idea of introducing the following features to
the buckminster reader system?

- change log export
- update cquery in existing workspace

I'd be willing to lend a hand if appropriate to help get to a seamless
integration with Hudson. Would it be best for me to just check out the
buckminster workspace and begin to get my bearings?

I'm definetly hoping that this feature can make it into buckminster and
I'll gladly add the necessary code into the hudson plugin once
buckminster supports that scenario. Unfortunately I wasn't able just yet
to write a patch for buckminster to allow update and persisting of the
change log because I wanted to enhance the target platform support
first, since that was personally the bigger pain for me (and because I
have a regular job of course ;P). So if you'd like to give it a shot on
the buckminster side, your help would be very much appreciated.

Best regards,
Johannes

Peter M. Murray schrieb:
>
> A tight integration between Hudson & Buckminster would be super cool.
> Right now, I'm testing Buckminster in Hudson by having Hudson check out
> the sources and pointing Buckminster at them as a local source for
> materialization. Besides all the problems incumbent with a static
> checkout list etc - and as has been pointed out elsewhere - Buckminster
> doesn't recognize that the sources have come from SVN originally, so I
> can't use the subversion latest revision qualifier replacement in the
> plugin/feature versions.
>
> Is there any movement on the idea of introducing the following features
> to the buckminster reader system?
>
> - change log export
> - update cquery in existing workspace
>
> I'd be willing to lend a hand if appropriate to help get to a seamless
> integration with Hudson. Would it be best for me to just check out the
> buckminster workspace and begin to get my bearings?
>
> Cheers,
>
> pete
>

I made some additions to the tutorial and I was hoping that you could
host the cquerys/mspecs/rmap along with the other buckminster demo
material to make this thing actually executable without any additional
files.
If the tutorial is missing anything or if I should do something
differently, please let me know.
The necessary files are attached to this mail.

Great!
We should make this part of the mailapp RCP tutorial.
So, a patch against the tutorial project is much appreciated.
(track IP etc.)
Then we can host the necessary files separately as well.

Johannes Utzig wrote:
> Hi,
>
> I made some additions to the tutorial and I was hoping that you could
> host the cquerys/mspecs/rmap along with the other buckminster demo
> material to make this thing actually executable without any additional
> files.
> If the tutorial is missing anything or if I should do something
> differently, please let me know.
> The necessary files are attached to this mail.
>
> http://wiki.eclipse.org/Building_an_RCP_application_with_hud son_(Buckminster)
>
>
> Best regards,
> Johannes

I'm not sure I understand you right. A patch against the tutorial
project? I already added a patch to make it buildable for the tutorial
and it has been applied, or are you talking about the MSPECs, CSPECs and
the RMAP? Should they live in the releng project of the tutorial? If so,
I can create a patch for that of course, just wasn't sure if that's what
you meant because the other demo queries and rmaps seem to live in http://www.eclipse.org/buckminster/samples...

If you guys are ok with the general approach and the tutorial can be
added to the buckminster examples section, I think it could
incrementally demonstrate more advanced features of CI. I could think of
-unit testing
-code coverage
-benchmarks
-metrics (like sonar)
-build triggers on commit action
-building specific branches/tags
-qualifier replacement

What do you think?

It's also quite handy IMO to see the current pains of buckminster
integration in real life.
-no changelog for checkout
-no update functionality (which forces the user to delete the workspace
prior to a build)
-no polling support (which makes it hard to trigger a build on commit
actions)
-plugin unit tests are currently difficult to execute

Until Peter has finished the 'Buckminster updating / change-list
generation' and issue 243293 (JUnit support) is resolved, the tutorial
can help explaining how that's currently done, and once those features
are in place, the tutorial can be an example on how those new
capabilities can be utilized.

Btw: The eclipse foundation has a hudson up and running as well, right?
Would it maybe be possible to set up an example job for the Mail-App RCP
and give anonymous users the right to view the job configuration?

Best regards,
Johannes

Henrik Lindberg schrieb:
> Great!
> We should make this part of the mailapp RCP tutorial.
> So, a patch against the tutorial project is much appreciated.
> (track IP etc.)
> Then we can host the necessary files separately as well.
>
> Johannes Utzig wrote:
>> Hi,
>>
>> I made some additions to the tutorial and I was hoping that you could
>> host the cquerys/mspecs/rmap along with the other buckminster demo
>> material to make this thing actually executable without any additional
>> files.
>> If the tutorial is missing anything or if I should do something
>> differently, please let me know.
>> The necessary files are attached to this mail.
>>
>> http://wiki.eclipse.org/Building_an_RCP_application_with_hud son_(Buckminster)
>>
>>
>> Best regards,
>> Johannes

Johannes Utzig wrote:
> Hi Henrik,
>
> I'm not sure I understand you right. A patch against the tutorial
> project? I already added a patch to make it buildable for the tutorial
> and it has been applied, or are you talking about the MSPECs, CSPECs and
> the RMAP? Should they live in the releng project of the tutorial?
That is what I had in mind.

>If so,
> I can create a patch for that of course, just wasn't sure if that's what
> you meant because the other demo queries and rmaps seem to live in
> http://www.eclipse.org/buckminster/samples...
>
The artifacts that need to be easily shareable (i.e. starting from
nothing) should be published there as well.

> If you guys are ok with the general approach and the tutorial can be
> added to the buckminster examples section, I think it could
> incrementally demonstrate more advanced features of CI. I could think of
> -unit testing
> -code coverage
> -benchmarks
> -metrics (like sonar)
> -build triggers on commit action
> -building specific branches/tags
> -qualifier replacement
>
> What do you think?
>
I think it is an excellent idea. We have to decide on if we want
something that builds up gradually (different instances), or if there is
one that includes everything. Maybe have two? The current mailapp
tutorial as it is to not overload it with stuff, and then one with
everything (and extra mayo :).

> It's also quite handy IMO to see the current pains of buckminster
> integration in real life.
> -no changelog for checkout
> -no update functionality (which forces the user to delete the workspace
> prior to a build)
> -no polling support (which makes it hard to trigger a build on commit
> actions)
> -plugin unit tests are currently difficult to execute
>
> Until Peter has finished the 'Buckminster updating / change-list
> generation' and issue 243293 (JUnit support) is resolved, the tutorial
> can help explaining how that's currently done, and once those features
> are in place, the tutorial can be an example on how those new
> capabilities can be utilized.
>
Issues understood - do you think that is worth while? Isn't it better to
wait for the proper solution and document that?

> Btw: The eclipse foundation has a hudson up and running as well, right?
> Would it maybe be possible to set up an example job for the Mail-App RCP
> and give anonymous users the right to view the job configuration?
>
Hm, not sure if that is doable (permission to view some jobs). Maybe
Thomas knows...
Meanwhile, would it be possible to store the job config in mailapps
releng component?

Henrik Lindberg schrieb:
> Johannes Utzig wrote:
>> Hi Henrik,
>>
>> I'm not sure I understand you right. A patch against the tutorial
>> project? I already added a patch to make it buildable for the tutorial
>> and it has been applied, or are you talking about the MSPECs, CSPECs
>> and the RMAP? Should they live in the releng project of the tutorial?
> That is what I had in mind.
>

Ok, I will create a patch for that and open a bugzilla.

>> If so, I can create a patch for that of course, just wasn't sure if
>> that's what you meant because the other demo queries and rmaps seem to
>> live in http://www.eclipse.org/buckminster/samples...
>>
> The artifacts that need to be easily shareable (i.e. starting from
> nothing) should be published there as well.
>

ok, that would indeed be the mspecs, cspecs and the rmap then.

>> If you guys are ok with the general approach and the tutorial can be
>> added to the buckminster examples section, I think it could
>> incrementally demonstrate more advanced features of CI. I could think
>> of -unit testing
>> -code coverage
>> -benchmarks
>> -metrics (like sonar)
>> -build triggers on commit action
>> -building specific branches/tags
>> -qualifier replacement
>>
>> What do you think?
>>
> I think it is an excellent idea. We have to decide on if we want
> something that builds up gradually (different instances), or if there is
> one that includes everything. Maybe have two? The current mailapp
> tutorial as it is to not overload it with stuff, and then one with
> everything (and extra mayo :).
>

That makes sense (although I hate mayo :)). Once the rest is in place I
can open a new bugzilla and attach a simple test fragment for example.
The advanced stuff can go into a new wiki page then.

>> It's also quite handy IMO to see the current pains of buckminster
>> integration in real life.
>> -no changelog for checkout
>> -no update functionality (which forces the user to delete the
>> workspace prior to a build)
>> -no polling support (which makes it hard to trigger a build on commit
>> actions)
>> -plugin unit tests are currently difficult to execute
>>
>> Until Peter has finished the 'Buckminster updating / change-list
>> generation' and issue 243293 (JUnit support) is resolved, the tutorial
>> can help explaining how that's currently done, and once those features
>> are in place, the tutorial can be an example on how those new
>> capabilities can be utilized.
>>
> Issues understood - do you think that is worth while? Isn't it better to
> wait for the proper solution and document that?
>

That depends a little on the timeline I'd say. Unit testing is crucial
for CI and it's currently not super easy to get it up and running. If
the JUnit support is in place relatively soon (including plugin unit
tests) then it would be easier to wait, however, if it will take some
months until the changes are done and actually released, then I think a
little guidance on how to set it up 'the hard way' would help people a lot.

>> Btw: The eclipse foundation has a hudson up and running as well, right?
>> Would it maybe be possible to set up an example job for the Mail-App
>> RCP and give anonymous users the right to view the job configuration?
>>
> Hm, not sure if that is doable (permission to view some jobs). Maybe
> Thomas knows...
> Meanwhile, would it be possible to store the job config in mailapps
> releng component?
>

I haven't tried it before, but I think it should be possible since
hudson is completely file based. I can zip up a minimal job (with just
the configuration) and the user should be able to extract that into
hudson's job directory. However, the configuration will have to include
the location of the MSPECs, CSPECs, RMAP, so I can't create the patch
until I know the final location of those files (i.e. until you actually
uploaded it to eclipse.org and told me their whereabouts).