Support multi module Maven projects

Details

Description

This is the feature I would like to see in the future. Below are the layout I can think for now:

Initially, Roo will create parent, core, and webapp projects

1. parent project:
This is pom project and the place where we call the Roo shell and use its command to create modules. Module's name format is something like (project-name)module??? or (project-name)plugin???. The modules list in this project pom is handled by Roo generator. The list contains: core, all module, and webapp projects. All the modules inside will inherit the parent project.

The pom of this project also has dependencies of application frameworks like spring, jpa, etc.

3. module project (can be one or many) (jar)
The module project contains the main components like controllers, models, services/daos, templates and resources which belong to the controllers of the module. Module project has the core project as its default dependency. The dependencies between module projects are managed by the developers.

4. webapp project (war)
It contains web configurations. It doesn't contain any component codes, templates, etc. But it contains the web resources that are shared between module projects. The dependencies in webapp are automatically handled by Roo generator during module project creation. The dependencies in webapp includes: core and all module projects. During the packaging, all module projects will be jar-ed and stored in WEB-INF/lib of the war.

Activity

"Multi-project-related support is not a priority for Roo at this time, as our focus is on single project productivity. If we support multi-project semantics a number of improvements would be necessary to related infrastructure (particularly classpath metadata abstractions, as we'd need to access the source code or bytecode for related projects) and therefore this request is not as simple as it initially appears."

If people would like to see multi-project support, would they please vote for ROO-163 and ROO-120 so we can gauge community interest levels. There is also ROO-91 for Any/Ivy-based build solutions, which some people might prefer instead of multi-project Maven support.

Ben Alex
added a comment - 13/Sep/09 7:50 PM As per my comments on ROO-163 :
"Multi-project-related support is not a priority for Roo at this time, as our focus is on single project productivity. If we support multi-project semantics a number of improvements would be necessary to related infrastructure (particularly classpath metadata abstractions, as we'd need to access the source code or bytecode for related projects) and therefore this request is not as simple as it initially appears."
If people would like to see multi-project support, would they please vote for ROO-163 and ROO-120 so we can gauge community interest levels. There is also ROO-91 for Any/Ivy-based build solutions, which some people might prefer instead of multi-project Maven support.

This would be something very desirable for us. Currently, we have over 150 modules and about 20 multi-module projects. It would also be important for our layering, since we don't bundle the web artifacts with the domain/service and repository artifacts.

Chuck Canning
added a comment - 15/Oct/09 8:14 AM This would be something very desirable for us. Currently, we have over 150 modules and about 20 multi-module projects. It would also be important for our layering, since we don't bundle the web artifacts with the domain/service and repository artifacts.

Sean Patrick Floyd
added a comment - 05/Nov/09 9:06 PM From my experience with large maven projects, I would say that Multi-Module support would be the single feature that will determine whether this technology will be used in large projects.
It would be nice to be able to configure the module like this:
project --topLevelPackage com.mycompany --multimodule true
// generate a root pom
persistence setup --provider HIBERNATE --database MYSQL --submodule persistence
// generate a sub module named persistence with root package ~.persistence

Oh, I forgot to add my suggestion. How hard would it be to have Roo just live in the projects and then you interact with your project similar to how you do things in the command line with maven with nested multi-module projects?

i.e.

mkdir wedding
cd wedding
roo

roo> project --topLevelPackage com.wedding --multiModule=true
creates a top level directory with a pom.xml that is packaging "pom" and an empty <modules> section

Obviously the syntax of using a multi-module Roo is pretty much the easiest part of it all. Echoing the comments above - I can't see Roo being used as more than a really cool toy if you are forced to throw everything into one big pom file.

Ryan Gardner
added a comment - 18/Mar/10 7:46 AM Oh, I forgot to add my suggestion. How hard would it be to have Roo just live in the projects and then you interact with your project similar to how you do things in the command line with maven with nested multi-module projects?
i.e.
mkdir wedding
cd wedding
roo
roo> project --topLevelPackage com.wedding --multiModule=true
creates a top level directory with a pom.xml that is packaging "pom" and an empty <modules> section
roo> create module 'foo-domain'
creates a directory "foo-domain"
roo> create module foo-webapp --packaging war [--archetype roo-webapp]
creates a directory foo-webapp with the war packaging, or if you wanted to use archetypes in this you could leave that part out and the roo-webapp archetype would create the webapp
roo> create module foo-services --multiModule=true
this creates a foo-services directory with a pom with packaging 'pom' and an empty modules section
cd foo-services
roo> create module foo-webservices
roo> create module foo-someotherservice
cd foo-someotherservice
roo>
(now - I'm not advocating that you have a big spider of a project - but it would be nice if you could use
then to work with the domain or the webapp you would change to those directories
cd foo-domain
roo> persistence setup --provider HIBERNATE --database POSTGRES
... etc.
Obviously the syntax of using a multi-module Roo is pretty much the easiest part of it all. Echoing the comments above - I can't see Roo being used as more than a really cool toy if you are forced to throw everything into one big pom file.

I totally see that supporting multi module projects come with the need for a lot of changes in the infrastructure and current code basis of Roo. Nevertheless I share the opinion, that without supporting the defacto way of developing "large" projects with maven could be a show-stopper for many projects/companies.

Nils Schmidt
added a comment - 23/May/10 10:42 AM I totally see that supporting multi module projects come with the need for a lot of changes in the infrastructure and current code basis of Roo. Nevertheless I share the opinion, that without supporting the defacto way of developing "large" projects with maven could be a show-stopper for many projects/companies.
If possible please put it on the schedule. Maybe for version 2.0?

I have to agree with the requesters. One of the reasons people shy away from Grails, Roo, Rails, etc is the assumption that you will do it all in one module. I think this would really put Roo on a very strong footing. The Grails team is already considering OSGi for modularization if I recall correctly. And Maven modules are really conceptually much simpler to me. Anyway I am putting my name in the list with those who think it is good to try to figure this out in a future release.

Ken Rimple
added a comment - 17/Jul/10 10:27 PM I have to agree with the requesters. One of the reasons people shy away from Grails, Roo, Rails, etc is the assumption that you will do it all in one module. I think this would really put Roo on a very strong footing. The Grails team is already considering OSGi for modularization if I recall correctly. And Maven modules are really conceptually much simpler to me. Anyway I am putting my name in the list with those who think it is good to try to figure this out in a future release.

This seems to have been popular for over a year, or open for over a year and recently popular , it's the major reason I hear people shying away from roo for any serious project. Myself included. Has there been any movement? Perhaps I need to take a better look at 1.1.0.rc1?

Nick Shaw
added a comment - 15/Oct/10 4:31 AM This seems to have been popular for over a year, or open for over a year and recently popular , it's the major reason I hear people shying away from roo for any serious project. Myself included. Has there been any movement? Perhaps I need to take a better look at 1.1.0.rc1?

I think as Roo becomes more popular, this issue is going to keep coming up. Furthermore, it stops Roo being used for real projects, though people may well use it to prototype initial cuts. I'd really like to see this addressed.

Alex Blewitt
added a comment - 07/Nov/10 1:21 PM I think as Roo becomes more popular, this issue is going to keep coming up. Furthermore, it stops Roo being used for real projects, though people may well use it to prototype initial cuts. I'd really like to see this addressed.

let me comment on this one as well.
I'm quite new to roo and after reading documentation I'm really excited about this framework!
But I also feel like multi-module support is a must for me to give roo a real try.
And as I read somewhere voting is important for decision making on the further developement of this project, so all you interested, please vote!
thanks.

Peter Butkovic
added a comment - 04/Jan/11 1:19 AM let me comment on this one as well.
I'm quite new to roo and after reading documentation I'm really excited about this framework!
But I also feel like multi-module support is a must for me to give roo a real try.
And as I read somewhere voting is important for decision making on the further developement of this project, so all you interested, please vote!
thanks.

Cemo Koc
added a comment - 23/Jan/11 1:48 AM Roo is really shining. However we can not give a real try at our business without this feature. Our legacy applications depend on only domain module of roo applications.

Another reason why this enhancement is desirable is to allow the conversion of large old-school J2EE apps (using EJB 2.x entity/session beans) to use modern JEE APIs. Imagine you have such a project, which will by definition be packaged as an EAR file (and therefore built by Maven as a multi-module project). Converting this app to use POJOs (i.e. POJOs for the session beans and JPA entities for the entity beans) would be a long process in which beans would be replaced incrementally between releases (creating a feature branch to do all the conversions en masse is a bad idea, as explained by Martin Fowler). So for a considerable period of time, the project needs to remain in EAR format in order to support the unconverted EJBs, while simultaneously allowing the addition (and usage) of the new JEE APIs such as JPA. However if Roo doesn't support multi-module projects, the JPA entities have to be generated and maintained by hand, instead of Roo doing all the heavy lifting.

Andrew Swan
added a comment - 24/Jan/11 7:23 PM Another reason why this enhancement is desirable is to allow the conversion of large old-school J2EE apps (using EJB 2.x entity/session beans) to use modern JEE APIs. Imagine you have such a project, which will by definition be packaged as an EAR file (and therefore built by Maven as a multi-module project). Converting this app to use POJOs (i.e. POJOs for the session beans and JPA entities for the entity beans) would be a long process in which beans would be replaced incrementally between releases (creating a feature branch to do all the conversions en masse is a bad idea, as explained by Martin Fowler ). So for a considerable period of time, the project needs to remain in EAR format in order to support the unconverted EJBs, while simultaneously allowing the addition (and usage) of the new JEE APIs such as JPA. However if Roo doesn't support multi-module projects, the JPA entities have to be generated and maintained by hand, instead of Roo doing all the heavy lifting.

I'd like to add my vote to this one. Roo looks great, but I would shy away from using it for real unless I can package the domain module into a separate .jar file for integration with other applications or back-end processes.

Shannon Madsen
added a comment - 03/Feb/11 12:30 PM I'd like to add my vote to this one. Roo looks great, but I would shy away from using it for real unless I can package the domain module into a separate .jar file for integration with other applications or back-end processes.

Lack of multi-module support in Roo inhibits it from using in real projects, unfortunately. As it is now, it's more like an inflatable toy despite the fact it allows you to build heterogeneous applications.
Please, here is my vote for the very anticipated feature.

Serge Belolipetski
added a comment - 17/Feb/11 1:27 PM Lack of multi-module support in Roo inhibits it from using in real projects, unfortunately. As it is now, it's more like an inflatable toy despite the fact it allows you to build heterogeneous applications.
Please, here is my vote for the very anticipated feature.

André Salvati
added a comment - 25/Feb/11 6:54 AM This also could be done for GWT add-on.
However, I don't know if the solution for a GWT multi module project is the same as a JSP multi module project.
http://www.summa-tech.com/blog/2011/02/22/structuring-gwt-modules-for-large-applications/
There are a lot of questions about this on foruns.
Should I open a different Jira for GWT?

A default option will be to create a parent pom and three modules: core, war, and ear. This is quite common in large enterprises. Of course, there will be fine-grained control over what modules a user wants.

We need to use a different type-binding library than JavaParser to support this feature.

Alan Stewart
added a comment - 01/Mar/11 7:28 PM A default option will be to create a parent pom and three modules: core, war, and ear. This is quite common in large enterprises. Of course, there will be fine-grained control over what modules a user wants.
We need to use a different type-binding library than JavaParser to support this feature.

Brad Murray
added a comment - 02/Mar/11 12:42 PM We 2 structures around our 2 major Roo projects atm. I'll outline just to offer ideas on different needs that you may or may not be able to meet.
Our integration project has
core
multiple modules that extend core (and depend on it), eg. publishing, document management interfaces, webservices interfaces
war that provides a simple ootb for config / testing /diagnostics
ear, for bundling and deploying it all on at Oracle Integration platform that uses Weblogic.
Our main app has.
common (one of our developers is trying to get a Maven overlay working ( http://maven.apache.org/plugins/maven-war-plugin/overlays.html ). It builds the wars fine, but the WTP support in Eclipse and STS seems to be incomplete for overlays, so its proving painful)
internal app. The stuff for internal stuff, with the common war overlayed.
external app. The public facing stuff, also with common war.

Why is not there any clarification regarding this issue from Roo Team despite of being most requested issue? At least a possible fix version of this issue can greatly help us to reconsider our near future goals.

Cemo Koc
added a comment - 05/Apr/11 5:56 AM Why is not there any clarification regarding this issue from Roo Team despite of being most requested issue? At least a possible fix version of this issue can greatly help us to reconsider our near future goals.

Please add support for multimodule maven projects. It's in spring philosophy (at least from my knowledge) to decouple and reuse specialized business modules. If roo it's only for applications that include everything in one place just make it public. In my opinion the roo user community will increase by order of magnitude if this issue will be addressed. I'm a great fan of spring ideas and projects. Please solve this! Thank you.

Elvis Ciocoiu
added a comment - 20/May/11 1:24 AM Please add support for multimodule maven projects. It's in spring philosophy (at least from my knowledge) to decouple and reuse specialized business modules. If roo it's only for applications that include everything in one place just make it public. In my opinion the roo user community will increase by order of magnitude if this issue will be addressed. I'm a great fan of spring ideas and projects. Please solve this! Thank you.

If it helps, I'm at the beginning of a new project and I wish Roo had multi moddule support. It will be deployed using an EAR to WebSphere, and we've had to create a single task only to configure the multimodule for it.

In fact, the use of Roo is being questioned due for this. Since I'm a big Roo fan and I'm in charge of development (for now ), we're going to give it another chance.

Javier Beneito Barquero
added a comment - 20/May/11 2:23 AM A two year old issue!
Is there any plan to resolve or even discard this request?
If it helps, I'm at the beginning of a new project and I wish Roo had multi moddule support. It will be deployed using an EAR to WebSphere, and we've had to create a single task only to configure the multimodule for it.
In fact, the use of Roo is being questioned due for this. Since I'm a big Roo fan and I'm in charge of development (for now ), we're going to give it another chance.

Now that 1.1.4 has been released, this is one of our top priority issues. However, supporting this feature is not trivial. It requires us to find a new type-binding solution, which JavaParser is currently, but is not adequate for multi-module Maven projects. Options being considered are JDK 6's mirror and annotation processing API's and Eclipse's JDT.

Major parts of Roo internal code are affected and we want to get it right to easily support JDK 7 and 8, and beyond

We are full time now on this ticket, along with JSF/Primefaces and services/DAO, so please stick with us
Alan

Alan Stewart
added a comment - 20/May/11 4:02 AM Now that 1.1.4 has been released, this is one of our top priority issues. However, supporting this feature is not trivial. It requires us to find a new type-binding solution, which JavaParser is currently, but is not adequate for multi-module Maven projects. Options being considered are JDK 6's mirror and annotation processing API's and Eclipse's JDT.
Major parts of Roo internal code are affected and we want to get it right to easily support JDK 7 and 8, and beyond
We are full time now on this ticket, along with JSF/Primefaces and services/DAO, so please stick with us
Alan

It would be good to know where this is on the product roadmap, so I can inform our readers before publication on what to do ("best practice" wise) about multi-module projects. Also, I haven't tried to do this yet, but why can't we just make it have a parent project later, point to a pom.xml above it, such as

<project...>

<groupId>org.sillyapp</groupId>

<artifactId>web-app</artifactId>

<parent>

<groupId>org.sillyapp</groupId>

<artifactId>parent</artifactId>

</parent>

Seems to me that you could still use Roo as the web app project but just not weave the Roo style coding into other sub-apps. Or could you? I guess the biggest problem would be namespace collisions with META-INF/spring/applicationContext*.xml right? Aren't there Maven plugins like Shade that let you deal with multiple descriptors with the same name in sub JARs? I remember something about that somewhere - how it folded the entries in multiple XML files together, but that does sound problematic.

Ken Rimple
added a comment - 24/Aug/11 7:46 AM Alan,
+1
It would be good to know where this is on the product roadmap, so I can inform our readers before publication on what to do ("best practice" wise) about multi-module projects. Also, I haven't tried to do this yet, but why can't we just make it have a parent project later, point to a pom.xml above it, such as
<project...>
<groupId>org.sillyapp</groupId>
<artifactId>web-app</artifactId>
<parent>
<groupId>org.sillyapp</groupId>
<artifactId>parent</artifactId>
</parent>
Seems to me that you could still use Roo as the web app project but just not weave the Roo style coding into other sub-apps. Or could you? I guess the biggest problem would be namespace collisions with META-INF/spring/applicationContext*.xml right? Aren't there Maven plugins like Shade that let you deal with multiple descriptors with the same name in sub JARs? I remember something about that somewhere - how it folded the entries in multiple XML files together, but that does sound problematic.
Ken

note - the project config is a little messed up as I'm seing while writing this up - due to the actions below. But the concept holds.

I thought it was due to the missing dependency, so I added a dependency in the web project on the services project. Still nothing (and probably not something the Roo shell would look at?).

I think this is a key thing to make happen, if at all possible, but I recognize how difficult it might be. Maybe we make this a follow-on JIRA? Maybe we say it isn't possible to use scaffolding against entities in an external module? I think that would be a shame but understandable.

Also, I think the POM files we are generating should be gone over for other issues:

don't make the groupId the package name of the child - that means the parent project and sub projects don't share a groupId, and usually they do. Have the child projects inherit the groupId from the parent (POM) project.

I'd generally not let them specify the groupId of the child - or default it to that of the parent. The top-level package could be used for component scan but the groupId should come from the parent POM.

add a <parent> tag with the groupId, artifactId and version of the parent POM so that we get inherited settings from it

define <dependencyManagement> and <pluginManagement> tags in the parent POM with the version of the artifacts, and using the <dependencies> section in each submodule. That way the child POMs are simpler.

strangely, if you do a mvn jetty:run from the parent POM, because you've installed Jetty there, it picks an arbitrary (non WAR) project to run. In my case it runs the services project (weird). You should move the plugin definition into a <pluginManagement> section so that you can inherit it in child poms, but not have to install it in the parent (and thereby not allow them to run the Jetty engine from a parent pom).

It would be nice to allow a specification of --module_dependencies so that you could automatically install those - for example in the web module you may want to include the dependencies of the services, repositories, etc..

Ken Rimple
added a comment - 03/Nov/11 1:14 AM Andrew,
Just FYI based on my view of trunk. I like what I'm seeing quite a bit in terms of the approach here with the module and module focus. Great idea.
I have a concern, maybe you've addressed this so far:
I've tried building a multi-module project ( http://db.tt/lTJq5lCJ to download my zip file) with:
web - simple mvc web module
services - a repository, entity...
I tried scaffolding a web controller using the entity from the services project. I get:
roo> module focus --moduleName web
web roo> web mvc scaffold --backingType ~.db.Person --class ~.web.PersonController
The specified entity can not be resolved to a type in your project
note - the project config is a little messed up as I'm seing while writing this up - due to the actions below. But the concept holds.
I thought it was due to the missing dependency, so I added a dependency in the web project on the services project. Still nothing (and probably not something the Roo shell would look at?).
I think this is a key thing to make happen, if at all possible, but I recognize how difficult it might be. Maybe we make this a follow-on JIRA? Maybe we say it isn't possible to use scaffolding against entities in an external module? I think that would be a shame but understandable.
Also, I think the POM files we are generating should be gone over for other issues:
don't make the groupId the package name of the child - that means the parent project and sub projects don't share a groupId, and usually they do. Have the child projects inherit the groupId from the parent (POM) project.
I'd generally not let them specify the groupId of the child - or default it to that of the parent. The top-level package could be used for component scan but the groupId should come from the parent POM.
add a <parent> tag with the groupId, artifactId and version of the parent POM so that we get inherited settings from it
define <dependencyManagement> and <pluginManagement> tags in the parent POM with the version of the artifacts, and using the <dependencies> section in each submodule. That way the child POMs are simpler.
strangely, if you do a mvn jetty:run from the parent POM, because you've installed Jetty there, it picks an arbitrary (non WAR) project to run. In my case it runs the services project (weird). You should move the plugin definition into a <pluginManagement> section so that you can inherit it in child poms, but not have to install it in the parent (and thereby not allow them to run the Jetty engine from a parent pom).
It would be nice to allow a specification of --module_dependencies so that you could automatically install those - for example in the web module you may want to include the dependencies of the services, repositories, etc..
Hope this helps.
Ken

When entering the backingType for the web mvc scaffold command, you can use tab completion to fill in firstly the module name then the type name. The "~" character can still be used as the top-level package of the relevant module. So in your case the command (once you've tabbed a few times and maybe typed some parts) might look like this:

I'll respond separately to your bulleted suggestions about POM files, but be aware that in the initial release at least, we're not going to be doing too many clever things with the contents of the various POMs (not only because it's expanding the scope but also because we don't want to second-guess how the user might want to set up their build).

Andrew Swan
added a comment - 03/Nov/11 8:19 PM Ken,
When entering the backingType for the web mvc scaffold command, you can use tab completion to fill in firstly the module name then the type name. The "~" character can still be used as the top-level package of the relevant module. So in your case the command (once you've tabbed a few times and maybe typed some parts) might look like this:
web roo> web mvc scaffold --backingType services|~.domain.Thing --class ~.controllers.ThingController
Where:
your Thing entity lives in the " domain " subpackage of your " services " module
the controller will be created in the "controllers" subpackage of your " web " module (which has the focus as indicated by the " web roo> " prompt).
N.B. You'll need revision 3ec094f060e41ae7666722233d3d4feea8d73922 or later.
I'll respond separately to your bulleted suggestions about POM files, but be aware that in the initial release at least, we're not going to be doing too many clever things with the contents of the various POMs (not only because it's expanding the scope but also because we don't want to second-guess how the user might want to set up their build).

It would be nice to allow a specification of --module_dependencies so that you could automatically install those - for example in the web module you may want to include the dependencies of the services, repositories, etc..

For the time being, the existing "dependency add" command will do this, but feel free to log this idea as a separate improvement (linking ROO-120 to it as a "depends on").

Andrew Swan
added a comment - 03/Nov/11 8:57 PM It would be nice to allow a specification of --module_dependencies so that you could automatically install those - for example in the web module you may want to include the dependencies of the services, repositories, etc..
For the time being, the existing " dependency add " command will do this, but feel free to log this idea as a separate improvement (linking ROO-120 to it as a "depends on").

Fantastic, Andrew. That was just what we needed. I'll test that out and then figure out where I can put it in Roo in Action - interesting thing is that it doesn't quite fit anywhere yet, but we're in final review so I can't add a chapter so it's gotta be a short segment somewhere. This is really great stuff and quite powerful.

Ken Rimple
added a comment - 04/Nov/11 3:48 AM Fantastic, Andrew. That was just what we needed. I'll test that out and then figure out where I can put it in Roo in Action - interesting thing is that it doesn't quite fit anywhere yet, but we're in final review so I can't add a chapter so it's gotta be a short segment somewhere. This is really great stuff and quite powerful.

Bruce Edge
added a comment - 09/Nov/11 8:57 PM Another side effect of this problem.
This is not related to scaffolding, rather to the ability to manually subclass an entity outside of the current project:
http://forum.springsource.org/showthread.php?117209-Subclassed-entities-java.lang.IllegalArgumentException-Unknown-entity
An annotation to disable the auto-add of the EntityManager would address this sub-problem, maybe @NoAutoEntityManager in the subclass could tell roo that it should not add it.

I think the issue has not completed.
Controller(MVC) or ManageredBean(JSF) must in the war module in 1.2 release.
But people need:
The module project contains the main components like controllers, models, services/daos, templates and resources which belong to the controllers of the module

gavinlau
added a comment - 25/Dec/11 3:53 AM I think the issue has not completed.
Controller(MVC) or ManageredBean(JSF) must in the war module in 1.2 release.
But people need:
The module project contains the main components like controllers, models, services/daos, templates and resources which belong to the controllers of the module