Forward port 0.1.1 incubating Maven poms to trunk

Details

Description

I improved the Maven poms in 0.1.1-incubating, and trimmed them down to a publishable set of dependencies.

My improvements:

1. reduce the redundancy in the deps section by not repeating deps in each sub-project.
2. use explicit dependency and module version #s which are required to not have "moving target" dependencies in the downstream applications and that allow each individual gora-* module to be included on its own (without having to include gora-core).

I'd like to forward port these to trunk and then update per trunk's dependencies.

Hey Chris can you provide any pointers of where I begin looking into this? Any good maven documentation that you're aware of? It's about time I got my head around the Maven stuff anyway and this amongst other issues, would be a good place to start. Thanks

Lewis John McGibbney
added a comment - 16/Nov/11 17:18 Hey Chris can you provide any pointers of where I begin looking into this? Any good maven documentation that you're aware of? It's about time I got my head around the Maven stuff anyway and this amongst other issues, would be a good place to start. Thanks

Hey Lewis, I was just thinking of grabbing the Maven poms I made for the 0.1.1-incubating release, which I know work with Maven Central and which I know work with the deps we have (except for Cassandra, and Thrift in that version, which you needed to mvn install by hand ahead of time). In 0.2-incubating onward, those deps are on newer versions of those libraries and are thus available in Maven Central.

Chris A. Mattmann
added a comment - 16/Nov/11 18:07 Hey Lewis, I was just thinking of grabbing the Maven poms I made for the 0.1.1-incubating release, which I know work with Maven Central and which I know work with the deps we have (except for Cassandra, and Thrift in that version, which you needed to mvn install by hand ahead of time). In 0.2-incubating onward, those deps are on newer versions of those libraries and are thus available in Maven Central.
You can find the poms here: http://svn.apache.org/repos/asf/incubator/gora/tags/0.1.1-incubating/

2) This is (part of) where we need to edit the location for the upgrade to for the cassandra.version, cassandra.thrift.version & hector.version. This obviously needs to be addressed in a separate issue but it is worth noting here as this issue therefore should block GORA-58 as well as GORA-22 (which we really need to get sorted out).

Lewis John McGibbney
added a comment - 17/Nov/11 17:35 - edited A couple of things here.
1) Withe regards to gora-sql tests we have no reference to an internal dependency for this nested under the <dependencyManagement> parent. Is it possible to add something like
<dependency>
<groupId>org.apache.gora</groupId>
<artifactId>gora-sql</artifactId>
<version>${project.version}</version>
<classifier>tests</classifier>
</dependency>
2) This is (part of) where we need to edit the location for the upgrade to for the cassandra.version, cassandra.thrift.version & hector.version. This obviously needs to be addressed in a separate issue but it is worth noting here as this issue therefore should block GORA-58 as well as GORA-22 (which we really need to get sorted out).
3) Does the stuff in
<build>
<testSourceDirectory>${basedir}/src/test/java</testSourceDirectory>
<sourceDirectory>${basedir}/src/main/java</sourceDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
...
</build>
need to be updated as well or is this fine as-is?
4) As far as I can see it is mainly stuff in
<properties>
<osgi.version>4.2.0</osgi.version>
<!-- Hadoop Dependencies -->
<hadoop.version>0.20.2</hadoop.version>
...
that needs to be updated? Can anyone clarify/confirm?
I understand you are up to your teeth just now Chris, but if you could provide insight into what needs to be addressed above, then I will get right on this. Thanks

OK so I undertook the exercise of seeing what can/could possibly be upgraded by doing searches on Maven Central. The following outcomes can be interpreted as

artifact id: current version: latest version

avro:1.3.2:1.3.3
hbase:0.90.0:0.90.4
cassandra-all:0.8.1:1.0.2
cassandra-thrift:ditto (by the way do we require this if we are shifting to Hector client???)
hector-core:0.8.0-1:1.0-1
guava:r09:10.0.1
commons-lang:2.5:2.6
jdom:1.1:1.1.2
sqlbuilder:2.0.6:2.0.10
hsqldb:2.0.0:2.2.4
mysql-connector-java:5.1.13:5.1.18
slf4j-api:1.58:1.6.4
slf4j-simple:ditto
slf4j-jdk14:ditto
log4j:1.2.15:1.2.16
junit:4.6:4.10

commons-logging:1.1.1: at newest, however I don't see this actually being used anywhere can someone clarify why this is present.
libthrift:0.5.0:0.7.0 however same comment as above

As we all know hadoop can stay at 0.20.2 for the foreseeable future. The concern I have is with compatibility! This is an area I do not have much experience and I'm trying to get used to it. Once this is done the ant/ivy can follow suit fine, I don't have a problem with that.

Further to this, as explained above, I have not looked for any plugins for the pom.xml or anything similar

Lewis John McGibbney
added a comment - 17/Nov/11 18:21 - edited OK so I undertook the exercise of seeing what can/could possibly be upgraded by doing searches on Maven Central. The following outcomes can be interpreted as
artifact id: current version: latest version
avro:1.3.2:1.3.3
hbase:0.90.0:0.90.4
cassandra-all:0.8.1:1.0.2
cassandra-thrift:ditto (by the way do we require this if we are shifting to Hector client???)
hector-core:0.8.0-1:1.0-1
guava:r09:10.0.1
commons-lang:2.5:2.6
jdom:1.1:1.1.2
sqlbuilder:2.0.6:2.0.10
hsqldb:2.0.0:2.2.4
mysql-connector-java:5.1.13:5.1.18
slf4j-api:1.58:1.6.4
slf4j-simple:ditto
slf4j-jdk14:ditto
log4j:1.2.15:1.2.16
junit:4.6:4.10
commons-logging:1.1.1: at newest, however I don't see this actually being used anywhere can someone clarify why this is present.
libthrift:0.5.0:0.7.0 however same comment as above
As we all know hadoop can stay at 0.20.2 for the foreseeable future. The concern I have is with compatibility! This is an area I do not have much experience and I'm trying to get used to it. Once this is done the ant/ivy can follow suit fine, I don't have a problem with that.
Further to this, as explained above, I have not looked for any plugins for the pom.xml or anything similar
Thanks

Now although neither maven-plugins:maven-findbugs-plugin:plugin:1.3.1 nor maven-plugins:maven-cobertura-plugin:plugin:1.3 are available on either of the above two repositories, they are available on http://search.maven.org/. If I add them manually they break the build even more!
I've attached the patch which includes all up-to-date dependency properties for record.

After manually downloading and installing both of the above plugins then running mvn clean, then mvn test, then mvn clean install I'm getting Gora to build successfully. There are a number of tests in which there are errors but no failures. Is there any way we can get the above two plugins uploaded to the maven central repos? This way we could add the configuration to the pom.xml which would mean that it is a case of fixing failing tests before moving on to other development?

Lewis John McGibbney
added a comment - 17/Nov/11 19:37 After manually downloading and installing both of the above plugins then running mvn clean, then mvn test, then mvn clean install I'm getting Gora to build successfully. There are a number of tests in which there are errors but no failures. Is there any way we can get the above two plugins uploaded to the maven central repos? This way we could add the configuration to the pom.xml which would mean that it is a case of fixing failing tests before moving on to other development?

Lewis, great work. You nailed down the major differences. It's interesting that we're failing on 2 maven plugins that aren't available in Central that seem to
be referenced from JDOM and JAXEN. One thing that we might do is an exclusion of org.jdom in the gora-hbase dependencies which seems to drive
both plugin incompatibilities. WDYT?

If exclusion fixes it, then my +1 to simply commit your forward port to trunk and then call this sucker done. Thoughts?

Chris A. Mattmann
added a comment - 17/Nov/11 20:42 Lewis, great work. You nailed down the major differences. It's interesting that we're failing on 2 maven plugins that aren't available in Central that seem to
be referenced from JDOM and JAXEN. One thing that we might do is an exclusion of org.jdom in the gora-hbase dependencies which seems to drive
both plugin incompatibilities. WDYT?
If exclusion fixes it, then my +1 to simply commit your forward port to trunk and then call this sucker done. Thoughts?

Please see attached for the updated patch. I'm not sure if Hector client will work with the Cassandra artefact's but this will hopefully become obvious in the not so near future. Even if this is the case we can simply downgrade the Cassandra dependencies. Can we please ensure that this patch is working OK as I manually installed the two maven plugins so I am unsure if all will be working after the exculusions have been added to gora-hbase modules.

Lewis John McGibbney
added a comment - 17/Nov/11 21:49 Please see attached for the updated patch. I'm not sure if Hector client will work with the Cassandra artefact's but this will hopefully become obvious in the not so near future. Even if this is the case we can simply downgrade the Cassandra dependencies. Can we please ensure that this patch is working OK as I manually installed the two maven plugins so I am unsure if all will be working after the exculusions have been added to gora-hbase modules.
Thanks

Lewis John McGibbney
added a comment - 22/Nov/11 14:24 I can confirm that the new patch still complains about the following missing dependencies
Path to dependency:
1) org.apache.gora:gora-hbase:jar:0.2-SNAPSHOT
2) org.jdom:jdom:jar:1.1.2
3) jaxen:jaxen:jar:1.1.3
4) maven-plugins:maven-findbugs-plugin:plugin:1.3.1
I am kinda stumped on this one as the link here [1] gets you to manually redirect here [2] .
Anyway, for clarity details of both the missing plugins can be found here [3] & [4]
[1] http://repo1.maven.org/maven2/
[2] http://search.maven.org/
[3] http://search.maven.org/#artifactdetails%7Cmaven-plugins%7Cmaven-findbugs-plugin%7C1.3.1%7Cjar
[4] http://search.maven.org/#artifactdetails%7Cmaven-plugins%7Cmaven-cobertura-plugin%7C1.3%7Cjar

Hopefully this is the final patch attached. It excludes the two rogue plugins from the org.jdom dependency, these were the ones causing trouble and failing my local build. In addition I've added the gora-sql test target to the pom. This can be removed should it be covered in a separate issue.

Lewis John McGibbney
added a comment - 23/Nov/11 09:50 Committed @ revision 1205346 in trunk.
I would please ask if Chris could finally check and close off if deemed so.
Thanks for checking this one over.