Patrick Hunt
added a comment - 20/Mar/10 15:04 Paolo thanks for following up! I send the release candidate vote to the zookeeper-dev mailing list - the link to the RC archive itself is this:
http://people.apache.org/~phunt/zookeeper-3.3.0-candidate-0/
would you like to create a JIRA/patch for this src vs sources naming issue? Seems pretty simple to fix. For 3.3.0 I will rename the jar file when uploading it to the maven repo.

"If you are not using Maven as your build system, and want something uploaded
to the Central Repository then you just need to make a bundle jar manually.
Please use the jar executable, not zip, pkzip or equivalent. The bundle
should have the following content:
pom.xml
foo-1.0.jar (or whatever artifact is referred to in the pom.xml)
foo-1.0-sources.jar
foo-1.0-javadoc.jar"
The names of the jar files inside the bundle must be built from the
<artifactId> and <version> in the pom.xml file, like this:
${artifactId}-${version}.jar
${artifactId}-${version}-sources.jar
${artifactId}-${version}-javadoc.jar"

Paolo Castagna
added a comment - 20/Mar/10 01:50 Where can I find the latest ZooKeeper release candidate?
The only Ant target which generates the pom.xml file is "package". I have removed the dependency on "create-cppunit-configure" just to see the pom.xml file generated by Ivy.
The pom.xml file generated by Ivy seems fine. The only problem I foresee is related to log4j, but everybody has that.
Ideally, you would like to have something like this in your pom.xml, however I do not know how to tell Ivy to generate this:
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.15</version>
<exclusions>
<exclusion>
<groupId>javax.jms</groupId>
<artifactId>jms</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jdmk</groupId>
<artifactId>jmxtools</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jmx</groupId>
<artifactId>jmxri</artifactId>
</exclusion>
<exclusion>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
</exclusion>
</exclusions>
</dependency>
A similar issue has been discussed here: https://issues.apache.org/jira/browse/HADOOP-6629
About sources vs. src:
"If you are not using Maven as your build system, and want something uploaded
to the Central Repository then you just need to make a bundle jar manually.
Please use the jar executable, not zip, pkzip or equivalent. The bundle
should have the following content:
pom.xml
foo-1.0.jar (or whatever artifact is referred to in the pom.xml)
foo-1.0-sources.jar
foo-1.0-javadoc.jar"
The names of the jar files inside the bundle must be built from the
<artifactId> and <version> in the pom.xml file, like this :
${artifactId}-${version}.jar
${artifactId}-${version}-sources.jar
${artifactId}-${version}-javadoc.jar"
– http://maven.apache.org/guides/mini/guide-central-repository-upload.html

Support for maven repo was contributed by Thomas & Hiram, I have limited/no experience with this to provide good answers to your questions. We've gotten a number of requests for Maven support, but not much help. We have limited maven experience so there may be some issues to shake out.

Until the release has been approved by the PMC we only publish the release candidate archive. It includes the generated pom and jars destined for maven deployment. That's not sufficient to test?

create-cppunit-configure is the c part of the build, it requires libraries such as cppunit to be installed. You might be able to get around this by using "ant compile-test" or "ant test-core-java" to compile/test the java code respectively.

I don't know about maven naming conventions, is there a pointer to docs on whether to use sources vs src?

Patrick Hunt
added a comment - 19/Mar/10 23:09 Hi Paolo, thanks for taking a look at this!
See ZOOKEEPER-537 for background.
Support for maven repo was contributed by Thomas & Hiram, I have limited/no experience with this to provide good answers to your questions. We've gotten a number of requests for Maven support, but not much help. We have limited maven experience so there may be some issues to shake out.
Until the release has been approved by the PMC we only publish the release candidate archive. It includes the generated pom and jars destined for maven deployment. That's not sufficient to test?
create-cppunit-configure is the c part of the build, it requires libraries such as cppunit to be installed. You might be able to get around this by using "ant compile-test" or "ant test-core-java" to compile/test the java code respectively.
I don't know about maven naming conventions, is there a pointer to docs on whether to use sources vs src?

Paolo Castagna
added a comment - 19/Mar/10 22:43 Are there ZooKeeper SNAPSHOTs artifact published somewhere, for people who want to test them before the "official" publication?
I tried to search here:
http://people.apache.org/~chirino/zk-repo/org/apache/hadoop/zookeeper/zookeeper/
https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/
but I did not find any.
I tried to compile ZooKeeper to check the generated POM file, but I have a problem with the target "create-cppunit-configure".
Will you publish sources and javadoc artifacts as Avro does? For example:
http://repo2.maven.org/maven2/org/apache/hadoop/avro/1.3.0/
Shouldn't be zookeeper-3.4.0-sources.jar instead of zookeeper-3.4.0-src.jar?

Recently, I 've uploaded a patch for hadoop-20 branch which uses maven-ant-task which signs artifacts using gpg and publishes them to the staging or snapshot repository based on the arguments passed. https://issues.apache.org/jira/browse/hadoop-6382

Giridharan Kesavan
added a comment - 20/Jan/10 03:51 Recently, I 've uploaded a patch for hadoop-20 branch which uses maven-ant-task which signs artifacts using gpg and publishes them to the staging or snapshot repository based on the arguments passed.
https://issues.apache.org/jira/browse/hadoop-6382

Patrick Hunt
added a comment - 19/Jan/10 21:38 Brian, is this different from what Avro is doing?
http://wiki.apache.org/hadoop/Avro/HowToRelease#line-80
I was planning to replicate what they do wrt publishing to maven.

@kay Unfortunately not, the build changes for maven deployment are in the trunk, so 3.3.0 will be the first release to support this.

If you are interested the artifacts please try the generated packages which are created as part of the current trunk tar build. This will ensure that the pkgs we deploy as part of 3.3.0 will work for everyone.

Patrick Hunt
added a comment - 05/Jan/10 20:33 @kay Unfortunately not, the build changes for maven deployment are in the trunk, so 3.3.0 will be the first release to support this.
If you are interested the artifacts please try the generated packages which are created as part of the current trunk tar build. This will ensure that the pkgs we deploy as part of 3.3.0 will work for everyone.

Patrick Hunt
added a comment - 22/Sep/09 17:49 linking ZOOKEEPER-224 to ZOOKEEPER-529 which adds support for ivy, which adds support for generating pom file and checksums of jar necessary to deploy to maven repo

Just a reminder - last I heard from hadoop PMC the maven specific files (poms etc...) need to be voted on as part of the release
process and tar artifact creation. So we may want to roll this into a 3.2.1 as well as 3.3 in order to get something out more quickly.

Patrick Hunt
added a comment - 29/Jun/09 20:01 Just a reminder - last I heard from hadoop PMC the maven specific files (poms etc...) need to be voted on as part of the release
process and tar artifact creation. So we may want to roll this into a 3.2.1 as well as 3.3 in order to get something out more quickly.

I am moving this issue to 3.3. Whatever changes we need to make in the build.xml (related to ivy) can be done as a part of this jira. We can just publish 3.2 release to maven (without any build changes) for now.

Mahadev konar
added a comment - 26/Jun/09 22:29 I am moving this issue to 3.3. Whatever changes we need to make in the build.xml (related to ivy) can be done as a part of this jira. We can just publish 3.2 release to maven (without any build changes) for now.

Doug Cutting
added a comment - 23/Jun/09 20:19 > but for the specific issue of getting our jars into the maven repository, we can just run a command after we do the release. right?
Moreover, I think you have to. I don't think either Maven or Ivy can sign things correctly.

sorry, i should have scoped my question better. i mean "why do we need ivy to push our release jars into the repository?". i can see how we can use ivy for other needs, but for the specific issue of getting our jars into the maven repository, we can just run a command after we do the release. right?

Benjamin Reed
added a comment - 23/Jun/09 19:48 sorry, i should have scoped my question better. i mean "why do we need ivy to push our release jars into the repository?". i can see how we can use ivy for other needs, but for the specific issue of getting our jars into the maven repository, we can just run a command after we do the release. right?

Ivy seems to be the best way to integrate Ant-based projects into the Maven world. You write an Ivy config that lists the jars you depend on, and Ivy fetches them, so that you no longer need to include them in svn. It can generate a POM file from its configuration, so that applications that depend on Zookeeper can easily get it and its dependencies. If you maintain your POM file manually, then each time you add or upgrade a lib you have to update multiple places, while with Ivy you only list your dependencies in one place. I think Ivy can also push releases into Maven repos, but I haven't tried that yet. I think this can also be done manually just by copying the .jar, .md5, .sha1, .pom, and .asc files to the right directory on people.apache.org.

Doug Cutting
added a comment - 23/Jun/09 18:50 > why do we need ivy?
Ivy seems to be the best way to integrate Ant-based projects into the Maven world. You write an Ivy config that lists the jars you depend on, and Ivy fetches them, so that you no longer need to include them in svn. It can generate a POM file from its configuration, so that applications that depend on Zookeeper can easily get it and its dependencies. If you maintain your POM file manually, then each time you add or upgrade a lib you have to update multiple places, while with Ivy you only list your dependencies in one place. I think Ivy can also push releases into Maven repos, but I haven't tried that yet. I think this can also be done manually just by copying the .jar, .md5, .sha1, .pom, and .asc files to the right directory on people.apache.org.
I did a simplisitic conversion of Avro to Ivy ( AVRO-53 , http://svn.apache.org/viewvc?view=rev&revision=785776 ). The build.xml now generates all of the files but the .asc. When I first try to do a release I'll learn whether this works!

Giridharan Kesavan
added a comment - 23/Jun/09 05:08 I'm working on getting the hadoop jars published to the maven repository using ivy.
In the case of zookeeper as well we can use ivy to publish zookeeper.jar.

i think to do it right it requires some work . If its just pushing some jar into apache maven using some commands, that can be done while doing the release. We should not hold up/delay the release if its too much work.

Mahadev konar
added a comment - 19/Jun/09 00:04 - edited i think to do it right it requires some work . If its just pushing some jar into apache maven using some commands, that can be done while doing the release. We should not hold up/delay the release if its too much work.

Benjamin Reed
added a comment - 18/Jun/09 23:51 In the next release can we get zookeeper.jar zookeeper-test.jar and bookeeper.jar published to maven? is there some simple procedure to apply to our built jar files to make them deployable?

Hadoop-core is yet to be published to the maven repo.
With HADOOP-3305 we have dependencies managed by ivy. Publishing hadoop-core to the mvn repo requires some more work
I will be workin on this soon.

Giridharan Kesavan
added a comment - 15/Apr/09 04:56 Hadoop-core is yet to be published to the maven repo.
With HADOOP-3305 we have dependencies managed by ivy. Publishing hadoop-core to the mvn repo requires some more work
I will be workin on this soon.

Todd Greenwood-Geer
added a comment - 14/Apr/09 19:35 https://issues.apache.org/jira/browse/HADOOP-3305
The hadoop issue( HADOOP-3305 ) has been resolved, and it appears that hadoop-core has been deployed to maven. Is there any possibility of deploying zookeeper to maven?

Nigel mentioned that we should be able to get some time from the contributors to core's build management. We currently use the hadoop core build/release process, it's a huge benefit to me not to have to reinvent the wheel and just use their vetted processes. I'll talk with Nigel to see what can be done wrt getting the work from 3305 "ported" into ZK build/release process. I'm going to hold off on any changes related to this issue (build or release) until 3305 settles, worst case we can port the work from 3305 ourselves.

Patrick Hunt
added a comment - 26/Nov/08 18:04 Thanks for the feedback Doug, I hadn't thought of that but it makes sense.
I definitely think we should support this, but it's quickly turning from extra work for me, to a lot of extra work.
I talked with Nigel who's the release manager for Hadoop Core. There is recent progress on
https://issues.apache.org/jira/browse/HADOOP-3305 which is very similar to this issue.
Nigel mentioned that we should be able to get some time from the contributors to core's build management. We currently use the hadoop core build/release process, it's a huge benefit to me not to have to reinvent the wheel and just use their vetted processes. I'll talk with Nigel to see what can be done wrt getting the work from 3305 "ported" into ZK build/release process. I'm going to hold off on any changes related to this issue (build or release) until 3305 settles, worst case we can port the work from 3305 ourselves.

Shouldn't the pom & metadata files be either included in svn or generated by the build? We shouldn't be publishing artifacts without a documented process in place so that any committer can build the next release. So the creation of this needs to be part of http://wiki.apache.org/hadoop/ZooKeeper/HowToRelease. We shouldn't generate new release artifacts after the release vote. Maybe, if the process is documented, and Pat's willing to create & publish these artifacts after the fact, an exception can be made for a few past releases, but going forward, this is not a good process.

Doug Cutting
added a comment - 25/Nov/08 17:29 Shouldn't the pom & metadata files be either included in svn or generated by the build? We shouldn't be publishing artifacts without a documented process in place so that any committer can build the next release. So the creation of this needs to be part of http://wiki.apache.org/hadoop/ZooKeeper/HowToRelease . We shouldn't generate new release artifacts after the release vote. Maybe, if the process is documented, and Pat's willing to create & publish these artifacts after the fact, an exception can be made for a few past releases, but going forward, this is not a good process.

Q: I update maven-metadata.xml and regenerate the md5/sha1?
A: yes. maven-metadata.xml is mainly used by the plugin download system maven has. it will consult this file to see what the latest version of the artifact is. So while it's not critical that the metadata is maintained, it is done for all maven deployed artifacts so ideally it should be done.

Q: And also create a new 3.0.1 subdir at the same level as the 3.0.0 subdir?
A: Yes

Q: Should I include 3.0.0 as well or just 3.0.1?
A: Yes please. That way all your releases are available via Maven.

Hiram Chirino
added a comment - 24/Nov/08 18:47 Q: How are the .sha1 files generated? What's the command?
A:
prompt> openssl dgst -md5 < inputfile
prompt> openssl dgst -sha1 < inputfile
Q: I update maven-metadata.xml and regenerate the md5/sha1?
A: yes. maven-metadata.xml is mainly used by the plugin download system maven has. it will consult this file to see what the latest version of the artifact is. So while it's not critical that the metadata is maintained, it is done for all maven deployed artifacts so ideally it should be done.
Q: And also create a new 3.0.1 subdir at the same level as the 3.0.0 subdir?
A: Yes
Q: Should I include 3.0.0 as well or just 3.0.1?
A: Yes please. That way all your releases are available via Maven.

Patrick Hunt
added a comment - 21/Nov/08 23:15 Ok, great. Thanks for the feedback Hiram. I will execute this process after the next release push (3.0.1). A few more questions after reviewing:
How are the .sha1 files generated? What's the command?
I update maven-metadata.xml and regenerate the md5/sha1?
And also create a new 3.0.1 subdir at the same level as the 3.0.0 subdir?
Should I include 3.0.0 as well or just 3.0.1?

1) Yeah.. same key used to sign the distro. Just so that folks who get the artifacts from maven can verify that it's from a trusted source.

2) The /www/people.apache.org/repo/m2-ibiblio-rsync-repository directory is the Apache Maven2 release repository. Only official releases should be pushed there. Artifacts deployed here will get mirrored to the maven central repository. You deploy to this the same way you deployed the release distro to people.apache.org:/www/www.apache.org/dist/hadoop/zookeeper. I would just scp to people.apache.org:/www/people.apache.org/repo/m2-ibiblio-rsync-repository

3) Yes. The entire directory structure and files contained within the http://people.apache.org/~chirino/zk-repo/ directory need to be preserved. If my directory had GPG signed all the artifacts (including poms), you would have been able to ssh into the people.apache.org machine and run:

4) Same implications that you have when your deploy your release distro to the people.apache.org:/www/www.apache.org/dist/hadoop/zookeeper directory. As long as the people.apache.org does not get hacked only Apache committers can deploy a signed zk jar. Just like with release distros, the onus of verifying jar signatures lies with the downstream user. You guys should document this well on your website along with the KEYS file they should validate against. And hope that the website hosting the KEYS file does not get hacked too (The chain of trust and security is so fragile!)

Hiram Chirino
added a comment - 19/Nov/08 18:21 Hi Patrick,
1) Yeah.. same key used to sign the distro. Just so that folks who get the artifacts from maven can verify that it's from a trusted source.
2) The /www/people.apache.org/repo/m2-ibiblio-rsync-repository directory is the Apache Maven2 release repository. Only official releases should be pushed there. Artifacts deployed here will get mirrored to the maven central repository. You deploy to this the same way you deployed the release distro to people.apache.org:/www/www.apache.org/dist/hadoop/zookeeper. I would just scp to people.apache.org:/www/people.apache.org/repo/m2-ibiblio-rsync-repository
3) Yes. The entire directory structure and files contained within the http://people.apache.org/~chirino/zk-repo/ directory need to be preserved. If my directory had GPG signed all the artifacts (including poms), you would have been able to ssh into the people.apache.org machine and run:
cp -r /x1/users/chirino/public_html/zk-repo/* /www/people.apache.org/repo/m2-ibiblio-rsync-repository
4) Same implications that you have when your deploy your release distro to the people.apache.org:/www/www.apache.org/dist/hadoop/zookeeper directory. As long as the people.apache.org does not get hacked only Apache committers can deploy a signed zk jar. Just like with release distros, the onus of verifying jar signatures lies with the downstream user. You guys should document this well on your website along with the KEYS file they should validate against. And hope that the website hosting the KEYS file does not get hacked too (The chain of trust and security is so fragile!)

Patrick Hunt
added a comment - 18/Nov/08 18:16 Hiram, couple things:
1) is the "GPG project KEY" different from the key I normally use to sign the release?
public release key here:
http://svn.apache.org/repos/asf/hadoop/zookeeper/dist/KEYS
2) how do I "deploy" it to the location you listed? What is that location?
3) do I need to name things in some particular way? What specific artifacts have to be deployed?
4) what are the security implications? what keeps someone from signing a zk jar with another key and deploying that?
Thanks!