From common-commits-return-14956-apmail-hadoop-common-commits-archive=hadoop.apache.org@hadoop.apache.org Mon Aug 22 18:32:12 2011
Return-Path:
X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org
Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
by minotaur.apache.org (Postfix) with SMTP id 243688F90
for ; Mon, 22 Aug 2011 18:32:12 +0000 (UTC)
Received: (qmail 1627 invoked by uid 500); 22 Aug 2011 18:32:11 -0000
Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org
Received: (qmail 1572 invoked by uid 500); 22 Aug 2011 18:32:11 -0000
Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help:
List-Unsubscribe:
List-Post:
List-Id:
Reply-To: common-dev@hadoop.apache.org
Delivered-To: mailing list common-commits@hadoop.apache.org
Received: (qmail 1565 invoked by uid 500); 22 Aug 2011 18:32:11 -0000
Delivered-To: apmail-hadoop-core-commits@hadoop.apache.org
Received: (qmail 1562 invoked by uid 99); 22 Aug 2011 18:32:11 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 Aug 2011 18:32:11 +0000
X-ASF-Spam-Status: No, hits=-2000.0 required=5.0
tests=ALL_TRUSTED
X-Spam-Check-By: apache.org
Received: from [140.211.11.131] (HELO eos.apache.org) (140.211.11.131)
by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 Aug 2011 18:32:09 +0000
Received: from eos.apache.org (localhost [127.0.0.1])
by eos.apache.org (Postfix) with ESMTP id 3AC80B0E;
Mon, 22 Aug 2011 18:31:49 +0000 (UTC)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: Apache Wiki
To: Apache Wiki
Date: Mon, 22 Aug 2011 18:31:49 -0000
Message-ID: <20110822183149.3922.22516@eos.apache.org>
Subject: =?utf-8?q?=5BHadoop_Wiki=5D_Update_of_=22HowToContribute=22_by_TomWhite?=
Auto-Submitted: auto-generated
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for ch=
ange notification.
The "HowToContribute" page has been changed by TomWhite:
http://wiki.apache.org/hadoop/HowToContribute?action=3Ddiff&rev1=3D58&rev2=
=3D59
Comment:
Updated with latest Maven changes
* You can run all the Common unit tests with {{{mvn test}}}, or a speci=
fic unit test with {{{mvn -Dtest=3D test=
}}}. Run these commands from the {{{hadoop-trunk}}} directory.
* For HDFS and MapReduce, you can run all the unit tests with the comma=
nd {{{ant test}}}, or you can run a specific unit test with the command {{{=
ant -Dtestcase=3D test}}} (for example {=
{{ant -Dtestcase=3DTestFileSystem test}}})
=
- =3D=3D=3D=3D Using Ant =3D=3D=3D=3D
- Hadoop HDFS and MapReduce are built by Ant, a Java building tool. (Common=
is build using Maven, see below.) This section will eventually describe ho=
w Ant is used within Hadoop. To start, simply read a good Ant tutorial. T=
he following is a good tutorial, though keep in mind that Hadoop isn't stru=
ctured according to the ways outlined in the tutorial. Use the tutorial to=
get a basic understand of Ant but not to understand how Ant is used for Ha=
doop:
- =
- * Good Ant tutorial: http://i-proving.ca/space/Technologies/Ant+Tutorial
- =
- Although most Java IDEs ship with a version of Ant, having a command line=
version installed is invaluable. You can download a version from http://an=
t.apache.org/.
- =
- After installing Ant, you must make sure that its networking support is c=
onfigured for any proxy you have. Without that the build will not work, as =
the Hadoop builds will not be able to download their dependencies using [[h=
ttp://ant.apache.org/ivy/|Ivy]].
- =
- Tip: to see how Ant is set up, run
- =
- {{{
- ant -diagnostics
- }}}
- =
=3D=3D=3D=3D Using Maven =3D=3D=3D=3D
- Hadoop Common is built using Maven. You need to use version 3 or later.
+ Hadoop built using [[http://maven.apache.org/|Apache Maven]], version 3 o=
r later. (Parts of MapReduce are still built using Ant, see the instruction=
s in the {{{INSTALL}}} file in {{{hadoop-mapreduce}}} for details.)
=
=3D=3D=3D Generating a patch =3D=3D=3D
=3D=3D=3D=3D Unit Tests =3D=3D=3D=3D
Please make sure that all unit tests succeed before constructing your pat=
ch and that no new javac compiler warnings are introduced by your patch.
=
+ For building Hadoop with Maven, use the following to run all unit tests a=
nd build a distribution. The {{{-Ptest-patch}}} profile will check that no =
new compiler warnings have been introduced by your patch.
- {{{
- > cd hadoop-common-trunk
- > ant -Djavac.args=3D"-Xlint -Xmaxwarns 1000" clean test tar
- }}}
- After a while, if you see
=
{{{
- BUILD SUCCESSFUL
+ mvn clean install -Pdist -Dtar -Ptest-patch
}}}
- all is ok, but if you see
=
+ Any test failures can be found in the {{{target/surefire-reports}}} direc=
tory of the relevant module. You can also run this command in one of the {{=
{hadoop-common}}}, {{{hadoop-hdfs}}}, or {{{hadoop-mapreduce}}} directories=
to just test a particular subproject.
- {{{
- BUILD FAILED
- }}}
- then please examine error messages in {{{build/test}}} and fix things bef=
ore proceeding.
=
Unit tests development guidelines HowToDevelopUnitTests
- =
- For building Hadoop Common with Maven, use the following to run all unit =
tests and build a distribution. The {{{-Ptest-patch}}} profile will check t=
hat no new compiler warnings have been introduced by your patch.
- =
- {{{
- mvn clean install -Ptar -Ptest-patch
- }}}
- =
- Any test failures can be found in {{{hadoop-common/target/surefire-report=
s}}}.
=
=3D=3D=3D=3D Javadoc =3D=3D=3D=3D
Please also check the javadoc.
=
{{{
- > ant javadoc
- > firefox build/docs/api/index.html
+ mvn javadoc:javadoc
+ firefox target/site/api/index.html
}}}
Examine all public classes you've changed to see that documentation is co=
mplete, informative, and properly formatted. Your patch must not generate =
any javadoc warnings.
- =
- For Common, use Maven to build the javadoc as follows:
- {{{
- mvn javadoc:javadoc
- firefox hadoop-common/target/site/api/index.html
- }}}
=
=3D=3D=3D=3D Creating a patch =3D=3D=3D=3D
Check to see what files you have modified with:
@@ -150, +113 @@
This way other developers can preview your change by running the script a=
nd then applying the patch.
=
=3D=3D=3D=3D Testing your patch =3D=3D=3D=3D
- Before submitting your patch, you are encouraged to run the same tools th=
at the automated Hudson patch test system will run on your patch. This ena=
bles you to fix problems with your patch before you submit it. The {{{test=
-patch}}} Ant target will run your patch through the same checks that Hudso=
n currently does ''except'' for executing the core and contrib unit tests.
+ Before submitting your patch, you are encouraged to run the same tools th=
at the automated Jenkins patch test system will run on your patch. This en=
ables you to fix problems with your patch before you submit it. The {{{dev-=
support/test-patch.sh}}} script in the trunk directory will run your patch =
through the same checks that Hudson currently does ''except'' for executing=
the unit tests.
=
- To use this target, you must run it from a clean workspace (ie {{{svn sta=
t}}} shows no modifications or additions). From your clean workspace, run:
+ Run this command from a clean workspace (ie {{{svn stat}}} shows no modif=
ications or additions) as follows:
=
{{{
+ dev-support/test-patch.sh /path/to/my.patch
- ant \
- -Dpatch.file=3D/path/to/my.patch \
- -Dforrest.home=3D/path/to/forrest/ \
- -Dfindbugs.home=3D/path/to/findbugs \
- -Dscratch.dir=3D/path/to/a/temp/dir \ (optional)
- -Dsvn.cmd=3D/path/to/subversion/bin/svn \ (optional)
- -Dgrep.cmd=3D/path/to/grep \ (optional)
- -Dpatch.cmd=3D/path/to/patch \ (optional)
- test-patch
}}}
+ =
- At the end, you should get a message on your console that is similar to t=
he comment added to Jira by Hudson's automated patch test system. The scra=
tch directory (which defaults to the value of {{{${user.home}/tmp}}}) will =
contain some output files that will be useful in determining what issues we=
re found in the patch.
+ At the end, you should get a message on your console that is similar to t=
he comment added to Jira by Jenkins's automated patch test system. The scr=
atch directory (which defaults to the value of {{{${user.home}/tmp}}}) will=
contain some output files that will be useful in determining what issues w=
ere found in the patch.
=
Some things to note:
=
@@ -173, +129 @@
* the {{{grep}}} command must support the -o flag (GNU does)
* the {{{patch}}} command must support the -E flag
* you may need to explicitly set ANT_HOME. Running {{{ant -diagnostics}=
}} will tell you the default value on your system.
- =
- For testing a patch in Hadoop Common, use the following command, run from=
the top-level ({{{hadoop-trunk}}}) checkout:
- {{{
- dev-support/test-patch.sh /path/to/my.patch
- }}}
=
Run the same command with no arguments to see the usage options.
=
@@ -219, +170 @@
=
When you believe that your patch is ready to be committed, select the '''=
Submit Patch''' link on the issue's Jira. Submitted patches will be automa=
tically tested against "trunk" by [[http://hudson.zones.apache.org/hudson/v=
iew/Hadoop/|Hudson]], the project's continuous integration engine. Upon te=
st completion, Hudson will add a success ("+1") message or failure ("-1") t=
o your issue report in Jira. If your issue contains multiple patch version=
s, Hudson tests the last patch uploaded.
=
- Folks should run {{{ant clean test javadoc checkstyle}}} (or {{{mvn clean=
install javadoc:javadoc checkstyle:checkstyle}}} in the case of Common) be=
fore selecting '''Submit Patch'''. Tests should all pass. Javadoc should =
report '''no''' warnings or errors. Checkstyle's error count should not exc=
eed that listed at [[http://hudson.zones.apache.org/hudson/job/Hadoop-trunk=
/lastSuccessfulBuild/artifact/trunk/build/test/checkstyle-errors.html|Check=
style Errors]] Hudson's tests are meant to double-check things, and not be=
used as a primary patch tester, which would create too much noise on the m=
ailing list and in Jira. Submitting patches that fail Hudson testing is fr=
owned on, (unless the failure is not actually due to the patch).
+ Folks should run {{{ant clean test javadoc checkstyle}}} (or {{{mvn clean=
install javadoc:javadoc checkstyle:checkstyle}}} in the case of Common or =
HDFS) before selecting '''Submit Patch'''. Tests should all pass. Javadoc=
should report '''no''' warnings or errors. Checkstyle's error count should=
not exceed that listed at [[http://hudson.zones.apache.org/hudson/job/Hado=
op-trunk/lastSuccessfulBuild/artifact/trunk/build/test/checkstyle-errors.ht=
ml|Checkstyle Errors]] Hudson's tests are meant to double-check things, an=
d not be used as a primary patch tester, which would create too much noise =
on the mailing list and in Jira. Submitting patches that fail Hudson testi=
ng is frowned on, (unless the failure is not actually due to the patch).
=
If your patch involves performance optimizations, they should be validate=
d by benchmarks that demonstrate an improvement.
=20