Awesome, thanks for working on this. I'll definitely kick the tires.Noticed the mvn version is 2.0.0-alpha, should we make it 2.0.0-alpha1in case we need to roll multiple alpha releases?

Thanks,Eli

On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>> thanks,> Arun>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

There are other HBase JIRAs related to 2.0.0-alpha that we are workingon, but I'd claim those are all our fault for breaking abstractions tosolve issues. In one case there's a new helpful 2.x API(ShutdownHookManager, thank you!) that we can eventually move to.

However, the minicluster changes are causing us some repeateddiscomfort. It will break, we'll get some help fixing up our tests forthat, then some time later it will break again, repeat. Perhaps wehave no right to complain, the minicluster isn't meant to be used bydownstream projects. If so then please disregard the complaint, butyour assistance in helping to fix the breakage again would be muchappreciated. And, if so, perhaps we can discuss what makes sense interms of a stable minicluster consumable for downstream projects?

Best regards,

- Andy

On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>> thanks,> Arun>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

> Arun,> > Awesome, thanks for working on this. I'll definitely kick the tires.> Noticed the mvn version is 2.0.0-alpha, should we make it 2.0.0-alpha1> in case we need to roll multiple alpha releases?>

I was thinking 2.0.1-alpha, or 2.0.0-alpha-1. Either way.

Arun

> Thanks,> Eli> > On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>> >> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>> >> The maven artifacts are available via repository.apache.org.>> >> Please try the release and vote; the vote will run for the usual 7 days.>> >> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>> >> thanks,>> Arun>> >> >> -->> Arun C. Murthy>> Hortonworks Inc.>> http://hortonworks.com/>> >>

> -1 (nonbinding), we are currently facing a minicluster semantic change> of some kind, or more than one:> > https://issues.apache.org/jira/browse/HBASE-5966> > There are other HBase JIRAs related to 2.0.0-alpha that we are working> on, but I'd claim those are all our fault for breaking abstractions to> solve issues. In one case there's a new helpful 2.x API> (ShutdownHookManager, thank you!) that we can eventually move to.> > However, the minicluster changes are causing us some repeated> discomfort. It will break, we'll get some help fixing up our tests for> that, then some time later it will break again, repeat. Perhaps we> have no right to complain, the minicluster isn't meant to be used by> downstream projects. If so then please disregard the complaint, but> your assistance in helping to fix the breakage again would be much> appreciated. And, if so, perhaps we can discuss what makes sense in> terms of a stable minicluster consumable for downstream projects?> > Best regards,> > - Andy> > On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>> >> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>> >> The maven artifacts are available via repository.apache.org.>> >> Please try the release and vote; the vote will run for the usual 7 days.>> >> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>> >> thanks,>> Arun>> >> >> -->> Arun C. Murthy>> Hortonworks Inc.>> http://hortonworks.com/>> >> > > > > -- > Best regards,> > - Andy> > Problems worthy of attack prove their worth by hitting back. - Piet> Hein (via Tom White)

How should we consider the suitability and stability of MiniMRClusterfor downstream projects?

On Wed, May 9, 2012 at 11:30 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> No worries Andy. I can spin an rc1 once we can pin-point the bug.>> thanks,> Arun>> On May 9, 2012, at 10:17 AM, Andrew Purtell wrote:>>> -1 (nonbinding), we are currently facing a minicluster semantic change>> of some kind, or more than one:>>>> https://issues.apache.org/jira/browse/HBASE-5966>>>> There are other HBase JIRAs related to 2.0.0-alpha that we are working>> on, but I'd claim those are all our fault for breaking abstractions to>> solve issues. In one case there's a new helpful 2.x API>> (ShutdownHookManager, thank you!) that we can eventually move to.>>>> However, the minicluster changes are causing us some repeated>> discomfort. It will break, we'll get some help fixing up our tests for>> that, then some time later it will break again, repeat. Perhaps we>> have no right to complain, the minicluster isn't meant to be used by>> downstream projects. If so then please disregard the complaint, but>> your assistance in helping to fix the breakage again would be much>> appreciated. And, if so, perhaps we can discuss what makes sense in>> terms of a stable minicluster consumable for downstream projects?>>>> Best regards,>>>> - Andy>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>>> The maven artifacts are available via repository.apache.org.>>>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>>>> thanks,>>> Arun>>>>>>>>> -->>> Arun C. Murthy>>> Hortonworks Inc.>>> http://hortonworks.com/>>>>>>>>>>>>>> -->> Best regards,>>>> - Andy>>>> Problems worthy of attack prove their worth by hitting back. - Piet>> Hein (via Tom White)>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

> Sounds good Arun.>> How should we consider the suitability and stability of MiniMRCluster> for downstream projects?>> On Wed, May 9, 2012 at 11:30 AM, Arun C Murthy <[EMAIL PROTECTED]>> wrote:> > No worries Andy. I can spin an rc1 once we can pin-point the bug.> >> > thanks,> > Arun> >> > On May 9, 2012, at 10:17 AM, Andrew Purtell wrote:> >> >> -1 (nonbinding), we are currently facing a minicluster semantic change> >> of some kind, or more than one:> >>> >> https://issues.apache.org/jira/browse/HBASE-5966> >>> >> There are other HBase JIRAs related to 2.0.0-alpha that we are working> >> on, but I'd claim those are all our fault for breaking abstractions to> >> solve issues. In one case there's a new helpful 2.x API> >> (ShutdownHookManager, thank you!) that we can eventually move to.> >>> >> However, the minicluster changes are causing us some repeated> >> discomfort. It will break, we'll get some help fixing up our tests for> >> that, then some time later it will break again, repeat. Perhaps we> >> have no right to complain, the minicluster isn't meant to be used by> >> downstream projects. If so then please disregard the complaint, but> >> your assistance in helping to fix the breakage again would be much> >> appreciated. And, if so, perhaps we can discuss what makes sense in> >> terms of a stable minicluster consumable for downstream projects?> >>> >> Best regards,> >>> >> - Andy> >>> >> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]>> wrote:> >>> I've created a release candidate for hadoop-2.0.0-alpha that I would> like to release.> >>>> >>> It is available at:> http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/> >>>> >>> The maven artifacts are available via repository.apache.org.> >>>> >>> Please try the release and vote; the vote will run for the usual 7> days.> >>>> >>> This is a big milestone for the Apache Hadoop community -> congratulations and thanks for all the contributions!> >>>> >>> thanks,> >>> Arun> >>>> >>>> >>> --> >>> Arun C. Murthy> >>> Hortonworks Inc.> >>> http://hortonworks.com/> >>>> >>>> >>> >>> >>> >> --> >> Best regards,> >>> >> - Andy> >>> >> Problems worthy of attack prove their worth by hitting back. - Piet> >> Hein (via Tom White)> >> > --> > Arun C. Murthy> > Hortonworks Inc.> > http://hortonworks.com/> >> >>>>> --> Best regards,>> - Andy>> Problems worthy of attack prove their worth by hitting back. - Piet> Hein (via Tom White)>

Downloaded the tarball and ran example mr jobs on a single nodecluster, also tried distributed shell jobs.

Best RegardsAhmed

On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>> thanks,> Arun>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

I was over the top initially to surprise. I'm sure the MR minicluster seems a minor detail.

Maybe it's worth thinking about the miniclusters differently? Please pardon if I am rehashing an old discussion.

Things like MRUnit for applications and BigTop for full cluster tests can help, but for as mentioned in the below annotation Pig, Hive, HBase, and other parts of the stack use miniclusters for local end to end testing in unit tests. As the complexity of the stack increases and we consider cross version support, unit tests on miniclusters I think will have no substitute.

As Hadoop 2 has been evolving there has been some difficulty keeping up with minicluster changes. This makes sense. The attention to stability to client APIs and such, and the lack thereof to the minicluster, I think is self evident. But the need to fix up tests unpredictably introduces some friction that perhaps need not be there.

Would a JIRA to discuss defining a subset of the minicluster interfaces as more stable be worthwhile?

> For this reason, in HDFS, we change MiniDFSCluster to LimitedPrivate and> not treat it as such:> > @InterfaceAudience.LimitedPrivate({"HBase", "HDFS", "Hive", "MapReduce",> "Pig"})> @InterfaceStability.Unstable> public class MiniDFSCluster { ...}> > On Wed, May 9, 2012 at 11:33 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:> >> Sounds good Arun.>> >> How should we consider the suitability and stability of MiniMRCluster>> for downstream projects?>> >> On Wed, May 9, 2012 at 11:30 AM, Arun C Murthy <[EMAIL PROTECTED]>>> wrote:>>> No worries Andy. I can spin an rc1 once we can pin-point the bug.>>> >>> thanks,>>> Arun>>> >>> On May 9, 2012, at 10:17 AM, Andrew Purtell wrote:>>> >>>> -1 (nonbinding), we are currently facing a minicluster semantic change>>>> of some kind, or more than one:>>>> >>>> https://issues.apache.org/jira/browse/HBASE-5966>>>> >>>> There are other HBase JIRAs related to 2.0.0-alpha that we are working>>>> on, but I'd claim those are all our fault for breaking abstractions to>>>> solve issues. In one case there's a new helpful 2.x API>>>> (ShutdownHookManager, thank you!) that we can eventually move to.>>>> >>>> However, the minicluster changes are causing us some repeated>>>> discomfort. It will break, we'll get some help fixing up our tests for>>>> that, then some time later it will break again, repeat. Perhaps we>>>> have no right to complain, the minicluster isn't meant to be used by>>>> downstream projects. If so then please disregard the complaint, but>>>> your assistance in helping to fix the breakage again would be much>>>> appreciated. And, if so, perhaps we can discuss what makes sense in>>>> terms of a stable minicluster consumable for downstream projects?>>>> >>>> Best regards,>>>> >>>> - Andy>>>> >>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]>>> wrote:>>>>> I've created a release candidate for hadoop-2.0.0-alpha that I would>> like to release.>>>>> >>>>> It is available at:>> http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>> >>>>> The maven artifacts are available via repository.apache.org.>>>>> >>>>> Please try the release and vote; the vote will run for the usual 7>> days.>>>>> >>>>> This is a big milestone for the Apache Hadoop community ->> congratulations and thanks for all the contributions!>>>>> >>>>> thanks,>>>>> Arun>>>>> >>>>> >>>>> -->>>>> Arun C. Murthy>>>>> Hortonworks Inc.>>>>> http://hortonworks.com/>>>>> >>>>> >>>> >>>> >>>> >>>> -->>>> Best regards,>>>> >>>> - Andy>>>> >>>> Problems worthy of attack prove their worth by hitting back. - Piet>>>> Hein (via Tom White)>>> >>> -->>> Arun C. Murthy>>> Hortonworks Inc.>>> http://hortonworks.com/>>> >>> >> >>

Have you seen the new MiniMRClientCluster class? It's meant to be whatyou describe - a minicluster which only exposes "external" APIs --most importantly a way of getting at a JobClient to submit jobs. Wehave it implemented in both 1.x and 2.x at this point, though I don'trecall if it's in the 1.0.x releases or if it's only slated for 1.1+

-Todd

On Wed, May 9, 2012 at 6:05 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:> Hi Suresh,>> The unstable designation makes sense. As would one for MiniMRCluster.>> I was over the top initially to surprise. I'm sure the MR minicluster seems a minor detail.>> Maybe it's worth thinking about the miniclusters differently? Please pardon if I am rehashing an old discussion.>> Things like MRUnit for applications and BigTop for full cluster tests can help, but for as mentioned in the below annotation Pig, Hive, HBase, and other parts of the stack use miniclusters for local end to end testing in unit tests. As the complexity of the stack increases and we consider cross version support, unit tests on miniclusters I think will have no substitute.>> As Hadoop 2 has been evolving there has been some difficulty keeping up with minicluster changes. This makes sense. The attention to stability to client APIs and such, and the lack thereof to the minicluster, I think is self evident. But the need to fix up tests unpredictably introduces some friction that perhaps need not be there.>> Would a JIRA to discuss defining a subset of the minicluster interfaces as more stable be worthwhile?>> Best regards,>> - Andy>>> On May 9, 2012, at 1:45 PM, Suresh Srinivas <[EMAIL PROTECTED]> wrote:>>> For this reason, in HDFS, we change MiniDFSCluster to LimitedPrivate and>> not treat it as such:>>>> @InterfaceAudience.LimitedPrivate({"HBase", "HDFS", "Hive", "MapReduce",>> "Pig"})>> @InterfaceStability.Unstable>> public class MiniDFSCluster { ...}>>>> On Wed, May 9, 2012 at 11:33 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:>>>>> Sounds good Arun.>>>>>> How should we consider the suitability and stability of MiniMRCluster>>> for downstream projects?>>>>>> On Wed, May 9, 2012 at 11:30 AM, Arun C Murthy <[EMAIL PROTECTED]>>>> wrote:>>>> No worries Andy. I can spin an rc1 once we can pin-point the bug.>>>>>>>> thanks,>>>> Arun>>>>>>>> On May 9, 2012, at 10:17 AM, Andrew Purtell wrote:>>>>>>>>> -1 (nonbinding), we are currently facing a minicluster semantic change>>>>> of some kind, or more than one:>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-5966>>>>>>>>>> There are other HBase JIRAs related to 2.0.0-alpha that we are working>>>>> on, but I'd claim those are all our fault for breaking abstractions to>>>>> solve issues. In one case there's a new helpful 2.x API>>>>> (ShutdownHookManager, thank you!) that we can eventually move to.>>>>>>>>>> However, the minicluster changes are causing us some repeated>>>>> discomfort. It will break, we'll get some help fixing up our tests for>>>>> that, then some time later it will break again, repeat. Perhaps we>>>>> have no right to complain, the minicluster isn't meant to be used by>>>>> downstream projects. If so then please disregard the complaint, but>>>>> your assistance in helping to fix the breakage again would be much>>>>> appreciated. And, if so, perhaps we can discuss what makes sense in>>>>> terms of a stable minicluster consumable for downstream projects?>>>>>>>>>> Best regards,>>>>>>>>>> - Andy>>>>>>>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]>>>> wrote:>>>>>> I've created a release candidate for hadoop-2.0.0-alpha that I would>>> like to release.>>>>>>>>>>>> It is available at:>>> http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>>>>>>>>> The maven artifacts are available via repository.apache.org.>>>>>>>>>>>> Please try the release and vote; the vote will run for the usual 7>>> days.>>>>>>>>>>>> This is a big milestone for the Apache Hadoop community -

On Wed, May 9, 2012 at 11:28 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>> On May 9, 2012, at 10:05 AM, Eli Collins wrote:>>> Arun,>>>> Awesome, thanks for working on this. I'll definitely kick the tires.>> Noticed the mvn version is 2.0.0-alpha, should we make it 2.0.0-alpha1>> in case we need to roll multiple alpha releases?>>>> I was thinking 2.0.1-alpha, or 2.0.0-alpha-1. Either way.>

2.0.1-alpha works for me.

Thanks,Eli

> Arun>>> Thanks,>> Eli>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>>> The maven artifacts are available via repository.apache.org.>>>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>>>> thanks,>>> Arun>>>>>>>>> -->>> Arun C. Murthy>>> Hortonworks Inc.>>> http://hortonworks.com/>>>>>>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

> Have you seen the new MiniMRClientCluster class? It's meant to be what> you describe - a minicluster which only exposes "external" APIs --> most importantly a way of getting at a JobClient to submit jobs. We> have it implemented in both 1.x and 2.x at this point, though I don't> recall if it's in the 1.0.x releases or if it's only slated for 1.1+

Do you mean the below?

/* * A simple interface for a client MR cluster used for testing.This interface * provides basic methods which are independent of the underlyingMini Cluster ( * either through MR1 or MR2). */ public interface MiniMRClientCluster { public void start() throws IOException; public void stop() throws IOException; public Configuration getConfig() throws IOException; }

This doesn't sufficiently encapsulate the mini MR cluster for thepurposes of a test rig. The issues we've seen are variations in whatconfiguration variables are required: their names, and theirsemantics, for finding information about how the cluster is set up.Let's take one basic case, how does one find the address of the jobtracker in a version agnostic way? For example, perhaps:

public InetSocketAddress getJobTrackerAddress();

or at a higher level of abstraction:

public JobTrackerInfo getJobTracker();

public TaskTrackerInfo[] getTaskTrackers();

and, since this a test rig, we'd like to terminate, perhaps abruptly,a task tracker, or launch replacements, or launch new ones.

public interface MiniHDFSClientCluster { public void start() throws IOException; public void stop() throws IOException; public Configuration getConfig() throws IOException; public NameNodeInfo[] getNameNodes(); public DataNodeInfo[] getDataNodes(); public DataNodeInfo startDataNode(...); public boolean stopDataNode(DataNodeInfo dn, boolean force); // Convenience method for getting the filesystem for the cluster // This needs some thought, because we have FileSystem in 1.xand FileContext in 2.x // Here we will use a hypothetical wrapper that uses reflection as needed public FileContext getFileContext(); }

- AndyOn Thu, May 10, 2012 at 11:23 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:> Hi Todd,>>> Have you seen the new MiniMRClientCluster class? It's meant to be what>> you describe - a minicluster which only exposes "external" APIs -->> most importantly a way of getting at a JobClient to submit jobs. We>> have it implemented in both 1.x and 2.x at this point, though I don't>> recall if it's in the 1.0.x releases or if it's only slated for 1.1+>> Do you mean the below?>> /*> * A simple interface for a client MR cluster used for testing.> This interface> * provides basic methods which are independent of the underlying> Mini Cluster (> * either through MR1 or MR2).> */> public interface MiniMRClientCluster {> public void start() throws IOException;> public void stop() throws IOException;> public Configuration getConfig() throws IOException;> }>> This doesn't sufficiently encapsulate the mini MR cluster for the> purposes of a test rig. The issues we've seen are variations in what> configuration variables are required: their names, and their> semantics, for finding information about how the cluster is set up.> Let's take one basic case, how does one find the address of the job> tracker in a version agnostic way? For example, perhaps:>> public InetSocketAddress getJobTrackerAddress();>> or at a higher level of abstraction:>> public JobTrackerInfo getJobTracker();>> public TaskTrackerInfo[] getTaskTrackers();>> and, since this a test rig, we'd like to terminate, perhaps abruptly,> a task tracker, or launch replacements, or launch new ones.>> public boolean stopTaskTracker(TaskTrackerInfo tracker, boolean force);>> public TaskTrackerInfo startTaskTracker(... /* some universal> public parameters TBD */);>> And, likewise for HDFS,>> public interface MiniHDFSClientCluster {> public void start() throws IOException;> public void stop() throws IOException;> public Configuration getConfig() throws IOException;> public NameNodeInfo[] getNameNodes();> public DataNodeInfo[] getDataNodes();> public DataNodeInfo startDataNode(...);> public boolean stopDataNode(DataNodeInfo dn, boolean force);> // Convenience method for getting the filesystem for the cluster> // This needs some thought, because we have FileSystem in 1.x> and FileContext in 2.x> // Here we will use a hypothetical wrapper that uses reflection as needed> public FileContext getFileContext();> }>> and, perhaps additionally a convenience method for corrupting blocks:>> public void writeBlock(Block block, byte[] data, long offset,> boolean updateChecksum) throws IOException;>> and so on.>> Best regards,>> - Andy>> Problems worthy of attack prove their worth by hitting back. - Piet> Hein (via Tom White)

This has been handled, I withdraw my objection. Otherwise 2.0.0-alphahas been working well for me, +1.

Best regards,

- Andy

On Wed, May 9, 2012 at 10:17 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:> -1 (nonbinding), we are currently facing a minicluster semantic change> of some kind, or more than one:>> https://issues.apache.org/jira/browse/HBASE-5966>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>> The maven artifacts are available via repository.apache.org.>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>> thanks,>> Arun>>>>>> -->> Arun C. Murthy>> Hortonworks Inc.>> http://hortonworks.com/>>>>>>>> --> Best regards,>> - Andy>> Problems worthy of attack prove their worth by hitting back. - Piet> Hein (via Tom White)

+1 I installed the build on a 6 node cluster and kicked the tires,didn't find any blocking issues.

Btw in the future better to build from the svn repo so the revision isan svn rev from the release branch. Eg 1336254 instead of 40e90d3c7which is from the git mirror, this way we're consistent acrossreleases.

hadoop-2.0.0-alpha $ ./bin/hadoop versionHadoop 2.0.0-alphaSubversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common-r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>> thanks,> Arun>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

Looking at the release tag vs the current state of branch-2, I havetwo concerns from the point of view of HDFS:

1) We reverted HDFS-3157 in branch-2 because it sends deletions forcorrupt replicas without properly going through the "corrupt block"path. We saw this cause data loss in TestPipelinesFailover. So, I'mnervous about putting it in a release, even labeled as alpha.

2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPCenvelope in branch-2, but didn't make it into this rc. So, that wouldmean that future alphas would not be protocol-compatible with thisalpha. Per a discussion a few weeks ago, I think we all were inagreement that, if possible, we'd like all 2.x to be compatible forclient-server communication, at least (even if we don't supportcross-version for the intra-cluster protocols)

Do other folks think it's worth rolling an rc1? I would propose either:a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 onbranch-2.0.0-alpha, so these are the only changes since rc0. Roll anew rc1 from here.or:b) Discard the current branch-2.0.0-alpha and re-branch from thecurrent state of branch-2.

-Todd

On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:> +1 I installed the build on a 6 node cluster and kicked the tires,> didn't find any blocking issues.>> Btw in the future better to build from the svn repo so the revision is> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7> which is from the git mirror, this way we're consistent across> releases.>> hadoop-2.0.0-alpha $ ./bin/hadoop version> Hadoop 2.0.0-alpha> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>> The maven artifacts are available via repository.apache.org.>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>> thanks,>> Arun>>>>>> -->> Arun C. Murthy>> Hortonworks Inc.>> http://hortonworks.com/>>>>

> Looking at the release tag vs the current state of branch-2, I have> two concerns from the point of view of HDFS:> > 1) We reverted HDFS-3157 in branch-2 because it sends deletions for> corrupt replicas without properly going through the "corrupt block"> path. We saw this cause data loss in TestPipelinesFailover. So, I'm> nervous about putting it in a release, even labeled as alpha.> > 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC> envelope in branch-2, but didn't make it into this rc. So, that would> mean that future alphas would not be protocol-compatible with this> alpha. Per a discussion a few weeks ago, I think we all were in> agreement that, if possible, we'd like all 2.x to be compatible for> client-server communication, at least (even if we don't support> cross-version for the intra-cluster protocols)> > Do other folks think it's worth rolling an rc1? I would propose either:> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.> or:> b) Discard the current branch-2.0.0-alpha and re-branch from the> current state of branch-2.> > -Todd> > On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>> +1 I installed the build on a 6 node cluster and kicked the tires,>> didn't find any blocking issues.>> >> Btw in the future better to build from the svn repo so the revision is>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>> which is from the git mirror, this way we're consistent across>> releases.>> >> hadoop-2.0.0-alpha $ ./bin/hadoop version>> Hadoop 2.0.0-alpha>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>> >> >> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>> >>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>> >>> The maven artifacts are available via repository.apache.org.>>> >>> Please try the release and vote; the vote will run for the usual 7 days.>>> >>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>> >>> thanks,>>> Arun>>> >>> >>> -->>> Arun C. Murthy>>> Hortonworks Inc.>>> http://hortonworks.com/>>> >>> > > > > -- > Todd Lipcon> Software Engineer, Cloudera

> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.I have merged HDFS-3157 revert.Do you mind taking a look at HADOOP-8285 and HADOOP-8366?

> Looking at the release tag vs the current state of branch-2, I have> two concerns from the point of view of HDFS:>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for> corrupt replicas without properly going through the "corrupt block"> path. We saw this cause data loss in TestPipelinesFailover. So, I'm> nervous about putting it in a release, even labeled as alpha.>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC> envelope in branch-2, but didn't make it into this rc. So, that would> mean that future alphas would not be protocol-compatible with this> alpha. Per a discussion a few weeks ago, I think we all were in> agreement that, if possible, we'd like all 2.x to be compatible for> client-server communication, at least (even if we don't support> cross-version for the intra-cluster protocols)>> Do other folks think it's worth rolling an rc1? I would propose either:> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.> or:> b) Discard the current branch-2.0.0-alpha and re-branch from the> current state of branch-2.>> -Todd>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>> +1 I installed the build on a 6 node cluster and kicked the tires,>> didn't find any blocking issues.>>>> Btw in the future better to build from the svn repo so the revision is>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>> which is from the git mirror, this way we're consistent across>> releases.>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>> Hadoop 2.0.0-alpha>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>>> The maven artifacts are available via repository.apache.org.>>>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>>>> thanks,>>> Arun>>>>>>>>> -->>> Arun C. Murthy>>> Hortonworks Inc.>>> http://hortonworks.com/>>>>>>>>>> --> Todd Lipcon> Software Engineer, Cloudera

> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.I have merged HDFS-3157 revert.Do you mind taking a look at HADOOP-8285 and HADOOP-8366?

> Looking at the release tag vs the current state of branch-2, I have> two concerns from the point of view of HDFS:>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for> corrupt replicas without properly going through the "corrupt block"> path. We saw this cause data loss in TestPipelinesFailover. So, I'm> nervous about putting it in a release, even labeled as alpha.>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC> envelope in branch-2, but didn't make it into this rc. So, that would> mean that future alphas would not be protocol-compatible with this> alpha. Per a discussion a few weeks ago, I think we all were in> agreement that, if possible, we'd like all 2.x to be compatible for> client-server communication, at least (even if we don't support> cross-version for the intra-cluster protocols)>> Do other folks think it's worth rolling an rc1? I would propose either:> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.> or:> b) Discard the current branch-2.0.0-alpha and re-branch from the> current state of branch-2.>> -Todd>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>> +1 I installed the build on a 6 node cluster and kicked the tires,>> didn't find any blocking issues.>>>> Btw in the future better to build from the svn repo so the revision is>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>> which is from the git mirror, this way we're consistent across>> releases.>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>> Hadoop 2.0.0-alpha>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>>> The maven artifacts are available via repository.apache.org.>>>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>>>> thanks,>>> Arun>>>>>>>>> -->>> Arun C. Murthy>>> Hortonworks Inc.>>> http://hortonworks.com/>>>>>>>>>> --> Todd Lipcon> Software Engineer, Cloudera

> Let me merge HADOOP-8285 and HADOOP-8366. Thanks.> Tsz-Wo> > > > ----- Original Message -----> From: Uma Maheswara Rao G <[EMAIL PROTECTED]>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Cc: > Sent: Monday, May 14, 2012 10:56 AM> Subject: RE: [VOTE] Release hadoop-2.0.0-alpha> >> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>> new rc1 from here.> I have merged HDFS-3157 revert.> Do you mind taking a look at HADOOP-8285 and HADOOP-8366?> > Thanks,> Uma> ________________________________________> From: Arun C Murthy [[EMAIL PROTECTED]]> Sent: Monday, May 14, 2012 10:24 PM> To: [EMAIL PROTECTED]> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha> > Todd,> > Please go ahead and merge changes into branch-2.0.0-alpha and I'll roll RC1.> > thanks,> Arun> > On May 12, 2012, at 10:05 PM, Todd Lipcon wrote:> >> Looking at the release tag vs the current state of branch-2, I have>> two concerns from the point of view of HDFS:>> >> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for>> corrupt replicas without properly going through the "corrupt block">> path. We saw this cause data loss in TestPipelinesFailover. So, I'm>> nervous about putting it in a release, even labeled as alpha.>> >> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC>> envelope in branch-2, but didn't make it into this rc. So, that would>> mean that future alphas would not be protocol-compatible with this>> alpha. Per a discussion a few weeks ago, I think we all were in>> agreement that, if possible, we'd like all 2.x to be compatible for>> client-server communication, at least (even if we don't support>> cross-version for the intra-cluster protocols)>> >> Do other folks think it's worth rolling an rc1? I would propose either:>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>> new rc1 from here.>> or:>> b) Discard the current branch-2.0.0-alpha and re-branch from the>> current state of branch-2.>> >> -Todd>> >> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>>> +1 I installed the build on a 6 node cluster and kicked the tires,>>> didn't find any blocking issues.>>> >>> Btw in the future better to build from the svn repo so the revision is>>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>>> which is from the git mirror, this way we're consistent across>>> releases.>>> >>> hadoop-2.0.0-alpha $ ./bin/hadoop version>>> Hadoop 2.0.0-alpha>>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>> >>> >>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>> >>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>> >>>> The maven artifacts are available via repository.apache.org.>>>> >>>> Please try the release and vote; the vote will run for the usual 7 days.>>>> >>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>> >>>> thanks,>>>> Arun>>>> >>>> >>>> -->>>> Arun C. Murthy>>>> Hortonworks Inc.>>>> http://hortonworks.com/>>>> >>>> >> >> >> >> -->> Todd Lipcon>> Software Engineer, Cloudera> > --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/

> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.I have merged HDFS-3157 revert.Do you mind taking a look at HADOOP-8285 and HADOOP-8366?

> Looking at the release tag vs the current state of branch-2, I have> two concerns from the point of view of HDFS:>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for> corrupt replicas without properly going through the "corrupt block"> path. We saw this cause data loss in TestPipelinesFailover. So, I'm> nervous about putting it in a release, even labeled as alpha.>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC> envelope in branch-2, but didn't make it into this rc. So, that would> mean that future alphas would not be protocol-compatible with this> alpha. Per a discussion a few weeks ago, I think we all were in> agreement that, if possible, we'd like all 2.x to be compatible for> client-server communication, at least (even if we don't support> cross-version for the intra-cluster protocols)>> Do other folks think it's worth rolling an rc1? I would propose either:> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.> or:> b) Discard the current branch-2.0.0-alpha and re-branch from the> current state of branch-2.>> -Todd>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>> +1 I installed the build on a 6 node cluster and kicked the tires,>> didn't find any blocking issues.>>>> Btw in the future better to build from the svn repo so the revision is>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>> which is from the git mirror, this way we're consistent across>> releases.>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>> Hadoop 2.0.0-alpha>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>>> The maven artifacts are available via repository.apache.org.>>>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>>>> thanks,>>> Arun>>>>>>>>> -->>> Arun C. Murthy>>> Hortonworks Inc.>>> http://hortonworks.com/>>>>>>>>>> --> Todd Lipcon> Software Engineer, Cloudera

> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.I have merged HDFS-3157 revert.Do you mind taking a look at HADOOP-8285 and HADOOP-8366?

> Looking at the release tag vs the current state of branch-2, I have> two concerns from the point of view of HDFS:>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for> corrupt replicas without properly going through the "corrupt block"> path. We saw this cause data loss in TestPipelinesFailover. So, I'm> nervous about putting it in a release, even labeled as alpha.>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC> envelope in branch-2, but didn't make it into this rc. So, that would> mean that future alphas would not be protocol-compatible with this> alpha. Per a discussion a few weeks ago, I think we all were in> agreement that, if possible, we'd like all 2.x to be compatible for> client-server communication, at least (even if we don't support> cross-version for the intra-cluster protocols)>> Do other folks think it's worth rolling an rc1? I would propose either:> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a> new rc1 from here.> or:> b) Discard the current branch-2.0.0-alpha and re-branch from the> current state of branch-2.>> -Todd>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>> +1 I installed the build on a 6 node cluster and kicked the tires,>> didn't find any blocking issues.>>>> Btw in the future better to build from the svn repo so the revision is>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>> which is from the git mirror, this way we're consistent across>> releases.>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>> Hadoop 2.0.0-alpha>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>>> The maven artifacts are available via repository.apache.org.>>>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!>>>>>> thanks,>>> Arun>>>>>>>>> -->>> Arun C. Murthy>>> Hortonworks Inc.>>> http://hortonworks.com/>>>>>>>>>> --> Todd Lipcon> Software Engineer, Cloudera

As soon as jira is back up and I can post an updated patch I'll mergeHDFS-3418 (also incompatible).On Mon, May 14, 2012 at 12:16 PM, Tsz Wo Sze <[EMAIL PROTECTED]> wrote:> I just have merged HADOOP-8285 and HADOOP-8366. I also have merged HDFS-3211 since it is an incompatible protocol change (without it, 2.0.0-alphaand 2.0.0 will be incompatible.)>> Tsz-Wo>>>> ----- Original Message -----> From: Tsz Wo Sze <[EMAIL PROTECTED]>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Cc:> Sent: Monday, May 14, 2012 11:07 AM> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>> Let me merge HADOOP-8285 and HADOOP-8366. Thanks.> Tsz-Wo>>>> ----- Original Message -----> From: Uma Maheswara Rao G <[EMAIL PROTECTED]>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Cc:> Sent: Monday, May 14, 2012 10:56 AM> Subject: RE: [VOTE] Release hadoop-2.0.0-alpha>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>> new rc1 from here.> I have merged HDFS-3157 revert.> Do you mind taking a look at HADOOP-8285 and HADOOP-8366?>> Thanks,> Uma> ________________________________________> From: Arun C Murthy [[EMAIL PROTECTED]]> Sent: Monday, May 14, 2012 10:24 PM> To: [EMAIL PROTECTED]> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>> Todd,>> Please go ahead and merge changes into branch-2.0.0-alpha and I'll roll RC1.>> thanks,> Arun>> On May 12, 2012, at 10:05 PM, Todd Lipcon wrote:>>> Looking at the release tag vs the current state of branch-2, I have>> two concerns from the point of view of HDFS:>>>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for>> corrupt replicas without properly going through the "corrupt block">> path. We saw this cause data loss in TestPipelinesFailover. So, I'm>> nervous about putting it in a release, even labeled as alpha.>>>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC>> envelope in branch-2, but didn't make it into this rc. So, that would>> mean that future alphas would not be protocol-compatible with this>> alpha. Per a discussion a few weeks ago, I think we all were in>> agreement that, if possible, we'd like all 2.x to be compatible for>> client-server communication, at least (even if we don't support>> cross-version for the intra-cluster protocols)>>>> Do other folks think it's worth rolling an rc1? I would propose either:>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>> new rc1 from here.>> or:>> b) Discard the current branch-2.0.0-alpha and re-branch from the>> current state of branch-2.>>>> -Todd>>>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>>> +1 I installed the build on a 6 node cluster and kicked the tires,>>> didn't find any blocking issues.>>>>>> Btw in the future better to build from the svn repo so the revision is>>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>>> which is from the git mirror, this way we're consistent across>>> releases.>>>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>>> Hadoop 2.0.0-alpha>>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>>>>>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>>>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/>>>>>>>> The maven artifacts are available via repository.apache.org.>>>>>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>>>>>> This is a big milestone for the Apache Hadoop community - congratulations and thanks for all the contributions!

Do we want to get MAPREDUCE-4067 in as well ? It affects folks who may bewriting their own AMs. Shouldn't affect MR clients though. I believe 2.0alpha doesn't freeze the Yarn protocols for the 2.0 branch, so probably notcritical.

One more thing on the rc tarball: the source artifact doesn't appearto be an exact svn export, based on a diff. For example, it includesthe README, NOTICE, and LICENSE files, as well as a few other thingswhich appear to be build artifacts (eghadoop-hdfs-project/hadoop-hdfs/downloads,hadoop-hdfs-project/hadoop-hdfs/test_edit_log, etc).

It seems like we _should_ have the various README style files, but weshouldn't have the test artifacts in our source release.

In order to get our source release to match svn, perhaps we shouldmove NOTICE, README, LICENSE, etc to the top level of our svn repo,such that a pure svn export would be a releaseable source artifact?

Can HDFS-3265 be included too?It seems like this was marked for inclusion but I can't seem to find thepatch in the branch-2.0.0-alpha tree.

Thanks,Kumar

Kumar Ravi

From: Todd Lipcon <[EMAIL PROTECTED]>

To: [EMAIL PROTECTED]

Date: 05/14/2012 11:21 PM

Subject: Re: [VOTE] Release hadoop-2.0.0-alpha

Hey Arun,

One more thing on the rc tarball: the source artifact doesn't appearto be an exact svn export, based on a diff. For example, it includesthe README, NOTICE, and LICENSE files, as well as a few other thingswhich appear to be build artifacts (eghadoop-hdfs-project/hadoop-hdfs/downloads,hadoop-hdfs-project/hadoop-hdfs/test_edit_log, etc).

It seems like we _should_ have the various README style files, but weshouldn't have the test artifacts in our source release.

In order to get our source release to match svn, perhaps we shouldmove NOTICE, README, LICENSE, etc to the top level of our svn repo,such that a pure svn export would be a releaseable source artifact?

IMO we should keep the new changes for 2.0.0-alpha to a minimum (justthings that impact client-server wire compatibility) and then plan a2.0.1-alpha ASAP following this release, where we can pull in everythingelse that went into branch-2 in the last couple weeks since the 2.0.0-alphabranch was cut.

Arun: do you have time today to roll a new RC? If not, I am happy to do so.

> As soon as jira is back up and I can post an updated patch I'll merge> HDFS-3418 (also incompatible).> > > On Mon, May 14, 2012 at 12:16 PM, Tsz Wo Sze <[EMAIL PROTECTED]> wrote:>> I just have merged HADOOP-8285 and HADOOP-8366. I also have merged HDFS-3211 since it is an incompatible protocol change (without it, 2.0.0-alphaand 2.0.0 will be incompatible.)>> >> Tsz-Wo>> >> >> >> ----- Original Message ----->> From: Tsz Wo Sze <[EMAIL PROTECTED]>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Cc:>> Sent: Monday, May 14, 2012 11:07 AM>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>> >> Let me merge HADOOP-8285 and HADOOP-8366. Thanks.>> Tsz-Wo>> >> >> >> ----- Original Message ----->> From: Uma Maheswara Rao G <[EMAIL PROTECTED]>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Cc:>> Sent: Monday, May 14, 2012 10:56 AM>> Subject: RE: [VOTE] Release hadoop-2.0.0-alpha>> >>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>> new rc1 from here.>> I have merged HDFS-3157 revert.>> Do you mind taking a look at HADOOP-8285 and HADOOP-8366?>> >> Thanks,>> Uma>> ________________________________________>> From: Arun C Murthy [[EMAIL PROTECTED]]>> Sent: Monday, May 14, 2012 10:24 PM>> To: [EMAIL PROTECTED]>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>> >> Todd,>> >> Please go ahead and merge changes into branch-2.0.0-alpha and I'll roll RC1.>> >> thanks,>> Arun>> >> On May 12, 2012, at 10:05 PM, Todd Lipcon wrote:>> >>> Looking at the release tag vs the current state of branch-2, I have>>> two concerns from the point of view of HDFS:>>> >>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for>>> corrupt replicas without properly going through the "corrupt block">>> path. We saw this cause data loss in TestPipelinesFailover. So, I'm>>> nervous about putting it in a release, even labeled as alpha.>>> >>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC>>> envelope in branch-2, but didn't make it into this rc. So, that would>>> mean that future alphas would not be protocol-compatible with this>>> alpha. Per a discussion a few weeks ago, I think we all were in>>> agreement that, if possible, we'd like all 2.x to be compatible for>>> client-server communication, at least (even if we don't support>>> cross-version for the intra-cluster protocols)>>> >>> Do other folks think it's worth rolling an rc1? I would propose either:>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>> new rc1 from here.>>> or:>>> b) Discard the current branch-2.0.0-alpha and re-branch from the>>> current state of branch-2.>>> >>> -Todd>>> >>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>>>> +1 I installed the build on a 6 node cluster and kicked the tires,>>>> didn't find any blocking issues.>>>> >>>> Btw in the future better to build from the svn repo so the revision is>>>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>>>> which is from the git mirror, this way we're consistent across>>>> releases.>>>> >>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>>>> Hadoop 2.0.0-alpha>>>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>>>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>>>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>>>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>>> >>>> >>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>>>>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to release.>>>>> >>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/

> Eli, is this done so I can roll rc1?> > On May 14, 2012, at 1:32 PM, Eli Collins wrote:> >> As soon as jira is back up and I can post an updated patch I'll merge>> HDFS-3418 (also incompatible).>> >> >> On Mon, May 14, 2012 at 12:16 PM, Tsz Wo Sze <[EMAIL PROTECTED]> wrote:>>> I just have merged HADOOP-8285 and HADOOP-8366. I also have merged HDFS-3211 since it is an incompatible protocol change (without it, 2.0.0-alphaand 2.0.0 will be incompatible.)>>> >>> Tsz-Wo>>> >>> >>> >>> ----- Original Message ----->>> From: Tsz Wo Sze <[EMAIL PROTECTED]>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>> Cc:>>> Sent: Monday, May 14, 2012 11:07 AM>>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>>> >>> Let me merge HADOOP-8285 and HADOOP-8366. Thanks.>>> Tsz-Wo>>> >>> >>> >>> ----- Original Message ----->>> From: Uma Maheswara Rao G <[EMAIL PROTECTED]>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>> Cc:>>> Sent: Monday, May 14, 2012 10:56 AM>>> Subject: RE: [VOTE] Release hadoop-2.0.0-alpha>>> >>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>>> new rc1 from here.>>> I have merged HDFS-3157 revert.>>> Do you mind taking a look at HADOOP-8285 and HADOOP-8366?>>> >>> Thanks,>>> Uma>>> ________________________________________>>> From: Arun C Murthy [[EMAIL PROTECTED]]>>> Sent: Monday, May 14, 2012 10:24 PM>>> To: [EMAIL PROTECTED]>>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>>> >>> Todd,>>> >>> Please go ahead and merge changes into branch-2.0.0-alpha and I'll roll RC1.>>> >>> thanks,>>> Arun>>> >>> On May 12, 2012, at 10:05 PM, Todd Lipcon wrote:>>> >>>> Looking at the release tag vs the current state of branch-2, I have>>>> two concerns from the point of view of HDFS:>>>> >>>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for>>>> corrupt replicas without properly going through the "corrupt block">>>> path. We saw this cause data loss in TestPipelinesFailover. So, I'm>>>> nervous about putting it in a release, even labeled as alpha.>>>> >>>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC>>>> envelope in branch-2, but didn't make it into this rc. So, that would>>>> mean that future alphas would not be protocol-compatible with this>>>> alpha. Per a discussion a few weeks ago, I think we all were in>>>> agreement that, if possible, we'd like all 2.x to be compatible for>>>> client-server communication, at least (even if we don't support>>>> cross-version for the intra-cluster protocols)>>>> >>>> Do other folks think it's worth rolling an rc1? I would propose either:>>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>>> new rc1 from here.>>>> or:>>>> b) Discard the current branch-2.0.0-alpha and re-branch from the>>>> current state of branch-2.>>>> >>>> -Todd>>>> >>>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>>>>> +1 I installed the build on a 6 node cluster and kicked the tires,>>>>> didn't find any blocking issues.>>>>> >>>>> Btw in the future better to build from the svn repo so the revision is>>>>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>>>>> which is from the git mirror, this way we're consistent across>>>>> releases.>>>>> >>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>>>>> Hadoop 2.0.0-alpha>>>>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>>>>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>>>>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>>>>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>>>> >>>>> >>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

On Tue, May 15, 2012 at 11:10 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> Any more HDFS related merges before I roll RC1?

I'm good as is. Thanks!

>> On May 15, 2012, at 10:05 AM, Arun C Murthy wrote:>>> Eli, is this done so I can roll rc1?>>>> On May 14, 2012, at 1:32 PM, Eli Collins wrote:>>>>> As soon as jira is back up and I can post an updated patch I'll merge>>> HDFS-3418 (also incompatible).>>>>>>>>> On Mon, May 14, 2012 at 12:16 PM, Tsz Wo Sze <[EMAIL PROTECTED]> wrote:>>>> I just have merged HADOOP-8285 and HADOOP-8366. I also have merged HDFS-3211 since it is an incompatible protocol change (without it, 2.0.0-alphaand 2.0.0 will be incompatible.)>>>>>>>> Tsz-Wo>>>>>>>>>>>>>>>> ----- Original Message ----->>>> From: Tsz Wo Sze <[EMAIL PROTECTED]>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>>> Cc:>>>> Sent: Monday, May 14, 2012 11:07 AM>>>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>>>>>>>> Let me merge HADOOP-8285 and HADOOP-8366. Thanks.>>>> Tsz-Wo>>>>>>>>>>>>>>>> ----- Original Message ----->>>> From: Uma Maheswara Rao G <[EMAIL PROTECTED]>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>>> Cc:>>>> Sent: Monday, May 14, 2012 10:56 AM>>>> Subject: RE: [VOTE] Release hadoop-2.0.0-alpha>>>>>>>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>>>> new rc1 from here.>>>> I have merged HDFS-3157 revert.>>>> Do you mind taking a look at HADOOP-8285 and HADOOP-8366?>>>>>>>> Thanks,>>>> Uma>>>> ________________________________________>>>> From: Arun C Murthy [[EMAIL PROTECTED]]>>>> Sent: Monday, May 14, 2012 10:24 PM>>>> To: [EMAIL PROTECTED]>>>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>>>>>>>> Todd,>>>>>>>> Please go ahead and merge changes into branch-2.0.0-alpha and I'll roll RC1.>>>>>>>> thanks,>>>> Arun>>>>>>>> On May 12, 2012, at 10:05 PM, Todd Lipcon wrote:>>>>>>>>> Looking at the release tag vs the current state of branch-2, I have>>>>> two concerns from the point of view of HDFS:>>>>>>>>>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for>>>>> corrupt replicas without properly going through the "corrupt block">>>>> path. We saw this cause data loss in TestPipelinesFailover. So, I'm>>>>> nervous about putting it in a release, even labeled as alpha.>>>>>>>>>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC>>>>> envelope in branch-2, but didn't make it into this rc. So, that would>>>>> mean that future alphas would not be protocol-compatible with this>>>>> alpha. Per a discussion a few weeks ago, I think we all were in>>>>> agreement that, if possible, we'd like all 2.x to be compatible for>>>>> client-server communication, at least (even if we don't support>>>>> cross-version for the intra-cluster protocols)>>>>>>>>>> Do other folks think it's worth rolling an rc1? I would propose either:>>>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>>>> new rc1 from here.>>>>> or:>>>>> b) Discard the current branch-2.0.0-alpha and re-branch from the>>>>> current state of branch-2.>>>>>>>>>> -Todd>>>>>>>>>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>>>>>> +1 I installed the build on a 6 node cluster and kicked the tires,>>>>>> didn't find any blocking issues.>>>>>>>>>>>> Btw in the future better to build from the svn repo so the revision is>>>>>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>>>>>> which is from the git mirror, this way we're consistent across>>>>>> releases.>>>>>>>>>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>>>>>> Hadoop 2.0.0-alpha>>>>>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>>>>>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55

On Tue, May 15, 2012 at 10:05 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> Eli, is this done so I can roll rc1?>> On May 14, 2012, at 1:32 PM, Eli Collins wrote:>>> As soon as jira is back up and I can post an updated patch I'll merge>> HDFS-3418 (also incompatible).>>>>>> On Mon, May 14, 2012 at 12:16 PM, Tsz Wo Sze <[EMAIL PROTECTED]> wrote:>>> I just have merged HADOOP-8285 and HADOOP-8366. I also have merged HDFS-3211 since it is an incompatible protocol change (without it, 2.0.0-alphaand 2.0.0 will be incompatible.)>>>>>> Tsz-Wo>>>>>>>>>>>> ----- Original Message ----->>> From: Tsz Wo Sze <[EMAIL PROTECTED]>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>> Cc:>>> Sent: Monday, May 14, 2012 11:07 AM>>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>>>>>> Let me merge HADOOP-8285 and HADOOP-8366. Thanks.>>> Tsz-Wo>>>>>>>>>>>> ----- Original Message ----->>> From: Uma Maheswara Rao G <[EMAIL PROTECTED]>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>> Cc:>>> Sent: Monday, May 14, 2012 10:56 AM>>> Subject: RE: [VOTE] Release hadoop-2.0.0-alpha>>>>>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>>> new rc1 from here.>>> I have merged HDFS-3157 revert.>>> Do you mind taking a look at HADOOP-8285 and HADOOP-8366?>>>>>> Thanks,>>> Uma>>> ________________________________________>>> From: Arun C Murthy [[EMAIL PROTECTED]]>>> Sent: Monday, May 14, 2012 10:24 PM>>> To: [EMAIL PROTECTED]>>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>>>>>> Todd,>>>>>> Please go ahead and merge changes into branch-2.0.0-alpha and I'll roll RC1.>>>>>> thanks,>>> Arun>>>>>> On May 12, 2012, at 10:05 PM, Todd Lipcon wrote:>>>>>>> Looking at the release tag vs the current state of branch-2, I have>>>> two concerns from the point of view of HDFS:>>>>>>>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for>>>> corrupt replicas without properly going through the "corrupt block">>>> path. We saw this cause data loss in TestPipelinesFailover. So, I'm>>>> nervous about putting it in a release, even labeled as alpha.>>>>>>>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC>>>> envelope in branch-2, but didn't make it into this rc. So, that would>>>> mean that future alphas would not be protocol-compatible with this>>>> alpha. Per a discussion a few weeks ago, I think we all were in>>>> agreement that, if possible, we'd like all 2.x to be compatible for>>>> client-server communication, at least (even if we don't support>>>> cross-version for the intra-cluster protocols)>>>>>>>> Do other folks think it's worth rolling an rc1? I would propose either:>>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>>> new rc1 from here.>>>> or:>>>> b) Discard the current branch-2.0.0-alpha and re-branch from the>>>> current state of branch-2.>>>>>>>> -Todd>>>>>>>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>>>>> +1 I installed the build on a 6 node cluster and kicked the tires,>>>>> didn't find any blocking issues.>>>>>>>>>> Btw in the future better to build from the svn repo so the revision is>>>>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>>>>> which is from the git mirror, this way we're consistent across>>>>> releases.>>>>>>>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>>>>> Hadoop 2.0.0-alpha>>>>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common>>>>> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55>>>>> Compiled by hortonmu on Wed May 9 16:19:55 UTC 2012>>>>> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888>>>>>>>>>>>>>>> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

On Tue, May 15, 2012 at 11:58 AM, Eli Collins <[EMAIL PROTECTED]> wrote:> Turns out 3418 is compatible, so no need to block on it.>> Thanks Arun!>> On Tue, May 15, 2012 at 10:05 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>> Eli, is this done so I can roll rc1?>>>> On May 14, 2012, at 1:32 PM, Eli Collins wrote:>>>>> As soon as jira is back up and I can post an updated patch I'll merge>>> HDFS-3418 (also incompatible).>>>>>>>>> On Mon, May 14, 2012 at 12:16 PM, Tsz Wo Sze <[EMAIL PROTECTED]> wrote:>>>> I just have merged HADOOP-8285 and HADOOP-8366. I also have merged HDFS-3211 since it is an incompatible protocol change (without it, 2.0.0-alphaand 2.0.0 will be incompatible.)>>>>>>>> Tsz-Wo>>>>>>>>>>>>>>>> ----- Original Message ----->>>> From: Tsz Wo Sze <[EMAIL PROTECTED]>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>>> Cc:>>>> Sent: Monday, May 14, 2012 11:07 AM>>>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>>>>>>>> Let me merge HADOOP-8285 and HADOOP-8366. Thanks.>>>> Tsz-Wo>>>>>>>>>>>>>>>> ----- Original Message ----->>>> From: Uma Maheswara Rao G <[EMAIL PROTECTED]>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>>> Cc:>>>> Sent: Monday, May 14, 2012 10:56 AM>>>> Subject: RE: [VOTE] Release hadoop-2.0.0-alpha>>>>>>>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>>>> new rc1 from here.>>>> I have merged HDFS-3157 revert.>>>> Do you mind taking a look at HADOOP-8285 and HADOOP-8366?>>>>>>>> Thanks,>>>> Uma>>>> ________________________________________>>>> From: Arun C Murthy [[EMAIL PROTECTED]]>>>> Sent: Monday, May 14, 2012 10:24 PM>>>> To: [EMAIL PROTECTED]>>>> Subject: Re: [VOTE] Release hadoop-2.0.0-alpha>>>>>>>> Todd,>>>>>>>> Please go ahead and merge changes into branch-2.0.0-alpha and I'll roll RC1.>>>>>>>> thanks,>>>> Arun>>>>>>>> On May 12, 2012, at 10:05 PM, Todd Lipcon wrote:>>>>>>>>> Looking at the release tag vs the current state of branch-2, I have>>>>> two concerns from the point of view of HDFS:>>>>>>>>>> 1) We reverted HDFS-3157 in branch-2 because it sends deletions for>>>>> corrupt replicas without properly going through the "corrupt block">>>>> path. We saw this cause data loss in TestPipelinesFailover. So, I'm>>>>> nervous about putting it in a release, even labeled as alpha.>>>>>>>>>> 2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC>>>>> envelope in branch-2, but didn't make it into this rc. So, that would>>>>> mean that future alphas would not be protocol-compatible with this>>>>> alpha. Per a discussion a few weeks ago, I think we all were in>>>>> agreement that, if possible, we'd like all 2.x to be compatible for>>>>> client-server communication, at least (even if we don't support>>>>> cross-version for the intra-cluster protocols)>>>>>>>>>> Do other folks think it's worth rolling an rc1? I would propose either:>>>>> a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on>>>>> branch-2.0.0-alpha, so these are the only changes since rc0. Roll a>>>>> new rc1 from here.>>>>> or:>>>>> b) Discard the current branch-2.0.0-alpha and re-branch from the>>>>> current state of branch-2.>>>>>>>>>> -Todd>>>>>>>>>> On Fri, May 11, 2012 at 7:19 PM, Eli Collins <[EMAIL PROTECTED]> wrote:>>>>>> +1 I installed the build on a 6 node cluster and kicked the tires,>>>>>> didn't find any blocking issues.>>>>>>>>>>>> Btw in the future better to build from the svn repo so the revision is>>>>>> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7>>>>>> which is from the git mirror, this way we're consistent across>>>>>> releases.>>>>>>>>>>>> hadoop-2.0.0-alpha $ ./bin/hadoop version>>>>>> Hadoop 2.0.0-alpha>>>>>> Subversion git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common

Thanks for posting the new RC. Will take a look tomorrow. Meanwhile,I'm going through CHANGES.txt and JIRA and moving things that didn'tmake the 2.0.0 cut to 2.0.1.

So, if folks commit things tomorrow, please check to put it in theright spot in CHANGES.txt and in JIRA. I'll take care of anythingcommitted tonight that would conflict with my change.

-Todd

On Tue, May 15, 2012 at 7:20 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> I've created a release candidate (rc1) for hadoop-2.0.0-alpha that I would like to release.>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc1/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>> thanks,> Arun>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

OK, the fixes to CHANGES.txt and JIRA are complete. Sorry for the mail bomb ;-)

-Todd

On Tue, May 15, 2012 at 10:30 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:> Thanks for posting the new RC. Will take a look tomorrow. Meanwhile,> I'm going through CHANGES.txt and JIRA and moving things that didn't> make the 2.0.0 cut to 2.0.1.>> So, if folks commit things tomorrow, please check to put it in the> right spot in CHANGES.txt and in JIRA. I'll take care of anything> committed tonight that would conflict with my change.>> -Todd>> On Tue, May 15, 2012 at 7:20 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:>> I've created a release candidate (rc1) for hadoop-2.0.0-alpha that I would like to release.>>>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc1/>>>> The maven artifacts are available via repository.apache.org.>>>> Please try the release and vote; the vote will run for the usual 7 days.>>>> thanks,>> Arun>>>>>> -->> Arun C. Murthy>> Hortonworks Inc.>> http://hortonworks.com/>>>>>>>> --> Todd Lipcon> Software Engineer, Cloudera

On Wed, May 16, 2012 at 12:04 AM, Robert Evans <[EMAIL PROTECTED]> wrote:> +1 I downloaded the binary ran a single node cluster and kicked the tires a bit with a few MR jobs. Everything worked.>>> On 5/15/12 9:20 PM, "Arun C Murthy" <[EMAIL PROTECTED]> wrote:>> I've created a release candidate (rc1) for hadoop-2.0.0-alpha that I would like to release.>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc1/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>> thanks,> Arun>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>>

On Tue, May 15, 2012 at 7:20 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> I've created a release candidate (rc1) for hadoop-2.0.0-alpha that I would like to release.>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc1/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>> thanks,> Arun>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

> I've created a release candidate (rc1) for hadoop-2.0.0-alpha that I would> like to release.>> It is available at:> http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc1/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>>+1, ship it

On Thu, May 17, 2012 at 8:16 AM, Thomas Graves <[EMAIL PROTECTED]> wrote:> +1, src tarball contents look good, builds, and ran some jobs on a single> node cluster using both built from source and binary.

+1 Downloaded the tarball and tested MapReduce and Distributed Shelljobs on a single node cluster. Everything works fine.

On Tue, May 15, 2012 at 7:20 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:> I've created a release candidate (rc1) for hadoop-2.0.0-alpha that I would like to release.>> It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc1/>> The maven artifacts are available via repository.apache.org.>> Please try the release and vote; the vote will run for the usual 7 days.>> thanks,> Arun>>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>

> I've created a release candidate (rc1) for hadoop-2.0.0-alpha that I would like to release.> > It is available at: http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc1/> > The maven artifacts are available via repository.apache.org.> > Please try the release and vote; the vote will run for the usual 7 days.

With 18 +1s (10 binding) and zero -1s, the vote passes. Thanks to all who voted.

Congratulations to everyone in the community for our first hadoop-2 release!

>> On May 15, 2012, at 7:20 PM, Arun C Murthy wrote:>> > I've created a release candidate (rc1) for hadoop-2.0.0-alpha that I> would like to release.> >> > It is available at:> http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc1/> >> > The maven artifacts are available via repository.apache.org.> >> > Please try the release and vote; the vote will run for the usual 7 days.>> With 18 +1s (10 binding) and zero -1s, the vote passes. Thanks to all who> voted.>> Congratulations to everyone in the community for our first hadoop-2> release!>> thanks,> Arun>> --> Arun C. Murthy> Hortonworks Inc.> http://hortonworks.com/>>>

NEW: Monitor These Apps!

All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext