As mentioned in HADOOP-9845, Protobuf 2.5 has significant benefits tojustify the upgrade.

Doing the upgrade now, with the first beta, will make things easier fordownstream projects (like HBase) using protobuf and adopting Hadoop 2. Ifwe do the upgrade later, downstream projects will have to support 2different versions and they my get in nasty waters due to classpath issues.

I've locally tested the patch in a pseudo deployment of 2.1.0-beta branchand it works fine (something is broken in trunk in the RPC layer YARN-885).

Now, to do this it will require a few things:

* Make sure protobuf 2.5.0 is available in the jenkins box* A follow up email to dev@ aliases indicating developers should installlocally protobuf 2.5.0

On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur <[EMAIL PROTECTED]> wrote:> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.>> As mentioned in HADOOP-9845, Protobuf 2.5 has significant benefits to> justify the upgrade.>> Doing the upgrade now, with the first beta, will make things easier for> downstream projects (like HBase) using protobuf and adopting Hadoop 2. If> we do the upgrade later, downstream projects will have to support 2> different versions and they my get in nasty waters due to classpath issues.>> I've locally tested the patch in a pseudo deployment of 2.1.0-beta branch> and it works fine (something is broken in trunk in the RPC layer YARN-885).>> Now, to do this it will require a few things:>> * Make sure protobuf 2.5.0 is available in the jenkins box> * A follow up email to dev@ aliases indicating developers should install> locally protobuf 2.5.0>> Thanks.>> --> Alejandro

On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur <[EMAIL PROTECTED]> wrote:> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.>> As mentioned in HADOOP-9845, Protobuf 2.5 has significant benefits to> justify the upgrade.>> Doing the upgrade now, with the first beta, will make things easier for> downstream projects (like HBase) using protobuf and adopting Hadoop 2. If> we do the upgrade later, downstream projects will have to support 2> different versions and they my get in nasty waters due to classpath issues.>> I've locally tested the patch in a pseudo deployment of 2.1.0-beta branch> and it works fine (something is broken in trunk in the RPC layer YARN-885).>> Now, to do this it will require a few things:>> * Make sure protobuf 2.5.0 is available in the jenkins box> * A follow up email to dev@ aliases indicating developers should install> locally protobuf 2.5.0>> Thanks.>> --> Alejandro

> In HBase land we've pretty well discovered that we'll need to have the> same version of protobuf that the HDFS/Yarn/MR servers are running.> That is to say there are issues with ever having 2.4.x and 2.5.x on> the same class path.>> Upgrading to 2.5.x would be great, as it brings some new classes we> could use. With that said HBase is getting pretty close to a rather> large release (0.96.0 aka The Singularity) so getting this in sooner> rather than later would be great. If we could get this into 2.1.0 it> would be great as that would allow us to have a pretty easy story to> users with regards to protobuf version.>> On Thu, Aug 8, 2013 at 8:18 AM, Kihwal Lee <[EMAIL PROTECTED]> wrote:> > Sorry to hijack the thread but, I also wanted to mention Avro. See> HADOOP-9672.> > The version we are using has memory leak and inefficiency issues. We've> seen users running into it.> >> > Kihwal> >> >> > ________________________________> > From: Tsuyoshi OZAWA <[EMAIL PROTECTED]>> > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> > Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; "> [EMAIL PROTECTED]" <[EMAIL PROTECTED]>; "> [EMAIL PROTECTED]" <[EMAIL PROTECTED]>> > Sent: Thursday, August 8, 2013 1:59 AM> > Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845> >> >> > Hi,> >> > About Hadoop, Harsh is dealing with this problem in HADOOP-9346.> > For more detail, please see the JIRA ticket:> > https://issues.apache.org/jira/browse/HADOOP-9346> >> > - Tsuyoshi> >> > On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur <[EMAIL PROTECTED]>> wrote:> >> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.> >>> >> As mentioned in HADOOP-9845, Protobuf 2.5 has significant benefits to> >> justify the upgrade.> >>> >> Doing the upgrade now, with the first beta, will make things easier for> >> downstream projects (like HBase) using protobuf and adopting Hadoop 2.> If> >> we do the upgrade later, downstream projects will have to support 2> >> different versions and they my get in nasty waters due to classpath> issues.> >>> >> I've locally tested the patch in a pseudo deployment of 2.1.0-beta> branch> >> and it works fine (something is broken in trunk in the RPC layer> YARN-885).> >>> >> Now, to do this it will require a few things:> >>> >> * Make sure protobuf 2.5.0 is available in the jenkins box> >> * A follow up email to dev@ aliases indicating developers should> install> >> locally protobuf 2.5.0> >>> >> Thanks.> >>> >> --> >> Alejandro>

> To move forward with this we need protoc 2.5.0 in the apache hadoop> jenkins boxes.>> Who can help with this? I assume somebody at Y!, right?>> Thx>>> On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark <[EMAIL PROTECTED]> wrote:>>> In HBase land we've pretty well discovered that we'll need to have the>> same version of protobuf that the HDFS/Yarn/MR servers are running.>> That is to say there are issues with ever having 2.4.x and 2.5.x on>> the same class path.>>>> Upgrading to 2.5.x would be great, as it brings some new classes we>> could use. With that said HBase is getting pretty close to a rather>> large release (0.96.0 aka The Singularity) so getting this in sooner>> rather than later would be great. If we could get this into 2.1.0 it>> would be great as that would allow us to have a pretty easy story to>> users with regards to protobuf version.>>>> On Thu, Aug 8, 2013 at 8:18 AM, Kihwal Lee <[EMAIL PROTECTED]> wrote:>> > Sorry to hijack the thread but, I also wanted to mention Avro. See>> HADOOP-9672.>> > The version we are using has memory leak and inefficiency issues. We've>> seen users running into it.>> >>> > Kihwal>> >>> >>> > ________________________________>> > From: Tsuyoshi OZAWA <[EMAIL PROTECTED]>>> > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> > Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; ">> [EMAIL PROTECTED]" <[EMAIL PROTECTED]>; ">> [EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> > Sent: Thursday, August 8, 2013 1:59 AM>> > Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release,>> HADOOP-9845>> >>> >>> > Hi,>> >>> > About Hadoop, Harsh is dealing with this problem in HADOOP-9346.>> > For more detail, please see the JIRA ticket:>> > https://issues.apache.org/jira/browse/HADOOP-9346>> >>> > - Tsuyoshi>> >>> > On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur <[EMAIL PROTECTED]>>> wrote:>> >> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.>> >>>> >> As mentioned in HADOOP-9845, Protobuf 2.5 has significant benefits to>> >> justify the upgrade.>> >>>> >> Doing the upgrade now, with the first beta, will make things easier for>> >> downstream projects (like HBase) using protobuf and adopting Hadoop 2.>> If>> >> we do the upgrade later, downstream projects will have to support 2>> >> different versions and they my get in nasty waters due to classpath>> issues.>> >>>> >> I've locally tested the patch in a pseudo deployment of 2.1.0-beta>> branch>> >> and it works fine (something is broken in trunk in the RPC layer>> YARN-885).>> >>>> >> Now, to do this it will require a few things:>> >>>> >> * Make sure protobuf 2.5.0 is available in the jenkins box>> >> * A follow up email to dev@ aliases indicating developers should>> install>> >> locally protobuf 2.5.0>> >>>> >> Thanks.>> >>>> >> -->> >> Alejandro>>>>>> --> Alejandro>

On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur <[EMAIL PROTECTED]> wrote:> pinging again, I need help from somebody with sudo access to the hadoop> jenkins boxes to do this or to get sudo access for a couple of hours to set> up myself.

Speaking of which -- I'd be totally willing to volunteer as a back upfor Giri for cases like these. I'm already maintaining Bigtop's Jenkinsinfrastructure. Please let me know if sharing creds (or somethingalong those lines) would be feasible.

On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur <[EMAIL PROTECTED]> wrote:> pinging again, I need help from somebody with sudo access to the hadoop> jenkins boxes to do this or to get sudo access for a couple of hours to set> up myself.

Have you asked on builds@ or filed an INFRA Jira issue or asked on #asfinfra?

> build slaves hadoop1-hadoop9 now has libprotoc 2.5.0> > > > -Giri> > > On Fri, Aug 9, 2013 at 10:56 PM, Giridharan Kesavan <> [EMAIL PROTECTED]> wrote:> >> Alejandro,>> >> I'm upgrading protobuf on slaves hadoop1-hadoop9.>> >> -Giri>> >> >> On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:>> >>> pinging again, I need help from somebody with sudo access to the hadoop>>> jenkins boxes to do this or to get sudo access for a couple of hours to>>> set>>> up myself.>>> >>> Please!!!>>> >>> thx>>> >>> >>> On Thu, Aug 8, 2013 at 2:29 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>>>> wrote:>>> >>>> To move forward with this we need protoc 2.5.0 in the apache hadoop>>>> jenkins boxes.>>>> >>>> Who can help with this? I assume somebody at Y!, right?>>>> >>>> Thx>>>> >>>> >>>> On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark <[EMAIL PROTECTED]>>>> wrote:>>>> >>>>> In HBase land we've pretty well discovered that we'll need to have the>>>>> same version of protobuf that the HDFS/Yarn/MR servers are running.>>>>> That is to say there are issues with ever having 2.4.x and 2.5.x on>>>>> the same class path.>>>>> >>>>> Upgrading to 2.5.x would be great, as it brings some new classes we>>>>> could use. With that said HBase is getting pretty close to a rather>>>>> large release (0.96.0 aka The Singularity) so getting this in sooner>>>>> rather than later would be great. If we could get this into 2.1.0 it>>>>> would be great as that would allow us to have a pretty easy story to>>>>> users with regards to protobuf version.>>>>> >>>>> On Thu, Aug 8, 2013 at 8:18 AM, Kihwal Lee <[EMAIL PROTECTED]>>>> wrote:>>>>>> Sorry to hijack the thread but, I also wanted to mention Avro. See>>>>> HADOOP-9672.>>>>>> The version we are using has memory leak and inefficiency issues.>>> We've>>>>> seen users running into it.>>>>>> >>>>>> Kihwal>>>>>> >>>>>> >>>>>> ________________________________>>>>>> From: Tsuyoshi OZAWA <[EMAIL PROTECTED]>>>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>>>>> Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; ">>>>> [EMAIL PROTECTED]" <[EMAIL PROTECTED]>; ">>>>> [EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>>>>> Sent: Thursday, August 8, 2013 1:59 AM>>>>>> Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release,>>>>> HADOOP-9845>>>>>> >>>>>> >>>>>> Hi,>>>>>> >>>>>> About Hadoop, Harsh is dealing with this problem in HADOOP-9346.>>>>>> For more detail, please see the JIRA ticket:>>>>>> https://issues.apache.org/jira/browse/HADOOP-9346>>>>>> >>>>>> - Tsuyoshi>>>>>> >>>>>> On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur <>>> [EMAIL PROTECTED]>>>>>> wrote:>>>>>>> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.>>>>>>> >>>>>>> As mentioned in HADOOP-9845, Protobuf 2.5 has significant benefits>>> to>>>>>>> justify the upgrade.>>>>>>> >>>>>>> Doing the upgrade now, with the first beta, will make things easier>>> for>>>>>>> downstream projects (like HBase) using protobuf and adopting Hadoop>>> 2.>>>>> If>>>>>>> we do the upgrade later, downstream projects will have to support 2>>>>>>> different versions and they my get in nasty waters due to classpath>>>>> issues.>>>>>>> >>>>>>> I've locally tested the patch in a pseudo deployment of 2.1.0-beta>>>>> branch>>>>>>> and it works fine (something is broken in trunk in the RPC layer>>>>> YARN-885).>>>>>>> >>>>>>> Now, to do this it will require a few things:>>>>>>> >>>>>>> * Make sure protobuf 2.5.0 is available in the jenkins box>>>>>>> * A follow up email to dev@ aliases indicating developers should>>>>> install>>>>>>> locally protobuf 2.5.0

> Ok. After installing protobuf 2.5.0 I can compile trunk.> But now I cannot compile Hadoop-2 branches. None of them.> So if I switch between branches I need to reinstall protobuf?> > Is there a consensus about going towards protobuf 2.5.0 upgrade in ALL> versions?> I did not get definite impression there is.> If not it could be a pretty big disruption.> > Thanks,> --Konst> > > > On Mon, Aug 12, 2013 at 3:19 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:> >> I've just committed HADOOP-9845 to trunk (only trunk at the moment).>> >> To build trunk now you need protoc 2.5.0 (the build will fail with a>> warning if you don't have it).>> >> We'd propagate this to the 2 branches once the precommit build is back to>> normal and see things are OK.>> >> Thanks.>> >> >> On Mon, Aug 12, 2013 at 2:57 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>>> wrote:>> >>> About to commit HADOOP-9845 to trunk, in 5 mins. This will make trunk use>>> protoc 2.5.0.>>> >>> thx>>> >>> >>> On Mon, Aug 12, 2013 at 11:47 AM, Giridharan Kesavan <>>> [EMAIL PROTECTED]> wrote:>>> >>>> I can take care of re-installing 2.4 and installing 2.5 in a different>>>> location. This would fix 2.0 branch builds as well.>>>> Thoughts?>>>> >>>> -Giri>>>> >>>> >>>> On Mon, Aug 12, 2013 at 11:37 AM, Alejandro Abdelnur <[EMAIL PROTECTED]>>>>> wrote:>>>> >>>>> Giri,>>>>> >>>>> first of all, thanks for installing protoc 2.5.0.>>>>> >>>>> I didn't know we were installing them as the only version and not>>>> driven by>>>>> env/path settings.>>>>> >>>>> Now we have a bit of a problem, precommit builds are broken because of>>>>> mismatch of protoc (2.5.0) and protobuf JAR( 2.4.1).>>>>> >>>>> We have to options:>>>>> >>>>> 1* commit HADOOP-9845 that will bring protobuf to 2.5.0 and iron out>> any>>>>> follow up issues.>>>>> 2* reinstall protoc 2.4.1 in the jenkins machines and have 2.4.1 and>>>> 2.5.0>>>>> coexisting>>>>> >>>>> My take would be to commit HADOOP-9845 in trunk, iron out any issues>> an>>>>> then merge it to the other branches.>>>>> >>>>> We need to sort this out quickly as precommits are not working.>>>>> >>>>> I'll wait till 3PM today for objections to option #1, if none I'll>>>> commit>>>>> it to trunk.>>>>> >>>>> Thanks.>>>>> >>>>> Alejandro>>>>> >>>>> >>>>> >>>>> On Mon, Aug 12, 2013 at 11:30 AM, Giridharan Kesavan <>>>>> [EMAIL PROTECTED]> wrote:>>>>> >>>>>> Like I said protoc is upgraded from 2.4 to 2.5. 2.5 is in the>> default>>>>> path.>>>>>> If we still need 2.4 I may have to install it. Let me know>>>>>> >>>>>> -Giri>>>>>> >>>>>> >>>>>> On Sat, Aug 10, 2013 at 7:01 AM, Alejandro Abdelnur <>>>> [EMAIL PROTECTED]>>>>>>> wrote:>>>>>> >>>>>>> thanks giri, how do we set 2.4 or 2.5., what is the path to both>> so>>>> we>>>>>> can>>>>>>> use and env to set it in the jobs?>>>>>>> >>>>>>> thx>>>>>>> >>>>>>> Alejandro>>>>>>> (phone typing)>>>>>>> >>>>>>> On Aug 9, 2013, at 23:10, Giridharan Kesavan <>>>> [EMAIL PROTECTED]>>>>>> >>>>>>> wrote:>>>>>>> >>>>>>>> build slaves hadoop1-hadoop9 now has libprotoc 2.5.0>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -Giri>>>>>>>> >>>>>>>> >>>>>>>> On Fri, Aug 9, 2013 at 10:56 PM, Giridharan Kesavan <>>>>>>>> [EMAIL PROTECTED]> wrote:>>>>>>>> >>>>>>>>> Alejandro,>>>>>>>>> >>>>>>>>> I'm upgrading protobuf on slaves hadoop1-hadoop9.>>>>>>>>> >>>>>>>>> -Giri>>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur <>>>>> [EMAIL PROTECTED]>>>>>>>> wrote:>>>>>>>>> >>>>>>>>>> pinging again, I need help from somebody with sudo access to>> the>>>>>> hadoop

NEW: Monitor These Apps!

All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext