The reason was that Increment was serialized differently (compared to all other mutations).In trunk that is no longer an issue, since the serialization logic is no longer part of the object to be serialized.-- Lars

> The reason was that Increment was serialized differently (compared to all> other mutations).> In trunk that is no longer an issue, since the serialization logic is no> longer part of the object to be serialized.>>> -- Lars>>>> ________________________________> From: Ted Yu <[EMAIL PROTECTED]>> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]> Sent: Wednesday, January 23, 2013 10:25 AM> Subject: HBASE-7114 Increment does not extend Mutation but probably should>> Hi,> I want to get opinion on whether we should proceed with HBASE-7114> 'Increment does not extend Mutation but probably should' in trunk.>> Is anyone using Increment.setWriteToWAL or Increment.getFamilyMap ?> For Increment.setWriteToWAL, are you using the Increment returned ?>> Your feedback would be appreciated.>

Well, I increment counters where the row key is a keyword and the qualifieris a country code, and in the post increment region observer I increment an"all countries" aggregative counter. These counters are divided to familiessuch as daily, weekly, hourly etc.So I get the family map to know which aggregative counter should Iincrement, then I piggyback onto the Result the "all countries" currentcount.On Jan 26, 2013 2:39 AM, "Ted Yu" <[EMAIL PROTECTED]> wrote:

By the way, did you guys handle the hbase bug that when an increment issent to region server and fails it still does it but throws an exception tothe client which causes it to do that increment again?Sent from my iPhone

On 26 בינו 2013, at 17:32, Amit Sela <[EMAIL PROTECTED]> wrote:

Well, I increment counters where the row key is a keyword and the qualifieris a country code, and in the post increment region observer I increment an"all countries" aggregative counter. These counters are divided to familiessuch as daily, weekly, hourly etc.So I get the family map to know which aggregative counter should Iincrement, then I piggyback onto the Result the "all countries" currentcount.On Jan 26, 2013 2:39 AM, "Ted Yu" <[EMAIL PROTECTED]> wrote:

Amit:

Can you tell us what operation you perform on the returned family map ?ThanksOn Thu, Jan 24, 2013 at 3:37 AM, Amit Sela <[EMAIL PROTECTED]> wrote:I'm using Increment.getFamilyMap in a postIncrement Observer.

> The all counters is on the same row?> > By the way, did you guys handle the hbase bug that when an increment is> sent to region server and fails it still does it but throws an exception to> the client which causes it to do that increment again?> > > Sent from my iPhone> > On 26 בינו 2013, at 17:32, Amit Sela <[EMAIL PROTECTED]> wrote:> > Well, I increment counters where the row key is a keyword and the qualifier> is a country code, and in the post increment region observer I increment an> "all countries" aggregative counter. These counters are divided to families> such as daily, weekly, hourly etc.> So I get the family map to know which aggregative counter should I> increment, then I piggyback onto the Result the "all countries" current> count.> On Jan 26, 2013 2:39 AM, "Ted Yu" <[EMAIL PROTECTED]> wrote:> > Amit:> > Can you tell us what operation you perform on the returned family map ?> > > Thanks> > > On Thu, Jan 24, 2013 at 3:37 AM, Amit Sela <[EMAIL PROTECTED]> wrote:> > > I'm using Increment.getFamilyMap in a postIncrement Observer.> > I'm running with HBase 0.94.2.> > > Amit.> > > On Thu, Jan 24, 2013 at 4:23 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:> > > The reason was that Increment was serialized differently (compared to> > all> > other mutations).> > In trunk that is no longer an issue, since the serialization logic is> > no> > longer part of the object to be serialized.> > > > -- Lars> > > > > ________________________________> > From: Ted Yu <[EMAIL PROTECTED]>> > To: [EMAIL PROTECTED]; [EMAIL PROTECTED]> > Sent: Wednesday, January 23, 2013 10:25 AM> > Subject: HBASE-7114 Increment does not extend Mutation but probably> > should> > > Hi,> > I want to get opinion on whether we should proceed with HBASE-7114> > 'Increment does not extend Mutation but probably should' in trunk.> > > Is anyone using Increment.setWriteToWAL or Increment.getFamilyMap ?> > For Increment.setWriteToWAL, are you using the Increment returned ?> > > Your feedback would be appreciated.

The reason was that Increment was serialized differently (compared toallother mutations).In trunk that is no longer an issue, since the serialization logic isnolonger part of the object to be serialized.-- Lars

The increment has many columns: impressions_country, clicks_country, etc.And these counters count events in our system. Since we don't have an "allcountries" event, I thought it would be best to do that with aRegionObserver (each row has it's own counters so no risk in going outsidethe region right ?) .On Jan 26, 2013 7:15 PM, "Asaf Mesika" <[EMAIL PROTECTED]> wrote:

> Why not have the Increment object have two columns: one for the country and> one for the allCountries ?>> Sent from my iPhone>> On 26 בינו 2013, at 18:54, Infolinks <[EMAIL PROTECTED]> wrote:>> Yes, of course. It's an all counter for the specific keyword.>> ב-26 בינו 2013, בשעה 18:40, Asaf Mesika <[EMAIL PROTECTED]> כתב/ה:>> The all counters is on the same row?>>> By the way, did you guys handle the hbase bug that when an increment is>> sent to region server and fails it still does it but throws an exception to>> the client which causes it to do that increment again?>>>> Sent from my iPhone>>> On 26 בינו 2013, at 17:32, Amit Sela <[EMAIL PROTECTED]> wrote:>>> Well, I increment counters where the row key is a keyword and the qualifier>> is a country code, and in the post increment region observer I increment an>> "all countries" aggregative counter. These counters are divided to families>> such as daily, weekly, hourly etc.>> So I get the family map to know which aggregative counter should I>> increment, then I piggyback onto the Result the "all countries" current>> count.>> On Jan 26, 2013 2:39 AM, "Ted Yu" <[EMAIL PROTECTED]> wrote:>>> Amit:>>> Can you tell us what operation you perform on the returned family map ?>>>> Thanks>>>> On Thu, Jan 24, 2013 at 3:37 AM, Amit Sela <[EMAIL PROTECTED]> wrote:>>>> I'm using Increment.getFamilyMap in a postIncrement Observer.>>> I'm running with HBase 0.94.2.>>>> Amit.>>>> On Thu, Jan 24, 2013 at 4:23 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:>>>> The reason was that Increment was serialized differently (compared to>>> all>>> other mutations).>>> In trunk that is no longer an issue, since the serialization logic is>>> no>>> longer part of the object to be serialized.>>>>> -- Lars>>>>>> ________________________________>>> From: Ted Yu <[EMAIL PROTECTED]>>>> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]>>> Sent: Wednesday, January 23, 2013 10:25 AM>>> Subject: HBASE-7114 Increment does not extend Mutation but probably>>> should>>>> Hi,>>> I want to get opinion on whether we should proceed with HBASE-7114>>> 'Increment does not extend Mutation but probably should' in trunk.>>>> Is anyone using Increment.setWriteToWAL or Increment.getFamilyMap ?>>> For Increment.setWriteToWAL, are you using the Increment returned ?>>>> Your feedback would be appreciated.>

Yep.I think it would be faster it you will change the Increment object in the preIncrement object and add the allCountries column to it.

When you do it in the PostIncrement method, you will do another Increment call, thus another :rowLock, write to WAL, etc.

You can benchmark it easily.

On 26 בינו 2013, at 20:01, Amit Sela <[EMAIL PROTECTED]> wrote:

> The increment has many columns: impressions_country, clicks_country, etc.> And these counters count events in our system. Since we don't have an "all> countries" event, I thought it would be best to do that with a> RegionObserver (each row has it's own counters so no risk in going outside> the region right ?) .> On Jan 26, 2013 7:15 PM, "Asaf Mesika" <[EMAIL PROTECTED]> wrote:> >> Why not have the Increment object have two columns: one for the country and>> one for the allCountries ?>> >> Sent from my iPhone>> >> On 26 בינו 2013, at 18:54, Infolinks <[EMAIL PROTECTED]> wrote:>> >> Yes, of course. It's an all counter for the specific keyword.>> >> ב-26 בינו 2013, בשעה 18:40, Asaf Mesika <[EMAIL PROTECTED]> כתב/ה:>> >> The all counters is on the same row?>> >> >> By the way, did you guys handle the hbase bug that when an increment is>> >> sent to region server and fails it still does it but throws an exception to>> >> the client which causes it to do that increment again?>> >> >> >> Sent from my iPhone>> >> >> On 26 בינו 2013, at 17:32, Amit Sela <[EMAIL PROTECTED]> wrote:>> >> >> Well, I increment counters where the row key is a keyword and the qualifier>> >> is a country code, and in the post increment region observer I increment an>> >> "all countries" aggregative counter. These counters are divided to families>> >> such as daily, weekly, hourly etc.>> >> So I get the family map to know which aggregative counter should I>> >> increment, then I piggyback onto the Result the "all countries" current>> >> count.>> >> On Jan 26, 2013 2:39 AM, "Ted Yu" <[EMAIL PROTECTED]> wrote:>> >> >> Amit:>> >> >> Can you tell us what operation you perform on the returned family map ?>> >> >> >> Thanks>> >> >> >> On Thu, Jan 24, 2013 at 3:37 AM, Amit Sela <[EMAIL PROTECTED]> wrote:>> >> >> >> I'm using Increment.getFamilyMap in a postIncrement Observer.>> >> >> I'm running with HBase 0.94.2.>> >> >> >> Amit.>> >> >> >> On Thu, Jan 24, 2013 at 4:23 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:>> >> >> >> The reason was that Increment was serialized differently (compared to>> >> >> all>> >> >> other mutations).>> >> >> In trunk that is no longer an issue, since the serialization logic is>> >> >> no>> >> >> longer part of the object to be serialized.>> >> >> >> >> -- Lars>> >> >> >> >> >> ________________________________>> >> >> From: Ted Yu <[EMAIL PROTECTED]>>> >> >> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]>> >> >> Sent: Wednesday, January 23, 2013 10:25 AM>> >> >> Subject: HBASE-7114 Increment does not extend Mutation but probably>> >> >> should>> >> >> >> Hi,>> >> >> I want to get opinion on whether we should proceed with HBASE-7114>> >> >> 'Increment does not extend Mutation but probably should' in trunk.>> >> >> >> Is anyone using Increment.setWriteToWAL or Increment.getFamilyMap ?>> >> >> For Increment.setWriteToWAL, are you using the Increment returned ?>> >> >> >> Your feedback would be appreciated.>>

Well, I increment counters where the row key is a keyword and the qualifieris a country code, and in the post increment region observer I increment an"all countries" aggregative counter. These counters are divided to familiessuch as daily, weekly, hourly etc.So I get the family map to know which aggregative counter should Iincrement, then I piggyback onto the Result the "all countries" currentcount.On Jan 26, 2013 2:39 AM, "Ted Yu" <[EMAIL PROTECTED]> wrote:

Amit:Can you tell us what operation you perform on the returned family map ?

Thanks

On Thu, Jan 24, 2013 at 3:37 AM, Amit Sela <[EMAIL PROTECTED]> wrote:

> I'm using Increment.getFamilyMap in a postIncrement Observer.> I'm running with HBase 0.94.2.>> Amit.>> On Thu, Jan 24, 2013 at 4:23 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:>> > The reason was that Increment was serialized differently (compared toall> > other mutations).> > In trunk that is no longer an issue, since the serialization logic is no> > longer part of the object to be serialized.> >> >> > -- Lars> >> >> >> > ________________________________> > From: Ted Yu <[EMAIL PROTECTED]>> > To: [EMAIL PROTECTED]; [EMAIL PROTECTED]> > Sent: Wednesday, January 23, 2013 10:25 AM> > Subject: HBASE-7114 Increment does not extend Mutation but probably> should> >> > Hi,> > I want to get opinion on whether we should proceed with HBASE-7114> > 'Increment does not extend Mutation but probably should' in trunk.> >> > Is anyone using Increment.setWriteToWAL or Increment.getFamilyMap ?> > For Increment.setWriteToWAL, are you using the Increment returned ?> >> > Your feedback would be appreciated.> >>

> Well, I increment counters where the row key is a keyword and the qualifier> is a country code, and in the post increment region observer I increment an> "all countries" aggregative counter. These counters are divided to families> such as daily, weekly, hourly etc.> So I get the family map to know which aggregative counter should I> increment, then I piggyback onto the Result the "all countries" current> count.>

Hi Andrew,I think you mentioned it before when I had questions about aggregationusing RegionObserver. I checked it out but since our use case is quitesimple and there is only one aggregation level (country), we went with thecoprocessor.However, someone did post a question about something similar to an OLAPcube and I recommended it to him with your recommendation :)On Jan 26, 2013 8:52 PM, "Andrew Purtell" <[EMAIL PROTECTED]> wrote:

> Out of curiosity, have you seen https://github.com/urbanairship/datacube ?>>> On Sat, Jan 26, 2013 at 7:14 AM, Amit Sela <[EMAIL PROTECTED]> wrote:>> > Well, I increment counters where the row key is a keyword and the> qualifier> > is a country code, and in the post increment region observer I increment> an> > "all countries" aggregative counter. These counters are divided to> families> > such as daily, weekly, hourly etc.> > So I get the family map to know which aggregative counter should I> > increment, then I piggyback onto the Result the "all countries" current> > count.> >>> --> Best regards,>> - Andy>> Problems worthy of attack prove their worth by hitting back. - Piet Hein> (via Tom White)>

+

Amit Sela 2013-01-26, 19:08

NEW: Monitor These Apps!

All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext