> Eh? That should not be the case, we should figure where 0.95.2 is> different from 0.94.> With which version of the 0.94 did you see this?>> -- Lars>>>> ----- Original Message -----> From: Kiru Pakkirisamy <[EMAIL PROTECTED]>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Cc:> Sent: Tuesday, September 24, 2013 9:46 PM> Subject: Re: Warning messages in hbase logs>> Are you running many concurrent clients ? I had a similar problem when> running on 0.94.x and I moved to 0.95.2 for this reason (see HBASE-9410)>> Regards,> - kiru>>> ________________________________> From: Vimal Jain <[EMAIL PROTECTED]>> To: [EMAIL PROTECTED]> Sent: Tuesday, September 24, 2013 9:36 PM> Subject: Warning messages in hbase logs>>> Hi,> I am seeing some warning message in Hregion log file for quite a few days.> The message is :>> *WARN org.apache.hadoop.ipc.HBaseServer: (responseTooSlow):>> {"processingtimems":12994,"call":"multi(org.apache.hadoop.hbase.client.MultiAction@602cdaf7> ),> rpc version=1, client version=29,> methodsFingerPrint=-1368823753","client":"> 192.168.20.31:42206>> ","starttimems":1380057021850,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}> *>>> Can someone please help in understanding what this warning message is ?>> --> Thanks and Regards,> Vimal Jain>

[HBASE-8806] - Row locks are acquired repeatedly in HRegion.doMiniBatchMutation for duplicate rows. (0.94.10)[HBASE-8639] - Poor performance of htable#getscanner in multithreaded environment due to DNS.getDefaultHost() being called in ScannerCallable#prepare() (0.94.9)[HBASE-9087] - Handlers being blocked during reads (0.94.11)-- Lars

Eh? That should not be the case, we should figure where 0.95.2 is different from 0.94.>With which version of the 0.94 did you see this?>>-- Lars>>>>>----- Original Message ----->From: Kiru Pakkirisamy <[EMAIL PROTECTED]>>To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>Cc:>Sent: Tuesday, September 24, 2013 9:46 PM>Subject: Re: Warning messages in hbase logs>>Are you running many concurrent clients ? I had a similar problem when running on 0.94.x and I moved to 0.95.2 for this reason (see HBASE-9410)> >Regards,>- kiru>>>________________________________>From: Vimal Jain <[EMAIL PROTECTED]>>To: [EMAIL PROTECTED]>Sent: Tuesday, September 24, 2013 9:36 PM>Subject: Warning messages in hbase logs>>>Hi,>I am seeing some warning message in Hregion log file for quite a few days.>The message is :>>*WARN org.apache.hadoop.ipc.HBaseServer: (responseTooSlow):>{"processingtimems":12994,"call":"multi(org.apache.hadoop.hbase.client.MultiAction@602cdaf7),>rpc version=1, client version=29, methodsFingerPrint=-1368823753","client":">192.168.20.31:42206>","starttimems":1380057021850,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}>*>>>Can someone please help in understanding what this warning message is ?>>-->Thanks and Regards,>Vimal Jain>--

[HBASE-8806] - Row locks are acquired repeatedly in HRegion.doMiniBatchMutation for duplicate rows. (0.94.10)[HBASE-8639] - Poor performance of htable#getscanner in multithreaded environment due to DNS.getDefaultHost() being called in ScannerCallable#prepare() (0.94.9)[HBASE-9087] - Handlers being blocked during reads (0.94.11)-- Lars

Eh? That should not be the case, we should figure where 0.95.2 is different from 0.94.>With which version of the 0.94 did you see this?>>-- Lars>>>>>----- Original Message ----->From: Kiru Pakkirisamy <[EMAIL PROTECTED]>>To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>Cc:>Sent: Tuesday, September 24, 2013 9:46 PM>Subject: Re: Warning messages in hbase logs>>Are you running many concurrent clients ? I had a similar problem when running on 0.94.x and I moved to 0.95.2 for this reason (see HBASE-9410)> >Regards,>- kiru>>>________________________________>From: Vimal Jain <[EMAIL PROTECTED]>>To: [EMAIL PROTECTED]>Sent: Tuesday, September 24, 2013 9:36 PM>Subject: Warning messages in hbase logs>>>Hi,>I am seeing some warning message in Hregion log file for quite a few days.>The message is :>>*WARN org.apache.hadoop.ipc.HBaseServer: (responseTooSlow):>{"processingtimems":12994,"call":"multi(org.apache.hadoop.hbase.client.MultiAction@602cdaf7),>rpc version=1, client version=29, methodsFingerPrint=-1368823753","client":">192.168.20.31:42206>","starttimems":1380057021850,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}>*>>>Can someone please help in understanding what this warning message is ?>>-->Thanks and Regards,>Vimal Jain>--

Vimal,Maybe you should try running the test case for HBASE-9410 (with some modifications) and report back.This way Lars/committers/contributors can work on this bug with confirmation.The test case makes a coprocessor call (which does nothing) on a table with regions in all the region servers.Feel free to send me any email in this regards.

Hi Kiru,Thanks for the reply.My understanding is not very clear with respect to these warning messagesin region server logs.When i read documentation , it says its related to slow query logs.So is it a issue in earlier versions of hbase and is fixed in 0.96 or is itan issue in my configuration and i have to set some parameters to preventthis?Please help in understanding the exact issue.On Sat, Oct 19, 2013 at 3:31 AM, Kiru Pakkirisamy <[EMAIL PROTECTED]> wrote:

Vimal,In my case, for a single client my queries take less than a second (sub-second performance is on what we were shooting for).But the same queries when run concurrently gives completely degraded performance. That is the reason I wrote that no-op test case which I have attached in the test HBASE-9410 bug. So many queries trying to execute on a region lead to some sort of lock contentions, is my guess which is causing the slow performance. My test case runs fine on 0.95.2 but on 0.94.x even for about 8 concurrent clients it is very high and I see these errors in the log.You should also try running the test case. You will see bad performance even though the coprocessor function does not do anything.