Ironically I just got a misleading error like this today. What happened wasI upgraded to hive 0.10.One of my programs was liked to guava 15 but hiveprovides guava 09 on the classpath confusing things. I also had a similarissue with mismatched slf 4j and commons-logger.On Thu, May 16, 2013 at 10:34 PM, Sanjay Subramanian <[EMAIL PROTECTED]> wrote:

But I made a small change in my query (I just removed the text marked in blue) that seemed to solve it at least for the test data set that I had….Now I need to run it in production for a days worth of data

Ironically I just got a misleading error like this today. What happened was I upgraded to hive 0.10.One of my programs was liked to guava 15 but hive provides guava 09 on the classpath confusing things. I also had a similar issue with mismatched slf 4j and commons-logger.On Thu, May 16, 2013 at 10:34 PM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

But I made a small change in my query (I just removed the text marked in blue) that seemed to solve it at least for the test data set that I had….Now I need to run it in production for a days worth of data

Ironically I just got a misleading error like this today. What happened was I upgraded to hive 0.10.One of my programs was liked to guava 15 but hive provides guava 09 on the classpath confusing things. I also had a similar issue with mismatched slf 4j and commons-logger.On Thu, May 16, 2013 at 10:34 PM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

I am using Hive 0.9.0+155 that is bundled in Cloudera Manager version 4.1.2Still getting the errors listed below :-(Any clues will be be cool !!!Thanks

sanjayFrom: Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>Date: Thursday, May 16, 2013 9:42 PMTo: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>Subject: Re: need help with an error - script used to work and now it does not :-(

:-( Still facing problems in large datasetsWere u able to solve this Edward ?Thankssanjay

But I made a small change in my query (I just removed the text marked in blue) that seemed to solve it at least for the test data set that I had….Now I need to run it in production for a days worth of data

Ironically I just got a misleading error like this today. What happened was I upgraded to hive 0.10.One of my programs was liked to guava 15 but hive provides guava 09 on the classpath confusing things. I also had a similar issue with mismatched slf 4j and commons-logger.On Thu, May 16, 2013 at 10:34 PM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

in the meantime why don't you breakup your single query into a series ofqueries (using CTAS semantics to create intermediate tables ).

The idea is narrow the problem down to a minimal size that _isolates theproblem_ . what you have there is overly complex to expect someone totroubleshoot for you. try to minimize the failure case. take out yourUDF's. Does it work then or fail? strip it down to the bare necessities!On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <[EMAIL PROTECTED]> wrote:

> I am using Hive 0.9.0+155 that is bundled in Cloudera Manager version> 4.1.2> Still getting the errors listed below :-(> Any clues will be be cool !!!> Thanks>> sanjay>>> From: Sanjay Subramanian <[EMAIL PROTECTED]>> Date: Thursday, May 16, 2013 9:42 PM>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Subject: Re: need help with an error - script used to work and now it> does not :-(>> :-( Still facing problems in large datasets> Were u able to solve this Edward ?> Thanks> sanjay>> From: Sanjay Subramanian <[EMAIL PROTECTED]>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Date: Thursday, May 16, 2013 8:25 PM> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Subject: Re: need help with an error - script used to work and now it> does not :-(>> Thanks Edward…I just checked all instances of guava jars…except those> in red all seem same version>> /usr/lib/hadoop/client/guava-11.0.2.jar> /usr/lib/hadoop/client-0.20/guava-11.0.2.jar> /usr/lib/hadoop/lib/guava-11.0.2.jar> /usr/lib/hadoop-httpfs/webapps/webhdfs/WEB-INF/lib/guava-11.0.2.jar> /usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar> /usr/lib/oozie/libtools/guava-11.0.2.jar> /usr/lib/hive/lib/guava-11.0.2.jar> /usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar> /usr/lib/hbase/lib/guava-11.0.2.jar> /usr/lib/flume-ng/lib/guava-11.0.2.jar> /usr/share/cmf/lib/cdh3/guava-r09-jarjar.jar> /usr/share/cmf/lib/guava-12.0.1.jar>> But I made a small change in my query (I just removed the text marked in> blue) that seemed to solve it at least for the test data set that I> had….Now I need to run it in production for a days worth of data>> Will keep u guys posted>>> ------------------------------------------------------------------------------------------------------------> SELECT> h.header_date_donotquery * as date_*,> h.header_id as *impression_id*,> h.header_searchsessionid as *search_session_id*,> h.cached_visitid *as visit_id* ,> split(h.server_name_donotquery,'[\.]')[0] *as server*,> h.cached_ip *ip*,> h.header_adnodeid *ad_nodes*,>> ------------------------------------------------------------------------------------------------------------>> Thanks>> sanjay>>> From: Edward Capriolo <[EMAIL PROTECTED]>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Date: Thursday, May 16, 2013 7:51 PM> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Subject: Re: need help with an error - script used to work and now it> does not :-(>> Ironically I just got a misleading error like this today. What happened> was I upgraded to hive 0.10.One of my programs was liked to guava 15 but> hive provides guava 09 on the classpath confusing things. I also had a> similar issue with mismatched slf 4j and commons-logger.>>> On Thu, May 16, 2013 at 10:34 PM, Sanjay Subramanian <> [EMAIL PROTECTED]> wrote:>>> 2013-05-16 18:57:21,094 FATAL [IPC Server handler 19 on 40222] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1368666339740_0135_m_000104_1 - exited : java.lang.RuntimeException: Error in configuring object>> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)>> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)>> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)

+

Stephen Sprague 2013-05-17, 18:18

-

Re: need help with an error - script used to work and now it does not :-(

HiI actually did all of the following- tested all UDFs…they return values correctly- tested left side of LEFT OUTER JOIN- tested right side of LEFT OUTER JOIN

But when I add that ON statement sh.date_seller=h.header_date

I start getting this error…and this script has had no change for 3 weeks….used to run fine in production and we did 15 days of aggregations using this script.Two days back we installed LZO compression on the production servers….Circumstancial…but the script is failing after that LZO jar install…Maybe totally unrelated

As we speak I am testing this script on my sandbox which I am fairly sure will work since I don't have LZO compression on my sandbox but I want to verify

What is CTAS semantics ? I don't know so please tell me… But even if I create intermediate tables, I will eventually need to join them…

in the meantime why don't you breakup your single query into a series of queries (using CTAS semantics to create intermediate tables ).

The idea is narrow the problem down to a minimal size that _isolates the problem_ . what you have there is overly complex to expect someone to troubleshoot for you. try to minimize the failure case. take out your UDF's. Does it work then or fail? strip it down to the bare necessities!On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:I am using Hive 0.9.0+155 that is bundled in Cloudera Manager version 4.1.2Still getting the errors listed below :-(Any clues will be be cool !!!Thanks

But I made a small change in my query (I just removed the text marked in blue) that seemed to solve it at least for the test data set that I had….Now I need to run it in production for a days worth of data

Ironically I just got a misleading error like this today. What happened was I upgraded to hive 0.10.One of my programs was liked to guava 15 but hive provides guava 09 on the classpath confusing things. I also had a similar issue with mismatched slf 4j and commons-logger.On Thu, May 16, 2013 at 10:34 PM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

ok. so it sounds like you are doing A/B testing then. so if it works inyour sandbox but doesn't in prod then you can slowing transform yoursandbox - one component at time - to look like your prod system until itbreaks. The last component you add then is an area of interest.

> Hi> I actually did all of the following> - tested all UDFs…they return values correctly> - tested left side of LEFT OUTER JOIN> - tested right side of LEFT OUTER JOIN>> But when I add that ON statement> * sh.date_seller=h.header_date*>> I start getting this error…and this script has had no change for 3> weeks….used to run fine in production and we did 15 days of aggregations> using this script.> Two days back we installed LZO compression on the production> servers….Circumstancial…but the script is failing after that LZO jar> install…Maybe totally unrelated>> As we speak I am testing this script on my sandbox which I am fairly> sure will work since I don't have LZO compression on my sandbox but I want> to verify>> What is CTAS semantics ? I don't know so please tell me… But even if I> create intermediate tables, I will eventually need to join them…>> Thanks> sanjay>> From: Stephen Sprague <[EMAIL PROTECTED]>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Date: Friday, May 17, 2013 11:18 AM>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Subject: Re: need help with an error - script used to work and now it> does not :-(>> in the meantime why don't you breakup your single query into a series> of queries (using CTAS semantics to create intermediate tables ).>> The idea is narrow the problem down to a minimal size that _isolates the> problem_ . what you have there is overly complex to expect someone to> troubleshoot for you. try to minimize the failure case. take out your> UDF's. Does it work then or fail? strip it down to the bare necessities!>>> On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <> [EMAIL PROTECTED]> wrote:>>> I am using Hive 0.9.0+155 that is bundled in Cloudera Manager version>> 4.1.2>> Still getting the errors listed below :-(>> Any clues will be be cool !!!>> Thanks>>>> sanjay>>>>>> From: Sanjay Subramanian <[EMAIL PROTECTED]>>> Date: Thursday, May 16, 2013 9:42 PM>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Subject: Re: need help with an error - script used to work and now it>> does not :-(>>>> :-( Still facing problems in large datasets>> Were u able to solve this Edward ?>> Thanks>> sanjay>>>> From: Sanjay Subramanian <[EMAIL PROTECTED]>>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Date: Thursday, May 16, 2013 8:25 PM>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Subject: Re: need help with an error - script used to work and now it>> does not :-(>>>> Thanks Edward…I just checked all instances of guava jars…except those>> in red all seem same version>>>> /usr/lib/hadoop/client/guava-11.0.2.jar>> /usr/lib/hadoop/client-0.20/guava-11.0.2.jar>> /usr/lib/hadoop/lib/guava-11.0.2.jar>> /usr/lib/hadoop-httpfs/webapps/webhdfs/WEB-INF/lib/guava-11.0.2.jar>> /usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar>> /usr/lib/oozie/libtools/guava-11.0.2.jar>> /usr/lib/hive/lib/guava-11.0.2.jar>> /usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar>> /usr/lib/hbase/lib/guava-11.0.2.jar>> /usr/lib/flume-ng/lib/guava-11.0.2.jar>> /usr/share/cmf/lib/cdh3/guava-r09-jarjar.jar>> /usr/share/cmf/lib/guava-12.0.1.jar>>>> But I made a small change in my query (I just removed the text marked>> in blue) that seemed to solve it at least for the test data set that I>> had….Now I need to run it in production for a days worth of data>>>> Will keep u guys posted>>>>>> -------------------------

+

Stephen Sprague 2013-05-17, 18:36

-

Re: need help with an error - script used to work and now it does not :-(

Thanks Stephen….I am really worried about this because now I have a backlog of days piling up for this script to process…A quick question….is there a way to do some remote debugging using some Eclipse-ish tool…u know how we debug java applications remotely and the debug point comes back and we can analyze line by line….so I was thinking if we can do that same thing for Hive source code and I can walk thru that…

ok. so it sounds like you are doing A/B testing then. so if it works in your sandbox but doesn't in prod then you can slowing transform your sandbox - one component at time - to look like your prod system until it breaks. The last component you add then is an area of interest.

CTAS is short for "Create Table <blah> AS"On Fri, May 17, 2013 at 11:25 AM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:HiI actually did all of the following- tested all UDFs…they return values correctly- tested left side of LEFT OUTER JOIN- tested right side of LEFT OUTER JOIN

But when I add that ON statement sh.date_seller=h.header_date

I start getting this error…and this script has had no change for 3 weeks….used to run fine in production and we did 15 days of aggregations using this script.Two days back we installed LZO compression on the production servers….Circumstancial…but the script is failing after that LZO jar install…Maybe totally unrelated

As we speak I am testing this script on my sandbox which I am fairly sure will work since I don't have LZO compression on my sandbox but I want to verify

What is CTAS semantics ? I don't know so please tell me… But even if I create intermediate tables, I will eventually need to join them…

To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>Subject: Re: need help with an error - script used to work and now it does not :-(

in the meantime why don't you breakup your single query into a series of queries (using CTAS semantics to create intermediate tables ).

The idea is narrow the problem down to a minimal size that _isolates the problem_ . what you have there is overly complex to expect someone to troubleshoot for you. try to minimize the failure case. take out your UDF's. Does it work then or fail? strip it down to the bare necessities!On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:I am using Hive 0.9.0+155 that is bundled in Cloudera Manager version 4.1.2Still getting the errors listed below :-(Any clues will be be cool !!!Thanks

But I made a small change in my query (I just removed the text marked in blue) that seemed to solve it at least for the test data set that I had….Now I need to run it in production for a days worth of data

Ironically I just got a misleading error like this today. What happened was I upgraded to hive 0.10.One of my programs was liked to guava 15 but hive provides guava 09 on the classpath confusing things. I also had a similar issue w

+

Sanjay Subramanian 2013-05-17, 18:45

-

Re: need help with an error - script used to work and now it does not :-(

First off. Note to self (and anybody else out there) Hive is a far cry frombetting the farm on. You better have a Plan B when it crumples. Justsayin.

Yeah, Sanjay, we'll have to wait for someone else to chime in on the javadebugging as i'm not the domain expert on that aspect.On Fri, May 17, 2013 at 11:45 AM, Sanjay Subramanian <[EMAIL PROTECTED]> wrote:

> Thanks Stephen….I am really worried about this because now I have a> backlog of days piling up for this script to process…> A quick question….is there a way to do some remote debugging using some> Eclipse-ish tool…u know how we debug java applications remotely and the> debug point comes back and we can analyze line by line….so I was thinking> if we can do that same thing for Hive source code and I can walk thru that…>> Thanks> sanjay>> From: Stephen Sprague <[EMAIL PROTECTED]>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Date: Friday, May 17, 2013 11:36 AM>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>> Subject: Re: need help with an error - script used to work and now it> does not :-(>> ok. so it sounds like you are doing A/B testing then. so if it works> in your sandbox but doesn't in prod then you can slowing transform your> sandbox - one component at time - to look like your prod system until it> breaks. The last component you add then is an area of interest.>> CTAS is short for "Create Table <blah> AS">>> On Fri, May 17, 2013 at 11:25 AM, Sanjay Subramanian <> [EMAIL PROTECTED]> wrote:>>> Hi>> I actually did all of the following>> - tested all UDFs…they return values correctly>> - tested left side of LEFT OUTER JOIN>> - tested right side of LEFT OUTER JOIN>>>> But when I add that ON statement>> * sh.date_seller=h.header_date*>>>> I start getting this error…and this script has had no change for 3>> weeks….used to run fine in production and we did 15 days of aggregations>> using this script.>> Two days back we installed LZO compression on the production>> servers….Circumstancial…but the script is failing after that LZO jar>> install…Maybe totally unrelated>>>> As we speak I am testing this script on my sandbox which I am fairly>> sure will work since I don't have LZO compression on my sandbox but I want>> to verify>>>> What is CTAS semantics ? I don't know so please tell me… But even if I>> create intermediate tables, I will eventually need to join them…>>>> Thanks>> sanjay>>>> From: Stephen Sprague <[EMAIL PROTECTED]>>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Date: Friday, May 17, 2013 11:18 AM>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Subject: Re: need help with an error - script used to work and now it>> does not :-(>>>> in the meantime why don't you breakup your single query into a series>> of queries (using CTAS semantics to create intermediate tables ).>>>> The idea is narrow the problem down to a minimal size that _isolates the>> problem_ . what you have there is overly complex to expect someone to>> troubleshoot for you. try to minimize the failure case. take out your>> UDF's. Does it work then or fail? strip it down to the bare necessities!>>>>>> On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <>> [EMAIL PROTECTED]> wrote:>>>>> I am using Hive 0.9.0+155 that is bundled in Cloudera Manager version>>> 4.1.2>>> Still getting the errors listed below :-(>>> Any clues will be be cool !!!>>> Thanks>>>>>> sanjay>>>>>>>>> From: Sanjay Subramanian <[EMAIL PROTECTED]>>>> Date: Thursday, May 16, 2013 9:42 PM>>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>> Subject: Re: need help with an error - script used to work and now it>>> does not :-(>>>>>> :-( Still facing problems in large datasets>>> Were u able to solve this Edward ?>>> Thanks>>> sanjay>>>>>> From: Sanjay Subramanian <[EMAIL PROTECTED]>

+

Stephen Sprague 2013-05-17, 19:01

-

Re: need help with an error - script used to work and now it does not :-(

> First off. Note to self (and anybody else out there) Hive is a far cry> from betting the farm on. You better have a Plan B when it crumples.> Just sayin.>> Yeah, Sanjay, we'll have to wait for someone else to chime in on the java> debugging as i'm not the domain expert on that aspect.>>> On Fri, May 17, 2013 at 11:45 AM, Sanjay Subramanian <> [EMAIL PROTECTED]> wrote:>>> Thanks Stephen….I am really worried about this because now I have a>> backlog of days piling up for this script to process…>> A quick question….is there a way to do some remote debugging using some>> Eclipse-ish tool…u know how we debug java applications remotely and the>> debug point comes back and we can analyze line by line….so I was thinking>> if we can do that same thing for Hive source code and I can walk thru that…>>>> Thanks>> sanjay>>>> From: Stephen Sprague <[EMAIL PROTECTED]>>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Date: Friday, May 17, 2013 11:36 AM>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>> Subject: Re: need help with an error - script used to work and now it>> does not :-(>>>> ok. so it sounds like you are doing A/B testing then. so if it works>> in your sandbox but doesn't in prod then you can slowing transform your>> sandbox - one component at time - to look like your prod system until it>> breaks. The last component you add then is an area of interest.>>>> CTAS is short for "Create Table <blah> AS">>>>>> On Fri, May 17, 2013 at 11:25 AM, Sanjay Subramanian <>> [EMAIL PROTECTED]> wrote:>>>>> Hi>>> I actually did all of the following>>> - tested all UDFs…they return values correctly>>> - tested left side of LEFT OUTER JOIN>>> - tested right side of LEFT OUTER JOIN>>>>>> But when I add that ON statement>>> * sh.date_seller=h.header_date*>>>>>> I start getting this error…and this script has had no change for 3>>> weeks….used to run fine in production and we did 15 days of aggregations>>> using this script.>>> Two days back we installed LZO compression on the production>>> servers….Circumstancial…but the script is failing after that LZO jar>>> install…Maybe totally unrelated>>>>>> As we speak I am testing this script on my sandbox which I am fairly>>> sure will work since I don't have LZO compression on my sandbox but I want>>> to verify>>>>>> What is CTAS semantics ? I don't know so please tell me… But even if I>>> create intermediate tables, I will eventually need to join them…>>>>>> Thanks>>> sanjay>>>>>> From: Stephen Sprague <[EMAIL PROTECTED]>>>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>> Date: Friday, May 17, 2013 11:18 AM>>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>>>> Subject: Re: need help with an error - script used to work and now it>>> does not :-(>>>>>> in the meantime why don't you breakup your single query into a series>>> of queries (using CTAS semantics to create intermediate tables ).>>>>>> The idea is narrow the problem down to a minimal size that _isolates the>>> problem_ . what you have there is overly complex to expect someone to>>> troubleshoot for you. try to minimize the failure case. take out your>>> UDF's. Does it work then or fail? strip it down to the bare necessities!>>>>>>>>> On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <>>> [EMAIL PROTECTED]> wrote:>>>>>>> I am using Hive 0.9.0+155 that is bundled in Cloudera Manager>>>> version 4.1.2>>>> Still getting the errors listed below :-(

+

Ramki Palle 2013-05-18, 16:54

NEW: Monitor These Apps!

All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext