Hi Stephan ,
Please use the following desc extended to see where is the table' s directory on hdfs. Here
is an example.
hive -e "desc extended hcatsmokeid0b0abc02_date252113 ;"
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter
in all the log4j.properties files.
Logging initialized using configuration in jar:file:/usr/lib/hive/lib/hive-common-0.10.0.21.jar!/hive-log4j.properties
Hive history file=/tmp/root/hive_job_log_root_201301211932_612003297.txt
OK
id int
name string
Detailed Table Information Table(tableName:hcatsmokeid0b0abc02_date252113, dbName:default,
owner:ambari_qa, createTime:1358814367, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:id,
type:int, comment:null), FieldSchema(name:name, type:string, comment:null)], location:hdfs://ambari1:8020/apps/hive/warehouse/hcatsmokeid0b0abc02_date252113,
inputFormat:org.apache.hadoop.hive.ql.io.RCFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.RCFileOutputFormat,
compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe,
parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[],
skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[],
parameters:{transient_lastDdlTime=1358814367}, viewOriginalText:null, viewExpandedText:null,
tableType:MANAGED_TABLE)
Time taken: 2.965 seconds
Hope this helps.
Hortonworks, Inc.
Technical Support Engineer
Abdelrahman Shettia
ashettia@hortonworks.com
Office phone: (708) 689-9609
How am I doing? Please feel free to provide feedback to my manager Rick Morris at rick@hortonworks.com
On Mar 2, 2013, at 1:59 AM, Stephen Boesch <javadba@gmail.com> wrote:
>
> I am struggling with a "no such file or directory exception " when running a simple query
in hive. It is unfortunate that the actual path were not included with the stacktrace:
the following is all that is provided.
>
> I have a query that fails with the following error when done as hive -e "select * from
<table>'". But it works properly when done within the hive shell. But at the same time,
doing hive> select * from <table2>;" fails with the same error message.
>
> I am also seeing this error both for hdfs files and for s3 files. Without any path information
it is very difficult and time consuming to track this down.
>
> Any pointers appreciated.
>
>
> Automatically selecting local only mode for query
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
> set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
> set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
> set mapred.reduce.tasks=<number>
> WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter
in all the log4j.properties files.
> Execution log at: /tmp/impala/impala_20130302095252_79ce9404-6af7-405b-8b06-849fe6c5328d.log
> ENOENT: No such file or directory
> at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:568)
> at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:411)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:501)
> at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:733)
> at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:692)
> at org.apache.hadoop.mapred.JobClient.access$400(JobClient.java:172)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:910)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:895)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:895)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:869)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:677)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Job Submission failed with exception 'org.apache.hadoop.io.nativeio.NativeIOException(No
such file or directory)'
> Execution failed with exit status: 1
> Obtaining error information
>