How to configure Hue for your Hadoop cluster

Hue is a lightweight Web server that lets you use Hadoop directly from your browser. Hue is just a ‘view on top of any Hadoop distribution’ and can be installed on any machine.

There are multiples ways (cf. ‘Download’ section of gethue.com) to install Hue. The next step is then to configure Hue to point to your Hadoop cluster. By default Hue assumes a local cluster (i.e. there is only one machine) is present. In order to interact with a real cluster, Hue needs to know on which hosts are distributed the Hadoop services.

Where is my hue.ini?

Hue main configuration happens in a hue.ini file. It lists a lot of options but essentially what are the addresses and ports of HDFS, YARN, Oozie, Hive… Depending on the distribution you installed the ini file is located:

At any time, you can see the path to the hue.ini and what are its values on the /desktop/dump_config page. Then, for each Hadoop Service, Hue contains a section that needs to be updated with the correct hostnames and ports. Here is an example of the Hive section in the ini file:

Then, if the Namenode is on another host than Hue, don’t forget to update in the hue.ini:

[hadoop]
[[hdfs_clusters]]
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://localhost:8020
# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
webhdfs_url=http://localhost:50070/webhdfs/v1

YARN

The Resource Manager is often on http://localhost:8088 by default. The ProxyServer and Job History servers also needs to be specified. Then Job Browser will let you list and kill running applications and get their logs.

[hadoop]
[[yarn_clusters]]
[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=localhost
# Whether to submit jobs to this cluster
submit_to=True
# URL of the ResourceManager API
resourcemanager_api_url=http://localhost:8088
# URL of the ProxyServer API
proxy_api_url=http://localhost:8088
# URL of the HistoryServer API
history_server_api_url=http://localhost:19888

236 Comments

hi, i have a question about how to configuration hue with federation namenode? Is Hue have a feature to contact a multiple namenode (federation) with one server hue, or do I have to create multiple hue on multiple namenode? Thank’s before

Sorry but i’m still confuse with your explanation, this is the scenario:
I have a double namenode(namenode 1 and namenode 2) and 5 datanode, i have install hue1 one namenode1 and hue2 on namenode2
I configure all namenodes on federation mode with the same datanode.
If i upload some data(data1) from namenode1 trough hue1, I can’t read the data1 through hue2, but if I configured hue2 to pointing on namenode1, sure it can read the data1 but i can’t upload any other data trough namenode2 or even read the data on namenode2.
if I pointing two webhdfs via pseudo-distributed.ini on two namenode on the single hue like this:
webhdfs_url=http://namenode1:50070/webhdfs/v1
webhdfs_url=http://namenode2:50070/webhdfs/v1
the service won’t up and give me an error message
So, what should I configure to make a single hue can read and upload the data from both namenode? Thank’s before

Hi, I am a cognos developer using cognos 10.2.2. I would like to know how I can connect my cognos with Hue GUI (Hive table) to generate meta data model and creates a report out of that. Appreciate your response. Thanks,

I want to config rdbms in hue.ini .
I know that in hue.ini is [librdbms]->[[databases]]->[[[mysql]]]
And Under CM I should use Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini.
But what’s the key?
For sqlite db it’s nice.name and for mysql ,it still nice.name?

hue is well done in pseudo distributed mode (CDH4) but I got error when configuring hue in multi node cluster.It always show “fail to create temporary ……..”whenever I am trying to upload file .Pls help me.

I am running hadoop-2.6,pig-0.14.0,Hive-1.1.0 on pseudo mode on Ubuntu 15.04. I have built hue based on http://gethue.com/how-to-build-hue-on-ubuntu-14-04-trusty/#comment-50596. and I am able to open the webpage, create a logID. However, I am unable to connect to HDFS. I tried my bit configuring hue.ini from above but connection never happen each time I start-up. Anybody around here, can you please advise. Thanks

Hi!
I’m trying to configure Hue with Hive and HBase. Though HBase runs fine with Hue after the configurations suggested here, Hive is not running on Hue and shows the configuration error “The application won’t work without a running HiveServer2. ” While I can access Hive through it’s shell easily, Hue doesn’t allow access to Hive.ANy help in this regard would be appreciated.

Hi Team,
Firstly I would thank you for providing a wonderful UI for hadoop. Please consider the below scenario:
1.I am connecting from my local system to Hue which is on a remote system.
2.Hue connects to cluster which is configured remotely(Hue and cluster are in different locations)

In this case can hue connect to cluster remotely ?If yes ,then will the data processing performance of this case be the same as hue present within in the cluster?

Thanks Manasa! Hue can connect to any cluster but of course the network needs to allow that (ie. firewall configurations). The data processing is done by the Hadoop cluster(s) and not Hue itself so you are just limited by the speed of the connection between the Hue installation and the cluster(s).

As you said,Hue won’t run easily on Windows. I am ready to experiment but couldn’t find anywhere the packages for windows or anything. Is it totally unsupported or difficult to install ? Request you to elaborate

Hi all,
I’ve installed Hue 3.9 version. On the first page in Hue. I can connect with a HUE user (created at first connection but I’ve got this warning (sorry, in French) :
“hadoop.hdfs_clusters.default.webhdfs_url : Valeur actuelle : http://172.26.122.135:50070/webhdfs/v1
Echec de la création du fichier temporaire “/tmp/hue_config_validation.15312085668819163275”
I can create/delete some folders but I can’t write files :
“Impossible d’effectuer l’opération. Note: you are a Hue admin but not a HDFS superuser, “hadoop” or part of HDFS supergroup, “supergroup”.”
I’ve followed your tutorial, tried several solutions found on the net. Any result …
I’m completly lost and would appreciate help
Thanks in advance

Thanks for your quick answer. I’ll test your command line tomorrow morning. Anyway, find here some additional informations. The 1rst problem is the system cannot write into /tmp (warning into administation page in my previous mesage). And if I navigate into file system, I cannot write into any directory (if connected with Hue ‘s user for exemple, I cannot write into /user/Hue). One more question before test your command line : Do I test it with Hadoop ‘s user or Hue’s ?
Once again, thanks for your help.

I’ve created a new user “HDFS” from Hue (as superuser) : same error on the administration page and writing a file on HDFS system.
Maybe something wrong in Hue.ini ? What can I check. Here is an extract from it :

# Webserver runs as this user
## server_user=hue
## server_group=hue

# This should be the Hue admin and proxy user
## default_user=hue

# This should be the hadoop cluster admin
# default_hdfs_superuser=hdfs

Does it mean that I’ve to create a user “hdfs” in Linux, in Hadoop group ?
Permissions for /tmp : 1777 but keep in mind that I can’t write a file anywhere in the system (/user/hdfs for instance)
Here is the msg in HUE :
Impossible d’effectuer l’opération. Note: you are a Hue admin but not a HDFS superuser, “hadoop” or part of HDFS supergroup, “supergroup”.
I can’t send you the complete msg (HTML format), it seems I’m blocked to access your web site.

Hi,
I’ve already test some M/R samples, Flume agents on my cluster. All work fine. I still have to install some other apps (oozie, thrift,…) but I would like to eradicate this error before to continue. No more idea ?

Yes I’ve created hdfs’ user in HUE and connected in HUE with. I don’t know what “btw’ means … Anyway, I’ve launched the “./supervisor” command from Linux with alternatively root & hue users. I’ve installed Hadoop with ‘hadoop’ user which belongs to ‘hadoop’ group. I’ve tried to add hue user to hadoop’s group. Yesterday, I’ve executed the ‘`touch foo && hadoop fs -put foo /tmp/foo` test. It walked but when I look at /tmp/foo file’s properties I can see that the file belongs to ‘hue’ user and to ‘supergroup’ group. Too, I’ve tried to add the ‘dfs.permissions.supergroup’ into core-site.xml . Without success. I hope this help. thanks again

I’ve tried creating a ‘hadoop’ user in hue too. Not the same error msg :
Default page : Valeur actuelle : http://My_Ip_Address:50070/webhdfs/v1
Echec de la création du fichier temporaire “/tmp/hue_config_validation.4572897978671048530”
When I try to add a new file : Impossible d’effectuer l’opération :
The requested URL could not be retrieved
The following error was encountered while trying to retrieve the URL: <a href="http://My_Ip_Address:50075/webhdfs/v1/tmp/xx.txt

Sorry, I don’t know how to check it. All I’m sure is that I’ve configure this port in any conf. file. I’ve done a pseudo-distributed intallation on only one machaine. Here is what I can see when I start dfs (start-dfs.sh commande) concerning datanode :

Sorry to give you so much informations.
As said in my previous message, when I test DataNode on : My_IP_Address:50075 ==> Ok
but when I test DataNode on : My_Server_Name:50075 ==> Failed
Where the system can retrieve my server name as mentionned in the error message ?
Is there a link with my main issue ?

Hi,
Seems to be solved ! It was du to a DNS issue. The error on administration page disappears et I can now create a file on HDFS either with ‘hadoop’ & ‘hue’ users
Thanks for your help and the time you ‘ve spent form me.

Hi again … May I ask another question ? About HBASE in Hue ? Once Hue worked last week, I tried to use HBASE (Hive was Ok). I received a message which made me think THRIFT was not installed. So I’ve installed it . Installation seems to be OK (thrift -version answers me 0.9.2). But HBASE does not work again. I think the reason is that I can’t launch thrist server. So, I would appreciate (once more) if you could help me or tell me where I could find specific help.

when i am trying to use Sentry from Hue, its throwing below error..
Could not connect to localhost:8038 (code THRIFTTRANSPORT): TTransportException(‘Could not connect to localhost:8038’
i have added [sentry] entry in hue.ini file and added added required entry in sentry.site.xml.
still error exist. Please help…
also after updating hue.ini and sentry.site.xml how to restart both the service.

I’ve tried to connect to localhost:8888. After it gets connected, it asks for username and password. I’ve tried these combinations username :- admin password :- admin and username :- cloudera password :- cloudera but it’s saying that these are invalid. Now I m stuck, please help me out which username and password should I use to get it connected.

I am beginner of hadoop environment. As of now we are already done configure hadoop, hive,pig,sqoop. We have 3 datanode,1 name node already configure.And now our status of this hadoop is still testing. I found this forum and i have question, do we really need to install hue in hadoop? what is the benefits of hue in hadoop?
Thanks in advance,

How do we setup Kerberos on a remote hue machine with no connectivity to Cloudera manager. Documentation talks about setting up hadoop security only using Cloudera manager. Do you have instructions on how do it manually?

Hi I followed the exact steps as u mentioned. I changes all the configurations in the ini file. But when i open the home page of hue I am getting errors in all the components. please help me. what shoulkd i do? I am using Ubuntu 15.04.

Hi ,
We are using one e-learning platform where our students(200-500) executing c, cpp ..ctc programming exercises on browser based editor .Now we are planning to integrate our hadoop with our platfom so can we use HUE .
Is is support more than 200 users at a time.
Is there any limit on users ?

It depends on your setup and user usage. Hue itself is getting more and more optimized, but the major bottleneck is the serving of the static files and downloading large files. Current “ballpark” number of users is more in the 50s, but 200s is the target for the end of this year.

(unavailable) Oozie Share Lib not installed in default location.
SQLITE_NOT_FOR_PRODUCTION_USE SQLite is only recommended for small development environments with a few users.
Hive The application won’t work without a running HiveServer2.
HBase Browser The application won’t work without a running HBase Thrift Server v1.
Impala No available Impalad to send queries to.
Spark The app won’t work without a running Livy Spark Server

These are the 6 configuration error i got while opening the HUE page. But I am solve one by one.

hadoop.hdfs_clusters.default.webhdfs_url Current value: http://localhost:50070/webhdfs/v1
Filesystem root ‘/’ should be owned by ‘hdfs’
desktop.secret_key Current value: Secret key should be configured as a random string. All
sessions will be lost on restart
SQLITE_NOT_FOR_PRODUCTION_USE SQLite is only recommended for small development environments
with a few users.
Hive Failed to access Hive warehouse: /user/hive/warehouse
HBase Browser The application won’t work without a running HBase Thrift Server v1.
Impala No available Impalad to send queries to.
Oozie Editor/Dashboard The app won’t work without a running Oozie server
Pig Editor The app won’t work without a running Oozie server
Spark The app won’t work without a running Livy Spark Server

I’ve exactly same issue with soniya, but I’ve fixed first two issues. For the rest, i still don’t know how to fix it. Especially for “Oozie Editor/Dashboard The app won’t work without a running Oozie server “, I’ve opened ooozie server manually in terminal and it’s working perfectly. My file directory is /usr/local/hadoop/hue/desktop/conf/pseudo-distributed.ini, and I’ve uncomment oozie_url=http://localhost:11000/oozie. But it still said the server is not running. So can you give me some hints to fix this problem?

I have pseudonode-distributed cluster with following components hiveserver2,hive and hue installed by using cloudera manager and also I have configured the hive to use mysql as metastore instead of usual derby metastore. I can see mysql databases in hive cli but i can’t see mysql dbs in HUE UI.

I have copied the hive-site.xml and placed it in the /etc/hue/conf directory and i have restarted the full cluster as well .. please let me know any other config need to be checked/changed.

Please I am in my last week of completing intro to hadoop and I need to do some word counting in command prompt, but because my hue is not configured so the codes cannot be run and any time I login to Hue I see this “Configuration files located in :/etc/hue/conf.empty ” and “Potential misconfiguration detected. Fix and restart Hue.” So I need help to fix it.

Hi,
I have installed Hue following you tutorial and works fine.
When i try to use Hive with hue no databases shown.
i’m sure that the HiveServer2 is running and here is my hue.ini conf for [beeswax]
hive_server_host=localhost
hive_server_port=10000
hive_conf_dir=/usr/local/hive/conf

Please let me know any other config need to be checked/changed.
Thanks

Your comment is awaiting moderation.
Hi,
I have installed Hue following you tutorial and works fine.
When i try to use Hive with hue no databases shown.
i’m sure that the HiveServer2 is running and here is my hue.ini conf for [beeswax]
hive_server_host=localhost
hive_server_port=10000
hive_conf_dir=/usr/local/hive/conf

Please let me know any other config need to be checked/changed.
Thanks

Hi Team, I’m doing a research on Cloudera and I really need help with the unexpected hanging at kill command. I’ve been waiting but it seems like hanging forever when executing a simple query with operations, i.e. count and join. I would like to know whether how can I fix it. I have been stuck with this problem for 3 months, trying on every solutions from google but it doesn’t help. Could you please help me?

im getting this error when access http://localhost:50070/webhdfs/v1
{“RemoteException”: “exception”:”UnsupportedOperationException”,”javaClassName”:”java.lang.UnsupportedOperationException”,”message”:”op=NULL is not supported”}}

our cluster have two namenode, one Active another Standby. the hadoop cluster installed with Ambari . Fs.defaultFS property config to hdfs://carmecluster auto in HDFS core-site.xml; how to config the hue fs.defaultFS ? urgent!

Hi,
I have installed and configured Hadoop1.2.1.
Then installed Hive. Everything works fine.
And I installed Hue, it show up in “http://127.0.0.1:8000”. Everything in my local machine only.
I have done every configuration setting u have mentioned in this web page.
But still i get error “Potential misconfiguration detected. Fix and restart Hue.”
I cant connect with Hive, and i get “Could not connect to localhost:10000”
Kindly reply me, ‘m a beginner

Hi,
How can we customize Hue to extend its capabilities ? I am trying to provide support for shape geometries(Polygon, LineString etc) in map UI. Is there any way to add an app or plugin to extend Hue ?

Hi, Is there any solution or features for hive server ha(high availability) now ? I keep the hive server list in zookeeper however hue can’t use zookeeper address for hive server configuration in the hue.ini

I have annother question may be a bug ? In the query editor page(3.11, hive),when i click the “view statistics” on the right of the table name , the popup page will not response until the data is ready.
sometimes, I just don’t want to wait the sample data,however, the View more is not usable,even the close button response nothing.

R support is through SparkR, which Hue requires a Livy Server installation to submit to. The Livy Server and your Spark executors will need R installed. To test if SparkR works, you can run $SPARK_HOME/bin/sparkR from a Spark Client

i am not able to access the hue web ui, i am using google cloud platform with cloudera manager,
ip_address:7180 CDM is running fine, but when i open hue web ui
in the browser this address poping ( instance.projectname.internal’s :8088 ) and
webpage not opening “instance.projectname.internal’s server DNS address could not be found”
i tried everything can you plzzz help
thank you

I would be greatful for your help. I’ve installed cloudera hadoop and each try I try to run Hue there is the following message: “potential misconfigfuration detected. Configuration files located in /var/run/cloudera-scm-agent/process/137-hue-HUE_SERVER”

Thanks for your reply. This is the whole message:
“Checking current configuration
Configuration files located in /var/run/cloudera-scm-agent/process/137-hue-HUE_SERVER
Potential misconfiguration detected. Fix and restart Hue”.

And “Oozie_email_server
Email notifications is disabled for Workflows and Jobs as SMTP Server is localhost”.

“Thanks for your reply. This is the whole message:
“Checking current configuration
Configuration files located in /var/run/cloudera-scm-agent/process/137-hue-HUE_SERVER
Potential misconfiguration detected. Fix and restart Hue”.

And “Oozie_email_server
Email notifications is disabled for Workflows and Jobs as SMTP Server is localhost”.

Hi,
I’m trying to run hue from a docker in VM. However, http://172.17.0.2:8000/ doesnt open and says 172.17.0.2 took too long to respond.
The ambari is running and I’m start hue using “service hue start”. But just that I can’t get it to open in browser. Can you help please.

Attempting to use WinSCP to edit the hue.ini file because I want to add in my credentials to access a postgres database stored on Amazon RDS. I was able to connect to the file via Win SCP as well as open it and make edits. When I tried to save the file it said I did not have the proper permissions and I should delete the existing file and load a new one. I have yet to try this and I am not sure if it will even allow me to do so. So, basically I want to know if this is feasible or should I go about editing this file a different way? I really don’t want to spin up another Amazon EMR cluster and attempt to figure the postgres connection at that time as I feel like this should be an easy edit post install.