Details

Description

For BIGTOP-547, I had done my testing where I running the MR job as root.

However, the helper script creates home directory for hard-coded set of users (root, jenkins, etc.) but not for other users. In particular, it doesn't create a directory for the user who is running the command.

A thought (in a form of Bruno pinging me ) has just occurred to me: wouldn't it be better for each component to advertise how exactly it need to initialize the HDFS via a json/xml/txt file dropped into a predetermined location describing exactly what needs to happen. We can them have a very generic script that simply operates on that.

Roman Shaposhnik
added a comment - 20/May/13 03:52 A thought (in a form of Bruno pinging me ) has just occurred to me: wouldn't it be better for each component to advertise how exactly it need to initialize the HDFS via a json/xml/txt file dropped into a predetermined location describing exactly what needs to happen. We can them have a very generic script that simply operates on that.

Mark Grover
added a comment - 20/May/13 18:36 Roman, I like that idea. However, I would personally be inclined towards still fixing this issue in this JIRA and working separately on the generic script on a separate JIRA.

For now to fix that jira, can we make the creation of that user optional? Or better, make it passed as a parameter? The latest patch does not seem give such option.
I don't always want to create a user directory for the user running that script. The user running that script may also not always be the one using Apache Hadoop (for instance when a vm gets created). So being able to pass the user as a parameter would be very nice (ie, creating home dir for various users when creating the vm).

Also, I am not sure we want to make that directory writable to the group or the world by default.

Bruno Mahé
added a comment - 30/May/13 08:44 For now to fix that jira, can we make the creation of that user optional? Or better, make it passed as a parameter? The latest patch does not seem give such option.
I don't always want to create a user directory for the user running that script. The user running that script may also not always be the one using Apache Hadoop (for instance when a vm gets created). So being able to pass the user as a parameter would be very nice (ie, creating home dir for various users when creating the vm).
Also, I am not sure we want to make that directory writable to the group or the world by default.

Bruno, I agree that the default behavior should be like it is today and only if -u uname is specified we should bother with the user creation. I'll update the patch to take care of that (and also the default permissions).

At the same time, I just don't want us to spend too much time polishing the current script since it is definitely getting replaced in Bigtop 0.7.0 with a more efficient implementation (BIGTOP-952).

Roman Shaposhnik
added a comment - 30/May/13 17:26 Bruno, I agree that the default behavior should be like it is today and only if -u uname is specified we should bother with the user creation. I'll update the patch to take care of that (and also the default permissions).
At the same time, I just don't want us to spend too much time polishing the current script since it is definitely getting replaced in Bigtop 0.7.0 with a more efficient implementation ( BIGTOP-952 ).