allow admins to force app deployment on a specific host

Details

Description

Currently you ask YARN to get slots on a host and it finds a slot on that machine -or, if unavailable or there is no room, on a host nearby as far as the topology is concerned.

People with admin rights should have the option to deploy a process on a specific host and have it run there even if there are no free slots -and to fail if the machine is not available. This would let you deploy admin-specific process across a cluster.

Activity

You can already ask for container exactly on a specific node setting relaxLocality to FALSE in the ResourceRequest. Though, this does not allow you to get a container if there is no capacity in the node.

Alejandro Abdelnur
added a comment - 06/Sep/13 14:29 You can already ask for container exactly on a specific node setting relaxLocality to FALSE in the ResourceRequest. Though, this does not allow you to get a container if there is no capacity in the node.

Steve Loughran
added a comment - 06/Sep/13 14:35 -Yes, and if you don't get that container it just stays in the queue -no notification to the AM. This is about being able to force things in without that wait and irrespective of space

Steve Loughran
added a comment - 07/Sep/13 19:39 yes. use case is deploying services cluster wide that use HDFS for provisioning and an AM to choreograph rollout, react to failures & addition of new nodes in the cluster

Alejandro Abdelnur
added a comment - 08/Sep/13 20:26 wouldn't make sense, instead having to deal with admin rights, to use a special queue for this and anybody in the queue ACL could manage such services?

-seems reasonable. The key thing is to ignore resource allocations, both in the demand of the app and the containers deployed -which lets people deploy static applications across the cluster, using YARN to push out the binaries & report failures to an AM

Steve Loughran
added a comment - 09/Sep/13 10:29 -seems reasonable. The key thing is to ignore resource allocations, both in the demand of the app and the containers deployed -which lets people deploy static applications across the cluster, using YARN to push out the binaries & report failures to an AM