hadoop-mapreduce-dev mailing list archives

Praveen,
There are many ways to prevent what you described...
I'm in the process of adding more docs, for now pls take a look at the
following older blog post for more details:
http://developer.yahoo.com/blogs/hadoop/posts/2011/03/mapreduce-nextgen-scheduler/
Arun
Sent from my iPhone
On Nov 25, 2011, at 11:55 PM, Praveen Sripati <praveensripati@gmail.com> wrote:
> Hi,
>
> Let's consider the following scenario
>
> -> The MR Job has an InputSplit on host h1 and h2
> -> AM makes a request to the Scheduler for a container on h1 and h2
> -> The scheduler responds with containers c1 and c2 on h1 and h2
> -> But the AM uses c1 and releases c2 after 15 minutes
>
> In this scenario c2 cannot be allocated to any of the AM by the scheduler
> for that span of 15 minutes and the c2 remains idle. And the released
> container is also not accounted into the Queues quota. How can such
> scenario be avoided?
>
> Also, it would be nice to specify a more complex requests to the scheduler,
> like a request for (h1 or h2), (r1 or r2). Add some dependencies between
> the requests.
>
> Thanks,
> Praveen