hadoop-common-dev mailing list archives

Hi,
this is in my mind since some time.
Since the global locking discussion in the nutch list, I would love
to inject my thoughts into this list.
First let me give a use case:
We have some nutch index search servers running as MapRunnable tasks.
The problem: how a client that want's to connect to this search
servers knows all the hosts that runs this runnable and may track
changes?
Once I had implement a kind of prototype for this use case and done
it like this:
I had start a rpc server on the search server client side (my tomcat
host) and all search servers was registering itself there. Than the
client was using this repository to lookup the search servers.
I think there could be a better more generalized solution.
Is it possible already possible or what are people thinking about
extending the jobtracker api to query tasktracker that run a named task?
This would be a smarter solution for the described usecase, also
people would be able to start a map runnable that runs a locking
service and would be able to connect to that.
However I agree with Doug's point of view that centralized locking
makes for this specific fetching use case no sense.
Any comments?
Stefan