[ https://issues.apache.org/jira/browse/HADOOP-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708035#action_12708035 ]
dhruba borthakur commented on HADOOP-3799:
------------------------------------------
I would like to figure out how to expose the filename in the API. It can be easily done when this API is invoked at the time when a new block is being allocated to a file. However, when the replicator takes a under-replicated block and tries to get a new replica location for that block, it is costly to the find the filename.
The filename can be deduced by traversing the INode-tree that is maintained by the namenode. But it is a costly operation beccause one has to traverse the entire branch starting from the specified INOde to the root. One option is to pass in the INode into the block-placement algorithm. f the algorithm needs the complete path name of the file in question, then it has to do the costly operation of generating the full path name of the file. However, this makes the interface kind-of less elegant. I am still debating how to do it right.
> Design a pluggable interface to place replicas of blocks in HDFS
> ----------------------------------------------------------------
>
> Key: HADOOP-3799
> URL: https://issues.apache.org/jira/browse/HADOOP-3799
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: BlockPlacementPluggable.txt
>
>
> The current HDFS code typically places one replica on local rack, the second replica on remote random rack and the third replica on a random node of that remote rack. This algorithm is baked in the NameNode's code. It would be nice to make the block placement algorithm a pluggable interface. This will allow experimentation of different placement algorithms based on workloads, availability guarantees and failure models.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.