Need to figure something for 0.20.0 release. Since we rely on a patched hadoop server, either must have a separate hbase, a patch for hbase and hadoop, or if we can (via reflection or such) detect if the hadoop server supports appends we could commit this and only maintain one thing.

Jonathan Gray
added a comment - 01/Jul/09 01:33 Need to figure something for 0.20.0 release. Since we rely on a patched hadoop server, either must have a separate hbase, a patch for hbase and hadoop, or if we can (via reflection or such) detect if the hadoop server supports appends we could commit this and only maintain one thing.

If dfs.support.append is set to true AND you have put an hadoop with hadoop-4379 into hbase CLASSPATH, this patch will make it so sync use the hadoop-4379 new syncFs call.

This is a bit tough to test. You have to do messing with different hadop jars. I played around a bunch with it and it seems to do the right thing. I'll just commit since it satisfies a probably exotic need (Last time I was running 4379, namenode shutdown because of a bad generation stamp. I don't think this a 4379 issue. Its probably something that can happen if you ahve dfs.support.append=true, a bug).

stack
added a comment - 11/Jul/09 00:16 If dfs.support.append is set to true AND you have put an hadoop with hadoop-4379 into hbase CLASSPATH, this patch will make it so sync use the hadoop-4379 new syncFs call.
This is a bit tough to test. You have to do messing with different hadop jars. I played around a bunch with it and it seems to do the right thing. I'll just commit since it satisfies a probably exotic need (Last time I was running 4379, namenode shutdown because of a bad generation stamp. I don't think this a 4379 issue. Its probably something that can happen if you ahve dfs.support.append=true, a bug).

dhruba borthakur
added a comment - 03/Aug/09 07:29 > Last time I was running 4379, namenode shutdown because of a bad generation stamp
I saw the same bug in our cluster. Have posted a fix as part of the new patch file attached to HDFS-200 . Attachment fsyncConcurrentReaders13_20.txt