1. A replica needs to be "detached" before it gets truncated so truncation won't modify snapshot. Sorry this is not described in the design document.
2. When finalizing a RUR, we need to make sure that the finalized data/meta files are located in the "finalized" directory. Check FSDataset#finalizeBlock to see how a replica is finalized.
3. Looks that we need to make sure that rbw#getBytesOnDiks() should be equal to rbw#getNumOfBytes() when stopping a writer..

Hairong Kuang
added a comment - 22/Sep/09 21:29 1. A replica needs to be "detached" before it gets truncated so truncation won't modify snapshot. Sorry this is not described in the design document.
2. When finalizing a RUR, we need to make sure that the finalized data/meta files are located in the "finalized" directory. Check FSDataset#finalizeBlock to see how a replica is finalized.
3. Looks that we need to make sure that rbw#getBytesOnDiks() should be equal to rbw#getNumOfBytes() when stopping a writer..

Tsz Wo Nicholas Sze
added a comment - 23/Sep/09 01:46 h627_20090922.patch: only able to fix (1).
> 2. When finalizing a RUR, we need to make sure that the finalized data/meta files are located in the "finalized" directory. Check FSDataset#finalizeBlock to see how a replica is finalized.
I cannot see any codes in FSDataset#finalizeBlock checking "finalized" directory. Could you give me more hints?
3. Looks that we need to make sure that rbw#getBytesOnDiks() should be equal to rbw#getNumOfBytes() when stopping a writer..
Then, it should be fixed in initReplicaRecovery(..), right?

> I cannot see any codes in FSDataset#finalizeBlock checking "finalized" directory. Could you give me more hints?
You need to move the replica from directory "rbw" to directory "finalized". This is done by FSVolume#addBlock(Block, File).

Hairong Kuang
added a comment - 23/Sep/09 18:21 > I cannot see any codes in FSDataset#finalizeBlock checking "finalized" directory. Could you give me more hints?
You need to move the replica from directory "rbw" to directory "finalized". This is done by FSVolume#addBlock(Block, File).

Tsz Wo Nicholas Sze
added a comment - 25/Sep/09 00:53 > It would be nice if the logic for finalizing a block is at FSDataset#finalizeBlock.
Yes, but I don't see any easy way to use it. Could you point out how to use finalizeBlock(..) in the patch?

OK, let me give a shot. With so many code changes, I agree it is not easy to figure it out.

This patch has a few changes to Nicholas' the most recent patch. His most recent patch has the right logic. My new patch simply makes it simpler by reusing existing code.
1. use bumpReplicaGS to bump RUR's GS to be recovery id;
2. refactor finalizeReplica that takes a ReplicaInfo to finalize a replica, and use finalizeReplica to finalize the rur;
2. remove the static method updateReplica and change updateReplicaUnderRecovery to non-static.

Hairong Kuang
added a comment - 26/Sep/09 02:07 OK, let me give a shot. With so many code changes, I agree it is not easy to figure it out.
This patch has a few changes to Nicholas' the most recent patch. His most recent patch has the right logic. My new patch simply makes it simpler by reusing existing code.
1. use bumpReplicaGS to bump RUR's GS to be recovery id;
2. refactor finalizeReplica that takes a ReplicaInfo to finalize a replica, and use finalizeReplica to finalize the rur;
2. remove the static method updateReplica and change updateReplicaUnderRecovery to non-static.

[exec] +1 overall.[exec][exec] +1 @author. The patch does not contain any @author tags.[exec][exec] +1 tests included. The patch appears to include 3 new or modified tests.[exec][exec] +1 javadoc. The javadoc tool did not generate any warning messages.[exec][exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.[exec][exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.[exec][exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.

Hairong Kuang
added a comment - 26/Sep/09 02:09 [exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.

Tsz Wo Nicholas Sze
added a comment - 05/Oct/09 23:20 > What was the reason to remove the following codes from h627_20090924.patch?
Oops, the codes are indeed committed but they are not used by HDFS-658 .