Filesystem Journaling Update II - July 2005

Softlinks and hardlinks are now supported and I cleaned up a few
more bugs. I can now run a buildworld loop and the mirror generated
by the journal stays in synch (diff -r reports no differences if I
idle the buildworld, wait a second or two for the journal to catch up,
and run it). I'm still not sure about modifications made via mmap()
but everything else seems to work.
In my buildworld timing tests a journal with a 1 megabyte memory buffer
mirroring to the same physical disk is able to run the buildworld in
2250 seconds. Without the journal it takes 2100 seconds. This is about
a 7% performance penalty, which is quite reasonable considering what
you get for the trouble.
We are very close to being able to use this for near real-time backups.
With a forward-running journal the idea would be to keep a base mirror
of the filesystem and use any remaining space to hold the journal files.
When the backup filesystem fills up, you run the earliest journaling
file into the mirror and remove it. Thus one can use 100% of the
available backup media. The disadvantage is that to restore the
filesystem the base mirror must be copied back and then all the
journaling files run to bring it up to date.
Once we get reverse-journaling (UNDO records) working, we can do it
the other way.. the mirror would contain the completely up to date
filesystem and the journal files would be used to 'revert' it to an
earlier state if necessary. This is safer because you can run the
filesystem closer to 100% full (when it fills up you just remove the
oldest journaling file), and you can restore a backup from the live
mirror without having to run through any journaling files.
To make the backup regimen work properly the originating machine needs
a buffering stage capable of using swap for robustness and needs the
two-way protocol to allow reconnects after connection failures at any
point in the pipe line. I am going to work on a userland buffering
stage and the (userland and kernel) two-way protocol next.
-Matt
Matthew Dillon
<dillon@xxxxxxxxxxxxx>