rdiff-backup

rdiff-backup backs up one directory to another. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special directory so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup can also operate in a bandwidth- efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back up to a remote location, and only the differences will be transmitted. It can also handle symlinks, device files, permissions, ownership, etc., so it can be used on the entire file system.

Recent releases

Release Notes: The --min-file-size and --max-file-size options were changed to agree with the man page. These options no longer include files, and will only apply to regular files. Python 2.2 compatibility was fixed. The program no longer crashes when the file system can't set the requested ACL. The error message that is shown if a regress fails due to a security violation was improved. The --override-chars-to-quote option was fixed. Numerous Windows fixes were made.

Release Notes: A typographical error in robust.py which broke
error reporting was fixed. Windows errors caused
by too long filenames are now ignored; the files
are not backed-up, but the backup process is no
longer halted.

Release Notes: Trailing spaces and periods are properly escaped
when the destination filesystem does not support
them. The start and end times are taken from the
same system so that the "elapsed time" statistic
is unaffected by servers in different time zones.
The program is now capable of backing up files
which the user cannot read, but can change the
permissions of. The --use-compatible-timestamps
option was added.

Recent comments

I think rsync and rdiff-backup both have strengths and weaknesses. rsync is often significantly faster than rdiff-backup, while rdiff-backup can use much less memory and be less disk intensive on large directories because it does not build the entire filelist ahead of time. And I have had some people tell me that they couldn't get rsync to work on their system, but rdiff-backup worked fine (I'm sure the reverse is also true).

Maybe the biggest feature that rdiff-backup has that rsync doesn't (besides the whole incremental backup thing) is support for various optional filesystem features, and the ability to back them up to a filesystem of a different type.

For instance, rdiff-backup can back up a filesystem that has ACLs, extended attributes, and various owners/groups, to a filesystem that doesn't have ACLs, EAs, or root permissions. With rsync you would lose that information, including all ownership. With rdiff-backup, everything will be preserved when you restore.

Rsync-based programs like your backupbuddy will probably inherit most of the strengths/weaknesses of rsync. Also rdiff-backup will in general use less disk space because it compresses file changes over time. Also newer versions of rdiff-backup keep SHA1 checksums of all regular files around, which can help with security auditing or checking the integrity of existing backups.

But anyway I'm sure both programs have their pros and cons, and I encourage anyone reading this to test both out for themselves.

Handy
Networked, incremental backups are definitely the way to go.
Sending things to another machine is much easier and more
reliable than tape. And it sucks to pull a backup file only to
discover you'd screwed it up before making a backup.
rdiff-backup avoids these problems.

It's also the only such tool I've seen that can handle Mac
OS X resource forks. (With 0.13.2 or better, IIRC.) These are
more important than you'd think - Quicken, for example, stores
important data in them.

Re: rdiff-backup has SERIOUS PROBLEMS
Hi, I'm sorry your experiences with rdiff-backup were so bad. rdiff-backup's error policy is stated here (http://www.nongnu.org/rdiff-backup/error_policy.html).
As it says, rdiff-backup does make an effort to recover from different types of errors in different ways, to leave the database in a consistent state in the face of unrecoverable errors, and reverse failed backup sessions (effecting a kind of atomicity between sessions).

Unfortunately, rdiff-backup archives can get corrupted (e.g. by disk errors or bugs in rdiff-backup) beyond the program's ability to automatically repair them. In these cases manual repair is necessary. But this is true of any backup program. Just a reminder, if you are using the program for something important, please use the stable version. The current unstable version is really old and doesn't include many CVS fixes. And anyway it's unstable.

Anyway, how rdiff-backup responds to network problems is testable.. Start a session and simulate a failure by killing the client or server, or by taking down the network connection. Then see what happens to the next session.

rdiff-backup has SERIOUS PROBLEMS
I've run rdiff-backup on a handful of systems, backing up on the same system, and remotely.

When a backup fails, rdiff essentially gives up responsibility for your backup's integrity. It will not preserve an operable, *usable* system state. It then becomes the admin's responsibility to *manually repair* the metadata which controls rdiff's behavior.

If you cannot successfully make those repairs, admins are advised in the documentation to *delete* the entire backup and start from scratch!

This is unacceptable. For rdiff-backup to become a serious and useful project, it needs to proactively take action when a failure occurs! For example, when there are network connectivity problems, rdiff-backup gives up and the backup is considered failed. *Further attempts* to backup will be rejected until you manually revert the metadata!

In other words: Don't go on vacation! When you've put your faith in rdiff-backup, and it fails while you're on that desert island, it will let you down: subsequent backups will fail too.

rsync is a much better solution, although you lose the sub-file increment capability of the rdiff algorithm. But disk space is cheap, and backups that repeatedly fail can cost you your job.