Using rsync to synchronize or mirror your home directory files to an USB stick

Using rsync to synchronize or mirror your home directory files to an USB stick

Before you modify and start using the script please consider the following:

OS dependencies

The script has been developed on OpenBSD and uses an USB stick formatted with OpenBSD UFS filesystem.
On my system with an IDE drive the hard disk is 'wd0' and the USB stick is 'sd0'.
When you use SATA disks 'sd0' will be your first hard disk. So please make sure you will use the correct device name for your USB stick.

For speed purpose the USB stick is mounted with what is called softdep in OpenBSD.

softdep
(FFS only.) Mount the file system using soft
dependencies. Instead of metadata being written
immediately, it is written in an ordered fashion to keep
the on-disk state of the file system consistent. This
results in significant speedups for file create/delete
operations. This option is ignored when using the -u
flag and a file system is already mounted read/write.
The options async and softdep are mutually exclusive.

For FreeBSD you have to use the -U option when you create the file system with newfs(8):

Code:

-U Enable soft updates on the new file system.

The rsync '--delete' option makes this script unsuitable for backing up purposes.

If you accidentally delete the picture of your boy/girl friend in his/her new swimsuit from your home directory, it will also be deleted from the USB stick upon the next run of this script.
Or in Jggimi's immortal words, a script that uses plain 'rsync' actually is a "backup data loss" script. This is one of them

If you really need a backup script you will have to wait for an next installment, that will use rsnapshot
From rsnapshot.org]

Quote:

rsnapshot is a filesystem snapshot utility for making backups of local and remote systems.

Using rsync and hard links, it is possible to keep multiple, full backups instantly available. The disk space required is just a little more than the space of one full backup, plus incrementals.

Security

In the current state, the script will also sync your private SSH keys in the .ssh directory, the passwords saved by your browser etc. So please keek the USB stick in a physical secure place or bad things may happen.

This paraphrase of jggimi's statement out of context is misleading. It takes a careful reading of this tutorial, I think, as well as some interpretation or inference of the intended meaning, to avoid walking away with the impression that rsync used alone (even without one of the --delete arguments) will delete files in the destination directory and that rsnapshot is the (as opposed to a) solution to that.

I believe it doesn't intend to say that at all. Your first phrasing is clear enough in only saying rsync --delete falls into jggimi's case of backing up your deletions: "The rsync '--delete' option makes this script unsuitable for backing up purposes." Then to me it would be clearer to follow up in your next tutorial with a modification of this script with different rsync options that make it a backup script rather than only a mirroring script (no --delete, but with --link-dest?) rather than proceeding immediately to an rsnapshot tutorial.

Myself, I don't want to go to rsnapshot but like to code my back up scripts myself. This lets me do something a little weird: I have a small disk laptop that I snapshot to an old Macintosh with a larger harddrive and some older files that never fit on the laptop. After I rsync from laptop to a subdirectory on the powermac I rsync from the powermac the full set of files minus some exclusions to a USB drive. Not sure this will fit into someone else's idea of a backup or snapshotting program. Then writing it myself also helps me to remember better what my "back up" (I'm still foggy on the rigorous definition of a back up -- sysadmins seem to have a stronger definition than I do) does exactly.

I like this phrasing of jggimi's too, but have to admit I've had occasion to want to backup some of my deletions, not to mention my moves. First, there are some things you genuinely do not want to exist anywhere. Second, in my half baked "backup" schemes I sometimes run out of space on the back up media. Yes, I suppose disks are cheap but not necessarily cheapenough for my taste. My back up space is sufficient again now, but I expect whenever I get around to using --link-dest in my script this may change eventually. Hmmm, I wonder how --link-dest and --delete interact, if at all. Maybe I will only have to delete selective big, old, no longer extant on the source drive files directly off the back up medium manually. Maybe you could argue that's a safer bet than rsync --delete in the first place.

Last edited by thirdm; 1st January 2014 at 07:26 PM.
Reason: correction and clarification after re-reading OP

Oh, one other thing. I found it useful in syncing to usb to first check that the right drive was mounted. You know, you can have a lot of different things you might stick in a usb port. I was doing this before on slackware by using fstab to mount to a certain directory based on disk identifier somehow IIRC and checking in script that that directory was mounted to an sdaX or something before proceeding. I think now that I'm back to OpenBSD I can check with the disk uuid from disklabel: e.g. disklabel sd0a | grep '^duid: 2319282e388bd9dc', but haven't added this back in yet.