What I would like to do is to provide a pre-defined subset of files from the source directory with the relative path and rsync them to the remote destination, deleting anything on the destination (including directories) that wasn't in the subset of files.

For example, given the above Source Directory:

My Pre-defined File List:

Dir1/File1-2
Dir3/File3-3

Destination directory:

Dir1/File1-1
Dir1/File1-2
Dir2/File2-1
Dir3/File3-1

I would expect to have only Dir1/File1-2, Dir3/File3-3, and no Dir2 in the destination when I'm done. I will also NOT have expected Dir1/File1-2 to be copied over again.

Things to Note:

These are very large files and I'm trying to prevent having to remotely copy them again if not needed.

The contents of the source sub-directories changes everyday

The pre-defined list changes everyday, but may/will include files in yesterday's list

There are 0 files in the Source Directory. It only contains sub-directories which contain the files needed to sync

The Sub-directories in the Source are pretty static. i.e. Dir1, Dir2 and Dir3 will always be there

I will keep plugging away to see if I can come up with the solution I'm looking for.

Update:

First of all, I want to thank jw013 for the formatting changes. I followed the help guideline, but didn't understand why is wasn't formatting correctly (first time poster).

Using the suggestions from Serge, I came up with the following command, but it still does not remove the extraneous files from the destination that are NOT in the File List.

3 Answers
3

where LISTFILE is a file that contains all files that should be synchronized to the destination directory. If your files do not change, you can use --ignore-existing instead of --update. Read the rsync manual page for details.

I'm not sure why certain other options are being suggested :P. The most "interesting" one is --ignore-existing. It may be worth making sure you understand the more friendly optimisations first - -a, or -u.

It looks like you're already using -a, which implies -t, which allows rsync to optimise by default. "Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time". If you're not using -a, it's often possible to use "-u or --update - skip files that are newer on the receiver".

If you want to benchmark the difference with the "quick check" optimisation disabled, you can use --checksum. (It's also useful if you want to forcibly resync all the existing files at some point in case of corruption. Enable -v and you'll be able to see which files had to be re-written).