Wednesday, December 19, 2012

Unison (was: rsync basics)

After writing this page, lvogdt turned me on to Unison, which of course is a much better way of syncing directories between two (or more?) machines/locations/shares:

See the Unison website.

As lvogdt wrote in the comment below, a package for openSUSE is available in the standard repos.

Here are my notes on the rsync manpage:



One of the things I've been meaning to do is review the rsync man page. So, let's get to it:

Typical usage is to transfer a directory hierarchy from one machine to another for the purpose of backup. A very commonly used option is -a (archive mode), which is the same as -rlptgoD. This, essentially, says to transfer anything and everything, and if it can't be transferred, then reproduce it. (For example, the -D parameter causes rsync to transfer/recreate special files, such as devices, fifos, etc.)

Reproducing everything may or may not be desirable. In my typical use scenario I will be syncing my working directory from a workstation to a laptop and vice versa, or backing it up to a back-up server, NAS, or external drive. In this scenario, I won't want or need the -D parameters. So archive mode is out.

A very important parameter to include is -u (--update), which prevents rsync from overwriting a file that exists in both the source and destination if the destination file's modification time is newer. To sync the two directories we first run the rsync from, say, the workstation to the laptop. Then, we execute the same rsync command again, but with the source and destination reversed. If a file existing at both locations has been modified, the -u parameter will ensure that the modified version is propagated correctly. However, if it has been modified in BOTH locations, the file with the most recent modification time will overwrite the other one.

(I was also concerned that rsync might delete files on the receiving side if they weren't present on the sending side, but it doesn't do that by default -- only if the --delete parameter is given.)

There's also the problem that, without the verbose option, rsync will produce little or no output. We won't have any idea what it's doing. So, we include the verbose option as well.

So we end up with the following parameter list:
-r  --recursive   recurse into subdirectories
-l  --links       copy symlinks as symlinks
-p  --perms       preserve permissions
-t  --times       preserve modification times
-g  --group       preserve group
-u  --update      skip any files which exist on the destination 
                  and have a modified time that is newer than the 
                  source file
-v  --verbose     increase verbosity

(Note that this rsync business is inherently dangerous. Though it most likely won't delete anything, it can easily overwrite stuff and cause unwanted data loss if you're not careful. For really important files, it's best to use some kind of revision control system.)

The only thing missing is a list of exclude patterns, since typically there will be certain subdirectories that I don't want rsync to send back and forth at all. So I create a file rsync-exclude that looks something like:
.*
/Bigfiles
/Desktop
/Music
/Pictures
/Public
/Templates
/Videos

My final rsync commands look like this (both are excluded on the laptop, since it doesn't have a fixed IP address):
$ rsync -rlptguv --exclude-from=rsync-exclude myuser@workstation: .
$ rsync -rlptguv --exclude-from=rsync-exclude . myuser@workstation:



1 comment:

  1. I'm using "unison", a real cool and simple GUI above rsync, svn, and other file synchronization tools:

    http://www.cis.upenn.edu/~bcpierce/unison/

    There is also a package for openSUSE available in the standard repos.

    ReplyDelete