Friday, July 20, 2007

Backing Up Subversion to Rsync.net

I've started using subversion to store all of my data, documents, photos, etc and I wanted a secure offsite backup that was easy to implement. After a bunch of trial and error, here's what I came up with:

  1. Create a subversion respository at .
  2. Create a post-commit hook to dump the current revision, gzip it and encrypt it.
  3. Rsync the encrypted files to an offsite backup provider like rsync.net nightly.
Here's the post commit hook script I use:
REPOS="$1"
REV="$2"

# dump the repository, bzip it and encrypt it
/usr/local/bin/svnadmin dump "$REPOS" --revision "$REV" --incremental \
| /usr/bin/gzip -c \
| /usr/bin/openssl aes-256-cbc -salt -pass pass: [password] \
> [backup directory]/commit-$REV.gz.aes
Some points worth mentioning:
  1. This backup method probably isn't suitable for a busy subversion server.
  2. Use gzip instead of bzip2. I tried bzip2 at first, but it was waaaay to slow.
  3. openssl encrypts data way faster than gpg (which I also tried at first).
  4. Change [password] to something long and tricky - I specified it on the command line to keep things simple and because I trust this host (I'm the only user). However, in a multiuser scenario ps will show your password so you should use an environment variable or something.
  5. You end up storing data twice which sucks. If you don't like that you could scp the encrypted file at every commit, but then every checkin will stall for the upload. I regularly checking 10MB+ files which take awhile to upload over DSL, which is why I rsync the data in a cronjob instead.
When you need to restore the server, all you need to do is download each commit, decrypt it and concatenate all of the files into 1 big dump file or load each dump file in sequence. Because I used dump files you don't have to worry about OS or CPU compatibilities when restoring.

0 comments: