You are using an outdated browser which puts all net citizens at risk. As an incentive to upgrade to a current and thus much more secure product (we recommend the free Firefox browser), you won't be able to visit this site in its cute design, but in this rather boring printer-ready version only. Thank you for considering a browser update!

Cloud Backup

Backups are important. Most people have learned that, some the hard way, and keep backups on external drives. This is a good idea, but it’s not enough. Physical desasters like fire or thieves in the house still have the potential to carve your name into the gravestone of deceased digital lives. Cloud backups to the rescue!

Requirements

The storage provider should be fast, unlimited and affordable. Furthermore, you want to keep at least some control as to where the data is actually being stored. (Jurisdiction in say US and EU differ significantly when it comes to data privacy.) As of now, the best match is Amazon S3.

The cloud backup software on the other hand has to encrypt the data before it is sent to the cloud. Moreover, subsequent backups should only store the bits that have changed meanwhile. And of course the software should be easy to use and as failsafe as possible.

Mac OS X

It’s been a fruitless task to find a suitable cloud backup software on Mac OS X ever since cloud storage appeared in the sky. But fear not, a relatively new kid on the block fills this gap amazingly well: Arq by Haystack Software.

Arq mimics Apple’s TimeMachine when it comes to ease of use, yet it’s still possible to tweak important parameters such as the physical storage region or the redundancy class (more reduntant = more expensive). All you have to do is create an Amazon S3 account, then launch Arq and answer the few question it initially asks. If you are on a high bandwith link, you can configure Arq to do hourly or daily backups. Nomadic users on the other hand may prefer to kick off backups manually from the Arq menubar item.

The overall volume (and thus cost) of cloud storage can be limited. Restoring files is as easy as dragging them out of the Arq window and even if the Mac you backup from gets lost in a sparkling blaze, it’s very easy to restore the files to any other Mac – provided you remember your encryption password or have a hardcopy of it safely tucked away in a safe place.

Arq is not freeware, but it’s very actively maintained and worth every cent.

Linux Servers

Failing backup systems offered by hosting providers are not unheard of and disasters may not stop at the doors of a server farm. Cloud backups therefore make sense for servers as well, at least for very essential data such as /etc or database dumps.

The list of available backup software is quite large but shrinks considerably if you are looking for a console tool with support for Amazon S3. A mature solution is duplicity by Ben Escoto and Kenneth Loafman, essentially a Python script which relies on rsync, GnuPG (for encryption), boto (for Amazon S3 connectivity) and NcFTP as an alternative storage backend.

The following wrapper script is an example for how to call duplicity as a daily cronjob. Check out the man page of duplicity for details. You should create a new set of GPG keys per server solely for use with duplicity as the passphrase for this GPG key is present in the wrapper script. Also make sure you set restrictive permissions on the wrapper script. To test your setup, use the --dry-run option which is handed through to duplicity.

#!/bin/bash

export AWS_URL='s3+http://myserver.mydomain.com'      # Amazon S3 bucket
export AWS_ACCESS_KEY_ID=''                           # Amazon S3 key ID
export AWS_SECRET_ACCESS_KEY=''                       # Amazon S3 access key
export GPG_ENCRYPT_KEY='root@myserver.mydomain.com'   # GPG key for encryption
export GPG_SIGN_KEY=''                                # GPG key ID (hex) for signing
export PASSPHRASE=''                                  # GPG key passphrase

duplicity                                                                \
  --full-if-older-than 1M                                                \
  --include /etc                                                         \
  --include /root                                                        \
  --exclude '**'                                                         \
  --exclude-other-filesystems                                            \
  --encrypt-key ${GPG_ENCRYPT_KEY}                                       \
  --sign-key ${GPG_SIGN_KEY}                                             \
  /                                                                      \
  ${AWS_URL}                                                             \
  --s3-european-buckets                                                  \
  --s3-use-new-style                                                     \
  --verbosity warning                                                    \
  $(if [ "$1" == '--dry-run' ]; then echo '--dry-run'; fi)               \
  $(if [ "$1" != '--dry-run' ]; then echo '--no-print-statistics'; fi)   \
                                                                         \
&&                                                                       \
                                                                         \
duplicity remove-all-but-n-full                                          \
  12                                                                     \
  ${AWS_URL}                                                             \
  --s3-european-buckets                                                  \
  --s3-use-new-style                                                     \
  --verbosity error                                                      \
  $(if [ "$1" != '--dry-run' ]; then echo '--force'; fi)                 \
                                                                         \
&&                                                                       \
                                                                         \
duplicity cleanup                                                        \
  --extra-clean                                                          \
  ${AWS_URL}                                                             \
  --s3-european-buckets                                                  \
  --s3-use-new-style                                                     \
  --verbosity error                                                      \
  $(if [ "$1" != '--dry-run' ]; then echo '--force'; fi)                 \
  >/dev/null

This is just an example which works for me, you use it at your own risk.

Other

I haven’t investigated other use cases such as Linux workstations or Windows. Feel free to share your killer apps by submitting a comment.

(Sven Schwyn)

(We are remaking our web presence and therefore comments are temporary disabled.)