The following is a guide on how to do encrypted backups to Amazon S3.
The dt-s3-backup.sh BASH script (http://blog.damontimm.com/bash-script-incremental-encrypted-backups-duplicity-amazon-s3/) combines the Duplicity backup utility, s3cmd, GPG and the Amazon S3 storage service to make a cheap, reliable and secure off-site backup storage solution.
You will require an Amazon S3 account (and a debit/credit card too).
The new Amazon S3 interface is pretty good - but you may also use the S3Fox Firefox extension.
Once you have created your Amazon S3 account (http://aws.amazon.com/s3/) you will need to locate your Access keys (http://aws-portal.amazon.com/gp/aws/developer/account/index.html?action=access-key).
Next you should make a GPG key to encrypt your backup. If you already have one you can use that but that means that you will need to have your private key on your server which isn't really desirable. You need to have the private key on the server because Duplicity needs to be able to both encrypt and decrypt the backup manifest so that it can do incremental backup.
To create a gpg key, run the following. It is best to do this on the server because that way you don't have to worry about importing or trusting.
# gpg --gen-key Please select what kind of key you want: (1) DSA and Elgamal (default) (2) DSA (sign only) (5) RSA (sign only) Your selection? 1 DSA keypair will have 1024 bits. ELG keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) 4096 Requested keysize is 4096 bits Please specify how long the key should be valid. 0 = key does not expire <n> = key expires in n days <n>w = key expires in n weeks <n>m = key expires in n months <n>y = key expires in n years Key is valid for? (0) Key does not expire at all Is this correct? (y/N) y ...The rest is up to you...
Make sure in the comment that you note that it is for your backups, so that you do not get it confused with your personal key.
After the key is finished generating (4096 bits worth of entropy takes a while!) then you need to export the private key and keep it in a safe place.
# gpg --list-keys /home/Daniel/.gnupg/pubring.gpg ------------------------------- pub 1024D/6BBDC0C6 2010-06-01 uid Daniel Devine (just doing this as a test) <firstname.lastname@example.org> sub 4096g/424CF9A7 2010-06-01 # gpg --output DanielDevine_Backup_public.gpg --armor --export 6BBDC0C6 # gpg --output DanielDevine_Backup_private.gpg --armor --export-secret-key 6BBDC0C6
The number you need after the --export option will be different for you, so substitute your own by observing the output of "gpg --list-keys"
Now that you have exported your keys into a format you can transfer you should run an MD5 sum on it before you transfer it off the server.
# md5sum DanielDevine_Backup_private.gpg 4343afa0bccb0878e123209766687614 DanielDevine_Backup_private.gpg
Then you need to transfer it off your server. I prefer to use SFTP for this because it is easiest and most secure. If you have a SSH server running on the remote machine then you have SFTP!
# sftp [YourUsername]@[your IP address] > PUT DanielDevine_Backup_private.gpg > exit
On the computer that you transferred the file to do a md5sum on the file and check if the output matches what it did on your server. If it doesn't then transfer the file again because the first copy you did is corrupt and you can't use it to decrypt your backup!
Then delete the exported private key from the server. (use "rm DanielDevine_Backup_private.gpg")
You can find the script at: http://github.com/thornomad/dt-s3-backup
or you download it and keep it up to date by using git.
git clone git://github.com/thornomad/dt-s3-backup.git
Once you have it you can copy/move the script to where you prefer to keep your backup/cron scripts. I just put it in a directory in root's home called "backupscripts". If you used git to obtain the script then it is best to copy rather than move. When you see than an update of the script is available you can use "git fetch" to get a new copy which you can merge your configuration into.
Before configuring the script use your distribution's package manager to get the dependencies for the script. One CentOS 5.5 with RPMFusion repositories you can run:
# yum install duplicity # yum install s3cmd # yum install gpg
The package names on Ubuntu/Debian are apparently the same. They should also be the same on Fedora.
On CentOS 5.5 the version of Boto is not new enough and will result in an error most likely to do with not being able to handle newer Amazon buckets. Download the newest version of Boto and install it.
# wget http://boto.googlecode.com/files/boto-2.0b3.tar.gz # tar xzvf boto-2.0b3.tar.gz # cd boto # python setup.py install
Now to finally configure the backup script!
Read the comments and fill in the variables. If you want more information on the variables look at the Duplicity man page ("man duplicity") for a better description. If it asks you to go through Duplicity setup then you must do that. Just follow the prompts.
Next you should add an entry to crontab. The following is a good little snippet you can add to the top of the file to guide you through scheduling tasks.
# * * * * * command to be executed # - - - - - # | | | | | # | | | | +----- day of week (0 - 6) (Sunday=0) # | | | +------- month (1 - 12) # | | +--------- day of month (1 - 31) # | +----------- hour (0 - 23) # +------------- min (0 - 59 # Remember to fill with 0's where appropriate. * means all so if you have * seconds its every second.
Else you can also use "@hourly" "@daily", "@weekly" etc. instead of filling out an exact time. I suggest backing up at least daily - it really doesn't take up much backup space if you configure dt-s3-backup.sh to do incremental backups.
If you get an error such as "Time too skewed" it is because your time is too different (mine was 2 hours in the future) from Amazon's as it can only be 9000ms different. This is a security thing on Amazon's part.
To get your system time right, install ntp.
# yum install ntp
To see the time difference between your system time and your BIOS clock:
Then to set up NTP do.
# ntpdate pool.ntp.org # /etc/init.d/ntpd start
To update your BIOS clock.
# hwclock --systohc
Then you should make sure NTP comes up on startup.
# chkconfig nptd on
If you imported an existing key and you get an error to the effect of "There is no assurance this key belongs to the named user" then you probably need to set the trust of the key by doing the following.
# gpg --edit-key [Key ID - eg. DS8F978D] > trust > 5 [or whatever the level for ultimate trust is] > save [not sure if this is needed] > quit)