Setting up Offsite Backups

BOA already has a backup system that creates copies of files in

/data/disk/o1/backups/

But this is on the same disk as the main site. It will not maintain backups if the primary disk fails. As a solution to this, I have created a new persistent drive and mounted it to that folder. At least then, if there is a failure on the primary drive, the backups are safe.

But this does not protect against a hacked server where everything is deleted. Or if the server and all drives are lost. So I set up a synchronisation to copy the backups to Amazon S3.

The CRON file looks like this

#!/bin/bash
echo Started ``date`` >> /tmp/sync-backups.log
find /data/disk/o1/backups/ -type f -maxdepth 1 -exec bash "/root/tools/move-to-s3.sh" "{}" \;
echo Finished ``date`` >> /tmp/sync-backups.log

It finds all files in the backup folder and runs another script. Then appends a date to a file in /tmp so I can see when it ran.

The guts of the copy utility (move-to-s3.sh) are as follows

#!/bin/bash

# Copy Backup file to S3 Bucket
#
# Filename is in the format
#
#    domain.com.au-20140520.133810.tar.gz
#
# And save to the format
#
#    domain.com.au/2014/05/20/domain.com.au-20140520.133810.tar.gz
#
# Keeping the same file name (so it can be identified if downloaded later)
# but into a date based folder structure based on the domain name

FILENAME=`basename ""`
echo $FILENAME

# Target - where to save it

TARGET=`echo $FILENAME | sed -r -e 's/(.*)\-([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])\.(.*)/\1\/\2\/\3\/\4/'`
echo "--> $TARGET"

s3cmd sync  s3://your-bucket-name/$TARGET/$FILENAME

Essentially I break the name up so that all the backups are saved in a manageable folder structure on S3. I have a rule in the S3 bucket to archive the files to glacier after a short period of time and delete them completely after many months.

You can obviously run the script directly against a backup file or part of the CRON.