Automated Backup Techniques

I recently found my self needing to conduct an automated backup for a client at work. Unfortunately, the only access available to the backup destination was ftp. Previously I’d used SCP or a subversion repo for backup management. Neither would be available in this instance, but I’ve detailed how you could use them.

Obvious there are many very convenient backup methods out there. Some free, others not. But this solution worked well for me, hopes it helps you too.

Wrapped Up in a Shell Script

The following examples are intended to be wrapped up in a bash shell script. Save them in a .sh file using your preferred text editor. I’d highly recommend nano. You can execute a shell script file by navigating to the directory that it is in and executing ./filename.sh

Website Backup via FTP

Automatically backing up to ftp is an interesting hurdle as there is a lot of communication between the client and the ftp server besides just the file transfer. Fortunately, it can be conquered with a somewhat simple shell script:

#!/bin/sh
USERNAME="ftpusername"
PASSWORD="ftppassword"
SERVER="ftp.mybackuphost.com"
FILE="/tmp/backup"# Cleanup and create backup buffer location

/bin/rm -fR $FILE
/bin/mkdir $FILE

# Generate backup tars
echo "Compressing Sites"
tar -czf $FILE/importantWebsite.tar.gz /path/to/website/
cd $FILE

# Login to remote server
echo "Starting File Transfers"
ftp -nu $SERVER <<EOF
user $USERNAME $PASSWORD
put importantWebsite.tar.gz
quit
EOF

As you can see, this script compresses the site (or file base) storing it in the tmp folder. You could select elsewhere if you want to retain a copy of your backups, but they could eat up space fast.

The crontab entry you will make below will determine how frequent backups are conducted. The above script will overwrite your previous backup each time it is conducted.

7 Day Backup

Alternatively, you can conduct backups in a 7 day loop so that each backup overwrites the backup from that same day last week. This following example shows how you could extend the script to backup multiple websites.

You could also use a different date pattern that would work in a pattern for longer periods. Just so long as it works out that a newer backup will eventually overwrite an old one. Or that you have something in place to cleanup older backups.

#!/bin/sh
USERNAME="ftpusername"
PASSWORD="ftppassword"
SERVER="ftp.mybackuphost.com"
DAYOFWEEK=`date +%A`
FILE="/tmp/backup"# Cleanup and create backup buffer

/bin/rm -fR $FILE
/bin/mkdir $FILE

# Generate backup tars
echo "Compressing Sites"
tar -czf $FILE/websiteA_$DAYOFWEEK.tar.gz /path/to/websiteA/
tar -czf $FILE/websiteB_$DAYOFWEEK.tar.gz /path/to/websiteB/
tar -czf $FILE/websiteC_$DAYOFWEEK.tar.gz /path/to/websiteC/
cd $FILE

# Login to remote server
echo "Starting File Transfers"
ftp -nui $SERVER <<EOF
user $USERNAME $PASSWORD
mput *.tar.gz
quit
EOF

Note the -i flag on ftp, it disables interactive prompts on multiple file transfers. Mput uses a filename pattern to globber a list of files and send them all in sequence.

Website Backup via SCP

If you have SSH access to your backup server, SCP may be a better (easier) route than FTP. Take this script for example:

#!/bin/sh
USERNAME="sshusername"
PASSWORD="sshpassword"
SERVER="sshHost"
DAYOFWEEK=`date +%A`
FILE="/tmp/backup"
DESTINATION="/pathToBackup/onDestinationServer"# Cleanup and create backup buffer

/bin/rm -fR $FILE
/bin/mkdir $FILE

# Generate backup tars
echo "Compressing Sites"
tar -czf $FILE/websiteA_$DAYOFWEEK.tar.gz /path/to/websiteA/
tar -czf $FILE/websiteB_$DAYOFWEEK.tar.gz /path/to/websiteB/
tar -czf $FILE/websiteC_$DAYOFWEEK.tar.gz /path/to/websiteC/

# Login to remote server
echo "Starting File Transfers"
scp -pw $PASSWORD $FILE/*.tar.gz $USERNAME@$SERVER:$DESTINATION

As you can see, a little shorter and very slick.

Subversion Version Control

This would have to be the coolest way to manage it, but requires a subversion repository and deploying a shell script on both the source and destination servers.

The benefit of this is that the backup will be incremental and conduct faster for code bases that don’t change frequently. This is also extremely handy if you want to deploy your backup to multiple destinations.

Source

Add and commit any new files on the source version to a subversion repo. I assume you’ve already created a repo and your files exist in it. You could also just commit as you make changes to your website (or other project).

#!/bin/sh
cd /Path/to/Files

svn add --force *
svn commit -m "Backup"

Destination

Then, on the destination, conduct an update on the repository. I assume that you’ve

already checked out the repository.

#!/bin/sh
cd /Path/to/Files

svn update

Automating the Execution of your Shell Script

To schedule your scripts to run automatically, you need to make an entry in your crontab file. You’ll likely need root or sudo privileges for this. Access your crontab file:

sudo nano /etc/crontab

Add a line for the shell script you want executed, the first 5 values is a time pattern (minute, hour, day of month, month, day of week), the next is which user to run it as, and the third is the command. Wikipedia or your favorite search engine has more details on editing the crontab file. For example, to run our backup script nightly at 2am as root we would add:

0 2 * * *   root   /path/to/script.sh > logFilePath.log

The tail end is to optionally append the output to a log file.

Tags: automated, backup, linux, techniques

You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.