Restore from offsite backups

We use Duplicity to perform offsite backups. Some backups are
encrypted with GPG before being shipped to an Amazon S3 bucket.

You will find the fingerprint of the key in hieradata/production.yaml within
the govuk-puppet repository. The key and passphrase are both
stored in encrypted hieradata in the govuk-secrets repository.
The same private key is used for all offsite backups.

Prerequisites for restoring backups

On the machine where you want to restore the backup:

VM requirements

For the backup and restore drill, you will restore and unpack a MySQL database
on a Vagrant VM.

On a fresh VM, you may require the following packages for this exercise:

You can use either your dev VM or if you have the space you can create a new mysql server VM using the following command:
vagrant up mysql-master-1.backend
This needs to be run from the root of the govuk-puppet repository
Access the new VM using:
vagrant ssh mysql-master-1.backend

Packages via apt-get

sudo apt-get install duplicity python-pip python-boto mysql-server

Python libs via pip

sudo pip install s3cmd

Set up GPG keys to decrypt backups

You will need access to production hieradata credentials to retrieve the AWS
credentials and GPG key to decrypt the backups.

This doesn’t seem to have any significant consequences and can be
ignored.

Restore a backup

Note: If performing this as part of the 2nd line drill with the whitehall
backup above, please note that this may require a lot of free disk space as
the whitehall database is large - ~10GB as of Sept 2017.

To make space, first drop your dev VM’s whitehall_development database. Note
after you import the sql, you will end up with a whitehall_production
database.

Extract the downloaded backup

cd /tmp
tar xvjf latest.tbz2

Extract the dump that you want to restore:

bunzip2 latest/foo.sql.bz2

Restore with:

sudo mysql < foo.sql

You will need to provide the password for mysql_root from the hieradata if running the mysql VM

This will restore the contents of file foo.sql to the database name that the
dump was taken from, creating it if it doesn’t exist.

Restore assets from offsite backups

This shows the example process of restoring files for Whitehall attachments.