A reasonable defense against all of these requires a minimum of three independent copies of the data. One or more of those copies must be powered down (offline) and at a different physical location (offsite). Remember, RAID is no substitute for a backup. Using a modern file system like BTRFS or ZFS with ECC RAM can help protect against bitrot. Surge protectors and UPS are essential protection from power irregularities. However, no protection system can guarantee absolute safety from a lightning strike, so always unplug during thunderstorms. If using RAID, a controller with built in battery backup helps prevent write failures during unexpected power outages.

A reasonable defense against all of these requires a minimum of three independent copies of the data. One or more of those copies must be powered down (offline) and at a different physical location (offsite). Remember, RAID is no substitute for a backup. Using a modern file system like BTRFS or ZFS with ECC RAM can help protect against bitrot. Surge protectors and UPS are essential protection from power irregularities. However, no protection system can guarantee absolute safety from a lightning strike, so always unplug during thunderstorms. If using RAID, a controller with built in battery backup helps prevent write failures during unexpected power outages.

Revision as of 03:50, 28 May 2015

There are two types of data: Backed up data, and data you're prepared to lose.

The process of backing up is dead simple. Deciding what to backup and getting into the habit of backing up is the trickier part.

A reasonable defense against all of these requires a minimum of three independent copies of the data. One or more of those copies must be powered down (offline) and at a different physical location (offsite). Remember, RAID is no substitute for a backup. Using a modern file system like BTRFS or ZFS with ECC RAM can help protect against bitrot. Surge protectors and UPS are essential protection from power irregularities. However, no protection system can guarantee absolute safety from a lightning strike, so always unplug during thunderstorms. If using RAID, a controller with built in battery backup helps prevent write failures during unexpected power outages.

What to Backup

First, decide on a place to keep important data. A directory named "data" or "stuff", or a second hard drive. We'll call this our "big data" directory. Most of your important things can be placed in subdirectories within this, such as:

Photos

Documents

Music

Movies

TV

Books/Audiobooks

Applications

Operating systems (see Install Kits below)

Chinese cartoons

Games

Porn

School files

Work files

Password databases

Wallpapers

Offline code repositories

Android roms

You may have your files in an unsorted mess, such as a catch-all "Downloads" directory. No problem! Make another subdirectory under big data:

Unsorted

Some files can't be moved to your big data directory because they either live in a specific place, or don't exist until you manually create them:

Application and Operating System configurations

Browser bookmarks

Offline email

Video game save files

Things in "My Documents"

To have a copy of these things in your big data directory, you may have to manually "export" or "backup" from the applications they're used with. Some of this you will be able to automate with a script. Don't forget to keep the script in your big data directory too!

You may want to include a datestamp in the file names of these things to keep multiple copies. If your password database becomes corrupt you'll want to restore an earlier version, rather than a perfect backup of the same corruption!

Install Kits

You should aim to be able to go from an unformatted drive to a complete working system, with all applications fully installed and configured, WITHOUT network access. If you need network access to download or activate anything then you don't really own it and don't have it backed up.

How to Backup

With all your data in a central location, you're ready to back it up.

Depending on your needs, investing in a file server, NAS or SAN may be worthwhile. A simpler option is to get yourself an external hard drive of sufficient capacity for you data. Encrypt it if you want.

Once it's mounted:

$ rsync -av --delete /path/to/mydata/ /path/to/externaldrive/

-v being verbose

--delete [optional] delete files on your /externaldrive/ which no longer exist in /mydata/

Keep the external drive unplugged and stored away when not in use. You don't want a power surge taking out your computer and your backup in one hit.

Cloud Storage

Cloud storage is a bad idea because if you:

don't have network access

have a slow connection

have a data limited connection which costs extra for excessive data

lose your password

get phished/hacked

get banned from the service

then you don't have a backup and have lost your data.

Other considerations:

Unencryped backups can be read the cloud provider

Cloud provider could go offline/shutdown without warning

Cloud provider may be in a different legal jurisdiction to you

When to Backup

This depends on how often your data changes. Once a week is a good. Once a month at least.

You may want to run an ad hoc backup when something important changes such as encryption keys or password databases.

Testing the Backup

Most of your backup (except drivers) can be tested in an isolated virtual machine to see if you've missed anything or if some of your install files require internet access.

If you're fairly confident in your backup, wipe your computer and test it. It's better to find out today that something was lost than finding out when it really matters. If this seems too scary, recheck your backed up data until it gives you confidence.