Linux: Tape backup for a home network

Sponsored Links

Introduction

In another article, I have discussed why tapes are still the most efficient form of backup. In this article, I detail how I used tapes in a Linux based home network for backup purposes, using a Seagate/Certance STT20000A IDE ATAPI Internal Travan TR-5 Tape drive. There are also several resources listed.

Read below for more ...

Background and Historical Notes

I have been backing up my personal stuff on tape for about 15 years. I used to make copies of my home directory (on UNIX, at work) to a QIC tape that had a whopping 60 MB capacity, running on an NCR TOWER (Motorola 680x0 CPU). Later, I would use 150 MB QIC tapes on the same architecture.

I then migrated to the same drives but on NCR System 3000 (Intel x86 CPU), and had to do byte swapping conversion on the data:

dd if=/dev/rmt/c0s0 conv=swab | cpio -icvmuld

Since QIC drives were expensive, this was out of my reach at home. But there were plenty of those at work, and my stuff was mainly on my machine at work. This high cost factor alone contributed to my being delayed in running UNIX/Linux at home for years.

When I started using Linux at home, I bought an HP Colorado 2.5GB/5GB Travan IDE tape drive. It worked well and I backed up stuff on it when my hard drive was just 2GB. I used several tapes, and backed up every week or two, normally overnight.

The tape drive was recognized as an IDE Tape, so the device was /dev/ht0, so in the dmesg output, or in /var/log/messages you should see something like:

A Real Life Example

In 2000, my home was broken into and the PC stolen! Those tapes came in handy with all my data on them. I only lost about a month's worth of data.

I bought a used Seagate 10GB/20GB Travan IDE tape drive off someone selling it on eBay for about 100$US, including several used tapes. I restored my home directory and everything was good to go. If I had bought another hard disk and copied stuff to it, all my data would have been gone forever! Remember that next time you say "buy another disk!"

All the computers on the home network share the server's drive using either Samba (for Windows clients) or NFS (for Linux clients). Each user is responsible for making sure that their files end up on the server, otherwise, they are not backed up.

I use a set of 6 tapes for weekly backup and 3 for monthly backups, all using cpio. The monthly ones are kept offsite, at a friend/co-worker's house. Even with removable hard disks, you can't justify the cost of 9 hard disks!

The Linux cron facility is used to start the backup in the early hours of the weekend. I even have another cron job the day before to check that the tape is inserted in the drive and that it has been rewind to the beginning of tape. The entire process is unattended, except for an email telling me the backup is done, and to change the tape, or to remind me of the monthly backup.

My backup is about 7 GB now for the home server, which contains my kids homework, my files, digital pictures, CVS repository, ..etc. So there still room on the tapes.

Configuration

In order to configure it with Linux, you need to specify the kernel argument hdx=ide-scsi (where x is the letter of the IDE position for that drive). You can specifiy this argument in the lilo or grub configuration file.

You will know that the tape is correctly installed and that it is recognized by the Linux kernel, when you run the dmesg command or view the /var/log/messages log, and see the following:

First a message that the IDE driver has recognized the drive.

hdd: Seagate STT20000A, ATAPI TAPE drive

Then, you should see the SCSI over IDE layer loaded, and it should recognize the tape as well:

Note the status bits at the end of the output. These are useful, but are often cryptic. Here is an explanation of the most important ones:

Status Bit

Description

BOT

The tape is positioned at the beginning of the first file.

EOT

A tape operation has reached the physical End Of Tape.

EOF

The tape is positioned just after a filemark.

WR_PROT

The tape (or drive) is write-protected. For some drives this can also mean that the drive does not support writing on the current medium type.

ONLINE

The drive has a tape in place and ready for operation.

DR_OPEN

Door is open. Depending on the type of drive, this usually means that the drive does not have a tape in place.

IM_REP_EN

Immediate report mode. This bit is set if there are no guarantees that the data has been physically written to the tape when the write call returns. It is set to zero only when the driver does not buffer data and the drive is set not to buffer data.

If you do not have a tape cartridge inserted in the tape drive, you will see:

DR_OPEN IM_REP_EN

If you have a tape cartridge in the tape drive, but the write protect tab is on, you will see:

BOT WR_PROT ONLINE IM_REP_EN

In order to rewind the tape, do the following:

# Rewind the tapemt -f /dev/st0 rewind

For actual backup, I do not use dump since it only dumps one filesystem, and does not cross the boundary. Therefore I will need two tapes to backup my system, which is kind of a waste. I could actually make the backups back to bak, but this complicates things if I am looking to read info from the tape, specially in the panic of a recovery situation. It is best to backup everything using cpio, with relative pathnames, and then restore whatever you need to wherever you need, even if partitions are different.

Therefore, I prefer to use cpio. Even tar is a bit primitive compared to cpio, since it has many nice features, such as preserving the modification time.

The -H crc option specifies that the backup will be in a portable format that can be read on other UNIX machines, and not only Linux.

If you are doing a full system backup, then you have to exclude some directories, such as /proc, /dev and /mnt.

To list the tape contents, you can do:

cpio -itv -I /dev/st0

To restore the tape contents, you can do:

cpio -icvmuld -I /dev/st0

Check the appropriate manual pages for the mt and cpio commands for more details (links below in the Resources section). You may also want to check the dd manual page as well, since you can do some neat tricks with it.

So, I will stay with tape for the forseable future, until optical technology catches up with disk size increases.

Resources and Links

Thanks to LinuxQuestions, the manual pages for Linux commands are available online, in HTML format. Here are the pages for

How to use your tape drive under Linux from the Swiss Federal Institute of Technology in Zurich. This is a good tutorial on tapes and Linux. Although it assumes you have SCSI and DAT drives, much of it will apply to IDE tapes using SCSI emulation. Ignore the parts about ejecting the tape, changing density and compression, and everything will work.

White Paper on DLT Tapes on Linux systems. This document discusses the use of Digital Linear Tape (DLT) under Linux. Although their stated goal in the document is about DLT, most of the document applies to any other tape device on Linux. It details how to use mt, tar, cpio and dump.

Thanks very much for your clear explanation of the tape backup procedure. I happened to have a SCSI STT20000 tape drive and running now SUSE Linux Enterprise Server 9. During my Novell 5 time I used Backup Exec, but this works great also. Your input on Internet was very much appreciated! Lost quite a bit of data recently; this will prevent a second time.

A better choice for creating the backup is to add the -depth flag to find. This allows read-only directories to be restored. (A directory is written to tape after its contents. So, the contents of a directory are restored and then the directory permissions are restored).

I strongly suspect that tape is still the best way to go. At least it is in my case, but then not everybody treats their home/SOHO operations like a professional software development facility.

Also, I don't believe in retiring equipment just because there's a sale at the discount market; and that goes for being coerced to upgrade by software vendors. If it ain't broke, don't fix it and if it does as it should, don't upgrade/replace it.

If you follow the latest trends, you hardly have time to configure everything before somebody is telling you that you need new equipment. Thank God they're not upgrading hammers and screwdrivers at every heartbeat -- we'd never get anything done.

Thanks, Khalid, I hope that your wisdom survives, with you and with all who have heard you.

Hi Khalid,
Thanks for the solution you have given here, but I would also want to know that how we can take incremental backup and how we can append the data by using a tape drive.
Since i am using a LTO3 tape drive and i have to perform daily backup.
It will be great if you will let me know to perform incremental backup and how to append data in the tape drive.
Since I have tried using "tar" command and it was not working.
Reply on my mail ID if possible.

I am using to backup Linux a software called Dmailer Backup Software , their software is free and their online storage service is also free but only up to 3gb .
But anyway what i like at it because its easier to use than other softwares and also much faster.