Cecil asked me to write up some usage notes about the new backup and restore scripts which are making their public debut in R5C7. These scripts were intended to be drop in replacements for the old ones. As a result they should generally be transparent to folks who don't do any personal backup hacking. For later KnoppMyth releases (R5D1, R5E50) please see the updates notes further down this thread.

Features of the new backup and restore scripts:

- A single unified and inclusive backup file for all filesystem stuff.

- Inclusive backup, and Selective restoration of files. This means that the custom version of /etc/X11/XF86Config-4 or /etc/webmin/miniserv.conf that you worked so hard on is automatically included in the backup.

- Multi-level rollover of backup files that keeps up to 10 old backups. You can freely purge these old backups if space is an issue. At the moment I've got a current backup and 6 sets of rollover files which are using a bit over 250Mb. That's roughly the equivalent of 7 minutes of video capacity.

- User customizable supplemental backup and restore lists. Not only is your custom /etc/X11/XF86Config-4 automatically saved by the backup, but you can have it automatically restored as well.

- Reverts to old style restore behavior if no new style backup is found. If this happens you will hear three beeps instead of the usual two.

Note that the asymmetry here is both intentional and purposeful. Having all of your /etc directory available for reference is good, blindly restoring all of it would be bad. This approach gives us the best of both worlds.

Things to keep in mind:

- /root and /home/mythtv should be fairly clean or the backups can get very large. For example, having a Subversion tree under one of these can cause serious backup bloat. This was already true with the old scripts, but keeping a deeper rollover stack makes it that much more important.

- Be careful about what you put in the supplemental backup and restore lists. Both in the size of the lists and their content. For example while restoring ./etc/hosts is probably quite safe, restoring ./etc/fstab is questionable, and ./etc/modules is probably a pretty bad idea. Trying to backup ./myth with these scripts would be amazingly, incredibly, spectacularly bad (because the backup files are written to /myth/backup and it would cause a feedback loop).

- The supplemental list files should be owned by and only writable for root.

- The compression of the SQL dump file done during rollover takes a fair amount of time. Give it a couple minutes before you assume that it's hung.

- Always check the results of your backup! This really goes for any backup script...

- Remember this isn't that magical . If you don't do a backup there won't be anything to restore. If you can't use the "Auto Upgrade" option of the installer, or fake it by creating the appropriate semaphore files after using the manual option, the 2nd stage setup won't run the restore automatically (you can do it manually). If you don't have the /myth partition mounted before entering the 2nd stage install (because you forgot to update /etc/fstab or reconfigure an LVM volume and mount it), the restore script won't be able to find the backup files. These were all equally true with the old scripts.

Preparations for the switchover (from KM version R5B7 or before):

- Make sure you don't have an old "/myth/backup/savedfiles.tar.bz2" file.

- Do an old style backup using mythbackup.

Code:

mythbackup

- Make a new style tarball. Doing this effectively converts an old style backup to a new style backup. Note that without a new style tarball the restore lists described below don't work, because we just run the old script in that case.

Code:

cd /tar jcvf /myth/backup/savedfiles.tar.bz2 ./etc ./root ./home

- Optional! Setup your supplemental backup and restore lists as described below. Remember, for the restore list to work, the script needs to find a new style unified backup, and the objects listed need to be included in that backup.

Using the supplemental backup and restore lists:

The new scripts allow you to specify additional files or directories to be backed up or restored, beyond what is specified by the builtin lists. This is done using two optional list files "/myth/backup/backup.list" and "/myth/backup/restore.list". These files do not exist unless you create them, and then should only contain the names of anything you want to add to the builtin lists for backup or restore. If one or both of the files doesn't exist, no additional entries are added to the corresponding list.

Each of the files is a whitespace seperated list containing the names of any additional objects to be backed up or restored when the corresponding script is run. The convention is to list one object per line. All the entries in these lists should start with a "./" followed by the full path name. This is a traditional safety mechanism which allows you to unpack a root directory based tarball under another directory without unintended consequences.

As described above, the default (builtin) backup list is "./etc ./root ./home". Given those three directories, about the only things I've found a need to add are personal scripts in the "/usr/local/bin" directory. As a result my current list looks like this:

Please note - Because they are already included in the backup by the default list, there is no need to specifically mention any files under the /etc, /root or /home directories.

As described above, the default (builtin) restore list is "./root ./home ./etc/mythtv/modules ./etc/lirc". This is intended to be as close as possible to what the old style backup and restore did. There are a number of additional files which I find it is generally safe and helpful to restore. As a result my current list looks like this:

When it comes to adding things to the restore list, it is far better to error to the side of caution. You can always get the files out of the backup bundle later, using something like this sequence of commands:

Attached is a script called "check_backup.sh" for verifying that a new style backup looks healthy and complete. It provides a fairly high degree of confidence that your freshly made backup really did what was expected and contains more or less what it should. I recommend running it either immediately after you backup or before you reboot to start an upgrade. Hopefully it will help you avoid that sinking feeling at the end of an upgrade, when you realize that your backup failed horribly without you noticing, and there isn't anything you can do about it but cry... If you cut and paste this from here please make sure you don't end up with spaces after the '\' line continuations.

(Note: In R5D1 and beyond this type of checking is done automatically everytime you backup or restore.)

root@mythbox:/myth/backup# sh check_backup.sh: command not foundne 2:: command not foundne 9:: No such file or directorymyth/backup: command not foundne 12:Checking saved filesChecking the integrity of the compressed file...: No such file or directory.h/backup: command not foundne 17:'heck_backup.sh: line 70: syntax error near unexpected token `do'heck_backup.sh: line 70: `for tbl in $(cat $EXPECTED_TABLES) ; do

Nevermind tjc... I figured it out... I originally pasted that script into WordPad and then copied it to my mythbox. I just copied it again, but this time directly into pico and it seems to work now. Sorry about that!

This may be obvious to most, but is there a way to drop the new backup / restore scripts onto an R5B7 install? Seems like it would be nice to be able to at least use this backup prior to doing my next upgrade.

- Boot from the CD as usual, but take option 6 (Exit) from the menu. This wil ask for confirmation then drop you to a command prompt.

- Type mount /dev/hda1 and hit enter. You can now access your root partition on /mnt/hda1. (Note from mihanson: this assumes your installation of R5B7 is on hda1. Substitute sda1 or whatever if you have a non-standard install)

- Multi-level rollover of backup files that keeps up to 10 old backups. You can freely purge these old backups if space is an issue. At the moment I've got a current backup and 6 sets of rollover files which are using a bit over 250Mb. That's roughly the equivalent of 7 minutes of video capacity.

I like this.

Every now and again someone accidently does a restore when they don't intend to.
Your cat playing with remote or walking on keyboard could do it.. etc. And you would
be left a bunch of orphan shows and other strangeness.

An idea for future releases of your backup script could be the option:

"Revert to the previous state"

where a backup is done before every restore that can be used to revert.

I agree with you completely about dangers, but I see a couple problems with the suggested approach.

First it could be kind of time consuming (run time not development time). Second, the restore always works from the "current" backup so doing another one before doing a restore would both turn it into a no-op and play poorly with upgrades where this is most commonly used. Changing this would require substantial re-engineering of something that was intended to so simple that it only has 2 or 3 more "moving parts" than a large rock.

On the other hand, one of the things on my to-do list is adding some kind of safety prompt when the restore is run from the menu. Possibly requiring you to enter the root password or the like. Given the potential for destruction it really seems like something that should be a privileged operation.

I would like to know how to restore "roll-back" to say two backups ago.

More specifically i need to go back until a certain table that dissappeared and another which was corrupted are backed up.

I would like to be able to restore single tables from the backups but a complete roll-back would be ok if I could roll-back everything except the program metadata like the show name and category etc. so i don't have to manually rename orphaned files by guessing.

Who is online

Users browsing this forum: No registered users and 2 guests

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum