Nice script, well presented and easy to understand. Works fine for me. I haven't been brave enough to wipe my system and try a full restore but listing it certainly seemed fine (no reports of any corruption). Only mods I made were the placement of the backup file on my external iomega drive and to include /usr/portage/distfiles (I wanted this because some packages I had to manually download). One mod I will do is to create the backup on the fast disk and then move it to my iomega as running bzip on that is mighty slow. I'll also use it as a basis for doing some others, like a daily backup of my data and so forth._________________Best Regards,
Jim Bannon
(When in doubt, try honesty!)

I found it usefull, but, correct me if I am wrong, to make the backup, it first creates the tar, and then compresses it right?
I can't afford to duplicate the space of my instalation, so I made some corrections to the script.
First, as you clearly need some files on the excluded directories, and tar wont accept to exclude the directory and include some file, I create a list with the excluded files, on this list I omit the files to be included on the tar. Then I pass the -X option and this created file to tar and modify the script to pipe the result from tar to gzip/bzip2...

PM me if you are interested on the sugested behavior...

Regards._________________The heart has reasons that the reason will never understand.

I have seen bad media or certain brands of cdr's cause the above "Unexpected EOF" type error. I have also seen the file size not be correct when doing a long listing on a linux machine (shows 20Mb instead of 1G) but read the correct file size on a windows machine. The only difference being a change in media to fix the problem and be able to read the tar.gz files on the linux machine. BTW, I was able to extract the needed files from the seemingly bad media by copying the tar balls to an nfs mount from a windows machine to a linux machine. Then I was able successully extract the files.

What type of cdr's is everyone using? I have had good luck with the sony brand.________________________________________________________
Oh how I love Linux, especially on gentoo.

Javoc, what are the files that are being excluded that you say are clearly needed? If they are needed then I would be interested in your modifications. I have not restored using this particular script yet but it appears to be viable. I have a spare hard drive so I think I will try to restore to it some time this week.________________________________________________________
Oh how I love Linux, especially on gentoo.

I found it usefull, but, correct me if I am wrong, to make the backup, it first creates the tar, and then compresses it right?
I can't afford to duplicate the space of my instalation, so I made some corrections to the script.
First, as you clearly need some files on the excluded directories, and tar wont accept to exclude the directory and include some file, I create a list with the excluded files, on this list I omit the files to be included on the tar. Then I pass the -X option and this created file to tar and modify the script to pipe the result from tar to gzip/bzip2...

PM me if you are interested on the sugested behavior...

Regards.

thanks for your pm. i had a similar version once. i don't like the fact of creating first the stage4.tar and then zipping it neither. but i chose transparency and simplicity for the script as others are using it too. give me some time, i'm testing your idea and maybe will change the script. a tar gets so big i almost have no choice ...

slydini wrote:

Javoc, what are the files that are being excluded that you say are clearly needed? If they are needed then I would be interested in your modifications. I have not restored using this particular script yet but it appears to be viable. I have a spare hard drive so I think I will try to restore to it some time this week.

don't worry, i haven't forgotten something. what he meant is the issue about tar not backing up files which are in a subfolder of an excluded folder_________________Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick

I just come accross this script but have no time right now to test it :/
But after reading the thread, I'm wondering one thing.

couldn't it be possible to generate directly backups CDs which include the LiveCD on the first one, so it could be possible to create an automated restauration script? Cos' the HUGE advantage I see with this script, is to recover a fucked up system while travelling and not having Internet access.
So if it was possible to only carry the backups CDs and knows that everything is in there... It would be perfect.

Javoc, what are the files that are being excluded that you say are clearly needed? If they are needed then I would be interested in your modifications. I have not restored using this particular script yet but it appears to be viable. I have a spare hard drive so I think I will try to restore to it some time this week.

As blinkeye said... those files are not left away in the script, it is a matter of implementation... tar does not allow you to use --exclude=<dir> and then try to explicity include a file that exists inside the excluded dir. So right now, the script first creates a tar with these files, and then appends the rest of the backup... simple, but as always it has a trade off, it first creates a full tar of the system (in my case around 8G ) writing it to disk *and* afterwards it compresses it. For me this was not an option as I cannot spare another 8G so I prefered to pipe the tar to gzip and only use the disk space to store the backup until I got to write it to a DVD.

The solution was to create an exclude file with where you explicitly list the files to exclude, and in creating this file you /omit/ the files you want to keep. Not so simple, but a little more efficient.

BlinkEye wrote:

but i chose transparency and simplicity for the script as others are using it too. give me some time, i'm testing your idea and maybe will change the script. a tar gets so big i almost have no choice ...

As you can see my sugestion is being taken into consideration... I hope it makes it to the stable branch _________________The heart has reasons that the reason will never understand.

If anyone else is having problems compressing with the bzip2 method please take the .tar and attempt to manually bzip2 it with the -s (--small) switch. I was having problems not being able to make it past 400mb and currently with the --small switch I have made it to : 1.3gb (almost done). The speed compressing also appears to be the exact same.

I guess this is supposed to be more memory-friendly and not thrash and stress your RAM so much.

I know my RAM isnt faulty cuz i just tested it with memtest86+ not too long ago... and I have plenty (1gig).

-Feld_________________< bmg505> I think the first line in reiserfsck is

If anyone else is having problems compressing with the bzip2 method please take the .tar and attempt to manually bzip2 it with the -s (--small) switch. I was having problems not being able to make it past 400mb and currently with the --small switch I have made it to : 1.3gb (almost done). The speed compressing also appears to be the exact same.

I guess this is supposed to be more memory-friendly and not thrash and stress your RAM so much.

I know my RAM isnt faulty cuz i just tested it with memtest86+ not too long ago... and I have plenty (1gig).

-Feld

please not that the issue about bzip2 and broken bzip2 files has nothing to do with the script.

Feld just dropped me a note and he "confessed" that his bzip2 isn't working - so, the --small switch does no good. i don't know what your problem is - i just tested the script over some of my mp3 files (resulting tar was 25G) and after 2.4GB of the resulting tar.bz2 i gave up. bzip2 is working flawlessly here and obviously is able to handle large files. i suggest you open a new thread about your issue ..._________________Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick

I just come accross this script but have no time right now to test it :/
But after reading the thread, I'm wondering one thing.

couldn't it be possible to generate directly backups CDs which include the LiveCD on the first one, so it could be possible to create an automated restauration script? Cos' the HUGE advantage I see with this script, is to recover a fucked up system while travelling and not having Internet access.
So if it was possible to only carry the backups CDs and knows that everything is in there... It would be perfect.

Just my 2 Euro cents
Ludo

if you backup the stage4 on another media than the to-be-fsck-harddisk you are set. boot to your fsck system with a regular livecd (whatever that might be) and untar the stage4. i thought about the solution of creating a livecd, but it's simply not worth the effort. i don't have a DVD burner yet and a stage4 of <670MB is simply very small. even if i had a DVD burner i don't gain anything if i'd create a livecd with the stage4: you get the same result just burning it on a cd/dvd and using another livecd. someone posted a link about creating a bootable livecd - so, if you really want to do it, just follow the link. i created a custom livecd, so, it's working. but if i fsck my harddisk/sytem i'd want a current stage4, so, i burn the cd/dvd if in need._________________Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick

yes. this is safe to ignore. such messages result from files which have changed/were deleted while tar is backing them up._________________Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick

Last edited by BlinkEye on Tue Apr 12, 2005 10:22 pm; edited 1 time in total

A question for BlinkEye: Is there any way you can eliminate the zip command in your script? What is the purpose of zip? I found that in order to use your script on most of my machines it is nescessary to install zip first. That kind of blows the "out of the box" goal out the window. I really like the script otherwise.

I have some older machines that never get updated at all that I would like to backup with your script but I can't because of the need for zip to be installed. These machines are not connected to the net in any way.

If not, it would be great if someone could create a live-cd that had zip and your script on it.________________________________________________________
Oh how I love Linux, especially on gentoo.

Now it makes sense. After looking at the script it looked like zip was a variable but I couldn't figure out why it was in the command list. So I checked on portage for a program called zip and sure enough there was one. I emerged zip and the script worked fine after that. So the last few days I have been trying to figure how to get that little zip program on my older "never been updated in almost 2 years" gentoo machines.

I should probably note for the beginners out there that this script or any other for that matter will give you the following error if you copy the script from a windows machine without converting the text file:

Code:

# ./stage4.sh
: No such file or directory.

You can use the dos2unix utility to convert any such files to the proper format.

Code:

emerge dos2unix
dos2unix stage4.sh

Not knowing the above caused great frustration for many hours on some of my older machines that were not on the network. I have a 40gig firelite usb drive that I use to Xfer files between machines. Anyways, just somthing to consider.________________________________________________________
Oh how I love Linux, especially on gentoo.

Just wanted to say thanks for an excellent script! I used it to convert my 2 identical hard drives into a RAID 1 array today, without the fear of having to reinstall gentoo again from scratch. It took me a few hours to shift all the data around, and get the disks synched, but it appears to have worked nicely

Damn, I tried to restore and as it was restoring I noticed the leading slashes. So now I am lost as how to get the .tgz file to extract without writing over the currently running Live CD filesystem???? I can't chroot into anything because there is nothing there to chroot too. It's like a catch-22. It makes more sense to have backed up each main filesystem sepearately then use relative pathes so I could restore to the /mnt/gentoo and /mnt/gentoo/boot directory. See what I mean?

If there is another way to restore using the Live-CD and a separate Data CD I am all ears.

Here is the commands I used to restore from my data-CD after I repatitioned and formated:

Damn, I tried to restore and as it was restoring I noticed the leading slashes. So now I am lost as how to get the .tgz file to extract without writing over the currently running Live CD filesystem???? I can't chroot into anything because there is nothing there to chroot too. It's like a catch-22. It makes more sense to have backed up each main filesystem sepearately then use relative pathes so I could restore to the /mnt/gentoo and /mnt/gentoo/boot directory. See what I mean?

no, i don't.

slydini wrote:

Here is the commands I used to restore from my data-CD after I repatitioned and formated:

I actually forgot to put the dot at the end of the tar command in my post to indicate the current directory. I did attempt to cp the tarball to /mnt/gentoo and then extract it but I got the same error. And just think about it a minute if you tar up your system using absolute pathes then / is the same root your running from the Live-CD!!!!!!! So absolute paths don't make any sense at all while booted to a Live-CD. The gentoo stage1-3 tarballs are done with relative paths which makes sense to me. However if there is another method for doing this I am ready to learn.

If anyone has the steps they use to restore using a regular gentoo-live-cd please post your steps. Creating the tarball with this script works great it's the restore steps that need to be clarified.

The error I am getting happens after about 6.5 minutes of extracted files scrolling up the screen then: