So I tried an emerge -pe world last night, but it ended up failing on GRUB. However, I thought to myself, nothing depends on GRUB, so it doesn't really matter if it failed to build. Further, I've already got all this stuff installed, so it shouldn't make much of a difference if stuff fails along the way, I want it to go all the way to the end.

Also, I've seen many people in the forums (including myself) that would like to resume an emerge -pe world. So, this morning, I wrote a couple scripts that (hopefully) solve the problem to some degree (and if anyone finds a problem, please correct me here).

This first one builds a list, and then emerges each item on the list, printing the failed packages to a file (/tmp/failedpkg)

Edit: Okay, I'm done re-merging my system. The first script I displayed worked (only two failures on my remerge. Grub and glibc. I guess the new binutils really doesn't work on glibc. I'll have to try the slightly older 2.12 version). So, I tweaked the scripts a little bit. I'll explain the changes from the original after the script. Here goes:

Update: Many thanks to fghellar, who suggested I replace my incredibly ugly method of retrieving the package names from emerge -pe world with something much nicer (I need to retake sed 101 ) (Also, it should be noted, that this is more correct than my version. My old version deleted the extra stuff around the package name, and there could theoretically be stuff in packages that I didn't anticipate (it deleted _beta, but it wouldn't delete _alpha), so you should update your old scripts even if you like ugly things):

Note also that the stuff between the 'FAILNAME' and the for is on one line, even though it wraps.

Continue Edit: Okay, so, it used to be you'd just type in the script name and it'd remerge your entire system. The way it's set up now, you can do that, and it will still remerge your entire system, or you can do:

./blah.sh PACKAGE

And it will only remerge the specified package. Similar changes will be posted shortly for the resuming script (along with a handy resume keyword)._________________They don't have a good bathroom to do coke in.

Last edited by Dolio on Fri Jul 26, 2002 5:11 am; edited 4 times in total

And here's the one we've all been waiting for (Similar code initially):
<EDIT>
This code has now been updated.

Update: Again, many thanks to fghellar for making things prettier and more correct (and probably faster, too):

Update 2: The code below is old. However, I'm about 80% sure it works as advertised. I wrote a new, fancier version of this, and it works from what I can tell. However, I haven't tested it at any length as of yet. So, I'm going to keep the old version up here, and post the new version below. Look there if you want something fancy.

I still haven't tested this one as thoroughly as the other one, so if someone could do that, it'd be helpful. Here's how you use this, let's assume you named it 'resmerge':

'resmerge': starts a new 'emerge -pe world' FROM SCRATCH, EVEN IF ONE WAS IN PROGRESS

'resmerge resume': attempts to resume a previous 'emerge -pe world', and, if not possible, begins a new one from scratch

'resmerge PACKAGE': same as the first one, except it attempts to emerge PACKAGE from scratch

'resmerge resume PACKAGE': same as the second one, only with the specified package.

If someone wants me to fiddle around with the logic at the start, I can, but I'm not really a bash scripting wizard by any stretch of the imagination (I kind of learn as I go along). Possible changes would be to have resuming be the default action, having to supply a parameter to start from scratch, being able to specify more than one package, and being able to type 'resmerge PACKAGE resume' instead of 'resmerge resume PACKAGE'. However, at the moment, things are the way they are because I'm lazy, and this was the (relatively) easy way to go.
</EDIT>

<EDIT again>
I should note that these two scripts now use different files than they originally did. The first script uses PACKAGE.lst and PACKAGE.failed, and the second uses PACKAGE.lst, PACKAGE.tmp and PACKAGE.tmp.2. Thus, you shouldn't run them simultaneously on the same package (I'm not sure why you would do that, but...).
</EDIT>

Note: I haven't tested this one as extensively as the previous one (right now I'm remerging with the 'continue at all costs' script, which is easier to test, as well), because I don't really feel like letting my system run for 3 hours until grub dies and see if the package list gets updated correctly. If someone would like to test this out (or can spot a problem right away), I'm sure we'd all be greatful.
Anyway, I hope this is of some use to people, and that I haven't screwed up the code too badly.

Cheers.

P.S.: How exactly do you use the list and list= tags? I can't figure out what the list= tags are for._________________They don't have a good bathroom to do coke in.

Last edited by Dolio on Sun Jul 28, 2002 2:12 am; edited 5 times in total

J0rd: I think I'm safe from that. Basically, what the scripts do is do an emerge -pe world, does various magic things to the output, and dumps all the package names to a file. Then, it goes through each package name in the file, and does "emerge --oneshot $package" which basically emerges each package by name, in order, without adding it to the world file. So dependencies shouldn't enter into it unless a: you're using the first script to emerge your system/a package from scratch (not a terribly good idea) or b: one of your already installed packages has a dependency that isn't merged, and it won't build (in which case, something is wrong with your system).

These should really only be used when you're rebuilding your system with all your dependencies already intact, and that's really the only time resuming is a problem (if you're emerging, and it cuts off, and stuff is missing, portage knows where you left off). However, I could be wrong, and if someone with more expertice out there knows otherwise, let me know and I'll correct the mistake.

debian: You're welcome.

all: Have a nice day._________________They don't have a good bathroom to do coke in.

Well, I thought about what debian said, and I think he/she/it is probably right about it being a useful portage package. However, I thought the script was a little rough around the edges to be submitted as it was, so I rewrote it to be a little nicer. This is not yet a finished work, as one part that I want to add (combining, I'll explain later) is not yet done. I just wanted to post this to get it out there, so people can maybe give it a go on a couple small merges to see if they can break it.

Note: I'd especially appreciate it if you have something in your system that breaks when emerging. If you don't mind, use this on it, and then see if the script resumes in the proper place (at the package that broke). As far as I know, this functionality has not been tested yet (because as far as I know, I don't have any package lines that break easily), and it's really the most important part.

So, here's how this script (rmerge, courtesy of debian) works (I'll have to write a usage for it too):

<EDIT>
Here's the 'final' code. I deleted the command line option explanation here, because the code now has a --help feature that explains everything, so you can read it there.

EDIT 7/31: I fixed a little bug in this code. There's two places where it occurs. In the remerging loop (if you have the older version) you'll see a

Code:

sed -e "s/$pkg//g" ...

which deletes the name of the package just merged from the list of packages that need to be merged. However, if you have a package named "man" and a package named "man-pages" (as there actually is), the sed for "man" will leave "-pages", which is a problem. The fix is to change the above to

Code:

sed -e "s/^$pkg\$//g"

as I have in the code below (Note again that it occurs twice in the script). This is only a significant bug if you acutally need to resume and packages you haven't merged yet have been mangled by this, but it's probably important, nonetheless.

Some of the code is kind of redundant (lots of copy-paste between the --separate and --combine cases), but I haven't really thought about modularizing it yet. I guess I'll look into submitting this somehow now (maybe then it will get some testing). I guess I might have to write an ebuild now .

Note that the default is now --combine, unlike what I just replaced, where --separate was the default.
</EDIT>

Again, thanks to fghellar, who contributed the much nicer looking sed command for extracting package names, and who also pointed me to the bash articles in the articles section of this site that basically allowed me to write this new, fancy script.

If you see any/find any errors with this script, please post them so I can make corrections. Once I add the final touches, I'll see about submitting it as debian suggested.

Enjoy._________________They don't have a good bathroom to do coke in.

Last edited by Dolio on Wed Jul 31, 2002 6:53 pm; edited 1 time in total

So I whipped up what I think is a satisfactory ebuild for my little script and submitted it for inclusion in the portage tree. The bug id is 5667, in case you want to track its progress._________________They don't have a good bathroom to do coke in.

I figured I should tell you all about this, because it might cause some confusion if you use this script.

I was rewriting the script to be a lot nicer (I think it's done now), as well as testing some of the existing things, and I came across a small problem. It goes like this. When you do emerge -pe world, emerge tells you what packages you need, as well as what versions. However, in some cases, there are newer versions of the packages you need, and when my script strips the package down to the name, that newest version will get installed, instead of the one you really need (which also tends to install other packages you don't want). The only real offender I've noticed with this is Gnome 2, which installs a bunch of bonobo stuff I don't want, even though I have -bonobo and don't acutally have anything that requires gnome 2.

So, there are currently two solutions to this problem. 1: you can use the script as is and remove the unwanted packages afterwards. See this post https://forums.gentoo.org/viewtopic.php?t=6755 for something that should help with that. 2: I could post the new, huge script here for you to use. It has options for solving this problem as an option (the solution is pseudo-dangerous, so it's not enabled by default). I'm trying to get the script added to portage, but it's kind of taking a while (I haven't really seen much activity), and the version I initially submitted didn't have the extra options, so I'm not sure if I'd be able to get the new script in there very fast.

Anyway, take your pick.

P.S.: Note the bug fix I made in the above script. It's sort of important if you want to keep using this version of the script._________________They don't have a good bathroom to do coke in.

None of these scripts work for me. I've also tried the newest version from the Bug Report. It seems like the sed command does not edit the stream at all! I've remerged sed but it did not help. Any idea?

None of these scripts work for me. I've also tried the newest version from the Bug Report. It seems like the sed command does not edit the stream at all! I've remerged sed but it did not help. Any idea?

I've got the same problem. Here's what I get:

Code:

# ./rmerge2 -f -s -e -p world
I would merge the following packages (in order):

#

If I leave out the -p, it does nothing. I'm using rmerge2-0.9.6.tar.gz, and I have created a /var/lib/rmerge2 directory as outlined in the bug page.