Just to make sure I'm on the right track here (I got a little confused between the two scripts emwrap.sh an tcupdate.sh)

I just did an emerge sync and found that there is a sys-kernel/linux-headers update. So if I want to update everything properly should I follow the steps below yanked from the first page of this post?

Code:

gcc-config -l # make sure of gcc before beginning
emwrap.sh -set
emwrap.sh -seb # We rebuild the TC and then the rest of the system
emwrap.sh -se # doing the 2nd system emerge minus the TC
emwrap.sh -wet # rebuilds TC against clean system
emwrap.sh -r # first build of the world files against clean TC system
emwrap.sh -r # second world rebuild and done

I have two computers (call them comp1 and comp2) running Gentoo, with the same MARCHs, gcc versions, etc. The only difference, in fact is that comp1 and comp2 differ slightly with respect to the world packages installed, with a large part common (lets call them world1 and world2)

Here's my plan to nail down the tool chains and rebuild the system/world packages on the two machines (without having to repeat steps on both of them):

On comp1, I have buildpkg as part of FEATURES, and I do exactly what the first page of this thread recommends:

Code:

gcc-config -l # make sure of gcc before beginning
emwrap.sh -set
emwrap.sh -seb # We rebuild the TC and then the rest of the system
emwrap.sh -se # doing the 2nd system emerge minus the TC
emwrap.sh -wet # rebuilds TC against clean system
emwrap.sh -r # first build of the world files against clean TC system
emwrap.sh -r # second world rebuild and done

On comp2, I then do these steps:

Code:

//copy the packages built on comp1 into the {$PKGDIR} of comp2
emerge -ek world
emerge --oneshot --nodeps <all packages in world2 not in world1, listed in the right order of dependencies>

Please let me know if what I plan to do is correct, and if not, please suggest what I should do instead.

Pseud That should work. If you grab sys-build.sh and on comp1 when your through do " emerge world -ep>basesys-build.lst " and on comp2 Edit sys-build.sh in "function get_build" where it says " wrld_lst system " and change "system to world " ==> wrld_lst world ".Then with sys-build on comp2 use your comp1 basesys list and run it twice. It should auotamatically filer out comp1s list from comp2. Oh and you can use -p with sys-build.sh to test._________________An A-Z Index of the Linux BASH command line

I've had a Jackass! installation going quite nicely for awhile now, and after reading several threads on the matter, decided to go ahead and upgrade to GCC 3.4.4. I also decided to use prelinking, so I read up on that, followed the steps, and then started up emwrap. First, I assume that -N works the same as it does in emerge? The help portion seemed to indicate so. In any case, I got linux-headers and glibc updated when I had to kill it. I figured the -c option would bring it up where it left off. Not so. It wants to re-do those. Is there any reason for this? I checked the versions, and they updated just fine. There's no reason I can see to do this. I don't mind all that much, as it's only another 1.5 hours or so, but it seems a bit strange.

Also, I read earlier in this thread that sys-build is still in beta, or something to that effect. Is it recommended to use it in addition to emwrap now, or is it not required?_________________Who needs reincarnation when you've got parallel universes?

I have noob question/or confusion about the topic, but then maybe it's a bit OT. I do alot of scientific programming, nothing on the scale of developing operating systems and most of it just needs to be useable for a few people, so it's a diferent kind of thing entirely than real package development. Still, I have alot of libraries (my own and third party) that are used by alot of diferent bits of code and such. Even so, so long as I don't link anything statically (or at least before I do), and so long as all the header files are in place and up2date, it's not gonna make 2 cents differnce what order I compile my object files in. They don't know about the other binary files anyway, just the headers. Then I have a bunch of object files... and I'm lazy and old fashioned and don't need to worry about space much anyway so I usually just statically link stuff then.

Now I can see if one code statically links a library and then the library changes then that maybe could problems if it needs to handle things the same way as some other code, but I thought pretty much everything in linux is dynamically linked these days. I can imagine that maybe recompiling the compiler itself is a diferent issue, but I have trouble seeing why world and even system packages excluding the tool-chain need to be built twice.

Anyway... I see 3 issues
1) recompiling the compiler maybe
2) Statically linking before the libraries get rebuilt (which I wouldn't think would happen)
3)... or just plain not having the latest headers in place because they haven't been downloaded and insalled by emerge yet (which surely could be solved by implementing some ebuild function to just install headers...)

EDIT: Actually I originally thought the whole idea of the emwrap.sh script was to make it so I could emerge the toolchain twice without needing to emerge to the whole world or system twice and so that after doing that I could emerge the rest of the world or system JUST ONCE without emerging the toolchain yet again in the wrong order anyway. Wow, if you really need to emerge world twice, then it seems like it would be easier to mount the system from another live distro or something, unmerge pretty much everything but the toolchain.. and then just re-emerge it all from scratch once.

Is it possible, just for my curiosity, to clear up my confusion at least in some handwaving way in a couple of sentences, or do I just have way too much to learn to understand this problem?

Ok this script started because of Rob_moss, during the early days gcc-3.4 testing, when it was still in CVS. Since he was being very conscinces about what built and what worked correctly after building by gcc-3.4 he was doing the famous " emerge system -e && emerge system -e && emerge world -e && emerge world -e ". This was to insure that any bugs were form gcc-3.4 and not screwed up libraries or binaries.

The other reason was, I wanted finer controll of emerge. While playing with the onlliners I also found that I could skip over "circulaur dependencies " and come back to them latter. I was also interested in combineihg a bunch of oneliners into something more usefull.

As to whether this script is needed or not ? Do you need to need faithfully rebuild your TC and all files after words no. Most of the time, 98% to 99% in my experince, if your running stable or testing an " emerge world -uD " works fine. Throw in a " emerge world -e " quarterly or bi-annually and your system should run fine. There is a big BUT in this. To take full advantage of gcc-3.4 you should rebuild everthing and if your going to do that then you might as well rebuild the TC in the correct order. So for most people emwrap.sh -suDb and a following emwrap.sh -wuD is fine._________________An A-Z Index of the Linux BASH command line

OK thanks, that helps make me much less paranoid, which I greatly apreciate. Don't get me wrong, I see the script as useful if nothing else just so I an isolate and build the toolchain before rebuilding everything else. And clearly that much is useful(at least since I know that includes the compiler). I'm even not shocked that the toolchain needs to be built in order, obviously the headers should be installed first at least, and if there's no way to pull in headers without emerging first then I'm not shocked a few more headers come in with the other parts and they thus need to be done in order too (although I'm curious if just doing an emerge --fetch wouldn't have the same effect...but maybe that doesn't really INSTALL the headers where they need to be; I still wonder if there's not a way to get around actually compiling in order... but then why not). I can also see that compiling the compiler with an updated compiler could be good. So toolchain twice, in order, I wont question.

As for building the rest of the world twice though... I wonder if this is just legend from some over-paranoid(no offense) guy who wanted to make sure no-one had any case for acusing him of causing incidental bugs, or can anyone actually do a checksum comparison or something which actually proves that some binary somewhere in the rest of the world actually does come out diferent the second time around? If so, can we trace down PRECISELY why for at least one example, and if that turns out to be just an issue of installing headers, can we see if an fetch or something would solve it? The world packages were only compiled once the first time and it all worked just fine, and it seems portage knows what order to compile them when it updates or remerges. I'm just a bit concerned that alot of people are getting overly worried about something that might be more myth than fact. I've seen posts that people are more worried about this issue than the fact that portage loses track of packages you have installed and updates them unecesarily because they are longer in the official tree (for example).

Note:
To do my propsed checksum experiement, the toolchain would have to not be touched between world compiles and/or it shoudl already be proven to be PERFECTLY (bit for bit) stable under rebuilds by checsum testing rebuilds of it. IE when the rest-of-the-world is rebuilt twice, I want to be dead sure the thing doing the rebuilding hasn't changed before trying to see if the results of the rest-of-the-world build change. Under those conditions, if someone can really prove that the-rest-of-the-world builds come out diferent the second time... that would be kinda nice to know. I'm not claiming this hasn't been done either. I wouldn't know.

Anyway, as I said, I'm really glad to hear I don't need to be sooooo paranoid about it. The rest is mostly just accademic curiosity. I'm not trying to say work on these scipts is useless or anytying either, just wanted to see if I should get all fanatic about it. Glad to hear that I should not. Thanks heilvc
.

As to whether this script is needed or not ? Do you need to need faithfully rebuild your TC and all files after words no. Most of the time, 98% to 99% in my experince, if your running stable or testing an " emerge world -uD " works fine. Throw in a " emerge world -e " quarterly or bi-annually and your system should run fine. There is a big BUT in this. To take full advantage of gcc-3.4 you should rebuild everthing and if your going to do that then you might as well rebuild the TC in the correct order. So for most people emwrap.sh -suDb and a following emwrap.sh -wuD is fine.

THANK YOU. I'd always wondered if running it so many times was actually necessary, since my system seemed perfectly stable._________________Who needs reincarnation when you've got parallel universes?

oh and stpanohvich, as for the -N option. Maybe I can answer that. I suggested it be added awhile back. I think heilvc was kinda like "why not, I'll throw it and maybe it will work right" I'm sure you can find his exact words a few pages back or in another thread or something. Anyway. I did scour over the script myself for awhile before requesting it and my conclusion was that the -N could indeed just be passed through and it should work like in emerge. I'm sure heilvc knows his own script well too and wouldn't have added it if he didn't think it might work. Anyway, the short answer is yes... it is just passed to the emerge command when deciding which builds to update and thus works just like in emerge.

Ive been sort of busy recently. Anyway as dagurasu said -N works. What dosent is -v. My filters wipe out all the output.

The point Im trying make about updateing your TC is play it safe. Do it when you have time do the full double double in case you get bit. My behaviour has changed . I use to just dive right in now I plan it a little. Ive given up thinking that the alatest greatest TC in going to make my Athlon=xp 2K frun like a optron 3500 _________________An A-Z Index of the Linux BASH command line

What have I missed? This is a dual-Opteron box running ~amd64 in case it makes a difference. I wonder if the parsing of line 165 in sys-build.sh is going wrong (bash 3.0-r12 is the latest of three installed versions). The line is:

Code:

for e in $(< build.lst);do

Up to now I've been running this much simpler script whenever I thought the tool chain needed a rebuild:

Its not the bash version. Im running the same on my box. I t might be Saturday before I have a chance to look at it but could you past your " emerge system -ep "? If your wondering about the $(< ...) that is just a builtin for "cat file".

No, seriously, I can't tell you because I have a stability problem at the moment which causes seg faults whenever I compiile anything big and heavy, such as glibc or gcc. Actually, last night it happend with binutils, so it isn't just big beasts that cause it. The upshot is that I have to emerge --resume from time to time, and of course quite a lot of compiling tasks are repeated._________________Rgds
Peter

I decided to "killall boinc" before running my one-liner (BOINC is a distributed-computing client which crunches numbers at nice=19). The script ran to completion! First time for several months that anything more than a trivial emerge has finished in one go.

wstn ~ # time tcrebuild
... (much logging snipped)

real 113m58.019s
user 83m43.994s
sys 52m19.423s

tcrebuild is what I've called my version of the script, which I quoted before:

So it seems to take a little under two hours, or about 2.3 times your 45 minutes
I have MAKEOPTS="-j5" in /etc/make.conf to make sure there's always something for each CPU to do._________________Rgds
Peter

What does 1066 represent? The year of the Norman invasion of England?
No...
The number of lines in my own, thoroughly revised, completely re-worked, not-necessarily-better-but-definitely-more-readable version of emwrap.sh.

I decided to start modifying the emwrap.sh text, because I thought the formatting was so horrible. It really breaks about every best-practices convention for writing readable, maintainable code. I was also disappointed that the "continue" option of emwrap.sh really meant the equivalent of the "resume" option of emerge, and the "resume" option of emwrap.sh was something completely different. The way that the build process could be broken into pieces---although a good idea---seemed to be implemented in a more-confusing-than-necessary manner. You may very well disagree. I also completely re-worked the "help" page, making it truly huge and descriptive (and I therefore also added a shortened "usage" page). Finally, I don't like naming executable shell scripts with a ".sh" extension. Why differentiate the script from any other command? Commands don't have extensions, so my personal belief is that shell scripts (and Perl scripts) shouldn't have extensions. Does the end-user care in what language the program/command is written?

This script adds some new functionality for compiling Tool Chain elements. Namely, if gcc is recompiled, then immediately after the emerge of gcc, gcc-config is run to switch all subsequent compilations to the new compiler (See the 2005.0 Stage 1/3 NPTL by Bob Predaina). ***This is completely untested in this script!*** It may completely bomb, it may have a minor bug, I do not know, because I haven't had the opportunity to test it. I have only used this script to update system and world components. So, if you use this for TC components, be very careful, and let me know what happens! Thanks.

Here it is:

EDIT:
Due to problems copy-pasting code, please get the code from the link graciously provided by dol-sen on the next page (page 10), or on the first page (thanks hielvc!).Failing that, I'll put the code here, but if the following file-integrity tests don't match, you may be experiencing the "copy-paste problem"! (And, unfortunately, after several versions, it no longer has 1066 lines... )

Code:

#!/bin/bash
#
# EMWRAP
#
# emwrap - A wrapper for 'emerge'. Type: $ emwrap --help
#
# 23-Aug-2005: * Extracted the 'emerge' from the SYSTEM+WORLD and the TC
# Version 4.2 updates---to eliminate a lot of common code. Also, fixed a
# WBMII bug in the selection of the GCC version with GCC-CONFIG.
# Thanks "sfp-a7x" (from the Gentoo Forums)!
# (There were a *lot* of modifications made for this version!)
#
# 18-Aug-2005: * When removing system packages from the world file in
# Version 4.1 "create_emerge_package_lists", I was sorting the contents
# WBMII so that I could use 'comm'. But I cannot re-order all the
# files---dependencies matter!
#
# 11-Aug-2005: * Completely re-coded "emwrap.sh", using several key ideas
# Version 4.0 from that script, to create this script, "emwrap".
# WBMII
#
#_______________________________________________________________________________
# ---------------------------------------------------
# Comments from the original---"emwrap.sh"---version:
# ---------------------------------------------------
#
# Date 06-24-05 Removed the "v" option to emerge, and added clean_up to
# remove dir created by emwrap Ver3.0.4
# Date 06-14-05 Version 3.0.2 fix function clean to clean up failed and sys
# list. Also added gcc-config and binutils-config to tc_filter.
# Date 05-08-05 Version 3.0.1 changed TC emerge || die, the old one didn't
# die. Also added -N.
# Date 05-02-05 Version 3.0.0 all functions
# Date 01-19-05 version 1.8.6 Hiel Van Campen
#
# This program is distributed under the terms of the GPL version 2
# Parts contributed by Ed Catmur <ed@catmur.co.uk> ;^), the parts that work,
# the help setup.
# Use at yee own risk. It works for me, but then I wrote it.
#
# Toolchain thang
# emerge linux-headers glibc && emerge glibc binutils gcc && emerge binutils gcc
#
# The build script for the edited wrld.lst-->build.lst. Anything that fails to
# emerge is copied to failed.lst To view the build.lst or failed.lst use
# "less failed.lst" or your favorite pager.
#
# Version 2.0.5 , 3-30-05
# Now does sytem builds and then world minus the system files big time saver
# remove -o option since it redundit, added emptytree system and world were
# the world build does not include the system files. Rewrote help info.
# 4-6-05 Cleaned up where files that are created are put, and worked on fetch opotion.
# 4-23-05 Added -c for continue, aka resume, works with all functions. Thanks dol-sen
#_______________________________________________________________________________

These ${GR}$me${Gr} options are passed through to 'emerge' (all leading hyphens are
removed, so "help" is the same as "--help" and "-h"). See the 'emerge' MAN
page for descriptions of their functions:

${CY}-f, fetch${Gr}
${CY}-u, update${Gr}
${CY}-D, deep${Gr}
${CY}-e, empty[tree]${Gr}
${CY}-p, pretend${Gr}
${CY}-N, new[use]${Gr}
${CY}-r, res[ume]${Gr} Resume the 'emerge' list from where it stopped.
To use, re-run the *exact same* command, but add the
"r" option. It will start from where it stopped,
without having to rebuild everything before. This
is analogous to the "--resume" in emerge.

The three '$me' build classes are unique to ${GR}$me${Gr} (at least one must be
specified, but any combination is allowed):

${CY}use, usage, opt[ions], <no args>${Gr} Displays this Usage/options reminder.
${CY}-h, help, man${Gr} Displays the huge help page.
${CY}nc, nocol[or]${Gr} Turns off all colored output.
${Mg}
IMPORTANT: The definition of SYSTEM and WORLD are therefore *different* here
than they are in 'emerge':

The Tool Chain (TC) is the group of packages that controls how the entire
system is built. It is composed of the following packages:

binutils[-config], gcc[-config], glibc, linux-headers, and portage.

If the Tool Chain (TC) is built using '$me', it is built in a special
order, and several packages are built twice. This procedure insures that
critical TC packages are built against the newest versions of the other
TC components.

'$me' checks if the TC is scheduled to be updated and if so, allows several
options for handling the TC packages. It uses an edited list of packages
generated by "emerge --pretend uD/e system/world", with the TC packages
removed, so that TC packages are considered seperately from all the other
system/world packages. The TC packages may then be 'emerge'd seperately,
or they may be ignored while all the remaining packages are 'emerge'd.

'$me' has two primary uses:

1. Do a system/world update *without* updating any TC packages that have
updates available. This allows you to keep your system current, while
maintaining a stable set of TC components upon which to build your
entire system.

2. Test if there is a TC update in "emerge system/world -uD", and if there
is, allow you several options for rebuilding your TC and all the
remaining non-TC packages.

The first use above (1) will be used most frequently. When you *do* want
to update your Tool Chain, the second use above (2) will allow you to
rebuild the Tool Chain packages first---in an optimal order---and then
optionally rebuild all the non-Tool-Chain packages of either the SYSTEM
or WORLD class.

The flags are chainable and determine whether you do a system or world
emerge, and whether you do a "deep update" emerge or an "emptytree" emerge.
There is also a "pretend" mode, which behaves the same as the pretend flag
in emerge. It is *strongly* recommended that you run your desired command
with the "-p" option first, and *if* everything looks good, *then* run the
command again without the pretend flag.

Remember, only *you* can prevent forest files! ;-)

Options:

These ${GR}$me${Gr} options are passed through to 'emerge' (all leading hyphens are
removed, so "help" is the same as "--help", "-h", and "h"). See the 'emerge' MAN
page for descriptions of their functions:

${CY}-f, fetch${Gr}
${CY}-u, update${Gr}
${CY}-D, deep${Gr}
${CY}-e, empty[tree]${Gr}
${CY}-p, pretend${Gr}
${CY}-N, new[use]${Gr}
${CY}-r, res[ume]${Gr} Resume the 'emerge' list from where it stopped.
To use, re-run the *exact same* command, but add the
"r" option. It will start from where it stopped,
without having to rebuild everything before. This
is analogous to the "--resume" in emerge.

The three '$me' build classes are unique to ${GR}$me${Gr} (at least one must be
specified, but any combination is allowed):

${CY}use, usage, opt[ions], <no args>${Gr} Displays a Usage/options reminder.
${CY}-h, help, man${Gr} Displays this huge help page.
${CY}nc, nocol[or]${Gr} Turns off all colored output.
${Mg}
IMPORTANT: The definition of SYSTEM and WORLD are therefore *different* here
than they are in 'emerge':

'$me' flag: 'emerge' equivalent:
-------------- --------------------
${MG}-ts ${Mg}=>${MG} system
${MG}-tsw ${Mg}=>${MG} world
${MG}-t ${Mg}=> <only TC packages>
${MG}-s ${Mg}=> <SYSTEM packages, but *not* TC packages>
${MG}-w ${Mg}=> <WORLD packages, but *not* TC packages
${MG} ${Mg} and *not* SYSTEM packages>
${MG}-sw ${Mg}=> <WORLD packages, but *not* TC packages>
${Gr}
The reason for the new definitions of "system" (doesn't include
TC packages) and "world" (doesn't include TC or SYSTEM packages)
is to give you maximum flexibility to compile each of the three
groups of packages separately. In reality, though, you will
most likely just use ${Cy}emwrap -uDsw${Gr} in place of the usual
${Cy}emerge -uD world${Gr}, to update all your packages---but not touching
any Tool Chain packages. Then, when you want to spend the time
to upgrade the TC, you can split the workload by first updating
all the TC packages with ${Cy}emwrap -et${Gr}, and then rebuilding all the
SYSTEM and WORLD packages with ${Cy}emwrap -esw${Gr}. (Or you could just
do it *all* in one fell swoop with ${Cy}emwrap -etsw${Gr}.)

# By default, assign the terminal color-codes:
# (If the "--nocolor" command-line option is found, then they
# will be cleared using "clear_terminal_color_codes", and the
# "--nocolor" flag will also be passed through to 'emerge'!)
set_terminal_color_codes

function get_all_tc_packages {
# For efficiency, use 'awk' rather than one giant 'grep' command:
# (Also, more specifically match with the RegEx to not get unintended
# packages (see comment at top of file) and added portage.)
# Use the Regular Expression variables declared at the top of the file:
awk "/$RE_hdrs|$RE_glibc|$RE_bin|$RE_bincfg|$RE_gcc|$RE_gcccfg|$RE_portage/"
}

function remove_all_tc_packages {
# For efficiency, use 'awk' rather than chained 'grep -vE' commands:
# (Also, more specifically match with the RegEx to not filter-out
# unintended packages (see comment at top of file) and added portage.)
# Use the Regular Expression variables declared at the top of the file:
awk "!/$RE_hdrs|$RE_glibc|$RE_bin|$RE_bincfg|$RE_gcc|$RE_gcccfg|$RE_portage/"
}

# If the user chooses a SYSTEM emerge, then the packages will be in the
# *exact* order of the 'emerge system' results.
#
# If the user chooses a SYSTEM + WORLD emerge, then the packages will be
# in the *exact* order of the 'emerge world' results.
#
# If the user chooses a WORLD emerge, then the packages will be the
# world-only (no system) packages, in the order they were in the
# 'emerge world' results.

{
# Read a line from "world_and_system_list.txt", and if it is *not* in the
# "system_list.txt" file, append it to the world_list file:
while read file_line ; do
# Hopefully, no meta-character will cause a problem:
if ! grep --line-regexp "$file_line" system_list.txt >/dev/null 2>&1 ; then
echo "$file_line" >>world_list.txt
fi
done
} < world_and_system_list.txt
}

function check_for_tc_updates {
# Test for TC packages:
linux_headers_pkg=$( grep $RE_hdrs world_and_system_list.txt )
glibc_pkg=$( grep $RE_glibc world_and_system_list.txt )
binutils_pkg=$( grep $RE_bin world_and_system_list.txt )
gcc_pkg=$( grep $RE_gcc world_and_system_list.txt )
#
# Even if there are newer versions of the FOLLOWING packages, they will
# *NOT* trigger a message telling you that there are TC package updates
# available and they will not be updated until an actual TC update is
# performed. The logic is, that although they are critical components,
# they alone do not justify a new TC build.
#
binutils_config_pkg=$( grep $RE_bincfg world_and_system_list.txt )
gcc_config_pkg=$( grep $RE_gcccfg world_and_system_list.txt )
portage_pkg=$( grep $RE_portage world_and_system_list.txt )

# In the case of SYSTEM and WORLD 'emerge's, this will be a FILENAME
# containing packages to merge. In the case of TOOLCHAIN 'emerge's, this
# will be a single package name. Which is it?:
if [ -s "$packages" ] ; then
# The argument is a FILE, put its contents into "package_list":
package_list=$( cat $packages )
else
# The argument is a single package name, put it into "package_list":
package_list="$packages"
fi

function emerge_tc_updates {
# Variables local to this function:
declare -i return_code=0 # Hold return status.
declare -i count=0
declare -i num_packages=0
declare tc_list
declare package
declare chost # The value of the CHOST in "/etc/make.conf".
declare gcc_current # The number of the current GCC (from "gcc-config").
declare gcc_version_number # The version number of the new GCC pkg.
declare gcc_regex
declare gcc_for_gcc_config # The gcc version handed to gcc-config.
declare loop_counter
# A global variable in the scope of the function call to:
# "emerge_packages_from_this_list" (because if I use the
# return code as a return value, it can only be <=255!):
declare -i successful_emerges=0

# Now that the Tool Chain list has been constructed---containing the
# appropriate list of Tool Chain packages to 'emerge' in the proper order
# ---I need to fill the "$gcc_pkg" variable with the name of the gcc
# package (even if it wasn't a package that originally needed updating).
# This is because *other* packages (glibc for example) will require gcc to
# be 'emerge'd! So, just fill *all* the package variables with their
# values from an 'emerge -pe' (since I no longer need their values as
# *flags*, as I used prior to this point):
linux_headers_pkg=$( grep $RE_hdrs alltools_list.txt )
glibc_pkg=$( grep $RE_glibc alltools_list.txt )
binutils_pkg=$( grep $RE_bin alltools_list.txt )
gcc_pkg=$( grep $RE_gcc alltools_list.txt )
binutils_config_pkg=$( grep $RE_bincfg alltools_list.txt )
gcc_config_pkg=$( grep $RE_gcccfg alltools_list.txt )
portage_pkg=$( grep $RE_portage alltools_list.txt )

function emerge_system_or_world {
# Variables local to this function:
declare -i num_packages
declare -i num_packages_total=0
# A global variable in the scope of the function call to:
# "emerge_packages_from_this_list" (because if I use the
# return code as a return value, it can only be <=255!):
declare -i successful_emerges=0

The 1066 lines, and my ultra-newbie bash-scripting abilities are keeping me from finding the very-likely-to-be-trivial bug, so please do it for me
Oh, and why does it say 1070 lines although I havent added any redundant newlines?_________________eschew obfuscation

Last edited by Pseud on Fri Aug 12, 2005 5:24 pm; edited 1 time in total

I just copy-pasted the code from my browser, and I found that the code above has four extra blank lines that are not in the original text! Line numbers 355, 899, 902 and 1057 should not be blank (if all four of those lines are blank lines in the source that you copy-pasted, then just delete them---but do it in reverse order so the line numbers don't change! i.e.: 1057, 902, 899, and 355). The only other discrepency that I see is that my original source has a blank line as the final line (I just think it makes the code easier to read), whereas the copy-pasted code I just did had no blank last line.

But, importantly, the copy-paste version that I just created still works fine (with the minor exception that the new blank line number 355 is inserted in the middle of the "./emwrap help" text).

So, my first suggestion would be to copy-paste the text again, and see if it changes. I'll edit the above text to remove the extra blank lines that were somehow created when I pasted the original text.

But do this: Make sure the code has a blank line as the last line of the file, and make sure that the source has exactly 1066 lines:

Code:

$ wc emwrap
1066 5470 45332 emwrap

and then I get the following MD5SUM:

Code:

$ md5sum emwrap
1f962cf917b7bbc1036698cdeb5b150d emwrap

If re-copy-pasting it doesn't work, maybe I can e-mail you a copy of the source, and we can figure-out where the problem is being introduced. I'll go make the edits to the above source now.

OK. I think I fixed all the discrepencies! I copy-pasted the above code twice, and both times it was an exact match.

I found that my original source had seven lines that had a trailing space. Since that obviously changes the character-count and the md5sum, I fixed that in my original. Now a copy-pasted version of the above source exactly matches my source code.

After you copy-paste the above code (make sure that there is one blank line after the "exit 0" line at the end), make sure that you get these exact results:

Code:

wc emwrap
1066 5470 45332 emwrap

Code:

md5sum emwrap
1f962cf917b7bbc1036698cdeb5b150d emwrap

Let me know if there are any more issues in getting the source. I have no idea why copy-pasting the source into the Gentoo Forum would have introduced extra blank lines... It would obviously be best if I could put the code somewhere to be downloaded. My code uses only spaces for indents, so the copy-paste should---theoretically---yield exactly the same source as the original. When I downloaded the emwrap.sh source (from the link on page 1), I found that the code was riddled with tabs---but completely inconsistently! Some lines used spaces to indent, some lines used tabs! In my book, that's just inexcusable. What coder doesn't check the code by turning-on invisible characters? Oh well...

Missed your post by a few minutes I guess, but this might be useful with the debugging:
I removed the four extra (blank) lines and added one blank line at the end like you suggested and the code still had the same issue. But the interesting this is, wc gives a different byte-count:

Code:

$ wc emwrap
1066 5470 46298 emwrap

So my guess is this: some spurious white space is also getting introduced, which is causing sytax errors (as bash is very intolerant of cavalier white-spacing).

Ok, I tried out the modified code, but again have the same problem. My wc byte-count is the same as my previous attempt (ok 1 more, but that's negligible)

Code:

$ wc emwrap
1066 5470 46299 emwrap

I notice that each line (in the forum post) has a trailing blank character (just select a portion with your mouse and check now ) ... is this how you intended it to be?
The reason I'm asking is that the difference in our byte counts is 966 ... which suggests that all of it is probably trailing blank spaces ...

[EDIT]
Well, following my hunch, I removed all trailing blank spaces, and got an exact match of both your wc and md5sum outputs, and emwrap -h worked ... so the fix is: just remove the trailing blank spaces from the forum post.
Oh and btw, thanks for your effort with the script--much appreciated!
[/EDIT]

[EDIT:2]
In case anyone wants to know how, I did the above removal of trailing blanks using Kate, my (favourite) text editor, using the Find-replace dialog that accepts regular expressions (so I asked it to replace the reg-ex " $" with ""). Of course, there should be a one-liner with sed or awk that does this, but I've procrastinated on learning them both till now, so I dont know
[/EDIT:2]_________________eschew obfuscation

That is very strange... This must be a function of the browser and/or OS. I see a blank character too, at the end of a line, when I select some text, but this character is converted into a newline when I paste the text into my editor (Vim). I guess we can assume that you are using Linux (since this is a Gentoo forum! ) I'm using Firefox as my browser, and I'm selecting all the text and opening Vim, going into "insert" mode, and pasting the text using my mouse's middle button. I think I remember hearing about trailing spaces before when people copy-pasted text from the forum... Just out of curiosity, if you remove all trailing spaces, do you get the correct 'wc' and 'md5sum' results? You could use the following perl one-liner:

Code:

$ perl -pi -e 's/ +$//' emwrap

What are the results after that?

Bruce.

EDIT: Another user said that he had leading spaces on all the lines. If---and only if---you have leading spaces on all the lines, this will remove one leading space from every line that begins with one or more spaces:

Code:

$ perl -pi -e 's/^ //' emwrap

But don't do that if you don't need to, because it will screw-up the indentation on an otherwise-good file (and the md5sum and wc won't match)!_________________BillyBear: I'm scared Poncho.
Poncho: Bullsh!t. You ain't afraid of no man!
BillyBear: There's something out there waiting for us, and it ain't no man. We're all gonna die.
<ominous music>
<enter Hillary Clinton>

Last edited by maguire on Wed Aug 24, 2005 4:22 pm; edited 1 time in total