The script is effectively just a wrapper to emerge (apart from needing to use -i to install to world.) It always runs an emerge -pv first so you can see what is about to happen, and any USE or SLOT changes (this is of course recommended practice.)

Using it on its own as update (or by shortcut eg update -s to sync first) means it will emerge -uD --changed-use world (or @system @world if you're on portage-2.2), then check GLSAs, depclean and revdep-rebuild. It will then run the config-updater if needed (one of: etc-proposals cfg-update dispatch-conf or etc-update - you can set your own in /etc/update; they are searched in that order, but just putting in configUpdater=/path/blah will override.)

It works with --jobs/-j in config, or at command-line. For single or toolchain packages, or if you're not using --jobs, or for revdep-rebuilds, haskell and python updates, you'll see that it makes it much easier to read einfo and ewarns. (This is useful if you want more output than --jobs, nicely filtered.) It also deals with failed packages by leaving them til the end and retrying. Any packages in the list which would need a failed package are also deferred til the end, when another pass will be made over all failed pkgs. It stops when either all the packages have been successfully emerged, or when no packages could be emerged in the pass.

This version has list and USE editing in conjunction with a dialog-based list editor. USE editing is only for single-file package.use and it complains if the pkg (or the slot being emerged) is listed twice. This covers my uses fine; we will add multi-file/multi-line soon, it's just not something I've needed. There is also special toolchain handling when you use -e. I've used this a couple of times already. It also does unmasking based on portage output, prompting you to confirm an unmask of required packages (which you can then unmask just for that version, or the slot/package name.)

We've been working on getting ABI upgrades automated, and it works fine. It picks up whenever a package needs a revdep-rebuild and also has a config file (/etc/warning) for more complicated cases like libpng. Since I have been advocating binhosts for a while, it seemed like a good time to make sure it worked ok. This script will deal nicely with binhosts across the web. (We needed this to test the ABI upgrades in chroot without constantly recompiling; it really is nice :D It's really odd to watch portage install packages as quickly as rpms on other systems.)

You just set PORTAGE_BINHOST and PKGDIR in make.conf (see man make.conf for info) Eg:

is what I would use for amd64. http://tinderbox.dev.gentoo.org/default-linux/ is the officially-unsupported one with less packages, but more secure compilation options.
Then you run with -g or -G. You can also put binHost=1 or binOnly=1 in /etc/update to do this on every run. See the new help for more details (update -h --help and -h config are three separate options.

You can also specify additional binhosts in <dir>/binHosts -- default dir is /root/emerge. It will use the portage binhost first, then check the hosts listed in the binHosts file, one per line. Thus we can use the gentoo tinderbox by default, and fall back to the experimental one for desktop or missing packages. We removed the genpkgindex support, since many web binhosts don't generate the file; we may add this back later. update -h config

Critical packages are protected from removal, see criticalPkg in update -h config; pre and post etc actions are new (testing appreciated, as ever.) If you want to add custom error handling, see update -h error (blame drescherjm ;)

It's been in development since January 2007, and it's been tested to death ;)
We've also debugged it on a couple of clean installs over screen, as well as in a chroot. It really does make life easier.

< input redirector <(..) process substitution as input to the while loop. Took me ages to get that out of #bash ;) It's the only way to avoid a subshell and still filter the output, afaik. It's used much more later in the script in installPkg.

edit: it's process subst, not cmd subst which is $(cmd) which is the new (approved ;) way to do `cmd` (nests better, yadda yadda..)
edit2: an example of nesting (thanks to twkm on #bash) is:

edit3: It turns out this is most likely due to running script with 'sh update'; when run as sh BASH goes into POSIX mode aiui; it doesn't support all the bashisms that the script needs. So if you aren't sure about chmod u+x'ing it, run it with 'bash update'.

Last edited by steveL on Tue Jul 03, 2007 2:57 pm; edited 1 time in total

This is output from a recent update (October 2011: upgrade of texlive from 2008 to 2011) to show unmasking behaviour, with a large set to unmask.

Note that I only have to confirm the unmask of one package, dev-texlive/texlive-latex, for it to also unmask all dependencies. Additionally since I used 'p' to always unmask the package or slot, not its version, there is nothing more to ask me.

The changes so far planned are to give full dependency information without two pretend emerges, switch to Bash 3.2
regexes, better CTRL-C handling, and some more sensible handling of blockers:
<dev>: I think it'd be great to have an optional feature to not unmerge until the new package is built when dealing with blockers.
Not sure if that can be handled by our script, but I'd like to add some sort of binary handling, and I know my coder mate has got an algorithm somewhere to handle unmasking interactively which a user suggested. It wouldn't take much to add a USE flag lookup, I'm sure.

So we're thinking to add that stuff; any other things you think should be added, please post in this topic, as we're going to start work on the next version soon.

Minor bugfix as it wasn't handling no-rebuild packages (showed up as php insisted on rebuild due to apache use flag being obsolesced, and i had only built it the day before..) Thanks to jakub and kloeri for bugfixing python

* Please download jdk-1_5_0-doc.zip from
* http://javashoplm.sun.com/ECom/docs/Welcome.jsp?StoreId=22&PartDetailId=jdk-1.5.0-doc-oth-JPR&SiteId=JSC&TransactionId=noreg
* and place it in /usr/portage/distfiles named as
* jdk-1_5_0-doc-r1.zip. Notice the r1. Because Sun changes the doc zip file
* without changing the filename, we have to resort to renaming to keep
* the md5sum verification working existing and new downloads.
*
* If emerge fails because of a md5sum error it is possible that Sun
* has again changed the upstream release, try downloading the file
* again or a newer revision if available. Otherwise report this to
* http://bugs.gentoo.org/67266 and we will make a new revision.

And after downloading the file (using -r to resume; note you could add -v or -q to r):

Code:

~ # update -r
resume: This will attempt to emerge all packages listed in /root/emerge/emergeList.
Installing to world
Targets: java-sdk-docs.
[ebuild N F ] dev-java/java-sdk-docs-1.5.0-r1 45,109 kB
There were fetch-restricted packages..
About to proceed (N to cancel)?

Man sorry about that; it's working fine on my stable install, but looks like the kind of thing you might get on Bash 3.1. (Regex changes were the major problem for getting the thing working in 3.2.) I've posted an updated version that should work okay on 3.1 as well (i had to add yet another regex variable at line 560.. I really don't like what they did to regexes in 3.2 </rant>)

If you still get errors please post them (as I'm curious anyhow) and try updating to 3.2 with emerge.

Er just to let you know there may be a minor bug with toolchain handling; I'll get it fixed tonight. I'm having a weird error with samba, passing LDFLAGS incorrectly, but it's a good example of the error handling:

Code:

~ # update
Emerging -uDN world then running revdep-rebuild.
You will be asked for confirmation; CTRL-C to abort at any time.
Resolving...

** net-fs/samba-3.0.24-r2 failed to install.
No packages compiled successfully on this run, aborting..
Run revdep-rebuild?
About to cancel (Y to proceed)?
Configuring search environment for revdep-rebuild

Checking reverse dependencies...

Packages containing binaries and libraries broken by a package update
will be emerged.

!!! SAMBA make everything error
!!! If you need support, post the topmost build error, and the call stack if relevant.
!!! A complete build log is located at '/var/log/portage/net-fs:samba-3.0.24-r2:20070521-204355.log'.

Skipped app-office/openoffice

You'll see it carried on with the other packages, in this case dev-util/git, checking that samba wasn't a dependency that git needed first. Then at the end, it retried samba which failed again. Since no packages were installed on that run, it stopped, and asked if I wanted to run the revdep (since there had been an error.) It did that, then printed out the last part of the error log for samba. If multiple packages had failed, it would have given me a list of tail commands to run to see the same output for each package, based on its logfile. (If you have PORT_LOGDIR="/path" in make.conf; I have it set at /var/log/portage which is standard.)

That's plenty for me to report a bug with (as well as my emerge --info of course :)

Done; it's on line 919 in the new version (and you'll see on l 957 that i changed it to use a path lookup rather than the /usr/sbin directory.) I added a regex so you don't see the message on every ebuild (unless you use -v of course.) There's a coupla bugfixes too.

Quote:

Thanks for the good work

Thanks for the feedback; cfg-update looks nice too, never tried it before :D

Just a minor tweak, as portage was being downgraded due to a bug that doesn't affect me USE="-*" destroys USE_EXPAND variables and according to the ChangeLog, 2.2.8 fixed some other bugs. So I added an environment variable check for $SKIP, which can be used to specify additional packages to skip on a temporary basis.
You can either use:

Code:

SKIP=sys-apps/portage update
update -r

Or, if you notice a package and want to skip it with resume you can do so eg:

Code:

update
export SKIP=sys-apps/portage
update -r

assuming that you cancelled the first update on seeing the downgrade message. Note that if SKIP is set at the time of resolving eg SKIP='foo/blah baz/bar' update then any resume will not need the variable set. You can also set it as above, after the initial resolve, but it will then need to be set for every resume (hence the export).
Please note that SKIP is only for temporary fixes and is not recommended as a general solution; the nc (no compile) and nr (no rebuild) arrays at the top of the script are for more permanent changes.
There was also a minor fix to the cfgUpdate regex to catch both types of message.

Bugfix to sort out blocker handling; we haven't had a block in ages so we didn't realise it wasn't working Anyhow fixed now, and sorted out SKIP so you can actually skip more than one package eg with SKIP='cat-foo/bar cat-baz/qux' update.
Good time to show how blockers are handled (basically it's up to you whether to continue, skipping the blocked package.) Hopefully for the next release we'll have the blockers sorted out automatically.

Code:

# update
Emerging -uDN world then running revdep-rebuild.
You will be asked for confirmation; CTRL-C to abort at any time.
Resolving...

** Downgrading app-portage/eix-0.8.8 - 2 of 10.
* As of >=eix-0.5.4, "metadata" is the new default cache.
* It's independent of the portage-version and the cache used by portage.

** Upgrading to net-misc/neon-0.26.3 from 0.26.1-r1 - 4 of 10.
Downloading neon-0.26.3.tar.gz
(112.64 KB/s) - `/usr/portage/distfiles/neon-0.26.3.tar.gz' saved [789289/789289]
* Elibtoolize: neon-0.26.3
Patching:.done
* Enabling SSL library thread-safety using POSIX threads...
* There are new features in this version; please beware that
* upstream considers the socks support experimental. If you
* experience test failures (eg, bug 135863) then try rebuilding
* glibc.
* Neon has a policy of breaking API across versions, this means
* that any packages that link against neon will be broken after
* updating. They will remain broken until they are ported to the
* new API. You can downgrade neon to the previous version by doing:
* emerge --oneshot '<net-misc/neon-0.26'
* You may also have to downgrade any packages that have already been
* ported to the new API.
Completed compile

I've missed out some packages that had nothing interesting of course. It's a good way of seeing what's going on in the tree imo, stuff you miss when there are thousands of lines of compiler output. Note that you can do something similar with portage's bashrc and iirc the MAKE var, but it's not as nice imo (although it is something to explore) since you direct to /dev/null and lose the log aiui.