Posted: Mon Aug 08, 2011 3:18 pm Post subject: So many new dependencies... (WARN: This is a rant)

Is it a Linux thing or a Gentoo thing?

When I first moved from Slackware to Gentoo oh so many years ago, it was because I was attracted by the idea of a package management system which allowed *ME* to pick which features I wanted on the system, and which ones I didn't...

I had that in Slackware, but I had to compile everything by hand into a DESTDIR, convert it into a tgz then install it using Slackware's rather primitive package manager.

Being able to set useflags and just let emerge take care of all of that was something almost magical - For me, having that choice and being able to roll stuff from source, including, patching and omitting things you want or don't want is what makes Linux awesome.

I feel very far from that ideal right now; The useflags available for actual options for each package (As opposed to asinine configuration 'thingies') has been decreasing, and where you maybe had an option for something, now it's a mandatory dependency.

This has increasingly been the state of things for a while, so what's triggered this post suddenly you may wonder?

I haven't had a chance to --update world for a while (I always dedicate a few hours to it like some sort of ritual as I know it will never Just Work), maybe a month and a bit has passed.

When I did, I suddenly have 97 packages marked [N] which are being pulled in!

What. The. Smeg.

The worst thing is that a large chunk of that stuff I can immediately see is stuff I explicitly Do Not Want, and I know has no place or use on my system.

Some of those 97 are due to asinine package reorganizations (e.g. Samba splits changing AGAIN, all these new 'virtuals' etc.), but many many more are due to existing packages having their dependencies changed in some way, which pulls in some new dependencies which weren't needed before, which then pull in more dependencies ad nauseum. And I'm not even covering the blockers!

Usually when this happens, I just copy the offending ebuild into my overlay and patch it back to how it was, but because I haven't updated in a month I have now been swamped - Now I will have to try and track down which of the few hundred updated packages are pulling in these new dependencies, which is made even more difficult by the horrid presentation of --tree with its multiple duplicate 'darkblue-on-black' entries.

A system update should be a relatively simple affair; Why does it have to be so difficult here?!

If I just blindly let Portage do what it wanted, I suppose, yes, it would be easier, but then why use Gentoo over a binary-based distro?

I'm very tempted to just never update world ever again, and just pull in updated ebuild packages from the tree as and when I want them!
I suspect most of these updated packages offer nothing new anyway, just fixes, which is why this constant dependency issue is particularly infuriating for me!

It's a lot of work to validate the packages with each other, and I think people get lazy over time. I have been bitten a few times when a package did have USE flags to disable a feature and later a package depended on a feature of that package... should that package also be made aware the feature doesn't exist on the dependency? At least portage has evolved to require USE dependencies on packages so things like this don't happen any more...

But it all ends up to upstream (or maintainer) trying to keep the package from breaking no matter how the dependency is built...

I suppose the other issue of having a "unified environment" and ensuring it stays that way is another reason for so many dependencies... Feature bloat upstream... sigh. Gentoo package maintainer to remove feature bloat? That's quite a bit of effort there too.

Must simplify things, use less X11/KDE/Gnome/Audio/video programs... Too many codecs...

If you find that something is not really required, be sure to open a bug - they'll often make it optional. If there's already a bug, post a comment saying that it works well for you too. If they don't want to (because it requires an unsupported patch, for example), bug upstream._________________“And even in authoritarian countries, information networks are helping people discover new facts and making governments more accountable.”– Hillary Clinton, Jan. 21, 2010

I do when I can and it is easy, but it already takes me long enough to hack things back myself; I don't want to have to waste time fighting someone else about it too. The problem (for lack of a better word) is the guidelines for 'offical' ebuilds are a lot stricter than what I'd allow in my overlay (For good reason! ).

It's funny tho' - Since day one I tried to keep dependencies to a minimum to avoid breakage problems (As cross-linking increases, complexity increases - Unnecessary complexity is Bad), but trying to maintain that minimal-dependency state is an incredible amount of work.

It has saved me on a few occasions where people here have gotten bitten by some obscure and unneeded .so breaking during an update, but phew the upkeep sometimes makes me wonder if it's just better to be a sheep!

While I'm here tho', I'd like to give a shout out to ian's 'demerge' utility; It has saved me from horrible horrible breakage several times now, where I've experimentally emerged updates only to find they break. Having an 'Undo' button for portage is friggin' awesome

It's not just upstream; In the past I've been hit by ebuilds updated in place (i.e. no version change) which have had optional deps changed to hard deps because a bug was posted about something in another package breaking without that dep.

hopefully the (somewhat new) USE deps in Portage will help that, just that the Gentoo devs need to insert and test...

and yes I hate those stealth ebuild changes, I suppose they are trying to save a recompile for those with slower machines... not sure if there's a way to automatically update USE flags for existing packages that had an option hardcoded off or on, but the newer ebuild now can enable/disable an option..._________________Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSDWhat am I supposed watching?

I know the feeling. One of my systems is very limited in terms of hdd-space, so to save a few MB, I've got FEATURES="... nodoc noinfo noman ...", thus the files aren't getting installed. This is ugly, because there are a lot of packages in the tree, forcing asciidoc/docbook/... although ready to use manpages are already in upstreams releases.

Removing those deps from the ebuilds works just fine, so I filed bugs, but most of them are still open and unanswered, though many have been correctly assigned.

So I'm spending CPU-cycles building stuff, just to have it deleted in the install phase and still have a few dozen MB of unneeded deps floating around. Bummer.

A few weeks ago, ~50 new Perl-related packages where introduced to the system, yes, it get's worse, imho. I'd love to keep track of all these packages and handle them myself, but I just don't have time for it _________________++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>.

Removing those deps from the ebuilds works just fine, so I filed bugs, but most of them are still open and unanswered, though many have been correctly assigned.

It wouldn't be so bad if it would be easy for the user to remove an unneeded dependency as one can also add user options to ebuilds with EXTRA_ECONF in /etc/portage/env if necessary.
Unfortunately, the corresponding request to add such a possibility to portage was put to death by Jakub's "The user is stupid so hinder him as much as you can" mentality. Maybe the corresponding bug report could be revived

Quote:

A few weeks ago, ~50 new Perl-related packages where introduced to the system, yes, it get's worse, imho.

It is not bad if new perl packages are introduced. In fact, a lot of important perl packages are still missing, so somebody really needing some perl modules might still have to do a lot of manual work. The flooding of perl module dependencies lies more in the (reasonable) upstream attitude to better depend on another module than to reinvent the wheel, so this is hardly something which can be avoided. Moreover, perl modules take practically no resources, so this is almost a non-issue.

I didn't say bad, it's just been a WTF moment, since I didn't install anything new, but still a whole bunch of new stuff popped up.

Quote:

The flooding of perl module dependencies lies more in the (reasonable) upstream attitude to better depend on another module than to reinvent the wheel

Well, it may be reasonale in some cases, but more often than not - at least in my experience - it's because some devs are just to lazy to do somethings themselves. Ie, I once had a python-package doing some reformatting of some RSS aka XML. It imported lots of stuff, which hasn't been on my system already, but as I did some digging through the code, I found that instead of importing stuff and writing a single line of code, it could have also done manually in <10 sloc.

Quote:

Moreover, perl modules take practically no resources, so this is almost a non-issue

On my 256Mb system, every kb counts - at least for me._________________++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>.

I blame pkgconfig needing glib which is retardation, but unfortunately, that's the way things are. At least there is now a replacement for pkgconfig called pkgconf that is smaller and doesn't require glib, but it's not in Gentoo yet

Yeah, but your comment still applies. So much stuff uses glib nowadays that it can't really be considered bloat anymore. Because instead of bits and pieces of it being duplicated in every app, they all use a central library. The only problem possibly are really, really constrained environments.

My issue is - Does it actually need glib?
Why does it suddenly need glib when it didn't before?

The other gripe I have is usually when something suddenly pulls in a new dependency, it very often has dependencies of its own and so on and so on which is what keeps happening to me and sparked the OP!

It's only a Commodore 64 emulator FFS!_________________If ~amd64 ebuilds are cutting edge, then git-9999 ebuilds are chainsaws.
"Not everyone can ride a unicycle, does that mean we should put another wheel on it?" - Lokheed

Making all in build
Making all in data
Making all in C64
Making all in C64DTV
Making all in C128
Making all in VIC20
Making all in PET
Making all in PLUS4
Making all in CBM-II
Making all in DRIVES
Making all in PRINTER
Making all in fonts
Making all in man
Making all in doc
Making all in html
You don't have a working TeX binary (tex) installed anywhere in
your PATH, and texi2dvi cannot proceed without one. If you want to use
this script, you'll need to install TeX (if you don't have it) or change
your PATH or TEX environment variable (if you do). See the --help
output for more details.

For information about obtaining TeX, please see http://www.tug.org. If
you happen to be using Debian, you can get it with this command:
apt-get install tetex-bin
make[2]: *** [vice.pdf] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1
emake failed

Yet, if you extract the tarball and have a look into it's doc/{html}, all the files which would be created are already there. It's not like this documentation looks different when built on a non-standard arch (ie MIPS), so there's absolutely no point in forcing TeX here - again._________________++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>.

That's what I do, and when the point I make is reasonable they usually are receptive, overall if I provide a patched ebuild.

The other day, for example, I found that kipi-plugins wanted to push libmediawiki and marble into my system, which I have no use for. All it took to fix this was a few minutes of tinkering around with the ebuild and a few lines in a bug report.

Easily fixed. Believe it or not, developers can't read your mind. Fixing most of the issues you all describe take much less writing than you are doing in this thread _________________Gentoo Handbook | My website

As I said above, I already filed quite a few dozen, but no, I don't share your experience regarding getting this resolved quickly - if at all.

But in all honesty, in the above example, I just don't get why an ebuild gets in the tree like this anyway. I mean, does the responsible dev try to bump this package, see it failing because of TeX and then just adds it to the deps without even looking if it's really needed? Is this really to much work to do?

It's okay for me, if someone doesn't care, but I'm not sure, if this person is on the right distro. I'm not trying to attack anyone in person, espacially since `equery m` doesn't give anything but the responsible herd in this case, I'm just trying to understand what happened here.

As I said above, I already filed quite a few dozen, but no, I don't share your experience regarding getting this resolved quickly - if at all.

But in all honesty, in the above example, I just don't get why an ebuild gets in the tree like this anyway. I mean, does the responsible dev try to bump this package, see it failing because of TeX and then just adds it to the deps without even looking if it's really needed? Is this really to much work to do?

It's okay for me, if someone doesn't care, but I'm not sure, if this person is on the right distro. I'm not trying to attack anyone in person, espacially since `equery m` doesn't give anything but the responsible herd in this case, I'm just trying to understand what happened here.

Well, it could be because of many reasons. Developers are not all the same and everyone has his/her motivations. Some of them may just be lazy animals. Some others will probably have TeX installed and this would be unnoticed to them. Gentoo packagers do have a very difficult job, and that is making sure that every single combination of USE flags will work on almost every imaginable scenario. That is not always possible and you always have to get a good balance between release dates and amount of testing. Both are on opposite/incompatible extremes. In other distros you choose the set of features, you build the binaries, the user just shuts up and use whatever you give to him. Much easier.

In any case, even people who like to do their job and do it well might not always feel in the mood of testing every single byte they write, and with experience and time the work becomes a bit mechanical. That's where users play their role, just like the critics and the public for a musician or an actor.

I don't know if any of these theories apply, but they are just things to think about._________________Gentoo Handbook | My website

Yeah, but imho Gentoo isn't the place for such behaviour. IMHO, do it right or don't do it at all.

From my pov, that behaviour seems to be an increasing problem. I can understand, that people don't have the time or aren't willing, but imho that's no reason to do a half-asses job. That doesn't work, on the contrary, it sometimes makes it even worse.

IMHO, that's one big problem with Gentoo, but it can't be easily solved, since it's way to hard/complicated/time consuming to fix this. I mean, I could spend some free time once in a while fixing things which annoy me, but I don't have the time to take the burden to get an account for sunrise, which wouldn't help me much anyway, since most of my problems are with packages from the main tree. Becoming a dev takes time and adds presure, which I can't stand up to and proxy-maintaining is imho not really different from filing bugs, still needing someone to look over the work.

Arch's AUR isn't perfect, but it has a much lower 'entrance-fee', so in the end, I'll keep fixing bugs on my machine, using a local overlay and try to keep up. But neither does this solve the problem for others, nor do I have time to fix packages I'm not interested in or I lack the experience and since I'm not the only one with this problem, private overlays are lying around everywhere from hundreds of people/projects - but yet, I can't easily use them, since there might be different ways to solve/ignore a problem and I would once again cherry-pick what I want. All in all, bummer._________________++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>.