Posted: Fri Dec 26, 2003 4:46 pm Post subject: The right use of lib64 directory name on Gentoo amd64.

I have been trying to get kdevelop to work om my amd64 and stumbled upon a problem I can't really solve without some feedback.
It turns out qt and kde install their libraries in /usr/qt/3/lib and /usr/kde/3/lib. But when I emerge kdevelop, the configure script detects my /lib64 and promptly decide that qt should be in /usr/qt/3/lib64 and informs me that I lack libqt.

I want to fix this, but the question is, is qt or kdevelop in error?
My qt is build as a 64bit library. But changing this into lib64 would probably break numerous ebuilds who havn't got a clue about 64 bit systems. OTOH, letting it install itself in lib will remove the very reason they first invented the lib64 convention, i.e. letting 32 and 64 bit libraries and applications live happily side by side.

An easy horribly ugly quick fix is just to create a symbolic link lib64 -> lib but this this would lead down to the long winding road of dll-hell.

Are there any knowledgeable people out there who have a good opinion on this matter? What have been decided for Gentoo? Should we use lib or lib64 for our 64-bit libraries?

gentoo is unable to comply with the new filesystem standard, and is not really multilib-aware. fixing this problem would require heavily editing portage itself, as well as checking every single ebuild one by one to see if it needs fixing.

the problem is this: the assumption is made that libraries will be in /lib for the main system architecture, regardless of bitdepth. the new FHS, with recent additions for multilib architectures, does the sane thing and states that 64bit libraries should be in lib64, with 32bit libraries in lib. that way, compatibility is maintained... shit, if gentoo was FHS compliant you could even install pre-compiled binaries from a 32bit install without problems (with libraries anyways... the binary in $PREFIX/bin would overwrite any previously installed 64bit binary). since gentoo isn't a binary dist this isn't as much of a plus... but as you can see, there are other issues as well. another one being that you cant have 32bit and 64bit versions of important applications that need it without jumping through hoops of fire and sacrificing your first born child....

a quick hack to get around some issues with compiling 64bit apps is to use the symlinks mentioned earlier. it's not so bad to use a hack when you take into consideration what would be needed to truely fix the problem:

1) fix portage so that there are no more hardcoded locations for dependency database location (not just /var/db/pkg but also /var/cache/edb and a few other misc details)
2) fix portage so that these db locations are chosen according to variables with a sane default of using the standard locations
3) fix portage to understand that there are CFLAGS/CXXFLAGS/LDFLAGS that should be mandatory for certain architectures, regardless of what you have yours set to or what gcc wants to do by default (for example, compiling 32bit apps on amd64 requires the -m32 option /everywhere/... also, 64bit apps need shared libs compiled with -fPIC which shouldn't be a problem but is worth mentioning anyways)
4) fix econf so that there is an optional filename suffix (configure allows for this, and in our case it would be very useful to have 32 as a suffix for 32bit installs... so mozilla is 64bit, and mozilla32 32bit. yaay, no name clashes or overwrites!)
5) fix econf so that lib directories can be changed via variables, with the sane default being FHS compliant and NOT the old default behavior! (we want 64bit libs in lib64, etc)
6) add an option parsing macro to portage that sets the above variables, with the main architecture as the default setting (64bit on amd64, for example). have this set proper database location with 32bit behavior being the same as the old behavior. have this set econf settings in a sane way.
7) rewrite every ebuild that runs configure directly to use econf
8 ) fix every ebuild broken by this change
9) rewrite every ebuild that doesn't make use of a configure script to do this checking
10) rewrite every ebuild that makes assumptions in install location in their package install functions
11) figure out how to handle environment variables and config files (especially for applications that load plugins of some sort that might have 32bit and 64bit versions installed.... if the config of one overwrite the other it might become unusable)
12) figure out how to handle headers installed by 32bit and 64bit apps that are installed at the same time
13) fix portage to be aware of both dependency trees AT ONCE
14) fix portage to be able to uninstall files properly somehow (a problem that comes to mind is this: 32bit version of app installs files. so does the 64bit version, installed later. after uninstalling the 64bit version, these files will be gone even tho the 32bit version needs them. but if you have awareness of both dep databases and let the files stay, uninstalling the 32bit version later on STILL wont uninstall those files because their timestamps will be different)
15) rewrite portage to not have hardcoded config file locations in any way whatsoever (like /etc/make.conf and /etc/portage/package.unmask)
16) add logic to portage to use seperate config files for each arch setting (for example, you probably dont want -march=pentium4 for your 64bit apps, but for your 32bit ones this enables things like SSE2, which the athlon64 supports.... there is no march option for 32bit code on a k8. you might also want different USE settings entirely, for example if you dont want to compile a complete feature-rich 32bit env and just want the bare minimum for an app or two)
17) fix all the gentoo specific apps broken by the above changes, like gentoolkit and ufed
18 ) remove the current compat stuff for 32bit......
19) rewrite all ebuilds that cant be done in a dual way so that they are forced not to be. this includes grub and lilo, which cannot be 64bit. this also includes kernel modules which MUST be the arch default. also, the base system will need to be redone in a way that avoids having a 32bit version of baselayout and other basic system deps without completely destroying the 32bit dependency tree
20) make glibc install correctly in all situations for all archs...
21) rewrite things like java-config and the opengl config app so that they're aware of the above changes so that you can choose a 32bit java without screwing up your 64bit apps, or so that you dont screw your system hard once nvidia supports 32bit apps in their 64bit drivers
22) fix multilib gcc, which currently cant compile a 32bit grub even if told to
23) figure out how to make multilib OPTIONAL for the large number of people that do not want a multilib system, and do not want to be forced to compile a multilib gcc that depends on both a 64bit and 32bit glibc, etc. note that if the default gcc is multilib, that increases the size of stages... you need 32bit glibc to have a multilib gcc. this might bother a few...
24) make things that depend on scripted language interpreters work without crazy dependency hell. a script should run the 64bit version, even if installed as a dependency for a 32bit emerge, and not install a 32bit interpreter
25) make it possible to use a 32bit interpreter instead of a 64bit one if you so choose... which means it'd have to be installed without the 32 suffix to work at all.
26) make emerge world -uDe disregard all of the above consideration of sane defaults ENTIRELY and instead figure out how to rebuild two dependency trees that might cross-depend on each other, using world files from both 32bit and 64bit trees.
27) make emerge system not work in 32bit mode, or somehow make it fundamentally different....
28 ) make gcc appear in both dependency trees if installed as multilib, just the 64bit one if not multilib, and still make it possible to install a 32bit version of gcc (this could be desired for various reasons..... the least of which is grub not compiling in 32bit with 64bit multilib gcc, as i'd hope this would be possible in the future. )

the list goes on. it pretty much would require a massive rewrite of almost all of gentoo. much of what i list can probably be done in a better way, but it'd still require some massive re-tooling of gentoo. much more so than i originally thought when I got the bright idea of tinkering to make gentoo a properly multilib capable system...

the simpler option if forcing a 32bit and 64bit build of everything is very very undesirable, and in many cases just would not work. (a previous proposal of doing this by gentoo dev 'tester' was denied)

I think the only option might be to get all the needed functionality into portage-ng, and then wait for a gentoo to be based on it in a handfull of years... maybe. tho it's possible longhorn will be out before then. heh ^^;