Posted: Thu Aug 23, 2012 3:47 pm Post subject: Gentoo is speed ... at least it use to be

There has always been a compromise of having a system that is fast vs one that is stable. As a long time gentoo user (2002, wow I just realized it has been 10 years since I built my first gentoo box from stage 1), the main reason I use gentoo vs any other distro was that gentoo was FAST, lean, and can be optimized for your particular system. Its founding is based on using an experimental fork of gcc (EGCS).

Today I feel Gentoo seems to have lost its speed heritage. Last night I was reading a Phoronix article about GCC 4.7 link-time optimization performance benchmarks and I was amazed of the potential this feature might have. Currently the stable gcc version is gcc 4.5.4 whose base was released over 2 years ago. GCC 4.6.3 is currently marked unstable. Even if GCC 4.6 were to be stabilized tomorrow, we would still be a major release behind with GCC 4.7.

For Gentoo to return to its speed roots, I think Gentoo needs to stay current with the stable release of GCC and have the unstable branch be based on the upcoming branch of GCC. GCC seems to have picked up its game and seems to be approaching ICC in terms of speed. Gentoo needs to better position itself to take advantage of GCC gains.

I know it is easy to spout all these idea and Gentoo is a small project vs Redhat or Ubuntu. I will be the first to admit I am no expert in linux especially in its programing. I am a structural biologist and use computers (even clusters) to figure out protein structures in order to solve many problems afflicting our lives. However, I am far from a linux novice.

I am appreciative of all the work the current and past developers do in maintaining and adding to the Gentoo distribution. I do not plan on leaving Gentoo anytime soon nor do I plan on stop sending in my occasional bug reports. Like many of you, I want Gentoo to be the best distro it can be.

If you are prepared to try out LTO, you might as well add gcc to your package.keywords and package.unmask files or even add the toolchain overlay. Because it is heavily experimental.

Putting the upcoming gcc release into ~arch would mean to break builds for a huge part of the tree on a regular basis._________________backend.cpp:92:2: warning: #warning TODO - this error message is about as useful as a cooling unit in the arctic

This comment isn't intended to argue with anything else in this thread or related threads, just an aside.

A while back Phorinx ran a benchmark which showed Gentoo not significantly faster than other distributions. In particular they were testing on AMD64 cpus, which were pretty new then. Which probably explains the results. Since there was only one or maybe two versions of AMD64 about, tailoring gcc with march=native would actually have no effect. The only way say Ubuntu could compile the code would be with march=athlon or whatever was valid then, and that would be pretty much the same as native. As the hardware platform gets more mature, and therefore more versions arrive (phenom, Intel i7, etc) there's more mileage in march=native. I'm not too sure how granular the precompiled Linux distros go.

Of course, there's mileage in stripping out cruft from the kernel and desktop, which will save on memory and maybe application loading performance. With memory coming in Gb chunks, there's not a lot of paging about, and application loading is, of course, usually a one-time issue and generally factored out of benchmarks._________________Greybeard

Gentoo used to be about speed long ago ~2000-2002 because of some gcc patches that it used by default. Today Gentoo is more about configurability, and in a sense hassle free upgrades.

A recent anecdote. I saw the kde packages from 4.9.0 in portage, and I noticed that the okular release can now annotate pdfs and save the annotations inside the pdf. Sure, enough, all of it "just worked" once I upgraded okular to the latest release. I attempted the same on ubuntu-12.04 - first added a backports ppa, then upgraded the okular package. Sure enough, it doesn't work even after the upgrade. And why is that? They decided to not compile okular against poppler-0.20, and there doesn't seem to be any sane way to get poppler-0.20 and a working ubuntu system at the same time.

Another recent anecdote. I suddenly decided I want samba on Gentoo. Enabled the USE flag, recompiled world, and there it is. Attempted the same on ubuntu-12.04 and guess what? The package is broken on ubuntu-12.04, and it's been so for several months already.

So yes, you can cry for gcc-4.7, but it is useless to me if half of what I try to do doesn't work. That said, there are reasons why gcc can not just be upgraded to the latest and greatest in Gentoo. The health of your Gentoo system crucially depends on it. It takes months and months before the problems with many packages are ironed out. If you want to follow the progress, look at the gcc porting bugs: gcc-4.6, gcc-4.7_________________emerge --quiet redefined | E17 vids: I, II | Now using e17 | e18, e19, and kde4 sucks :-/

As already mentioned. It takes time to test most of the pkgs in the tree that they will work with a new compiler. gcc-4.7 is in the tree and has been going through some tinderbox runs to find the pkgs that fail to compile. There have already been some bugs submitted and fixed for 4.7 compatibility. After they have been fixed they still need proper testing to ensure they actually work, not just compile error free.

Best advice, if you want to test out the newest and greatest. Get a chroot or VM setup to do some testing and help get things fixed so it can make it's way into the unmasked testing stage and subsequently stabilized. When you find it stable enough for your liking, you can migrate your primary system to it._________________Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

Posted: Mon Aug 27, 2012 6:21 am Post subject: Re: Gentoo is speed ... at least it use to be

barureddy wrote:

There has always been a compromise of having a system that is fast vs one that is stable. As a long time gentoo user (2002, wow I just realized it has been 10 years since I built my first gentoo box from stage 1), the main reason I use gentoo vs any other distro was that gentoo was FAST, lean, and can be optimized for your particular system. Its founding is based on using an experimental fork of gcc (EGCS).

Today I feel Gentoo seems to have lost its speed heritage. Last night I was reading a Phoronix article about GCC 4.7 link-time optimization performance benchmarks and I was amazed of the potential this feature might have. Currently the stable gcc version is gcc 4.5.4 whose base was released over 2 years ago. GCC 4.6.3 is currently marked unstable. Even if GCC 4.6 were to be stabilized tomorrow, we would still be a major release behind with GCC 4.7.

For Gentoo to return to its speed roots, I think Gentoo needs to stay current with the stable release of GCC and have the unstable branch be based on the upcoming branch of GCC. GCC seems to have picked up its game and seems to be approaching ICC in terms of speed. Gentoo needs to better position itself to take advantage of GCC gains.

I know it is easy to spout all these idea and Gentoo is a small project vs Redhat or Ubuntu. I will be the first to admit I am no expert in linux especially in its programing. I am a structural biologist and use computers (even clusters) to figure out protein structures in order to solve many problems afflicting our lives. However, I am far from a linux novice.

I am appreciative of all the work the current and past developers do in maintaining and adding to the Gentoo distribution. I do not plan on leaving Gentoo anytime soon nor do I plan on stop sending in my occasional bug reports. Like many of you, I want Gentoo to be the best distro it can be.

If you want to make Gentoo fast, I suggest installing it on ZFS. The Adaptive Replacement Cache algorithm does wonders for performance.

Also, we fixed the main GCC 4.6 stabilization blocker a month or two ago. It will likely be stabilized as soon as the toolchain team signs off on it. Until then, you could keyword it locally on stable. That is what I did.

Unless you run a real critical system, it is no problem to use gcc-4.7.1 with -flto
Sure, many packages (about half of them) will not compile with -flto, but you realize this when you compile them
There is also some packages failing with gcc-4.7, so you should still keep a copy of gcc-4.6 for these. Currently, for me these are firefox and libreoffice. firefox had been fixed with gcc-4.7, but now is broken again, and libreoffice compilation fails nonreproducible at random places - I was trying to nail it down for hours and was surprised that with gcc-4.6 there were not such issues.

Perhaps GCC 4.7.x shouldn't be hardmasked, but IMO, SO MUCH STUFF breaks with it, even without -flto that maybe it should. with all those breakages, it certainly shouldn't be in "stable". IMO, the fact that Gentoo "stable" is quite up-to-date and yet quite stable, but I *CAN* install single unstable or hardmasked packages is one of Gentoo's biggest selling points.

IMO, unless you're on god-awful hardware (which would make using Gentoo a pain unless you're using a binhost) or you NEED insane performance (protip: unless you're a bank or a stock trading company you DON'T), using Gentoo for the speed means you're using Gentoo for the wrong reasons.

Gentoo is about the flexibilitiy and the cleanliness, not the speed (IMO). Sure the speed is a nice bonus, but that's not what it's about.

I have 1,746 packages (including firefox and libreoffice) installed on my laptop, everything against gcc-4.7.1 without LTO. If anything fails to compile, it most probably needs a

Code:

CFLAGS="${CFLAGS} -fpermissive"

in /etc/make.conf.

Ok, I admit it, I am too busy right now to test with LTO, but for now the message is clear:

grep -A 1 lto /usr/portage/sys-devel/gcc/metadata.xml wrote:

<flag name="lto">Add support for link-time optimizations (unsupported, use
at your own risk).</flag>

And finally, the performance benchmarks on phoronix do not really show any benefit of LTO. Quite the contrary. The Dhrystone 2 test show a better result, but the rest are nearly equal or even worse than without LTO._________________systemd - The biggest fallacies

It still is...on this Core 2 Duo, at least. Having spent the last few years running mostly Slackware and Debian, I'm amazed at how much faster some of the larger applications are in Gentoo. I'm assuming it's march=native that achieves this, because I built generic x86_64 binaries in Slackware (for copying to various machines) and obviously Debian's packages are all generic. With monsters like libreoffice it may also be that useless libraries aren't being loaded, while they're all being dragged in by one-size-fits-all builds in other distros. In any case, whatever the reasons, libreoffice in Gentoo launches so quickly I don't get any time to admire Larry on the splash screen...easily four or five times faster than Debian (amd64 testing)...and it remains much more responsive (snappier feeling), as does the whole desktop, especially apps like inkscape and so on.

Oh, and Debian's using gcc 4.7.1. They're probably using it very conservatively, but the point is, my Gentoo ~amd64 systems with no rice at all (just plain old -O2) are a heck of a lot faster than equivalent Debian amd64 testing boxes; and slightly faster than Slackware64. So in addition to the flexibility and ease of maintenance and other advantages that Gentoo enjoys regardless of speed, Gentoo is still faster, too.

Speed is a subjective thing. I noticed that my system gets slower and slower, but I think big parts of it are also caused by the software, not the optimization.
Kernel gets bigger, Desktop gets heavier, of course there's nothing the gentoo developers can do about it.

I tried many distros. And believe me it doesnt take a day to come back to gentoo Even Slackware which is really fast when installing as few as possible packages. (Xorg~300mb) but who wants 1 year old software )

Speed is a subjective thing. I noticed that my system gets slower and slower, but I think big parts of it are also caused by the software, not the optimization.
Kernel gets bigger, Desktop gets heavier, of course there's nothing the gentoo developers can do about it.

No they can't: they spend what little free time they can devote to the project maintaining a vast package tree, some of them doing some rather excellent work on toolchains and crossdev (which beats the pants off any other cross-compiler setup imo), and others usually helping maintain the packages upstream, officially or not. Gentoo gives the best bug-reports, since it's as near to vanilla upstream as possible by convention, and being source-based, the developers also have a lot more expertise with things like autotools, and how to do things the right way wrt patches (such as fixing the problem in the build-system and sending the patch upstream), than most.

Users can however do a lot about the way things are going, if motivated. Examples that spring to mind are Gentoo hardened, which a few years ago was dead in the water til Xake, Zorry, kernelOfTruth and a few others put effort into making it work on their own, and Walter's work on mdev replacement for udev.

Personally I'd love to get rid of consolekit and policykit on my boxen, and just have pam. A mate who's tried it said KDE really didn't like it, but I'd gladly put some effort in if someone who knows the software (I don't, nor do I code C++, but I do know C) wanted to take it on, and ofc they couldn't do something like that alone: they'd need testers and admin/network support, as well as at least one more person to help with code.

Quote:

I tried many distros. And believe me it doesnt take a day to come back to gentoo :-)

Heh, yeah: there's countless "I'm leaving gentoo" threads: standard practice is to say "Good luck, and see you when you come back" and they usually do. :)

-fpermissive
Downgrade some diagnostics about nonconformant code from errors to warnings. Thus, using -fpermissive will allow some nonconforming code to compile.

It essentially makes the compiler less strict about what constitutes a valid program. Sometimes it can be a very good flag, but if you're using it, it can be pretty dodgy unless you know exactly why you're using it. As far as building a Gentoo system goes, I'd be of the opinion that it shouldn't ever be necessary, but I'm a noob, so take that with a mountain of salt._________________CFLAGS=" -O999999"

I just tested activating LTO on GCC 4.7 and I only had to disable it for 72 out of 1800 packages (I run a KDE desktop with quite many multimedia useflags enabled).

But to be frank: LTO is expected to become really useful with GCC 4.8, so my testing is more out of curiosity than anything else. I want to try using GCC with LTO at work, though. If you have a model which runs for a week, 10% better performance can be a lot . That’s also why I tested it at home: I wanted to see it in a safe environment (well, my main home computer… the convenience of Gentoo sure dilluted my sense of “safe”) before hitting it with scientific code _________________Being unpolitical means being political without realizing it. - Arne Babenhauserheide ( http://draketo.de )