If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Benchmarking LLVM/Clang 3.2, GCC 4.8, DragonEgg Compilers

11-09-2012, 11:00 AM

Phoronix: Benchmarking LLVM/Clang 3.2, GCC 4.8, DragonEgg Compilers

Earlier this week I shared some updated benchmarks of the latest development code for LLVM/Clang 3.2 on an Intel Core i7 processor. Now from this same setup to complement the LLVM 3.1/3.2 benchmarks are results of the GCC 4.7.2 compiler, the latest GCC 4.8 development snapshot, and benchmarks of GCC when using LLVM's DragonEgg 3.1/3.2-SVN optimizer plug-in.

Comment

Again these benchmarks are close to worthless, not only does Michael entirely fail to disclose any flags used on the vast majority of tests making them totally useless, but on those few he provides flags he doesn't set a -On optimization level which again makes them useless.

This is a compiler performance benchmark, if you are going to benchmark performance you need to supply optimization levels as there is no point whatsoever to benchmark compilers when they don't optimize which is the default in GCC.

Also, if you want to measure the best performance the compiler can achieve you will use the highest optimization level (-O3 in this case), it's true that in certain cases the heuristics fail and -O2 produces faster code but in general -O3 will beat -O2, and if you only benchmark one optimization level then it should of course be -O3.

For a while it looked like Michael was getting a clue as we saw some benchmark tests where he supplied flags for the vast majority if not all benchmarks and also set -On flags but now he is back to doing these totally useless tests. He is obviously clueless.

In this benchmark only the following have optimization flags set (and can thus be used to draw conclusions from):

The rest can be discarded as totally worthless as there is no optimization flag set and in many cases not even any flags at all declared. These tests do nothing but highlight Michael's lack of understanding at best, wilfull ignorance at worst when it comes to compilers.

Comment

That's really interesting that GCC+DragonEgg is generating such faster binaries than Clang. More than a few cases show GCC+DragonEgg to be as fast as main GCC. That implies that the backend is not a limiting factor (that is, that LLVM optimizations are near as good as the more mature GCC ones in most cases, OpenMP support not withstanding), but rather that something funny is going on with Clang (the frontend) and how it's lowering code into LLVM IR.

And that doesn't sound right to me at all. If it isn't something fishy with the builds of the compilers themselves, this is probably some kind of simple bug in Clang that once fixed should have a pretty serious impact on Clang-compiled binary performance.

Alternatively, it may possibly be a weakness in the LLVM constant folding and propagation, which GCC does a lot of in the frontend to implement things like constexpr (and does far more than constexpr allows, which has been a problem for folks using GCC and then porting to more standards-conforming compilers). I'd be surprised if such a simple set of optimizations was the problem, but a weakness there does propagate down to almost every other optimization you perform, so that could be it.

Short version being that it looks like it should be a straight forward fix once the problem is identified, rather than some major architectural limitation or large set of missing features of Clang/LLVM.

Comment

That's really interesting that GCC+DragonEgg is generating such faster binaries than Clang. More than a few cases show GCC+DragonEgg to be as fast as main GCC. That implies that the backend is not a limiting factor (that is, that LLVM optimizations are near as good as the more mature GCC ones in most cases, OpenMP support not withstanding), but rather that something funny is going on with Clang (the frontend) and how it's lowering code into LLVM IR.

You are confused, the DragonEgg benchmarks where done with '-fplugin-arg-dragonegg-enable-gcc-optzns' as declared on the benchmark information page (first page). This means that it enables GCC's plugin architecture to not only to use LLVM's optimization backend, but _also_ GCC's optimization backend. This is conceptually the 'best of both worlds' as it uses all optimizations from both LLVM _and_ GCC so good results are actually what one would expect, in practice though previous benchmarks have shown poor results so seeing good results from this is now very positive.

Again, since he passes the option '-fplugin-arg-dragonegg-enable-gcc-optzns' it enables the GCC optimization backend _along_ with LLVM's, so there's no mystery.