Re: How to compile apps to bc files with the new llvm-gcc4?

Unless I'm missing something, the problem lies directly with the fact that you are trying to do a link-stage operation with the GCC frontend. GCC, by default, probably runs "ld" or another system linker, which grabs the executable from binutils. This linker hasn't been modified (yet) to include support for .bc files, but you're compiler is emitting byte code to the *.o files. That is why the linker barfs, and tries to interpret as a script.

I am actually planning on working on integration between binutils' ld and LLVM to allow just this functionality. I'll be working on this over the holidays, and hopefully will get something working by mid January. Until then, directly using the LLVM linking facilities to handle the bytecode is probably the best if not only option.

-Chandler

PS: I'm still kinda new, so someone else may come along and correct me, but that's the best I can figure out...

Re: How to compile apps to bc files with the new llvm-gcc4?

Chandler/Domagoj,

On Sun, 2006-12-10 at 10:53 -0500, Chandler Carruth wrote:
> Unless I'm missing something, the problem lies directly with the fact
> that you are trying to do a link-stage operation with the GCC
> frontend. GCC, by default, probably runs "ld" or another system
> linker, which grabs the executable from binutils. This linker hasn't
> been modified (yet) to include support for .bc files, but you're
> compiler is emitting byte code to the *.o files. That is why the
> linker barfs, and tries to interpret as a script.

Correct. In a discussion with Domagoj on IRC, it turns out that he
actually doesn't want to link a native executable. He just wants to get
a single linked .bc file for each program. llvm-gcc3 used to do this
because it linked with gccld which produces a .bc file and a shell
script for running that bc file with lli. I suggested to him that he
change his makefiles to simply link the llvm-gcc4 produced bytecode
files with gccld as did llvm-gcc3.

> I am actually planning on working on integration between binutils' ld
> and LLVM to allow just this functionality. I'll be working on this
> over the holidays, and hopefully will get something working by mid
> January. Until then, directly using the LLVM linking facilities to
> handle the bytecode is probably the best if not only option.

We look forward to LTO in binutils. This will be a great addition to
LLVM (and binutils) that will make using bytecode more transparent than
ever.

>
> -Chandler
>
> PS: I'm still kinda new, so someone else may come along and correct
> me, but that's the best I can figure out...

Re: How to compile apps to bc files with the new llvm-gcc4?

Reid Spencer wrote:

> Chandler/Domagoj,
>
> On Sun, 2006-12-10 at 10:53 -0500, Chandler Carruth wrote:
>
>>Unless I'm missing something, the problem lies directly with the fact
>>that you are trying to do a link-stage operation with the GCC
>>frontend. GCC, by default, probably runs "ld" or another system
>>linker, which grabs the executable from binutils. This linker hasn't
>>been modified (yet) to include support for .bc files, but you're
>>compiler is emitting byte code to the *.o files. That is why the
>>linker barfs, and tries to interpret as a script.
>
>
> Correct. In a discussion with Domagoj on IRC, it turns out that he
> actually doesn't want to link a native executable. He just wants to get
> a single linked .bc file for each program. llvm-gcc3 used to do this
> because it linked with gccld which produces a .bc file and a shell
> script for running that bc file with lli. I suggested to him that he
> change his makefiles to simply link the llvm-gcc4 produced bytecode
> files with gccld as did llvm-gcc3.

Here's where my issue about DTRT with llvm-gcc (or whatever your prefix
to llvm tools happens to be) resurfaces. I'm keenly aware that Chris
thinks that one really ought to be careful with having llvm-gcc emit
bytecode. But frankly, if that's the case, what's the point of emitting
byte code in the first place?

It might be a good interim step to hack the frontend so that llvm-ld
gets invoked (and I'm aware of problems like passing '-gcse' to the ld
phase when the frontend collects the arguments.) Even if emitting
bytecode only happens with "-O4", you still run into the same problem at
link time. Unless you override LD in your makefile or some other hack,
which, BTW, doesn't necessarily work either.

>>I am actually planning on working on integration between binutils' ld
>>and LLVM to allow just this functionality. I'll be working on this
>>over the holidays, and hopefully will get something working by mid
>>January. Until then, directly using the LLVM linking facilities to
>>handle the bytecode is probably the best if not only option.
>
>
> We look forward to LTO in binutils. This will be a great addition to
> LLVM (and binutils) that will make using bytecode more transparent than
> ever.

While that would be cool, it still remains that they potentially aren't
going to be installed in the right place for gcc to pick them up, or
worse, you get mismatches between the distro's concept of binutils vs.
llvm-gcc's concept of binutils. That becomes the headache that I suspect
we're all trying to avoid.
_______________________________________________
LLVM Developers mailing list
[hidden email]http://llvm.cs.uiuc.eduhttp://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Re: How to compile apps to bc files with the new llvm-gcc4?

Hi,

On 12/11/06, Scott Michel <[hidden email]> wrote:
> Here's where my issue about DTRT with llvm-gcc (or whatever your prefix
> to llvm tools happens to be) resurfaces. I'm keenly aware that Chris
> thinks that one really ought to be careful with having llvm-gcc emit
> bytecode. But frankly, if that's the case, what's the point of emitting
> byte code in the first place?

For my application, byte-code emission was the crucial reason why I
decided to go with LLVM rather than other front-ends. In general, I
think I made a good decision, but when I need to spend 2 days to compile
a simple app to bytecode files or a week to fix code after
ISA/types/streams/interface change, it gets me thinking....

> It might be a good interim step to hack the frontend so that llvm-ld
> gets invoked (and I'm aware of problems like passing '-gcse' to the ld
> phase when the frontend collects the arguments.) Even if emitting
> bytecode only happens with "-O4", you still run into the same problem at
> link time. Unless you override LD in your makefile or some other hack,
> which, BTW, doesn't necessarily work either.

I hacked a replacement for llvm-collect2 (explained below), but such
hackery is really not a way to go in the long term. I'm very hopeful
that Chandler's work will solve the problem for good.

on the code compiled with llvm-collect2 and llvm-collect2 called by the
script, you might notice a difference in the libraries it gets linked
with, so in my case it links with a library that doesn't exist on the
system, probably the script doesn't handle paths/LD_LIBRARY_PATH
correctly. At that point I didn't have the patience to try to figure
exactly what's going on, so I just added a symbolic link:
$ ln -s /lib64/ld-linux-x86-64.so.2 /lib/ld64.so.1

[ collect2 patches welcome ;-) ]

I successfully compiled 20 or so apps, some fairly large ones (like wine)
with the script.

Finally, I need to mention that all the credit for the idea to write
such script goes to Reid. All the bugs are exclusively mine :-)

Re: How to compile apps to bc files with the new llvm-gcc4?

Domagoj Babic wrote:
> I hacked a replacement for llvm-collect2 (explained below), but such
> hackery is really not a way to go in the long term. I'm very hopeful
> that Chandler's work will solve the problem for good.

Interim (pontoon) bridges are the way forward for most successful
campaigns; often, you can't wait for the engineers to erect the
permanent bridge if you're looking to make progress. If it works for the
Navy and the Marine Corps, it'll probably work for us. :-)

Actually, this may be a much cleaner interim solution that the hack I
was planning inside the gcc4. It does require that the frontend's
gcc/gcc.c LINKER_NAME define be set appropriately. Getting that set
correctly might be an interesting configure hack, but easier than
hacking command specs in gcc/gcc.c.