ecls-list

Dear ECL developers,
two years ago, I made cl-launch to successfully support dumping of
asdf systems with ECL 0.9i plus a patch I wrote for the builder. But
trying the latest ECL from debian (actually it is somewhat outdated,
0.9j-20080306-5) or from git (from two days ago), and it now breaks
horribly when trying to dump a system. My patch has bitrotten, and
what's now in asdf-ecl.lisp is wildly different.
I'm trying to port cl-launch to the new ECL mechanism, and am doing
things wrong, not being familiar with ECL. What is the right way to
debug? How do I get ECL to dump a backtrace before it dies? Or even
better, to enter an interactive debugger?
At one point, I was having ECL reusing objects from a previous
compilation and was confused. At this point, I'm befuddled as it dies
with the uninformative error message:
;;; Warning: COMPILE-FILE warned while performing #<ASDF:COMPILE-OP
NIL 168542448> on
#<ASDF:CL-SOURCE-FILE "header" "cl-launch-prefix" 168378656>.
but no warning is printed. If I try to insert a break after that
happens, it also dies.
Also, to save me a bit of work, what is the name of the file created
by (asdf::make-build :cl-launch-program :type :program) ?
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
An excessive knowledge of Marxism is a sign of a misspent youth.
-- John McCarthy

On Wed, Oct 29, 2008 at 12:34 AM, Faré <fahree@...> wrote:
> two years ago, I made cl-launch to successfully support dumping of
> asdf systems with ECL 0.9i plus a patch I wrote for the builder [...]
> My patch has bitrotten, and
> what's now in asdf-ecl.lisp is wildly different.
Yes, a lot of things have changed, fixing design errors and adding new
functionality to support building monolithic systems that coexist with
non-monolithic ones, avoiding name collisions and many other things
that made the previous asdf:make-build of little use.
> I'm trying to port cl-launch to the new ECL mechanism, and am doing
> things wrong, not being familiar with ECL. What is the right way to
> debug? How do I get ECL to dump a backtrace before it dies? Or even
> better, to enter an interactive debugger?
> At one point, I was having ECL reusing objects from a previous
> compilation and was confused. At this point, I'm befuddled as it dies
> with the uninformative error message:
> ;;; Warning: COMPILE-FILE warned while performing #<ASDF:COMPILE-OP
> NIL 168542448> on
> #<ASDF:CL-SOURCE-FILE "header" "cl-launch-prefix" 168378656>.
> but no warning is printed. If I try to insert a break after that
> happens, it also dies.
This is a problem with ASDF itself, not really the ECL environment nor
our extensions. ASDF captures all compiler conditions (i.e. errors and
warnings) and thus will prevent you from entering the debugger. I do
not remember whether there are options to enter the debugger from
ASDF, but you may try setting c::*compiler-break-enable* to T before
building anything
> Also, to save me a bit of work, what is the name of the file created
> by (asdf::make-build :cl-launch-program :type :program) ?
I cannot tell you just like that, without knowing the platform and
your configuration settings. But the ANSI standard function
COMPILE-FILE-PATHNAME exists for this reason. Just pass it the same
type as you would pass to make-build.
Juanjo
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28009 (Spain)
http://juanjose.garciaripoll.googlepages.com

Dear Juanjo,
thanks a lot for your support.
I tried not one but two different ways of getting cl-launch to run with ECL:
* one (simplified from previous cl-launch thanks to the new asdf-ecl)
that calls the last builder directly and sometimes fails at runtime.
* one that fully relies on make-build and writes temporary asd files,
sometimes fails at compile-time.
With the first solution, ecl seems to always build an executable, but
sometimes this executable will fail during initialization, seemingly
trying to use variables before they have been defined, or otherwise
running initialization code out of order.
With the second solution, the linker seems to fail to find
initialization code that ecl expects, as in
1> (C::BUILDER :PROGRAM
#P"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/cl-launch-6621-program"
:LISP-FILES (#P"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libcl-launch-6621-prefix.a"
#P"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libclt-asd.a"
#P"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libcl-launch-6621-program.a"))
/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libclt-asd.a(ECLINITjQdZQR.o):
In function `init_lib_CLT_ASD':
/dev/shm/tmp/fare/ECLINITjQdZQR.c:40: undefined reference to
`_eclI7Z7omJ7_2XkTlSy'
collect2: ld returned 1 exit status
An error occurred during initialization:
(SYSTEM "gcc -o
\"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/cl-launch-6621-program\"
-L\"/usr/local/stow/ecl/lib/\" \"/dev/shm/tmp/fare/ECLINITUeSclS.o\"
\"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libcl-launch-6621-prefix.a\"
\"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libclt-asd.a\"
\"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libcl-launch-6621-program.a\"
-lecl -lpthread -ldl -lm -lgc -lgmp") returned non-zero value 1.
This might actually be the same bug, whereas some initialization
functions are missing but expected for asdf systems, except that in
one case some introspection routine just skips over those that were
forgotten, but the linker catches them statically.
If you're interested, you can try for yourself.
mkdir /tmp/foo
cd /tmp/foo
wget http://fare.tunes.org/files/cl-launch/cl-launch_2.09.sh
apt-get install clisp gcl gclcvs cmucl sbcl
./cl-launch_2.09.sh -l 'ecl' -B tests
Adding 76 as an argument after tests will start directly at the first
test that borks, which you can directly invoke this way:
./cl-launch_2.09.sh -B redo_test sh ecl exec noupdate dump noinc system noinit
You have a full log in tests.log, which can be made more verbose by
uncommenting the ;#+ecl line towards the bottom of cl-launch.
Note that I left the second implementation of build-and-dump wired in,
but you may switch implementation of build-and-dump by switching which
is correctly named and which is named xxx-build-and-dump.
> Yes, a lot of things have changed, fixing design errors and adding new
> functionality to support building monolithic systems that coexist with
> non-monolithic ones, avoiding name collisions and many other things
> that made the previous asdf:make-build of little use.
I'm glad this happened. But it looks like there are still bugs, and I
hope that with cl-launch it will become easier to detect and squash
those bugs, as the same programs are run with various implementations.
>> At one point, I was having ECL reusing objects from a previous
>> compilation and was confused. At this point, I'm befuddled as it dies
>> with the uninformative error message:
>> ;;; Warning: COMPILE-FILE warned while performing #<ASDF:COMPILE-OP
>> NIL 168542448> on
>> #<ASDF:CL-SOURCE-FILE "header" "cl-launch-prefix" 168378656>.
>> but no warning is printed. If I try to insert a break after that
>> happens, it also dies.
>
> This is a problem with ASDF itself, not really the ECL environment nor
> our extensions. ASDF captures all compiler conditions (i.e. errors and
> warnings) and thus will prevent you from entering the debugger. I do
> not remember whether there are options to enter the debugger from
> ASDF, but you may try setting c::*compiler-break-enable* to T before
> building anything
>
Thank you for this tip. However, it didn't help. I never got into the
debugger and never got a backtrace :-(
I think the unprinted warning is about the redefinition of functions
already loaded by cl-launch (as cl-launch tries to compile itself as a
prefix to what follows), but I am not sure how to confirm the
hypothesis.
>> Also, to save me a bit of work, what is the name of the file created
>> by (asdf::make-build :cl-launch-program :type :program) ?
>
> I cannot tell you just like that, without knowing the platform and
> your configuration settings. But the ANSI standard function
> COMPILE-FILE-PATHNAME exists for this reason. Just pass it the same
> type as you would pass to make-build.
>
Well, as previously reported (two years ago, when I first released
cl-launch support for ecl - we don't get any younger, do we?),
COMPILE-FILE-PATHNAME is not compliant, because it accepts a set of
parameters different from COMPILE-FILE, whereas the standard mandates
that any extension supported by COMPILE-FILE must be supported by
COMPILE-FILE-PATHNAME. I have to call the former with :type :object
and the latter with :system-p t. That's most confusing, and a remnant
from the ancient history of ECL, but I got around it with ugly #+ecl
in cl-launch.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
The ultimate result of shielding men from the effects of folly is
to fill the world with fools. -- Herbert Spencer

On Thu, Oct 30, 2008 at 6:43 PM, Faré <fahree@...> wrote:
>>> Also, to save me a bit of work, what is the name of the file created
>>> by (asdf::make-build :cl-launch-program :type :program) ?
>>
>> I cannot tell you just like that, without knowing the platform and
>> your configuration settings. But the ANSI standard function
>> COMPILE-FILE-PATHNAME exists for this reason. Just pass it the same
>> type as you would pass to make-build.
>>
> Well, as previously reported (two years ago, when I first released
> cl-launch support for ecl - we don't get any younger, do we?),
> COMPILE-FILE-PATHNAME is not compliant, because it accepts a set of
> parameters different from COMPILE-FILE, whereas the standard mandates
> that any extension supported by COMPILE-FILE must be supported by
> COMPILE-FILE-PATHNAME. I have to call the former with :type :object
> and the latter with :system-p t.
Not really. You are probably using a too old version.
COMPILE-FILE-PATHNAME works fine with the _same_ set of options as
COMPILE-FILE
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help. Top level.
> (compile-file-pathname "foo")
#P"foo.fas"
> (compile-file-pathname "foo" :system-p t)
#P"foo.o"
Thanks for the other files and the detailed description of the
problem. I will download them and investigate it during the following
days -- cannot promise full commitment, though, as I will be
travelling.
Juanjo
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28009 (Spain)
http://juanjose.garciaripoll.googlepages.com

2008/10/30 Juan Jose Garcia-Ripoll <juanjose.garciaripoll@...>:
>> COMPILE-FILE-PATHNAME is not compliant, because it accepts a set of
>> parameters different from COMPILE-FILE, whereas the standard mandates
>> that any extension supported by COMPILE-FILE must be supported by
>> COMPILE-FILE-PATHNAME. I have to call the former with :type :object
>> and the latter with :system-p t.
>
> Not really. You are probably using a too old version.
> COMPILE-FILE-PATHNAME works fine with the _same_ set of options as
> COMPILE-FILE
>
> ECL is free software, and you are welcome to redistribute it
> under certain conditions; see file 'Copyright' for details.
> Type :h for Help. Top level.
>> (compile-file-pathname "foo")
> #P"foo.fas"
>> (compile-file-pathname "foo" :system-p t)
> #P"foo.o"
>
Oh, OK. I was confused. COMPILE-FILE-PATHNAME indeed implements a
superset of what COMPILE-FILE supports. Discrepancies this way seem to
be permitted.
> Thanks for the other files and the detailed description of the
> problem. I will download them and investigate it during the following
> days -- cannot promise full commitment, though, as I will be
> travelling.
>
Take your time. I believe that this is a bug in ECL and that when this
bug is fixed, cl-launch will work fine with ECL. Actually, I believe
that anyone trying to use make-build with ASDF systems may be
experiencing the same problem and expecting the same solution, not
just the few users of cl-launch.
Thanks a lot for your support!
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
A real person has two reasons for doing anything ... a good reason and
the real reason.

On Thu, Oct 30, 2008 at 10:06 PM, Faré <fahree@...> wrote:
>> Thanks for the other files and the detailed description of the
>> problem. I will download them and investigate it during the following
>> days -- cannot promise full commitment, though, as I will be
>> travelling.
>>
> Take your time. I believe that this is a bug in ECL and that when this
> bug is fixed, cl-launch will work fine with ECL.
I just read your blog entry. The actual problem has not even been
identified and you are stating this is a bug of the implementation. Is
there any need on stating as true something that has not been yet
confirmed? You do not even know what is causing the problem, whether
it is ECL or just that your implementation of cl-launch got outdated
by current changes.
Juanjo
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28009 (Spain)
http://juanjose.garciaripoll.googlepages.com

2008/10/31 Juan Jose Garcia-Ripoll <juanjose.garciaripoll@...>:
> On Thu, Oct 30, 2008 at 10:06 PM, Faré <fahree@...> wrote:
>>> Thanks for the other files and the detailed description of the
>>> problem. I will download them and investigate it during the following
>>> days -- cannot promise full commitment, though, as I will be
>>> travelling.
>>>
>> Take your time. I believe that this is a bug in ECL and that when this
>> bug is fixed, cl-launch will work fine with ECL.
>
> I just read your blog entry. The actual problem has not even been
> identified and you are stating this is a bug of the implementation. Is
> there any need on stating as true something that has not been yet
> confirmed? You do not even know what is causing the problem, whether
> it is ECL or just that your implementation of cl-launch got outdated
> by current changes.
>
I did update cl-launch. cl-launch is not the software that tries to
use linker symbols that have not been generated or otherwise fails to
generate or register initialization routines. If I'm using the
available interface incorrectly (which I don't think I am, but hey,
that's possible), at the very least ECL is coming up with a much
lower-level error than should be. In other words, a bug in cl-launch
is quite possible, but a bug in ECL (or possibly the underlying C
compiler or linker toolchain, or the asdf-ecl contrib - but it looks
like ECL to me) is quite certain.
That said, I've been known for jumping to conclusions and opening my
big fat mouth. And sometimes publicly saying mea culpa for it.
/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libclt-asd.a(ECLINITu45oWA.o):
In function `init_lib_CLT_ASD':
/dev/shm/tmp/fare/ECLINITu45oWA.c:40: undefined reference to
`_eclI7Z7omJ7_BAUArSy'
collect2: ld returned 1 exit status
An error occurred during initialization:
(SYSTEM "gcc -o
\"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/cl-launch-10110-program\"
-L\"/usr/local/stow/ecl/lib/\" \"/dev/shm/tmp/fare/ECLINITz2jMy9.o\"
\"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libcl-launch-10110-prefix.a\"
\"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libclt-asd.a\"
\"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libcl-launch-10110-program.a\"
-lecl -lpthread -ldl -lm -lgc -lgmp") returned non-zero value 1.
I swear the code inserting a dangling reference to ECLINITu45oWA isn't
on cl-launch's side! Maybe either asdf-ecl or build-static-library is
failing to link in the master object for the .asd files themselves, or
something? I admit I don't understand this part of the code.
In any case, it looks pretty reproducible whenever dumping asd systems
is involved, as in
cl-launch -l ecl -B tests 76
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
We don't want tradition. We want to live in the present and the only history
that is worth a tinker's dam is the history we make today. -- Henry Ford

> /home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libclt-asd.a(ECLINITu45oWA.o):
> In function `init_lib_CLT_ASD':
> /dev/shm/tmp/fare/ECLINITu45oWA.c:40: undefined reference to
> `_eclI7Z7omJ7_BAUArSy'
> collect2: ld returned 1 exit status
> An error occurred during initialization:
> (SYSTEM "gcc -o
> \"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/cl-launch-10110-program\"
> -L\"/usr/local/stow/ecl/lib/\" \"/dev/shm/tmp/fare/ECLINITz2jMy9.o\"
> \"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libcl-launch-10110-prefix.a\"
> \"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libclt-asd.a\"
> \"/home/fare/.cache/lisp-fasl/ecl-8.10.0__cvs_2008-07-12_18_54_-linux-pentium3/dev/shm/tmp/foo/libcl-launch-10110-program.a\"
> -lecl -lpthread -ldl -lm -lgc -lgmp") returned non-zero value 1.
I don't know the details of cl-launch. However, as an ECL user, I've
met similar
error with an old (buggy) Lisp code I was porting to ECL. In all
cases, it was the
result of a bug in the old Lisp code -- undefined functions,
undeclared globals, etc.
I'm not saying cl-launch is at fault -- only that the occasions where I found
similar problems came from bugs in the input Lisp source code. Maybe
it might help
a little if ECL emits warnings on used but undefined functions/globals, and does
not die in those cases.
-- Gaby

On Fri, Oct 31, 2008 at 7:16 PM, Juan Jose Garcia-Ripoll
<juanjose.garciaripoll@...> wrote:
> On Thu, Oct 30, 2008 at 6:43 PM, Faré <fahree@...> wrote:
>> If you're interested, you can try for yourself.
>>
>> mkdir /tmp/foo
>> cd /tmp/foo
>> wget http://fare.tunes.org/files/cl-launch/cl-launch_2.09.sh
>> apt-get install clisp gcl gclcvs cmucl sbcl
>> ./cl-launch_2.09.sh -l 'ecl' -B tests
>
> Did this on my Mac OS X, using ECL sources from GIT or CVS
> repositories (master and stable branches respectively, so nothing
> fancy) and the first test that fails is #86. It fails because the
> value of the variable HEADER in BUILD-AND-DUMP is NIL and you are
> applying ENSURE-LISP-FILE on it. I cannot try on linux because my big
> quadcore box died this morning out of a hard disk failure.
I should add that I tried both from the command line and from a
running ECl, using the same commands that your log file shows. The
reason for running them from the toplevel instead of using -eval and
-load command line arguments is that the latter will _never_ enter the
debugger. This is designed so on purpose, since entering the debugger
in a script that is being executed god knows where and how is
unreasonable.
If you really need to enter the debugger, a workaround is to enclose
the command in a HANDLER-CASE statment as in
ecl -norc -eval '(setf si::*break-enable* t)' -eval '(handler-case
(cos #\a) (error (c) (invoke-debugger c)))'
Juanjo
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28009 (Spain)
http://juanjose.garciaripoll.googlepages.com

Dear JJ and ECL hackers,
the good news is that I if I rm -rf ~/.cache/lisp-fasl/ecl* everytime
I can make standalone executables with ECL and cl-launch and ASDF
systems.
The bad news is that the bug is still there, though it is now narrowed
down to a bad interaction with the object cache. Somehow the asdf+ecl
combo must be somehow doing the wrong thing: either ASDF wrongly
reuses object files from a previous version of same-named source when
run with ECL, or ECL fails to agree to the proper initialization
routine name as it reuses object files from previous compilation of
same source, or maybe ASDF is getting confused between files named .o
but compiled with different parameters? More investigation is required
to find out what's happening.
My next test, the compiling of exscribe, fails on a stream error while
trying to compile fare-utils/basic-strings. I'll investigate whether
the bug is mine.
I'll upload version 2.11 of cl-launch that now almost runs perfectly with ECL.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
You cannot teach a man anything; you can only help him find it for himself.
-- Galileo Galilei