Embeddable Common-Lisp

Recent Content

I just saw a bunch of old news resurrected both in my news reader and in Planet Lisp. This was probably due to the software upgrade in Sourceforge: ECL is now using the new development platform that Sourceforge provides and during the migration several unexpected events took place (lost git repositories, this issue with the news, etc). Fortunately this new platform is easier to use, it provides http access to git, CVS is still there, bug reporting is much nicer and I can kill spam comments in bug reports, which is great. Please make the best use of it and accept once more my apologies for the zombie news.

It seems that Windows now implements a kind of input history for any application that uses the console. This is nice, because it allows ECL users that directly work on a terminal window to recall previous commands, using just the arrow keys -- a limited form of what "readline" does for you --.

On a side note, the next release of ECL will ship with a small script to install quicklisp in the installation directory. It will be as simple as using (require :ecl-quicklisp) from the Common Lisp prompt. ECL will download the installation file and run it for you with the appropriate paths.

Sorry for the multiple posting -- I had some problems with SourceForge news submission today.

Known issues

This release is the first one with the new multithreading library,
which no longer relies on the POSIX mutexes, condition variables and
semaphores. Instead, ECL makes use of libatomic-ops to implement
userspace routines for process communication (mailboxes), resource
sharing (locks, condition variables, counting semaphores) and fast
spinlocks.

Due to the new implementation, it is likely that some corner cases
may appear during use. In this case we would like to ask you to
report a reproducible test case to ECL's bug tracker and we will
provide a solution as soon as possible, with a new release, if
needed.

In addition to this, the following problems persist:

Cygwin's library is still broken: fork/exec fails to reload the
cygwin library, or ECL's compiled libraries in a completely random
fashion. For this reason we recommend using ext:system instead of
ext:run-program in that platform.

In Windows ECL comes with bytecodes compiler by default, because C
compilers are normally not avaiable. Unfortunately several
libraries out there are not prepared for this. If you plan to use
quicklisp and have a C compiler accessible to ECL, you may use
(ext:install-c-compiler) to switch back to the Lisp-to-C compiler.

Changes since last release

Some highlights of this release are:

The multithreading library.

Complete support for MOP.

Speed improvements in areas such as slot accessors.

Common Lisp code can now trap and capture Unix interrupts, though
the processing is asynchronous for all but the critical ones.

ECL's manual, which is found here http://ecls.sourceforge.net/new-manual/ is being updated on two fronts. The most important part is to document features that were long present but were not explained in the documentation. This includes for instance character types, support for Unicode or external formats, which were erroneously absent from the manual for several years.

The second set of improvements consists on the documentation of its C/C++ API. The manual now lists the C equivalents of almost all Standard Common Lisp functions and types, and in the following days we will add also the functions and macros which are proper just to the C part, such as conversion from all C types to Lisp and back, memory allocation, etc.

I am sorry. Really, I am. I cannot follow you all in all your usual communication channels: comp.lang.lisp, the ecl forum, #lisp irc channel... But please consider this:

First of all, ECL is maintained in free time. In this sense, it is more beneficial for you if the maintainers spend more time coding than trolling about lisp.

Second, an IRC channel is not a place to look for support. Sourceforge offers you a bug reporting facility (https://sourceforge.net/tracker/?group_id=30035), a feature request database (https://sourceforge.net/tracker/?group_id=30035&atid=398056), a mailing list (http://news.gmane.org/gmane.lisp.ecl.general/)...

Response times are usually very fast, and only in the most difficult cases (it involves legacy code, requires some nonstandard architecture, etc), or when a release is on queue, does it takes longer.

In particular, in order to ease your life, bug reporting and feature requests are now possible for people without an account in Sourceforge. This is an experiment and I will try to keep it that way as long as spammers allow. Please make good use of it.

Finally, if you find something does not work the way you expect it, instead of spending a week on it and then complaining about it anywhere, just leave a message in any of those communication channels I mentioned above. It will be more productive for you and for the free software community in general.

Ah, and before I forget it: a good bug report for ECL needs at least the following information: 1) platform information and operating system, 2) configuration options, 3) if it failed at configuration time, the log, 4) if it failed at build time (i.e. during "make") a log of that, 5) if it fails with a standard (quicklisp library), steps to reproduce it, 6) if it fails with your code, ideally some sample to reproduce it.

Not long ago a user reported problems with ECL when running under Windows with the Chinese codepage 936. I was really surprised to verify that indeed, ECL was aborting on any multibyte character... but only when typed in a terminal window (the so called Windows console).

A bit of investigation revealed that the Microsoft C runtime is quite broken (http://alfps.wordpress.com/2011/12/08/unicode-part-2-utf-8-stream-mode/) or at least not very POSIX / ANSI conformant. Roughly, some versions of the runtime act differently when the standard input or output of a program is connected to a console. In this case they try to be very clever and translate the Unicode input/output generated by the console into the selected codepage. This also leads the CRT to reject multibyte characters because read() demands a buffer with as many bytes as a whole character needs. For instance, if a character with codes F7 8C is input at the console, read(0, buffer, 1), instead of returning F7, will abort. Or put some other way: characters cannot be read one by one.

All these problems disappear when the user forgets about POSIX/ANSI file descriptors and works with the Windows API and indeed there are MSDN blogs out there asking you to do so... And so we did.

As of now, ECL incorporates new ANSI streams for handling the Windows console. Currently they cannot be created by the user and are activated by ECL whenever the input or output of a program are connected to a console. The stream is interactive and read, write, listen, etc, they all work as expected.

This also has the added benefit that ECL now detects which codepage is used by the console and in that case it activates by default the expected external format --- which you are free to correct with (SETF STREAM-EXTERNAL-FORMAT) IIRC.

I would like to ask people working with Windows to test and help debug this new feature.

Probably nobody cares about this, but there has been some activity in ECL in various fronts. I periodically report that to the ECL mailing list, but this seems to lead other people to think the project is silent or dead. From now on, expect a little bit more noise through the ECL news RSS.

The biggest change ECL will suffer with the upcoming release is the multithreading library. So far we were relying on the POSIX primitives for that (pthread_mutex_lock, condition variables, etc), but this has proven to be a nightmare to maintain. The primitives are not interruptible and they do not interact well with mp:process-interrupt, a feature unfortunately everybody expects in a Lisp implementation.

The upcoming library will be based on a combination of hardware-coded atomic operations (via libatomic-ops) with a new waiting queue that makes clever use of the operating system facilities. The result should be a library that is fair (different threads have equal opportunities), it recognizes fast paths (when a synchronization object is free it is quickly acquired), and in the worst case it is not CPU consuming (relies on sigsuspend or SleepEx to really sleep a waiting thread).

Everything is right now in git/CVS (http://ecls.sourceforge.net/download.html) and under heavy testing with new dedicated tests. Help is welcome from anybody who is interesting in ensuring stability.

Another side effect is that "make check" now works in ECL. I have abandoned the development of "ecl-tests" and instead the tests are now integral part of ECL. In order to save space, the ANSI test suite is not shipped with ECL but rather automatically downloaded thanks to ECL's new components: ecl-curl and ecl-minitar. The test suite also allows downloading of quicklisp and automated testing of selected libraries.

In the end this was the last missing ingredient to resucitate our compiler farm, which is now up and running again: http://ecls.sourceforge.net/logs.html

Expect a new release soon, and also a few more posts on other upcoming features.

Cheers,

Juanjo

P.S.: I would like to thank the ELS organizers and steering committee for a wonderful meeting in Zadar. Great location, amazing food, and even better discussions.