In 1991, I was working at Lucid Inc., and our newest product,
Energize, was an integrated development environment for C and C++ on
Unix. The design of this development environment involved very tight
integration between the various tools: compilers, linkers, debuggers,
graphers, and editors. So of course we needed a powerful editor to tie
the whole thing together, and it was obvious to all of us that there
was only one editor that would do: Emacs.

At the time, the current version of GNU Emacs from the FSF was
Emacs 18. There was another version of GNU Emacs called Epoch, that
had been developed by Alan M. Carroll (and later by NCSA). Epoch was a
set of patches to Emacs 18 that gave it much better GUI support (Emacs
18 was very much a tty program, with GUI support crudely grafted on as
an afterthought.)

For the last few years, Emacs 19 had been due to be released ``real
soon now,'' and was expected to integrate the various features of Epoch
in a cleaner way. The Epoch maintainers themselves saw Epoch as an
interim measure, awaiting the release of Emacs 19.

So, at Lucid we didn't want to tie ourselves to Emacs 18 or on
Epoch, because those code bases were considered obsolete by their
maintainers. We wanted to use Emacs 19 with our product: the idea was
that our product would operate with the off-the-shelf version of Emacs
19, which most people would already have pre-installed on their system
anyway. That way, Energize would make use, to some extent, of tools
you already had and were already using.

The only problem was, Emacs 19 wasn't done yet. So, we decided we
could help solve that problem, by providing money and resources to get
Emacs 19 finished.

Even though Energize was a proprietary, commercial product, all of
our work on Emacs (and on GCC and GDB) was released under the GPL. We
even assigned the copyright on all of our work back to the FSF, because
we had no proprietary interest in Emacs per se: it was just a tool that
we wanted to use, and we wanted it to work well, and that was best
achieved by making our modifications to it be as freely available as
possible. (This was one of the earliest, if not the earliest, example
of a commercial product being built to a significant extent out of
open source software.)

Well, our attempts to help the FSF complete their Emacs 19 project
were pretty much a disaster, and we reached the point where we just
couldn't wait any longer: we needed to ship our product to customers,
and our product needed to have an editor in it. So we bundled up our
work on GNU Emacs 19, called it Lucid Emacs, and released it to the
world.

This incident has become famous as one of the most significant
``forks'' in a free software code base.

When Lucid went out of business in 1994, and I came to Netscape, I
passed the torch for the maintenance of Lucid Emacs to Chuck Thompson
(at NCSA) and Ben Wing (at Sun), who renamed it from ``Lucid Emacs''
to ``XEmacs.''

To this day, XEmacs is as
popular as FSFmacs, because it still provides features and a design
that many people find superior to the FSF's version.

I attribute Lucid Emacs's success to two things, primarily:

First, that my focus was on user interface, and an attempt to
both make Emacs be a good citizen of modern GUI desktops, and to make
it as easy for new users to pick up Emacs as any other GUI editor;

Second, that I ran the Lucid Emacs project in a much more
open, inclusive way than RMS ran his project. I was not just
willing, but eager, to delegate significant and critical
pieces of the project to other hackers once they had shown that they
knew what they were doing. RMS was basically never willing to do
this with anybody.

Other things that helped Lucid Emacs's success, but were probably
less important than the above:

We gave the users what they wanted first. People had been
anticipating Emacs 19 for years, and we stopped dragging our feet and
finished it. So this got us a lot of users up front. However,
XEmacs's current popularity can't be attributed to this, not since
1993, anyway.

Lucid Emacs was technically superior in many ways. This won
us the mindshare of many good developers, who preferred working with
Lucid Emacs to FSF Emacs. It would be nice if technical superiority
was all that mattered, but realistically, the other factors were
probably more important than this one, as far as number of users is
concerned.

The following messages, from the Lucid Emacs mailing lists in
1992 and 1993, comprise the bulk (if not the entirety) of the public
discussions between the Lucid and FSF camps on why the split happened
and why a merger never did.

The current XEmacs maintainers have a much more pusillanimous summary
of this history on their
XEmacs
versus GNU Emacs page.

-- jwz, 11-Feb-2000.

Dramatis Personae:

Richard Stallman: creator of Emacs, founder of the FSF.

Jamie Zawinski: that's me, I did most of the hacking on Lucid
Emacs and managed the project. Near the end of my tenure, dozens of
people were working on Lucid Emacs on a regular basis, and I reviewed
and tested all the code they emailed me, as well as stewarding all
the mailing lists. (This was in the days before public CVS
servers...)

Richard Mlynarik: prolific contributor to Lucid Emacs: he did
almost all of the work of merging changes that were made to the FSF
version back into the Lucid version. He was the largest contributor
to Lucid Emacs who wasn't actually getting paid to work on it.

Marc Andreessen: you've probably heard of this guy, but you
probably didn't know that he used to be one of the maintainers of
Epoch at NCSA.

Joseph Arceneaux: the fellow initially employed by both the
FSF and by Lucid to hack on Emacs 19, before I started working on it.

Jim Blandy: employed by FSF to hack on their version of Emacs
19, after Arceneaux.

Kyle Jones: author of VM, one of the Emacs mail readers.

Dave Gillespie: author of ``calc,'' a scientific calculator
application that runs inside of emacs; calc was hit harder by
Lucid Emacs compatibility problems than most other Emacs packages.

I've rearranged the lisp directory so that there are only a handful
of files at top-level; everything has been moved to subdirectories.
This should speed up the automatic generation of load-path, since it
will have to stat() 10x fewer files.

You can now input and display all iso8859-1 characters. If you set
the variable ctl-arrow to something that is non-nil and non-t then
high-bit characters will be displayed in the current font instead of
in octal.

If you load x-compose.el, you can do Sun/DEC-like compose processing.
See the comment in that file.

I am curious as to what people think of the likelyhood of EMACS and Epoch
merging into a single release.

Isn't that what's called Emacs 19?

Further, it would be nice if any new Lucid-emacs features
get rolled in. It's getting confusing with Emacs 18.X, Epoch, Emacs 19,
and Lucid Emacs based on 19 all floating around at the same time.
For instance, I make frequent use of ^Z2 to dup a screen in Epoch..it's
something else in lucid emacs, and I haven't used Emacs 19 [1] but I'm
wondering if it's yet another thing.

[1] I gather 19 really isn't widely available yet.

Anyways, I think competition is a good thing and all, but
I'm also hoping it doesn't start getting out of hand. Are we going
to have three different versions of Emacs under the GNU license?
From my perspective as a relatively new user of Emacs (about 1 year
and 9 years of Vi before that), I can keep the differences straight
in my mind, and there's ways of binding keys so that everything
looks the same, etc. However, if I'm explaining something about Emacs
versions to someone who's new to all of this, I can see contortions on
their face, etc :-) It's starting to look like the history of
some rock bands (like what's happened over the years to the band "Yes").

Would it be worthwhile for people to work towards a common
GNU Emacs goal?

What does Richard Stallman think of all of this?

I'm just tossing this out there. If there's going to be
2 or three GNUish Emacii, fine, I can keep that straight. I just
personally hope there isn't any more splintering. Perhaps there
should be an Emacs Summit Meeting? Anyways, this is all from a user's
perspective who is wondering what is going on, but who doesn't
necessarily know all of the history behind it...so keep any flames
informative rather than outright hostile :-)

Would it be worthwhile for people to work towards a common
GNU Emacs goal?

I don't think so. If anything I think it would be worthwhile to
consider pulling Emacs apart into its components, much as OS
developers have pulled apart operating systems. There are some
good ideas embodied in Emacs. I think they could be improved by
distilling them out of the monobloc (to horribly mix metaphors).
But this isn't the place for such discussions.

From: dansmith@autodesk.com (Daniel Smith)
Further, it would be nice if any new Lucid-emacs features
get rolled in. It's getting confusing with Emacs 18.X, Epoch,
Emacs 19, and Lucid Emacs based on 19 all floating around at the
same time. For instance, I make frequent use of ^Z2 to dup a
screen in Epoch..it's something else in lucid emacs, and I haven't
used Emacs 19 [1] but I'm wondering if it's yet another thing.

How many incompatibilities are really present, yet?

Emacs 18.58 and Epoch 4.0 are identical, except for the Epoch
extensions.

Emacs 19 isn't out yet.

So current incompatibilities include exactly these:

Differences between Epoch and Lucid X commands and related
interfaces.

Differences in general between Emacs 18.58 and Lucid Emacs
19.

The Lucid folks are, I think, doing a good job of handling the latter
problem, while the former problem could be temporarily solved with an
Epoch emulator package for Lucid (an idea which has been tossed around
on help-lucid-emacs) or vice versa.

[1] I gather 19 really isn't widely available yet.

Isn't available at all yet, basically.

Marc

--
Marc Andreessen
Epoch Development Team
University of Illinois at Urbana-Champaign
marca@ncsa.uiuc.edu

The long delay in releasing Emacs 19 is the FSF's fault. (In some
sense, therefore, mine.) While it's regrettable that there are
multiple versions, I can't blame people for filling the gap that the
FSF left. One of the goals of the copyleft is to allow people to do
this--so that one central maintainer's lapse does not hold back the
rest of the community.

However, the people who wrote the Lucid version didn't try to to make
the changes usable for us. They charged ahead without discussing the
design with me or with the main Emacs maintainer. They reimplemented
things we had already done. I asked them to do some work making parts
usable for us, but they said no.

The result is that not much of the code in Lucid's version will ever
appear in the FSF's version. Some of the features are worth having,
but need to be reimplemented. If you'd like this to happen sooner,
one way is to volunteer to do some of the work.

a) It would be _really_ useful if someone familiar with emacs19, leamcs
and epoch could give everyone an idea of what the major differences are.
Not so much details, but places where the major abstractions differ.
For instace lemacs `faces' and epoch `styles' look fairly similar,
converting would be irritating but not suicide-inducing. I havn't
looked at screen creation etc in lemacs in detail but from what I have
seen it treats screens as parallel to windows while epoch treats them as
parallel to buffers (current-screen vs selected-screen etc), if that
difference goes deep it might be very painful.

b) What would really be good would be emacs19 simulators for both epoch
and lemacs. I'm less interested in getting at existing epoch code than
in giving my own code a future path.

I read RMS's message with sadness. Over the past two years I have
tried very hard to figure out a way to work with FSF on Emacs, and for
a lot of that time I thought we were working together, but apparently
I failed. For those of you who wish to read about that effort, you can
read the postscript to this note, which outlines Lucid's involvement -
a sort-of history of the events.

We still want to work with FSF and have our contributions become part
of FSF Emacs, and we are eager to hear suggestions on how to proceed.

-rpg-

P.S. RMS's letter paints an unflattering and unfair picture of Lucid's
involvement with Emacs, and in particular on the motives and efforts
of our hackers.

The Free Software Foundation is a remarkable organization - for many
years I have supported it personally (and still do), and my company
supports it, its goals, and its products. This support is in terms of
time, effort, and dollars.

RMS says:

However, the people who wrote the Lucid version didn't try to to make
the changes usable for us. They charged ahead without discussing the
design with me or with the main Emacs maintainer. They reimplemented
things we had already done. I asked them to do some work making parts
usable for us, but they said no.

This is not how we at Lucid remember events, although it may be a difference
without a distinction. I'll recount the events as I recall them and let the
reader supply the characterization.

Lucid set a course 4 years ago to develop a C/C++ programming environment,
using Emacs as a key component. We needed a small number of specific
things:

it had to run under X using multiple windows

it had to support multiple fonts (at least a little)

it had to support embedded glyphs (icons)

it had to support attaching active regions to contiguous blocks of
characters (to support a hypertext-like facility)

it had to support menubars

it had to support readonly regions of text

My goals with respect to FSF were these:

do anything (including paying money) to get our changes into the
official FSF sources

not sacrifice quality or the schedule (which was lengthy, and therefore
apparently safe)

For the first two years we prototyped the functionality we needed in an
X-widget-based text editor and in Epoch.

By the Summer of 1990, Emacs 19 appeared to be right around the corner, so
we entered discussions with the person at FSF whom we were led to believe
was the official Emacs 19 coder - Joe Arceneaux. It appeared that our goals
and FSF's were in line, and so he proposed that we pay him a consulting fee
to speed up Emacs 19 development plus get our changes in. We agreed, wrote a
contract, and volunteered manpower to help.

The contract period started October 1990, was to last approximately 9
months. We were to pay him approximately $50k. He was paid on milestones,
on which he appeared to deliver. In addition, we provided him office space
and his own Sparcstation at Lucid. (At the conclusion of this period, it
appeared that Joe Arceneaux was no longer the official Emacs 19 coder.)

In the Spring of 1991 it became apparent that the effort was going badly -
Emacs 19 as a text editor was slow, and full of bugs. The only people at
Lucid who dared use it were Arceneaux and myself. Moreover, Emacs 19
performed hideously in the new areas outlined above. Its behavior was
especially bad with respect to active regions -- even worse than our
seat-of-the-pants prototype in Epoch.

Nevertheless, we persevered: We re-arranged our Emacs team and priorities
to work more closely with Arceneaux, and we were confident the arrangement
could work. To be safe, I put 4 of our best hackers on it with
Arceneaux. By Summer of last year we became aware that RMS had
hired/appointed Jim Blandy as the official Emacs 19 person.

At that time, one of our hackers went out to Cambridge to talk RMS about how
to proceed, and it appeared we reached a reasonable working arrangement.
But, as time went on we determined that the FSF code for Emacs 19 was too
unwieldy, bug-ridden, and convoluted to work with, so we started rewriting
parts of it.

In addition to not listening to any design ideas we had, RMS's policy was
to require us to provide source diffs and to explain every changed line.
Because our efforts towards release were mounting, and we were re-writing
massive amounts of the system, source diffs didn't make sense, but RMS and
Blandy were unwilling or unable to work at a higher level. I traveled to
Cambridge to work things out, and my team had numerous phone conversations
with RMS.

One area that was completely rewritten was the implementation of active
regions. Various attempts were made to discuss the design with RMS,
including a 2 hour telephone conversation from one of my hackers, but he
was completely unwilling to consider another point of view. At first, he
apparently didn't understand the differences between attaching active
regions to text and attaching fonts to text, and when he finally did, he
was so wedded to his design for putting fonts in the text that he wanted
us to implement BOTH his design for fonts and ours for other active region
support.

Our hacker wanted RMS to cooperate to the extent that we would first
implement our version to support all of these features, and see how well it
worked, but he wouldn't even agree to that. It is worth noting that with
the original FSF version, in spite of some rather elaborate tree-balancing
code and other hackery for precomputing display characteristics, Emacs
redisplay spent more than 40% of its time in the active regions code. In
our much simpler version it spends significantly less than 1% of the
redisplay time in the corresponding code, even for buffers with a few
thousand active regions. (This isn't to say that the original design
couldn't be made to work, but several man-months of effort had gone into
their original version, and it took less than 2 man-months to implement and
debug the code that we shipped with our version. Finally, our version is
approximately 1/2 the number of lines of the original, it is more general,
and it uses less memory.)

Early last Fall, after spending approximately $70k on Arceneaux, we let
his contract expire, but we did put a leased 38.4 kb digital line and an
X terminal in Arceneaux's apartment in San Francisco (35 miles away from
Lucid), so he could continue to work on Emacs 19 for FSF with Lucid
support.

When we learned last Summer that FSF had established an Emacs steering
committee, and no one from my organization was put on that, or even
informed of its existence, we came to the conclusion that the FSF had
no interest in working with us, and we stopped trying. The punch line
of RMS' statement is true -- we wouldn't do things they way he wanted.
But we didn't ``charge ahead'' without talking to FSF. While we did
reimplement some things that they had already done, the things we
reimplemented were completely broken. It's true that in the end our
work has been unusable by the FSF, but that seems to us to have more
to do with their willingness to use it than with anything else.

RMS concludes:

The result is that not much of the code in Lucid's version will ever
appear in the FSF's version. Some of the features are worth having,
but need to be reimplemented. If you'd like this to happen sooner,
one way is to volunteer to do some of the work.

I have trouble imagining how anyone could have worked with with them
on this. We may have failed in this attempt, but we have a history of
trying, and we have succeeded in other areas.

In addition to working on Emacs, and contributing to Cygnus initial
implementations of some new features for gdb, such as a malloc that
supports multiple heaps, code to save the internal gdb ``symbol
table'' state for a given executable file, and a various C++ debugging
features, (as well as paying $100K to have these features officially
incorporated into gdb), during the past year we dedicated
approximately 1 person-year, spread over 3 people, to producing a new
Emacs manual. In addition we have spent a lot of time trying to
figure out how to work amicably with FSF.

We have also supported FSF and RMS by selecting several FSF components
for our product, an endorsement of their quality and usefulness.
Since 1987 Lucid has regularly contributed code (and occasionally
money) to FSF. Lucid paid over $200k to support FSF products over the
last 2 years. We contributed over 8 hacker-years to improve FSF
products.

I personally am a member of LPF (not related to FSF, but related to
RMS). I travel about debating various people in RMS's name, and I
have actively supported RMS in various private ways. I couldn't
figure out how to work with FSF and RMS even though I had years to do
it, hundreds of thousands of dollars to spend, and 8 man-years to
volunteer. What would you have done differently?

We offer a free Emacs 19 to anyone who wants it, under exactly the
same conditions as the FSF, providing commercial support to our paying
customers (all improvements arising out of this support become part of
the free version as well, of course), as well as answering questions
and incorporating changes received from users on the net, and we will
continue to develop it. We will rewrite the redisplay code, and we
will very probably do work to internationalize Emacs. We also are
using scarce documentation resources on the Emacs manual (copyright
still held jointly with RMS). However, unlike the FSF, we will also
listen to ideas from other people. For one thing, if someone
implements an improved version of a feature, we aren't arrogant or
inflexible enough to refuse to consider using it. The hackers that
worked on our version are concerned about doing a good job for the
sake of Emacs, not just for the sake of our company or their egos.

By this time I've ceased expecting that we will work closely with FSF
on Emacs, but we would still like to do so. As far as we can tell,
the reason that FSF won't use any of our work is that they aren't
interested in working *with* us (rather they want us to work *for*
them, doing precisely what they tell us to do the way they say to do
it, and on their own schedule). Perhaps they assume that our designs
and implementations are substandard, but as far as I know they have
never really looked at what we have done.

One reason it didn't is that Lucid hired away the maintainer and put
him to work adding new features that you wanted, instead of getting
the program into shape for release. No one discussed with me how this
would affect the maintenance of Emacs. In fact, I wasn't even told
until a couple of weeks before he was going.

The features Lucid wanted him to implement were useful. But at that
time, I wanted to get Emacs 19 out the door--not delay the release for
additional features.

I couldn't change the plans, so I had to make the best of them. I
suggested a design to him, one oriented toward editing formatted
text--a feature I wanted Emacs to have, eventually.

But I had to find someone else to work on getting Emacs 19 ready for
release. There was only one person available--Jim Blandy. He is
careful and methodical, but not fast, and he hadn't worked on such a
large program before. It took him a while to get into the swing of
things. Also, he was unhappy to leave his previous project, which was
a desktop system.

(To be fair, there have been other causes of delay as well, compounded
with these.)

Since then, Lucid has sometimes tried to cooperate with us. But the
attempts have not worked.

There have been communication failures. For example, last summer
Gabriel found out about an Emacs Steering Committee which no one from
Lucid had been invited to join. Apparently I wasn't invited, or even
told--I don't recall ever hearing about such a committee. I think
something got garbled in transmission.

But if I had set up a committee, and I had looked for who should be on
it, I wouldn't have thought of Lucid. I had been told they had
stopped employing Arceneaux, but I didn't know they were continuing to
work on a version of Emacs. I thought they weren't doing anything.

Last summer I was told that someone there would work on a project that
was needed for Emacs--updating the manual, I think. But in September,
when I asked him about progress, the he told me he had dropped the
project; other higher priority work took all his time.

Later I learned that Lucid had people other than Arceneaux working on
a version of Emacs. In December, Gabriel was in this area, and he
told me that he wanted Lucid's programming work in Emacs to fit into
the FSF Emacs release, and he told me how to reach the programmers
there. I was glad to discover a source of additional hands for the
work. So I called them, wanting to discuss the design.

But they told that they had already chosen the design and done so much
implementation that they wouldn't consider redoing any of it. They
had a deadline and had to finish following the path they had set out
on.

They had not tried to talk with Jim Blandy to find out what he was
doing, working on the Emacs 19 release. I'm not sure they knew he was
the person doing it. If I'd known what they were working on, I could
have told them they should speak with him.

When people offer the FSF changes to programs that the FSF maintains,
the FSF maintainer always has to consider them line by line. That's
part of the maintainer's task. And the maintainer might reject some
of the changes, or ask for rewriting--that's also part of the
maintainer's task.

But mainly we didn't even get to this stage. Problems became visible
at the general design level.

During that conversation I found out about the problem with slowness
of interval processing. I called Arceneaux and looked at his code,
and found a localized bug in the balancing of binary trees. If you
created all the intervals from front to back, it would do no
balancing, leaving the tree maximally unbalanced. The effect was a
rather roundabout linear search. As luck would have it, the Lucid
application usually created intervals from front to back, so they
always saw the worst possible behavior. Anyway, the bug was simple
once found. This all took a few days.

The Lucid programmers had seen the slowness and decided that the basic
approach was at fault. So they decided to rewrite the interval code
from scratch. They said they didn't need to support editing large
files of formatted text; for their usage, a small number of intervals
was enough. So they wrote a faster, straightforward linear search.

They were willing to offer FSF the code for this--but it isn't
useful. No matter how cleanly written, a linear search becomes a
disaster for large files with lots of intervals.

When I explained why the binary tree approach was not inherently slow,
they said it was too late--the code worked for their purposes, and
because of their deadline, they couldn't consider redoing anything
that worked well enough for their needs.

This is what I mean by charging ahead. If they had told me the
problem they saw before choosing to start from scratch, I could have
told them where the problem was, and they could have made a solid
binary tree implementation. But they didn't tell me.

So since then I've been giving Joe Arceneaux more supervision, and I
expect he will provide an interval implementation for Emacs 20. Not
19, because I don't want to delay that for these features.

I don't think Gabriel wanted this to happen. I think he had good
intentions but didn't explain them to the troops. They did what they
thought they were supposed to do, which was to implement a certain set
of features--with no particular requirement to discuss the design with
me.

I hope that Lucid would like to rectify the situation, but it's
entirely in their hands.

during the past year we dedicated
approximately 1 person-year, spread over 3 people, to producing a new
Emacs manual.

This might be a very useful thing. But why didn't you tell me sooner?

Last summer I was put in touch with someone at Lucid who was going to
work on updating the Emacs manual--documenting the new features
already implemented, I understood this to mean. (Later he told me he
had had to drop the project.) But your message today was the first I
had heard of a whole new manual. Is it free? Can I read it?

I'm sorry you feel it is hard to figure out how to work with me.
Sometimes I am hard to work with. Sometimes I'm stubborn. But I do
manage to work with a lot of other people.

Emacs is full of ideas contributed by others, some of whom contributed
the code as well. But you can't expect to bat 1000. Odds are I won't
like every single idea you suggest.

If you let me choose a subset of your suggestions, I'll be glad to
accept some of them. But if you make me choose all or none, most
likely I'll have to choose none. I don't think you meant to do that,
but when you don't include me in the plans until too late, that is the
effect.

There are some good ideas in what Lucid did. I've already accepted
one of them: I've asked Joe Arceneaux to implement temporary
highlighting intervals that are not "part of the text", in addition to
permanent intervals that are part of it. It was hearing how Lucid was
going to use this that made me realize it was important.

But I can't use this idea the easy way, by using Lucid's code, because
that code isn't designed to fit in with permanent intervals support.
Having temporary intervals is a good idea, but that doesn't make the
lack of permanent intervals a good idea.

This is why I would like people to adapt the Lucid code--so that I can
combine some of your ideas with the other ideas I want to use.

If you would like to work with me, the first step should be to tell me
what sort of thing you want to work on, before you start work. The
second step is to discuss the design with me before you commit to a
particular design. Then we can consider various alternatives and work
out a good combination of ideas.

So since then I've been giving Joe Arceneaux more supervision, and I
expect he will provide an interval implementation for Emacs 20. Not
19, because I don't want to delay that for these features.

What I understand from that is that if I want these features now
I should continue to use lucid-emacs as my Emacs 19, and wait
for a future Emacs 20 which might be some kind of a merge with
lucid-emacs, leading to one FSF Emacs.

So since then I've been giving Joe Arceneaux more supervision, and I
expect he will provide an interval implementation for Emacs 20. Not
19, because I don't want to delay that for these features.

What I understand from that is that if I want these features now
I should continue to use lucid-emacs as my Emacs 19, and wait
for a future Emacs 20 which might be some kind of a merge with
lucid-emacs, leading to one FSF Emacs.

Using the phrase "these features" implies that one set of features is
being discussed. That's not really true.(though there is some
overlap).

For exampe, Emacs 20 will support including formatting in the files
you edit, and also support temporary highlighting intervals. Lucid
Emacs supports only the latter. (Of course, our other versions don't
support either.)

As the person somewhat in the middle this extended phlegm, it strikes
me that both accounts of this history contain a certain degree of
inaccuracy.

Be that as it may, I would like to urge the parties involved to forget
all this and again make efforts to join forces. Lucid has some
talented hackers which can and have done good things for emacs (in
fact, version 19 already uses some of their code), and if the "bosses"
can work out their politics, the community will benefit.

I would like to urge the parties involved to forget
all this and again make efforts to join forces.

I heartily agree. I just a mere user but, from the outside, this is
starting to look like the Sun/AT&T vs OSF battle for the "One true
Unix".

GNU Emacs became a de facto standard because it had features nothing
else had and because it was widely available. Right now, it seems
that Lucid Emacs has those two qualities while GNU Emacs, alas, isn't
available. Please! Think of the community and the delays that would
be introduced by the confusion. Let's face it; Lucid is not going to
drop their version since sales depend on it. On the other hand, the
FSF is not satisfied with it and is going to redo some coding and
implement other features.

Consider this suggestion: let's all work towards making Lucid Emacs
the long awaited "official" Emacs 19, learn from it, and integrate the
FSF's long term plans into Emacs 20.

Thanks to Kyle Jones, we now have a pair of newsgroups bidirectionally
gatewayed into the bug-lucid-emacs and help-lucid-emacs newsgroups.
They are alt.lucid-emacs.bug and alt.lucid-emacs.help.

I expect that many of you will want to unsubscribe from the mailing lists,
and read the messages in newsgroup form instead. However, I would advise
you to wait a few weeks before unsubscribing, so that we know that the
gateway is functioning properly.

We wanted the newsgroups to be gnu.lucid-emacs.bug and gnu.lucid-emacs.help,
but the FSF folks are not willing to allow the creation of those groups.
In fact, it would appear that they are unwilling even to allow us to
announce the existence of our version of emacs on their newsgroups.

We're considering making a committment to Epoch or Lucid emacs and I
was curious about why you decided to make the switch - beyond the fact
that the hilit package makes the two comparable. Are there things
about Lucid emacs you prefer over Epoch? Any thoughts you have on the
subject would be appreciated.

Hans,

I made the switch from emacs 18.57 to epoch 3.2 a while back, the
biggest reason being the multiple screens from one process. Having
the ability to edit multiple buffers of text, and have more than one
on the display at once was very important. One other consideration in
making the switch was region hilighting via a mouse drag. As more
people started writing code for epoch, features like displaying a
filename at the point in another screen, man pages in another screen
with hyper-text like See Also sections, a nifty info-mode and so on
became part of my .emacs. Epoch gave the the x-support that seemed
intuitive to have in this environment. One the source code hilighting
came out, I was hooked. I cannot live without it. It is the one
thing that kept me from moving to lemacs sooner. The reason I wanted
to stop using Epoch was it has some intermittent bugs, like spontaneous
quitting, killing random xterms when it did quit, and just an overall
level of unstableness, not unusable, just annoying.

Lemacs probably has a similar level of annoyance, but it also has some
very nice features. I like the menubar (alot). I was using the bam
package (I forgot by whom) when I was using epoch. It gave me the same
functionality the the menu bar in lemacs does. The hilighting package
in several time faster under lemacs than under epoch (although they
are different packages, hilit vs brightlight, I get the same
functionality at 1/4 the cost) The interface into x with lemacs is
much cleaner, I get, out of the box, functionality that one would
expect in an x-based editor that would take several packages under
epoch. Things like double and triple clicks for word and line
hilighting and the menubar. Although I am not an elisp programmer,
the overall interface to lemacs is much cleaner than with epoch. It
makes (sorry guys) epoch look like a hack.

I made the switch from emacs 18.57 to epoch 3.2 a while back ...
The reason I wanted to stop using Epoch was it has some
intermittent bugs, like spontaneous quitting, killing random xterms
when it did quit, and just an overall level of unstableness, not
unusable, just annoying.

Umm, folks, Epoch 4.0 has been out for quite a while now and is quite
a bit more stable than 3.2 was. Spontaneous quitting, killing random
xterms, et al certainly ought to all be fixed in 4.0 (if they're not,
PLEASE let me know).

People should keep in mind that the successor of Emacs 18 does not
come from Lucid. Its is GNU Emacs 19. I still can't say exactly when
public release is going to be, but we are about to start testing at a
number of sites.

GNU Emacs 19 will support a broad spectrum of machines, like Emacs
18. Ensuring this is the purpose of the testing we are about to do.

(Please don't volunteer; we have enough pretesters, and if more people
offer, dealing with those messages will slow things down.)

Once we make sure it is indeed working reliably on various different
systems, we will have a public beta test release.

Emacs 19 does support adding properties to ranges of text, and using
these to switch fonts. In the future, the Epoch people will help
merge support for variable-width fonts.

Meanwhile, I have almost finished updating the Emacs Lisp manual. Its
next edition will describe Emacs 19. There will be an announcement
when we know when this edition will be available.

People should keep in mind that the successor of Emacs 18 does not
come from Lucid. Its is GNU Emacs 19. I still can't say exactly when
public release is going to be, but we are about to start testing at a
number of sites.

Free software is a great thing - freedom of distribution and use, and
freedom from price. It's such a great thing that some companies
recognize the value of giving things away, as a mechanism of goodwill,
of ``paying back'' the community it serves, and as a mechanism to show
potential customers the quality and style of software you can expect
from those companies.

Four years ago we embarked on a development project one of whose
central portions would be Emacs, but an Emacs with enhanced
capabilities - active regions, properties of text, fonts, color,
menubars, pop-up menus, and true X integration. Free Software
Foundation at the time was beginning to work on Emacs 19. Lucid paid
the author of the then-current Emacs 19 prototype to help us get it
ready. Well, $50,000 poorer and 18 months older, we were no closer to our
goal than when we started.

So, I assigned 5 of our best developers to work on Emacs 19, and last
summer we released it to the world. It runs on a bunch of platforms,
it is actively maintained (it is the centerpiece of our user interface
for our commercial product), thousands of people use it, we are
working with volunteers around the world, and - the best part - we
operate just like FSF on it: it's free, source code and all, there's
an updated manual for it, and we incorporate all reasonable changes
just the way FSF does.

The difference is, we have Emacs 19, it is well-tested, and its users
love it. Another difference is, we have 35 developers and we don't
need handouts to keep maintaining Emacs - it's just good business.

The concept of free software doesn't belong to anyone. The dream of
those who value free software is that the idea will be so compelling
that others will pick up the standard and improve it, that others will
take more time, do more things, champion the cause so that the
community get more because fresh blood is applied to it.

We waited for years and years for Emacs 19. The idea of free software
is a good one, and if you cannot live up to its needs, you should let
go. Free software is an idea so good that you shouldn't smother it by
claiming you own it. Free Software Foundation doesn't own the concept
of free software, and it doesn't own the exclusive rights to work on
Emacs.

Wait for the private tests, wait for the beta tests, wait for general
release, wait, wait, wait. Or, FTP it from /pub/gnu/lucid on
labrea.stanford.edu today and use it this afternoon. Help us move on
from Emacs 19 to Emacs 20.

Over two years ago, you hired the FSF staff person who I had charged
with getting Emacs 19 out the door soon with the features it had
then. I got about two weeks notice that he was leaving.

While you set him to work implementing new features that I had planned
for Emacs 20, I had to try to get Emacs 19 back on track. It took a
while for the new maintainer to come up to speed. The result: a long
delay. Emacs 19 would have come out two years ago if not for this.

Later you assigned programmers to work on another version of Emacs
without telling me about it. I found out months later by luck. Why
didn't you tell me? Why didn't you ask them to discuss the design
with me before they implemented it? Whatever the reason, you didn't,
which was not a very cooperative way of working.

Well, we have finally dealt with the problem caused two years ago, and
what happens? Now you are asking people to make up their minds that
GNU Emacs 19 is dead, quickly, before it has a chance to crack the
shell.

Overall your conduct leaves something to be desired.

I would be happy to see someone capable take over the work on Emacs
from the FSF. I might have been happy to see Lucid do so, if you had
approached it openly and with a good spirit.

Lucid can implement things. And some of them are good features. But
some of them are clunky. If you had asked the programmers to work
with me, I think it would be better. (People will soon be able to
compare the design of analogous features and form their own opinions.)

Above all, you've shown a bad spirit. I want Emacs to be in more
worthy hands than that. I think there are possibilities.

I have just about finished updating the Emacs Lisp manual. In the
next three weeks I'll update the user's manual.

I went back and found the mail exchange that you and I had about
Lucid's involement with Emacs 19 and FSF. I still think we tried as
hard as any reasonable organization can to work with you and FSF. I am
still saddened and puzzled by your peculiar memory of it. I reproduced the
exchange below so people can see it (again) and judge for themselves our
relative behavior.

Our Emacs is used widely and I have yet to hear anything but
embarassingly high praise for it. We only want to help the community,
but I think you just couldn't let go enough to allow us to help. There
are things that I think need to be done to it to make it better, but
unfortunately you and I continue to duplicate effort. Everyone loves a
race, I suppose.

Earlier I did not mention in public how you contributed
to the delay in Emacs 19, because I was being polite.
(Yes, I really do that sometimes.) I took the blame on myself
for the sake of amity.

But now that you are using our mailing lists to actively urge people
not to work with us, I have the feeling that it is too late to seek
amity. What you tell the public does not match what you actually do.

This is a handy cheap shot. It is always easy to make this
accusation, and no one can ever disprove it.

But I can point out that

I gave the work on Emacs to other people within the FSF,

I've given maintenance on other programs I wrote
to people outside the FSF, and

I'm raising funds now to give away the maintenance of GCC
to someone outside the FSF.

I politely kept quiet last year about your role in the Emacs 19 delay,
even when you were criticising me publicly,
in the hope of achieving cooperation with you.

I wouldn't have done these things if I were the sort of person whose
ego would never let go of a project. I hope I will find someone
suitable to hand over Emacs maintenance to. Someone competent,
cooperative and upright.

I don't think I've found someone yet. You would be capable enough if
you were sometimes willing to listen. But at every step of the way
you have given cooperation only lip service. It takes special gall to
present my earlier politeness as indication I am "misremembering" what
happened.

I believe in cooperating with people who approach me trying to
cooperate. At one point I hoped you were that person. But at this
point, your actions say no.

To the users:

I'm not going to ask you to stop cooperating with Lucid. Since Lucid
Emacs is free software, it is not wrong for you to cooperate with them
about it.

But I do appeal to you of GNU Emacs to continue working with the FSF
on GNU Emacs, and not to listen when Lucid asks you to stop.

It's always fun talking with you, Richard. Listening is a skill many
of us could stand to learn. I'll try to learn it if that's what you
think my problem is. Shall we try an experiment? I am ready and more
than eager to try to make this work one more time - I've got over a
million $ into Emacs already. Shall we cc our conversations on the design
of our cooperative efforts to this list?

Shall we try an experiment? I am ready and more
than eager to try to make this work one more time - I've got over a
million $ into Emacs already.

My understanding is that it is technically very hard to merge the two
versions now.

Lucid Emacs is set up to use and require an X toolkit; our version
does not use one. I think it is too expensive to require a toolkit.
Especially since we have already got most of the features we want,
without a toolkit.

In principle, I like the idea of supporting both modes of
operation--with toolkit and without. But the programmers at Lucid
told me last summer that using a toolkit required far-reaching changes
in many parts of Emacs, including making C-g fail to interrupt a
running program (something I worked hard a couple of years ago to make
work right).

So it sounds impractical to support both modes--at least, unless
something is done to change the requirements of the toolkit.

I don't blame Lucid for this. If anyone made a mistake, it was those
who designed the X toolkits.

But here is an idea. You mainly support Suns. They have threads.
More and more other systems have threads.

Would it be possible to use a separate thread to do the synchronous
interaction with the toolkit, and make it mimic the usual kind of
asynchronous input interface Emacs expects? And hide that within a
toolkit interface file that fits into GNU Emacs 19 in a modular way?
This might also make C-g work right in the toolkit case.

Perhaps this will make it possible after all to support both toolkit
and non-toolkit modes of operation. Perhaps then our versions could
be merged.

Two other areas of difference are in the design of menus and of
events. Lucid Emacs has separate opaque data types for these. GNU
Emacs uses pre-existing Lisp objects for the same job: keymaps serve
as menus, and ordinary numbers, symbols and lists serve as input
events. It should not be very hard to write Lisp functions to imitate
the Lucid functions, using these data structures, and then support
both interfaces.

A final area of difference is in intervals. But if that is the
only discrepancy, I'm sure it won't be hard to deal with.

Lucid Emacs is set up to use and require an X toolkit; our version
does not use one. I think it is too expensive to require a toolkit.
Especially since we have already got most of the features we want,
without a toolkit.

There are features that you don't have, such as guaranteed correct
interaction with X resources and app-defaults files, and the ability to
build other widgets into the emacs process.

Also, using a toolkit improves portability, since whoever ported the toolkit
has done all of the hairy select() hacking for you already. (But if you
want to compile without support for X (which is different from running
without X) then this is not an issue, because you have to do this
system-dependent hacking anyway.)

I think the overhead of using Xt is not anything to be concerned about;
the speed cost associated with Xt is insignificant, and the increase in
executable size is both A: small, and nonexistent if you are using shared
libraries, and B: proven to be acceptable by the already widespread use of
this toolkit and toolkits derived from it.

In principle, I like the idea of supporting both modes of
operation--with toolkit and without. But the programmers at Lucid
told me last summer that using a toolkit required far-reaching changes
in many parts of Emacs, including making C-g fail to interrupt a
running program (something I worked hard a couple of years ago to make
work right).

The event loop was rewritten to support the Xt-based command loop, true; but
it was rewritten in what I believe is a much more retargetable way. It is
not dependent on Xt; it just happens that currently Xt is the only backend
supported. I'm sure it would be much easier to write a home-grown non-Xt
backend for the event-loop with lemacs than it would be to make the FSF v19
command loop work with Xt. (In fact, that's why I designed it this way.)

Perhaps I was unclear in a previous message, but you seem to be confusing an
implementation detail of the C-g handing with the user-visible behavior. I
spent a lot of time ensuring that the emacs command loop, including its
handling of C-g as an elisp-level interrupt, is identical in behavior to
emacs 18 from the user's point of view. We would certainly not break things
badly enough that C-g didn't work!

I don't blame Lucid for this. If anyone made a mistake, it was those
who designed the X toolkits.

Mistakes were made in the design of the X toolkits, to be sure. But I don't
think they have adversely affected Lucid Emacs (aside from the time I had to
spend to work around them.)

There is a segment of the community that wants an Emacs that uses a
toolkit. The people who use Lucid Emacs are part of that segment.

There is also a segment of people whose machines won't support that.
I think there are a large number of such sites with limited memory and
no shared libraries, now using GNU Emacs. I know of some myself. I
also know I have always had complaints about how large Emacs is, from
people whose machines are small and overloaded.

Lucid won't get many complaints from these people. (If you don't like
liver, you won't complain to the chef that the chopped liver has liver
in it.) But that doesn't mean they are few in number.

There is also a segment of users whose machines don't have X windows.

If all three segments are large, a program that appeals only to one of
them can get an enthusiastic reception. I think that's what Lucid has
found. But if the FSF follows suit, a large number of users will be
unhappy.

So I would like add the capability to use a toolkit, without giving up
the ability to work without one. I'd like to add this in a modular
way, for the sake of future maintenance.

My beliefs and opinions about Lucid's changes in input structure are
all based on what Lucid's staff told me last summer. I was told the
toolkit insisted on calling Emacs as a subroutine, and would not
deliver another event until Emacs returned to it. I recall noting out
that this would stop C-g from working, and that someone confirmed
this. I asked if the toolkit provided any way to peek ahead, and was
told no.

This information dissuaded me from the goal of merging the two
versions. Other Lucid programmers agreed that it was impossible to
support both modes of operation because the changes were too
pervasive.

Now I think you are saying it is possible, and that you have a clean
design to propose. I'm glad to hear it.

Can you write up a technical proposal for how to change the Emacs
command loop and/or input reading routines in preparation for
supporting toolkit, non-toolkit X, and no-X-at-all modes of operation?

My beliefs and opinions about Lucid's changes in input structure are
all based on what Lucid's staff told me last summer. I was told the
toolkit insisted on calling Emacs as a subroutine, and would not
deliver another event until Emacs returned to it. I recall noting out
that this would stop C-g from working, and that someone confirmed
this. I asked if the toolkit provided any way to peek ahead, and was
told no.

Who told you this? If you think it was me, then we had a very serious
miscomunication, because none of this is true, and I have never believed
it to be true. And I'm the one who wrote basically all of the code in
question.

Can you write up a technical proposal for how to change the Emacs
command loop and/or input reading routines in preparation for
supporting toolkit, non-toolkit X, and no-X-at-all modes of operation?

Even better, I can point you at a working implementation, liberally
commented. I find it upsetting that, despite all of the criticism you
heap upon us, it is clear that you have never even *run* Lucid Emacs.

Here is a description of what Lucid Emacs currently does at the lowest
level:

My beliefs and opinions about Lucid's changes in input structure are
all based on what Lucid's staff told me last summer. I was told the
insisted on calling Emacs as a subroutine, and would not
deliver another event until Emacs returned to it. I recall noting out
that this would stop C-g from working, and that someone confirmed
this. I asked if the toolkit provided any way to peek ahead, and was
told no.

Who told you this?

I don't know which person at Lucid told me this. But that doesn't
seem important. What matters is that it isn't so. I'm glad, because
it was discouraging news at the time.

The event_stream mechanism sounds good. Could you send me the code
for that part of Lucid Emacs? I think we can install it, if you will
do the job of separating it out from the mass of the rest. (This is
something you can do much more easily than I.) I doubt it can
be installed verbatim, since we've probably made changes in the
surrounding context, but at least it sounds like a job worth doing.

The event_stream mechanism sounds good. Could you send me the code
for that part of Lucid Emacs? I think we can install it, if you will
do the job of separating it out from the mass of the rest. (This is
something you can do much more easily than I.) I doubt it can
be installed verbatim, since we've probably made changes in the
surrounding context, but at least it sounds like a job worth doing.

I'll send you the files, but I doubt there is much surrounding context
left at all.

Where else are changes needed to work with your toolkit-use code?

It's hard to say; our changes have been pervasive. As you said, a merge
will be a very large task.

It's hard to say; our changes have been pervasive. As you said, a merge
will be a very large task.

It's almost impossible to merge one program with another as a unit,
when there is a lot of difference. By far the easiest way is to find
cohesive "unit changes" in one program--various parts that were
changed together and need each other to work, but are separate from
other changes--and think about them one at a time.

The merging in of the changes is most efficiently done by us; the
extraction is most efficiently done by you.

What part of the job of producing a merged version is Lucid
willing to do?

If the work of merging is up to the FSF, then I can say how we will do
it. We'll try to make a version that has all the capabilities of both
versions, with as little work as possible. In many cases, we have
both implemented pretty much equivalent functionality; we will stick
with our implementations of these, while merging in capabilities that
we don't have. (These mainly relate to Xt use, and things needed to
support it.)

Once such a merged version exists, what would happen then? Will you
work on that version, together with us?

I hope so. If you keep working on new features in the current Lucid
version, each feature will require considerable rewriting to fit into
the merged version. That way the job of merging would be unending.

We tried very hard to make the event-stream model general enough to support
other window systems, and to keep the X-specific code well isolated. There
are currently a lot of Xisms in lemacs, but they're almost entirely related
to redisplay, which we haven't done much work on yet.

I've always thought that a native NeXTstep port would be great, because I
think NeXTstep is a nice window system. Certainly the design won't have
been truly vindicated until it is used with a window system other than X.
But I suspect that a Windows NT port will be more useful...

And we could also write wrapper functions for the next which
implemented whatever X calls emacs was making.

I know of someone who has done this (simulate Xt) for another window system.
I think it's a pretty poor way to go about it, though it might be easier in
the short term.

However, I wonder how gross and low level these calls are. If they are
things like moveXY(x,y); drawString(font, "string"); we could do those
easily.

Lemacs doesn't exercise Xlib very heavily (there are probably some
abstractions missing, like getting the pixel-size of a string in a particular
font, but that's easy to fix), and all of the Xt stuff is isolated under the
event-stream callbacks.

-- Jamie

Newsgroups:

gnu.emacs.help

Date:

1 Mar 1993 17:59:57 -0500

From:

rms@gnu.ai.mit.edu (Richard Stallman)

Subject:

Emacs 19

There is also a segment of users whose machines don't have X
windows.

Does this imply that FSF's emacs-19 might support other windowing
systems?

I was talking about people with no window system.

In principle, we could support other window systems.
In practice, we don't have the resources to do so--sorry.
It would take a lot of work and would in no way enhance
the GNU system.

It's almost impossible to merge one program with another as a unit,
when there is a lot of difference. By far the easiest way is to find
cohesive "unit changes" in one program--various parts that were
changed together and need each other to work, but are separate from
other changes--and think about them one at a time.

The merging in of the changes is most efficiently done by us; the
extraction is most efficiently done by you.

I am totally uninterested in doing the merge the other way, if for no
other reason than that I believe the people at Lucid working on Emacs
are far more competent and more far tasteful programmers, and aren't
wasting large amounts of FSF resources which would much, much better
have been spent elsewhere.

Working on programs with people like Jamie Zawinski (or many other
excellent hacker types) is also vastly easier than trying to cooperate
with RMS on anything.

Our concerns about merging Lucid Emacs with FSF v19 are that there is
insufficient motivation on the part of FSF to compromise. In particular, we
are concerned that FSF will decide not to include features that we consider
critical, or that those features will have an interface that is quite
different from what we need and are implemented in a way that preclude
supporting the interface that we need.

However, if there was a version of emacs from the FSF that did everything that
we require from Lucid Emacs, then we would use it. Also, if v19 has a better
interface to something than what we designed, we would be motivated to use it.
(In general, a change in the interfaces will make the merge harder, and make
it take longer, but it doesn't preclude our switching. An implementation or
interface that eliminated functionality important to us would preclude a
merge.)

Here are the things (that we can think of at the moment) that we believe we
need in a merged version, in no particular order:

We don't want to lose our tight integration with the Xt event loop, so that
we can link arbitrary widgets into emacs.

We want our implementation of keymaps to be used: we want them to be an
abstract data type, not something like "if the third element of the alist is
a cons whose car is a vector of length 7, then it represents an aliased
indirection into the sixth element of the alist..." (NOTE: Having support
for "secondary" data types, similar to structure subtypes in Common Lisp,
would be quite acceptable, but alists are not.)

We don't want any ASCII limitations: all X chords should be bindable.
We want our extensions to the syntax for the define-key function to work.

We want input and display of all ISO-8859-1 characters.

We want ICCCM compliance.

We want a menubar, popup menus, and dialog boxes, and the ability to use the
native toolkit widgets to implement it to guarantee that they comply with
local look-n-feel customs.

We want the menubar to be buffer-local, and update efficiently.

We want the menubar to display the keybindings corresponding to the commands
listed in it, and update this dynamically.

We want to avoid compiling pathnames into the emacs executable: it should
figure out where things are at run-time based on the location of the
executable.

We do not want menus implemented in terms of keymaps or anything odd like
that. This sort of false generality just makes things harder to maintain.

We want our implementation of extents and faces, or something with a
superset of the functionality and performance characteristics.

I'm glad that you would be willing to use a merged version *if*
the FSF does all the work of producing it.

But this sounds like a statement that Lucid is not willing to do any
of the work necessary to produce a merged version. I'm sad to hear
that. At the same time, Lucid sounds very exigent regarding what
features we should include and exclude.

Thus, we have a combination of stringent demands, with a refusal to
help accomplish them. The combination sets up a situation where
cooperation is predictably unlikely. We will certainly merge in the
features that seem important (those we don't have already), but
whether we merge in every last thing Lucid wants, depends on our
resources and on how generally useful the features seem to us.

We can install more of the features Lucid wants if Lucid helps do the
job.

A couple of specific points are disturbing in another way: they seem
to insist that we remove some of the features of GNU Emacs 19--such
as, its unification of menus with keymaps, and its use of standard
Lisp data types rather than creating new opaque types.

Is there any chance you could accept a compromise? Such as support
for the interfaces that you now use, as well as the ones we prefer?

But this sounds like a statement that Lucid is not willing to do any
of the work necessary to produce a merged version.

This is not the case; we are willing to help work towards a merge.
We are not willing to do it ourselves unless you say that we get to
be the final arbiter of what goes in. Without that, we do not, at
this time, have any confidence that we won't be wasting our time.

The facts speak for themselves: we have tried very hard to work with
the FSF. We have spent hundreds of thousands of dollars and several
man-years on it. Clearly we have not done so successfully, regardless
of where the fault for the failure lies, but we did try, that is
undeniable.

We don't think it's too much to ask for you to make the first move as
a show of faith. We've been burned before. We'd like to see some
commitment from your side before committing any more money or resources
to this endeavor.

At the same time, Lucid sounds very exigent regarding what features
we should include and exclude.

Whether you include or exclude them from your version of emacs is your
decision. But these are features we need, and if they are not in your
version of emacs, then your version of emacs will simply not be adequate
for our purposes.

A couple of specific points are disturbing in another way: they seem
to insist that we remove some of the features of GNU Emacs 19--such
as, its unification of menus with keymaps, and its use of standard
Lisp data types rather than creating new opaque types.

You call them features, we call them bugs. We have contractual obligations
to maintain the version of emacs that we distribute, and the design of the
data structures used has a great bearing on how difficult that is.

Is there any chance you could accept a compromise? Such as support
for the interfaces that you now use, as well as the ones we prefer?

Working on a version of Emacs and working with the FSF are not
identical. In the past, Lucid has done a lot of work on Lucid Emacs,
but only rarely tried to work with the FSF.

Your latest message states the intention to do some work on merging in
the future, which is good. The emotional tone, shows a contrary
attitude. I'm not sure what to make of this mixed message.

The FSF will merge in some features of Lucid Emacs in any case, now
that I see they are reasonable to use. Specifically, those that
pertain to generic interfaces to window-system support code, followed
by the support for an X toolkit as a compile-time option.

In principle, there's no capability that we would object to supporting
in some fashion. Some we would do the work to add, but some we would
not (if we think it is of minor importance or that we have a better
capability already). Thus, merging of the latter class depends on
what Lucid does.

I need to study your implementation of menus and keymaps. I am away
all next week.

My worry is that the two abstractions should not be mixed up unless
there is a fundamentally good reason to. Implementation ease or
implementation shortcut is not a reason for me. But, I'll look at it.

As an avid Epoch and NeXTstep user, I'd be interested in hearing more about
the possibility of generalizing the windowing interface of Emacs 19.

I'm not familiar with Lucid Emacs, so I don't have a feel for whether
this will be necessary, practical, or even desirable.

From rms/lucid discussion, it wasn't entirely clear whether or not supporting
both the vanilla X & Xt toolkit designs would require enough
restructuring to make another layer of indirection worthwhile.

I'd like to work on native NeXTstep support (if such an interface could
be implemented).

Based on what I've heard in the recent discussion, it ought to be
possible for GNU Emacs to support Xt in the future as an optional
feature. The first step towards this will be supporting a generic
interface for reading input events. GNU Emacs 19 already has a
generic event structure, much like the Lucid one, so the main job is
supporting general hooks for interface to the program that does the
reading.

The second step might be to support a new kind of frame that
interfaces via Xt instead of via Xlib, and perhaps to create a set of
clean hooks for that interfacing.

Does this in any way imply that a merge of the two versions is getting any
closer?

GNU Emacs incoroporates many of the features of Lucid Emacs. We are
using sections of Lucid's code, and have benefited from their advice.

... and have deliberately implemented other features in an
incompatible way, so as better to screw all Emacs users.

We have a Lucid emulation package (lucid.el) which may or may not be
terribly effective; we'll have to see it in use.

Not terribly much. (The real advances in Lucid's code are in areas in
which Stallman has deliberately chosen to implement incompatible
functionality, even given a Lucid precedent.)

The FSF's "lucid.el" consists largely of

aliases of functions named "*screen*" in Lucid Emacs and Emacs 18
but renamed "*frame*" in FSF 19 (this is a desirable name change,
but I believe the Lucid Emacs authors preferred compatibility
over better nomenclature at the time.) I imagine that Lucid may
change also, given a precedent for user pain.

some functions which it is totally insane not to include as
part of basic Emacs (keymap-parent, map-keymap, etc)
"What's data abstraction anyway?"

a couple of miscellaneous utilities (eg add-timeout)

Not anything really effective on the whole, I'd imagine, to deal with
the real incompatibilities (the event-stream, faces and extents.)
As far as I can see, RMS' whole course has been to make everything to
do with the real issues as divergent as possible by systematically
substituting worse for better.

Some features of Lucid we will probably never have; Stallman doesn't
like some of them. Since the people with real influence over the
designs of the products (Zawinski, Stallman) are both rather stubborn,
I wouldn't hold my breath for a complete unification.

My aim wasn't a "unification", either because the RMSmacs code is
simply perverse and wrong in many areas (eg events, keymaps) or
because it has diverged (deliberately) too far from Lucid's code for a
simple effort to be made by a non-window-system kind of guy (faces,
screens, etc.)

My aim was to incorporate as many compatible changes made in the FSF
code (many bug fixes, some features, many dubious kludges like
find-file-name-handler) as possible into the exiting Lucid Emacs
distribution, both to take advantage of good work done by FSF Emacs
hackers and to make all future source-compare-and-merge operations
simpler (if anybody has the resolve, technical ability and time to
perform them).

Several people have bashed GNU and RMS lately about their alleged
disregard for established precedent. Just to set the record straight,
the Lucid folks are no less guilty of this particular sin. Lucid
threw out RMS's event and keymap model, for example. Maybe Lucid's
approach was better (not entirely clear; each system has plusses and
minusses). But RMS was there first, and just because Lucid Emacs was
released earlier is no reason to fault RMS for sticking to his original
scheme.

GNU Emacs 19 makes a much greater attempt at backward compatibility.
One little thing I noticed recently is that the unread-command-char
variable is still supported, *as well as* an alternative that holds
events instead of chars. Lucid simply deleted the old variable,
and it was a giant pain for me to fix all the references to it in
Calc. I sure wish Lucid had put compatibility at a higher priority.

Lucid threw out RMS's event and keymap model, for example. Maybe Lucid's
approach was better (not entirely clear; each system has plusses and
minusses).

What do you think is better about RMS's event and keymap model, beyond
the fact that it was there first (i.e., compatibility)?

One little thing I noticed recently is that the unread-command-char
variable is still supported, *as well as* an alternative that holds
events instead of chars. Lucid simply deleted the old variable,
and it was a giant pain for me to fix all the references to it in
Calc. I sure wish Lucid had put compatibility at a higher priority.

The new event model in Lucid Emacs is a portability problem, to be sure.
I did what I could to remain compatible with the old model (keeping
last-command-char as the ASCII equivalent of last-command-event, for
example.) But I could not think of a way to keep unread-command-char
and still *make progress*.

My goal was to make emacs a better editor than it was before. Perhaps you
don't think that removing the restriction to the ASCII character set was
worth the pain; I think it was.

If you don't like the design decisions I made, please suggest better
ones. Converting to ASCII and back loses information. If a package were
to read and unread a character as ASCII, it could easily cause the wrong
command to be executed. I could put this hack in, and it would mostly
work most of the time, but it's the wrong thing. There are arguments in
favor of improving software by adding kludge on top of kludge instead of
stepping back and reevaluating the original decisions. I chose not to
take that approach. Maybe you disagree. But I gave you an editor more
than a year ago that I think was a vast improvement over what I started
with.

What do you think is better about RMS's event and keymap model, beyond
the fact that it was there first (i.e., compatibility)?

One: Simple keystroke events are represented the same as they were
before, which goes a long way toward compatibility with Emacs 18.
The mapping between strings and event vectors is much more natural,
leading GNU Emacs 19 to provide still more compatibility in that area.
Keymaps, while not backward-compatible, are fairly close.

Two: The most common events are atomic objects, which means he doesn't
need to resort to explicit memory management of event objects to keep
from garbage collecting all day long. The bulk of events don't need
to be collected because they're just symbols or integers.

Three: Events and keymaps are built from plain Lisp objects, which
makes them easier to examine during debugging, and easier to construct
and modify if necessary.

Four: Events are more usage-oriented, less X-oriented. This will
probably make for an easier transition to other window systems in
the future.

Those are the things inherent to the representation that I like. He
also has some features that you don't have, but that you could add
without losing your event model. The handling of down, up, and drag
events is nice. The Lucid model acts on the click, whereas RMS's
model usually acts on the release. This seems to work fine in
practice and allows him to generate drag events for the user.
(Although he doesn't yet, he could also generate double-click events
on the second of two consecutive clicks---hint hint, nudge nudge.)
It also fixes the stupid problem where mouse commands leave a
message behind only to have it erased when the release-event is
processed.

On the other hand, your method has its own advantages:

A: Keystroke events carry position and timestamp information, which
allows keys to emulate extra mouse buttons in a pinch.

B: All events have the same representation, which has a certain
elegance to it. (This elegance is muffled by the need to keep the
garbage collector away from event management, though.)

C: There was something else I liked, but I can't think of it offhand.
It will come back to me eventually.

Overall though, I seem to have found more items in RMS' favor.
Before I made the list, I was thinking of the two systems as about
equal in merits.

But I could not think of a way to keep unread-command-char
and still *make progress*.

GNU Emacs simply has both. If there is a character in unread-command-char,
it stuffs it into the input stream, and if there are events in
unread-command-events, it stuffs those, too. There is some order
of priority, but it doesn't really matter.

You could still have retained compatibility by using the name
"unread-command-char" instead of "unread-command-event" for your
current variable, and allowing it to contain integers as well as
event objects.

My goal was to make emacs a better editor than it was before. Perhaps you
don't think that removing the restriction to the ASCII character set was
worth the pain; I think it was.

Maybe RMS' system has evolved since you first saw it. He represents
function keys as symbols, and he can also represent a full set of
modifier bits on printable keys. In terms of keys themselves, both
systems are equally expressive as far as I can see. Yours does
include time stamps, though, which could conceivably be handy. (Not
enough to choose them over the simplicity of RMS' model, though, in
my opinion.)

If a package were
to read and unread a character as ASCII, it could easily cause the wrong
command to be executed.

Yes, but there are about a zillion places in Calc where I stuff an
event that I *know* is a printable character, and I had to change
every single one of these.

Your model does not detect incompatible code, either, because older
programs can simply go ahead and store something in "unread-command-char"
and then silently fail.

But I gave you an editor more
than a year ago that I think was a vast improvement over what I started
with.

What do you think is better about RMS's event and keymap model, beyond
the fact that it was there first (i.e., compatibility)?

One: Simple keystroke events are represented the same as they were
before, which goes a long way toward compatibility with Emacs 18.

Yes, I realize this; that's why I said "beyond compatibility." I wanted
to know what you meant besides this.

The mapping between strings and event vectors is much more natural,
leading GNU Emacs 19 to provide still more compatibility in that area.

Keys and buffer-characters are not the same thing. Strings are made
of the stuff that buffers are made of. Keys are not. Show me how to
represent Control-Shift-A or Control-Backspace in a string.

Two: The most common events are atomic objects, which means he doesn't
need to resort to explicit memory management of event objects to keep
from garbage collecting all day long.

You don't need to resort to explicit memory management. You can drop the
events on the floor and they will be garbage collected like any other lisp
object. It's simply more efficient to reuse them (this probably only
matters in one place in all of emacs: the single event that is reused by
the main command loop; and even there it hardly matters at all.)

It such a common misconception that you *need* to explicitly manage events
that I'm starting to regret that I made it possible at all...

Three: Events and keymaps are built from plain Lisp objects, which
makes them easier to examine during debugging, and easier to construct
and modify if necessary.

And harder to programmatically tell apart from other things ("eventp" is
unpleasantly heuristic.) But perhaps they should be less opaque; give me
examples where you've wanted this.

Four: Events are more usage-oriented, less X-oriented. This will
probably make for an easier transition to other window systems in
the future.

I very carefully designed the event structures to expose only those
components that emacs would need when running under *any* window system.
They are not X specific at all; why do you say they are?

The Lucid model acts on the click, whereas RMS's model usually acts on
the release.

Lucid delivers both up and down events, and you can bind commands to either.

This seems to work fine in practice and allows him to generate drag events
for the user. (Although he doesn't yet, he could also generate double-click
events on the second of two consecutive clicks---hint hint, nudge nudge.)

Handling of double click and drag events is controverial and varies from
window system to window system. That's why I didn't build them into the
C substrate, but instead made them be processed up in lisp. (But this is
a tangential point; if you want to talk more about it, send me mail.)

It also fixes the stupid problem where mouse commands leave a message
behind only to have it erased when the release-event is processed.

Hey, this is just a *bug*, it has nothing to do with the event model per se...

B: All events have the same representation, which has a certain
elegance to it. (This elegance is muffled by the need to keep the
garbage collector away from event management, though.)

See above.

GNU Emacs simply has both. If there is a character in unread-command-char,
it stuffs it into the input stream, and if there are events in
unread-command-events, it stuffs those, too. There is some order
of priority, but it doesn't really matter.

You could still have retained compatibility by using the name
"unread-command-char" instead of "unread-command-event" for your
current variable, and allowing it to contain integers as well as
event objects.

The reason I didn't do this is that the canonical use of unread-command-char
is something like this:

If the user types some key with an unambiguous ASCII equivalent, this works
fine. But what if the user types Control-Shift-A or Shift-Linefeed, which
have no ASCII equivalent. Right now, lemacs's read-char signals an error when
C-Sh-A is typed, and returns 10 for Sh-LFD (the same as LFD). This means that
either this code would fail to work with many otherwise legitimate keystrokes,
or would unread LFD instead of Sh-LFD, which could have a different binding.
Likewise, when this code reads and unreads \010, we lose the distinction
between Control-h and Backspace.

Again, maybe you think that compatibility is more important than the feature
that BS and C-h aren't the same thing. That's a perfectly legitimate opinion.
But I can't think of a way to have both.

Using the FSF19 model, an int would be returned for C-a and an "event" would
be returned for C-Sh-A. But what about BS/C-h? Which one returns an event?
Both are equally bad choices.

Maybe RMS' system has evolved since you first saw it. He represents
function keys as symbols, and he can also represent a full set of modifier
bits on printable keys. In terms of keys themselves, both systems are
equally expressive as far as I can see.

The last time I checked, C-h, C-Shift-H, BS, C-BS, and C-Shift-BS were not
distinguishable in the FSF version. Has this changed?

Yes, but there are about a zillion places in Calc where I stuff an
event that I *know* is a printable character, and I had to change
every single one of these.

I understand that you've had massive compatibility problems. But in my
defense, I think that Calc is about as atypical an emacs application as you
can find. Most packages just don't work that way; by far the most common
use of unread-command-char is to unread an arbitrary character read from
the user.

Your model does not detect incompatible code, either, because older
programs can simply go ahead and store something in "unread-command-char"
and then silently fail.

Yes, this is true. However, it does issue a compilation warning. I could
make setq signal an error when storing into unread-command-char, but that
seems pretty perverse...

My first impressions of FSF v19 vs. Lucid Emacs 19: I liked having
scroll bars and having horizontally-split windows, selective display
etc. working, but the FSF menus are *tacky*. IMHO it's high time FSF
emacs began to use XT and widgets properly -- anyone who has X has XT
anyway. Lucid's event and keymap implementation is generally better.

In the Lisp library, however, FSF emacs was much more polished. Lucid
Emacs's Lisp is (except for that which had to be changed due to
changes in Lucid's C code) largely v18 with a hodgepodge of seperate
packages tied on.

The GNU documentation was better, too.

I also liked the scrollbars, but the menus were horrid. And why did they
choose to not go with motif or lucids motif style widgets for the menus
and the scrollbars?

I would love the next version of LucidEmacs to have the capability to
place vertical & horizontal scroll bars (motif style) on each edit area.
Any plans for this?

Keys and buffer-characters are not the same thing. Strings are made
of the stuff that buffers are made of. Keys are not. Show me how to
represent Control-Shift-A or Control-Backspace in a string.

But some keys are. In fact, statistically nearly all keys that are
actually typed are. Likewise, many key sequences dealt with by
programs are. It's silly not to incorporate this important fact
into a program's concept of events.

Lucid Emacs does this anyway, by allowing strings in some contexts
where event vectors are expected. But it's a kludge---only some
contexts allow this, where others require you to use "event-to-character"
and its inverse.

RMS' model represents all the key distinctions yours does, but the
subset of keys which are ASCII characters are represented by their
ASCII codes. It's very natural. All vector operations in Lisp apply
to strings as well, which can be thought of simply as special vectors
optimized to store only ASCII characters. So the relation between
strings and event vectors also becomes very natural in RMS' scheme.

It's not that your system doesn't have its own advantages; it's
just that I've seen some claims that your system is obviously
and completely superior, which is simply not the case.

You don't need to resort to explicit memory management. You can drop the
events on the floor and they will be garbage collected like any other lisp
object. It's simply more efficient to reuse them [...]

I know, and that is commendable. It's purely a matter of efficiency,
and as you say it only matters in one point in the command loop. But
because of that one little thing, little uglies show up all over
the place in Lemacs' event functions. I'm thinking of "next-event",
which takes a blank event object to be filled in rather than simply
returning a new event object. That's inelegant and un-Lispish; it's
an expedient that isn't necessary in RMS' model.

It such a common misconception that you *need* to explicitly manage events
that I'm starting to regret that I made it possible at all...

I never misunderstood; I just interpreted your design as deriving
from a *need* to manage events in the command loop for performance
reasons. If this kludge had been hidden inside the command loop
it wouldn't have mattered, but in your design, "next-event" *must*
take an event to be filled in rather than consing one up itself,
"last-command-event" *must* be copied because it will be scribbled
on by the next input, and so on. Again, your event model is hardly
crippled by any of this, but it *is* inelegant in a system whose
main virtue, as you've argued, is its elegance. RMS' model neatly
sidesteps the whole issue. That, in its way, is elegant.

And harder to programmatically tell apart from other things [...]

Lists are used for all kinds of things right now in Lisp, and it
doesn't seem to hurt anything. Lambda expressions can't be
distinguished from lists except by context, but that's okay because
context is enough. Likewise for events and keymaps. You know it's
an event because you got it from a place that has events in it.

But perhaps they should be less opaque; give me
examples where you've wanted this.

I'm tempted to, but I will resist---it misses the point. In some
contexts, like large object-oriented software systems, opaque
data types are a virtue. In others, flexibility and malleability
are the prime virtues. I think Emacs falls in the latter class.
For a system like that you don't want to wait for examples of why
it would be good to have access to a particular thing, you want to
give access to as much as possible *on the assumption* that some
ingenious soul will later find a way to use it.

That's why I worry a little bit about the fact that RMS' model
forgets the time stamps on keystroke events. I have only a vague
sense of when these would be useful, but I have absolute faith
that people, given this information, will be able to do cool things
with it. (I had misremembered your event model as storing mouse
positions in keystroke events, too, which could also be useful
though less portable between window systems.)

I very carefully designed the event structures to expose only those
components that emacs would need when running under *any* window system.
They are not X specific at all; why do you say they are?

I went back and looked over your events, and, though they are X-like
in character, I agree that they are no more X-dependent than GNU's.

The Lucid model acts on the click, whereas RMS's model usually acts on
the release.
Lucid delivers both up and down events, and you can bind commands to either.

Same in GNU Emacs 19, just that commands are now traditionally bound
to up instead of down. But the way the default bindings works sets
a pretty strong precedent for the way people will do their own
bindings.

It also fixes the stupid problem where mouse commands leave a message
behind only to have it erased when the release-event is processed.

Hey, this is just a *bug*, it has nothing to do with the event model
per se...

Not with the event model itself but with the decision to process on
up instead of down events. Emacs clears the echo area before processing
a command; if the down click is an active operation then the up click
acts like a null command with the side effect of clearing the last
message. Fixing the bug with active down-clicks would require a
kludge, but with active up-clicks the whole problem goes away.

You could still have retained compatibility by using the name
"unread-command-char" [etc...]

The reason I didn't do this is that the canonical use of unread-command-char
is something like this:
[Reading a character and stuffing it back]

Again, maybe you think that compatibility is more important than the feature
that BS and C-h aren't the same thing. That's a perfectly legitimate opinion.
But I can't think of a way to have both.

I understand that the "read-char"/"unread-command-char" idiom now loses
information. Nobody is asking you to forego an "unread-command-event"
feature, just that providing a backward-compatible feature *as well*
will allow code to hobble along until someone has time to fix it.
You don't have to give one up for the other. Just have both, and then
everybody is happy in both the short term and the long term.

The last time I checked, C-h, C-Shift-H, BS, C-BS, and C-Shift-BS were not
distinguishable in the FSF version. Has this changed?

Yes. They are all distinguishable.

Using the FSF19 model, an int would be returned for C-a and an "event" would
be returned for C-Sh-A. But what about BS/C-h? Which one returns an event?
Both are equally bad choices.

The event for C-Sh-A is a number with a few high-end modifier bits set.
The event for C-h is 8, and the event for BS is the symbol "backspace".
This is rather like Lucid, where C-h is treated as a regular control
character and BS is treated as a function key. You both had to make
the choice, and you both happened to make the same choice.

RMS' model is more irregular in the way it handles modifier bits and
things. Lucid Emacs is much more orthogonal there, which is nice.
But I expect the two systems will be equally convenient to deal with
in practice.

I could
make setq signal an error when storing into unread-command-char, but that
seems pretty perverse...

I'll second that! I thought of it, too, but I didn't want to suggest
it in public. :-)

Will the next release of Lucid Emacs contain most of the Emacs Lisp
changes to FSF v19.8? Obviously some things aren't compatible with
the way Lucid Emacs works, but how about the rest?

My first impressions of FSF v19 vs. Lucid Emacs 19: I liked having
scroll bars and having horizontally-split windows, selective display
etc. working, but the FSF menus are *tacky*. IMHO it's high time FSF
emacs began to use XT and widgets properly -- anyone who has X has XT
anyway. Lucid's event and keymap implementation is generally better.

In the Lisp library, however, FSF emacs was much more polished. Lucid
Emacs's Lisp is (except for that which had to be changed due to
changes in Lucid's C code) largely v18 with a hodgepodge of seperate
packages tied on.

Lists are used for all kinds of things right now in Lisp, and it
doesn't seem to hurt anything. Lambda expressions can't be
distinguished from lists except by context, but that's okay because
context is enough. Likewise for events and keymaps. You know it's
an event because you got it from a place that has events in it.

I don't think you'll find much anybody involved in the Lisp world
(apart from those with their tastes set in concrete circa 1972) who
believe that the ad-hoc mess of "vectors of length 259 of lists whose
first element is the symbol `indirect' and whose tail is an alist of
conses whose ..." is any substitute for actual structure objects (of
varous degrees of opacity.)

It is one thing to argue that such perversions might make sense for
some reason of compatibility, but quite another to claim that
representing hash-tables and keymaps and events and so forth in terms
of conses and vectors and symbols is in any way a feature. (Why
aren't windows and processes and window configurations represented in
such a way in FSFmacs anyway?)

By the way, FSFmacs doesn't allow "(fillarray (make-keymap))" any
more. This "incompatibly broke" a bunch of code I wrote.

Lucid Emacs does this anyway, by allowing strings in some contexts
where event vectors are expected. But it's a kludge---only some
contexts allow this, where others require you to use "event-to-character"
and its inverse.

I believe that all functions which take event vectors also take strings,
and all functions which take events also accept ASCII integers.

unread-command-event must be an event, not an ASCII integer. I have no
particular objection to making it able to take an integer as well, except
that I don't see it as particularly useful (and the event- accessor
functions would need to be modified to work on ASCII integers as well, or
all code which references unread-command-event would have to be modified to
check to see if it is really an integer, either of which I would consider
an ugly wart.)

All functions/variables which return a key or sequence of keys return
events or vectors of events. I think this is a good thing because of
uniformity: keys are events, not sometimes events and sometimes integers.
It makes less work for the programmer.

YES, it is incompatible. YES, it makes more work for people porting their
code. I'm talking about purity of design, not about compatibility with
existing code.

RMS' model represents all the key distinctions yours does, but the
subset of keys which are ASCII characters are represented by their
ASCII codes. It's very natural.

It makes more work for the programmer because the type of the key changes.

I know, and that is commendable. It's purely a matter of efficiency,
and as you say it only matters in one point in the command loop. But
because of that one little thing, little uglies show up all over
the place in Lemacs' event functions. I'm thinking of "next-event",
which takes a blank event object to be filled in rather than simply
returning a new event object. That's inelegant and un-Lispish; it's
an expedient that isn't necessary in RMS' model.

You're right; the arg to next-event should be optional.

Lists are used for all kinds of things right now in Lisp, and it
doesn't seem to hurt anything.

(Mly's reply said exactly what I would have said to this.)

For a system like that you don't want to wait for examples of why
it would be good to have access to a particular thing, you want to
give access to as much as possible *on the assumption* that some
ingenious soul will later find a way to use it.

This is far from clear to me. The fact that "everybody knows keymaps are
just alists/vectors" was a big problem. Lots of code got written that used
aset and fillarray instead of the "advertised" interface of define-key,
making it harder to add power to keymaps (in both lemacs and FSFmacs.)

It also fixes the stupid problem where mouse commands leave a message
behind only to have it erased when the release-event is processed.

Hey, this is just a *bug*, it has nothing to do with the event model
per se...

Not with the event model itself but with the decision to process on
up instead of down events. Emacs clears the echo area before processing
a command; if the down click is an active operation then the up click
acts like a null command with the side effect of clearing the last
message. Fixing the bug with active down-clicks would require a
kludge, but with active up-clicks the whole problem goes away.

Well I really think we're getting off the track here, but it is my opinion
that if one *does* choose to bind a command to a downstroke (whether that's
the prescedent or not), and there is no command on the upstroke, then the
upstroke should not cause the minibuffer to be cleared. This used to work
in lemacs, but it's broken right now.

I understand that the "read-char"/"unread-command-char" idiom now loses
information. Nobody is asking you to forego an "unread-command-event"
feature, just that providing a backward-compatible feature *as well*
will allow code to hobble along until someone has time to fix it.
You don't have to give one up for the other. Just have both, and then
everybody is happy in both the short term and the long term.

I think that having existing code hobble along in way that sometimes
loses information is not a good solution. I have never subscribed to the
Unixlike idea that "mostly working most of the time" is good enough. I
think it's better for incompatible changes to cause code to not work at
all than to lose in obscure ways. Otherwise it will never get fixed.

If I could see a way to cause existing code to work 100% without
compromising the design, I would do it. If I thought that compatibility
with v18 was important enough to compromise the design, I would do it.

The event for C-Sh-A is a number with a few high-end modifier bits set.
The event for C-h is 8, and the event for BS is the symbol "backspace".

Then in FSF19, there are really *three* representations of events: ASCII
codes; ASCII-like codes with bits beyond #7 set; and symbols. In lemacs,
there is but a single representation (with strings allowed as well in the
interest of compatibility). The FSF model is good for compatibility, but
I think my model is a cleaner (and "more Lispy") way of doing things.

In the Lisp library, however, FSF emacs was much more polished. Lucid
Emacs's Lisp is (except for that which had to be changed due to
changes in Lucid's C code) largely v18 with a hodgepodge of seperate
packages tied on.

For example?

The more important questions however is will Lucid and FSF Emacs keep
jumping each other's minor version number? Lucid's next release should be
Lucid Emacs 19.27 just to see if the FSF jumps to 19.>28.

It is one thing to argue that such perversions might make sense for
some reason of compatibility, but quite another to claim that
representing hash-tables and keymaps and events and so forth in terms
of conses and vectors and symbols is in any way a feature.

GNU Emacs doesn't have hash-tables of any sort, as far as I can tell.
If it did you're right that they would work better as primitive Lisp
objects. It remains to be seen whether GNU Emacs 19 will suffer from
using alists instead of hash tables to store function key and mouse
maps. If the maps get big enough, it will.

The GNU keymaps are admittedly pretty baroque.

(Why aren't windows and processes and window configurations represented in
such a way in FSFmacs anyway?)

Windows and processes probably contain data that is more conveniently
dealt with on strictly a C level. There might also be funny business
in there with the garbage collector; there certainly is with buffers
and markers.

I've often wondered why window configurations are opaque. It has
occasionally been frustrating that they were.

By the way, FSFmacs doesn't allow "(fillarray (make-keymap))" any
more. This "incompatibly broke" a bunch of code I wrote.

Very true. I see why they had to do it, but it's still a shame that
keymaps couldn't be 100% backward compatible.

I believe that all functions which take event vectors also take strings,
and all functions which take events also accept ASCII integers.

I just did "(setq unread-command-event ?a)" and it gave an error.

unread-command-event must be an event, not an ASCII integer. I have no
particular objection to making it able to take an integer as well, except
that I don't see it as particularly useful [...]

Nor do I, on its own terms. My suggestion was that, if the variable
allowed this and still retained the name "unread-command-char", then
existing Lisp code would be able to function more or less. Not
perfectly, but it would work until the various authors had time to
redo it properly.

I'm talking about purity of design, not about compatibility with
existing code.

And I appreciate the purity of your design. It would be kludge for
Lucid Emacs to return strings from "this-command-keys" sometimes.
But it's not a kludge in GNU Emacs, because a vector of ASCII events
in GNU Emacs would be a vector of byte-sized integers, for which
strings can serve as functional substitutes.

There is a kludge for meta characters in the GNU system, though.
I can see it possibly coming back to haunt them some day.

It makes more work for the programmer because the type of the key changes.

Could be, although I think it's rarely going to matter much. GNU
Emacs 19 does provide a few accessors for getting at things like the
modifiers of any key.

Lots of code got written that used
aset and fillarray instead of the "advertised" interface of define-key,
making it harder to add power to keymaps (in both lemacs and FSFmacs.)

True, and I've certainly been guilty of that myself.

Honestly, I think the balance of merits is a lot more interesting
on the issue of events than on keymaps. Lucid's keymap model works
perfectly well, and even I, as devil's advocate, only managed to
come up with half-hearted arguments in favor of vectors and conses.
But RMS' event model has a nice feel to it. It still could be nice
to add extra stuff to events in the future, which his model can do
for mouse events but not for keystrokes. But there are other
advantages that weigh against it.

I think it's better for incompatible changes to cause code to not work at
all than to lose in obscure ways. Otherwise it will never get fixed.

Then perhaps Lucid's command loop should yell if it sees that
unread-command-char has been bound to anything.

It is one thing to argue that such perversions might make sense for
some reason of compatibility, but quite another to claim that
representing hash-tables and keymaps and events and so forth in terms
of conses and vectors and symbols is in any way a feature.

I believe that all functions which take event vectors also take strings,
and all functions which take events also accept ASCII integers.

I just did "(setq unread-command-event ?a)" and it gave an error.

That's a variable. (And I explained in my last message why I didn't make
it able to take integers: because, being a variable, it "returns" a value
as well as taking an "argument", and I don't want integers to be "returned"
for events.)

Nor do I, on its own terms. My suggestion was that, if the variable
allowed this and still retained the name "unread-command-char", then
existing Lisp code would be able to function more or less. Not
perfectly, but it would work until the various authors had time to
redo it properly.

I understand. And my argument against this was the such code would only
work most of the time, and the existing bug would be obscured.

It makes more work for the programmer because the type of the key changes.

Could be, although I think it's rarely going to matter much.

It may well be that the existing code will only fail in rare cases, that most
users don't encounter. But I don't think that's a good thing. For code which
uses read-char and unread-command-whatever to be *correct* in the FSF model,
it must accept that events can be integers or symbols. Existing code which
assumes they are integers will work if all you type is keys which are
represented integrally, and those are the majority. But the code is still
incorrect, and will lead to obscure errors that are unlikely to be noticed for
a long time.

; I think it's better for incompatible changes to cause code to not work at
all than to lose in obscure ways. Otherwise it will never get fixed.

Then perhaps Lucid's command loop should yell if it sees that
unread-command-char has been bound to anything.

(I guess you mean "set", not "bound.") Yes, I think that's a good idea.

I believe that all functions which take event vectors also take strings [...]

I just did "(setq unread-command-event ?a)" and it gave an error.

That's a variable.

... which was my point: Unless the symmetry between events and
characters is carried all the way through, there will always be
something that slips through the cracks. Even if the functions can
all be fixed, there will be variables that break and can't easily
be fixed (for the reasons you stated).

Then perhaps Lucid's command loop should yell if it sees that
unread-command-char has been bound to anything.

(I guess you mean "set", not "bound.") Yes, I think that's a good idea.

... which was my point: Unless the symmetry between events and
characters is carried all the way through, there will always be
something that slips through the cracks. Even if the functions can
all be fixed, there will be variables that break and can't easily
be fixed (for the reasons you stated).

Rather than seeing this as an omission of the design, which "slips through
the cracks" implies, I see this as a piece of backward compatibility which
is simply not provided. Lemacs uses strings and ASCII codes only for
compatibility; if I could get away with it without breaking every .emacs
file that has ever existed, lemacs wouldn't use them at all. The backward
compatibility is there in places where it wouldn't adversely affect the core
of the design, which I think is clean and consistent. The compatibility
kludges are not complete, because I don't think they are worth the price.

I don't think you'll find much anybody involved in the Lisp world
(apart from those with their tastes set in concrete circa 1972) who
believe that the ad-hoc mess of "vectors of length 259 of lists whose
first element is the symbol `indirect' and whose tail is an alist of
conses whose ..." is any substitute for actual structure objects (of
varous degrees of opacity.)

No, they went ahead and implemented monster lisp environments that
collapsed under their own weight during the 80'ties. The techniques
needed to survive in those environments are not the same as for a lean
and mean lisp environment such as GNU Emacs.

Apart from that, I would agree that Jamie Zawinski's event model is
better. It is just that the benefit of having a `clean' event model
is much smaller than the problems of having two different event
models. Keymaps are not a problem, since GNU emacs can emulate
Lucid's interface.

The more important questions however is will Lucid and FSF Emacs keep
jumping each other's minor version number? Lucid's next release should
be Lucid Emacs 19.27 just to see if the FSF jumps to 19.>28.

Jimb and I don't really have any good ideas what to do about this; he says
that he didn't pick 19.7 because the last lemacs was 19.6, but that by some
great, cosmic coincidence, there really were 6 alpha releases of FSF's
version...

I'll probably call the next lemacs 19.7, and just hope that people mention
which one they mean. `emacs-version' is "19.6 Lucid", after all.

I don't think you'll find much anybody involved in the
Lisp world (apart from those with their tastes set in
concrete circa 1972) who believe that the ad-hoc mess of
gt; "vectors of length 259 of lists whose first element is the
symbol `indirect' and whose tail is an alist of conses
whose ..." is any substitute for actual structure objects
(of varous degrees of opacity.)

No, they went ahead and implemented monster lisp environments that
collapsed under their own weight during the 80'ties. The techniques
needed to survive in those environments are not the same as for a lean
and mean lisp environment such as GNU Emacs.

I personally find it very difficult to argue in favor of the GNU
keymap structure. It seems to me that building keymaps using the
simple types has the following arguments in its favor:

It requires no changes to allocation code or GC code - things do break.

It requires no new read syntax or printed representation.

You don't need to worry about thinking of all the necessary
primitives beforehand, since people can roll their own.

It's obvious, just by looking at the structure, how they work.

You get leverage from the ability to apply existing routines (assq,
etcetera) to them.

But those are kind of stretched.

Is is really that hard to write allocation, parsing, and unparsing
code? ("Are we not hackers?" :-) )

Come on, it's not hard to come up with a good set of primitives. A
keymap is a function mapping events onto commands. How many things
do you need to do with that?

It's *not* obvious how to use them any more! Did you know that a
binding for t in an GNU keymap structure acts like a default,
catch-all binding, providing the only way to create a keymap that
catches *all* events? And exactly what does it mean to nconc one
keymap onto another? Are you *sure*?

You can no longer effectively apply existing utility routines like
assq to an GNU keymap. assq doesn't know about the vectors. The
keymaps are too hairy to be operated on by anything but code which
knows it's dealing with a keymap.

It's a pain to upgrade. The keymap changes are probably the
largest source of broken a lot of code in GNU 19. Anyone who
thinks it's not going to happen again is a doodyhead.

It restricts your implementation. If you expect people to poke
around in the data structure in the raw, this discourages you from
making reasonable improvements, like the Lucid reverse indices.

I have yet to hear a single mildly persuasive argument from anyone in
favor of the GNU keymap structure.

RMS is this great programmer. He's got GCC. He's got Emacs. He's
got the MacArthur. He's got this luxurious pad on the fourth floor of
posh NE43. I go to work for him, hoping some of that will rub off.
It turns out that the single most common theme in his thought is,
"That's too much work." "That's going to take too long." "Don't
bother."

Is that what it takes? Is this the punchline of my lifelong pursuit
of hackerdom? I'm apprenticing myself to a blacksmith, just in case.
:-)

My first impressions of FSF v19 vs. Lucid Emacs 19: I liked having
scroll bars and having horizontally-split windows, selective display
etc. working, but the FSF menus are *tacky*. IMHO it's high time FSF
emacs began to use XT and widgets properly -- anyone who has X has XT
anyway. Lucid's event and keymap implementation is generally better.

In the Lisp library, however, FSF emacs was much more polished. Lucid
Emacs's Lisp is (except for that which had to be changed due to
changes in Lucid's C code) largely v18 with a hodgepodge of seperate
packages tied on.

The GNU documentation was better, too.

I also liked the scrollbars, but the menus were horrid. And why did they
choose to not go with motif or lucids motif style widgets for the menus
and the scrollbars?

Motif cost money! Is the lucid widget stuff free? What are the
licensing restrictions on this?

; I would love the next version of LucidEmacs to have the capability to
place vertical & horizontal scroll bars (motif style) on each edit area.
Any plans for this?

That would be a very welcome addition, that would help convert a lot
of anti-emacs types around here.

The more important questions however is will Lucid and FSF Emacs keep
jumping each other's minor version number? Lucid's next release should be
Lucid Emacs 19.27 just to see if the FSF jumps to 19.>28.

At the rate that GNU bug fix versions are comming out, maybe 27 is too
small a nnumber for Lucid to avoid confusion with a soon to exist GNU
version.

The release of GNU Emacs 19 from the FSF does not mean that Lucid Emacs is
going away. We will continue developing it, and merging in bug fixes and new
features from the FSF version as appropriate; we do not have plans to discard
the functionality that RMS has chosen not to include in his version.

The long awaited Great Redisplay Rewrite is well underway, in the new guise of
the Great Epoch Merge. Most of this work will end up being done by the Epoch
developers, but we're in close communication.

Within two months we expect to have a version of Lucid Emacs with the new
redisplay, which can run all Epoch 4 code without alteration (excepting those
areas where lemacs is incompatible with v18, that is.)

Lucid Emacs will run on dumb ttys shortly thereafter (if not at the same time.)

I will probably release a new version of lemacs in a few weeks which contains
some bug fixes and portability improvements, and a bunch of internal
improvements that aren't user visible. The next release after that will
probably be in a few months, when the Epoch merge has reached a point of
stability.

The latest released Lucid Emacs is 19.6, and can be found on
labrea.Stanford.EDU in pub/gnu/lucid/; see the README.

There is a version of Lucid Emacs in beta test now; send me mail if you'd like
to help out with it (hackers only, please.)

I also liked the scrollbars, but the menus were horrid. And
why did they choose to not go with motif or lucids motif style
widgets for the menus and the scrollbars?

Motif cost money! Is the lucid widget stuff free? What are the
licensing restrictions on this?

Lucid Emacs can be compiled with Motif if you have it, but also comes with a
simple set of Motif-lookalike widgets used for the menubar and popup menus.
This code is Copylefted; all you need is Xt. (Even if you have Motif, I
recommend that you use the Lucid widgets, since they're loads faster (though
not 100% compatible).)

I would love the next version of LucidEmacs to have the
capability to place vertical & horizontal scroll bars (motif
style) on each edit area. Any plans for this?

Will the next release of Lucid Emacs contain most of the Emacs Lisp
changes to FSF v19.8? Obviously some things aren't compatible with
the way Lucid Emacs works, but how about the rest?

Probably not, because I don't have time. The current beta version of
lemacs has had many FSF changes merged in to it, along with a lot of
internal improvements. Most of the files in the lemacs lisp/prim/
directory have been merged against their FSF counterparts, but the
rest of the lisp library has not.

This is something that a volunteer could easily do, since it doesn't
require intimate knowledge of lemacs internals...

The GNU documentation was better, too.

Likewise, I don't have much time to hack documentation if you want all of
these redisplay bugs fixed...

I'm also still looking for some noble volunteer to add "configure" support
to lemacs. This should be quite simple with the current lemacs beta, since
at this point, the lemacs s/ and m/ directories are basically identical to
their GNU19 counterparts.

I recall a message in February where you posted a list of the features
you would find essential in order to be able to use Emacs 19. You
asked for the FSF to show the will to merge the two versions of Emacs
by doing some of the work.

I did this. Emacs 19 contains several features merged from Lucid
Emacs, such as the X selection code and faces. It also contains
comparable menu and event features with emulation for the Lucid
interfaces. It contains features comparable with intervals, and
emulation for the Lucid interval functions should be easy to add.
Almost all of the features on that list are now provided.

So it would seem proper for you to pitch in and merge things such as
toolkit output, sound output, and dialogue boxes into Emacs 19 and end
the split.

Your posting today seems to describe quite different plans, however.
Are you willing to end the split, or not?

Date:

Thu, 27 May 93 16:45:19 PDT

From:

Jamie Zawinski <jwz@lucid.com>

To:

rms@gnu.ai.mit.edu (Richard Stallman)

Cc:

rpg@lucid.com

Subject:

Re: the future of Lucid Emacs

We have been doing a lot of merging with your sources, but in the direction
of taking your new features and putting them in our source tree. As this
results in the two emacses being closer together, I don't think our current
strategy is incompatible with an eventual merge with your version of emacs;
it will help in the long run.

We need to talk about this more on our end (some people who should be
involved are on vacation for a few days) and will send you mail later.

Please don't use motif-style scrollbars. Remember, this is emacs, so
I'd like to be able to control what mouse buttons are bound to in the
scrollbars. I don't care if they look like motif scrollbars, just that
they don't have to act like them.

I hate motif (and Open Look, for that matter) scrollbars because I'd
like one button to scroll up and another to scroll down, and another to
scroll absolutely ala xterm scrollbars. It annoys me that I have to
click before the gizmo or after the gizmo. It really anoys me when the
gizmo moves up past my mouse cursor and the same clicking which was
scrolling up now starts to scroll down.

I'll bet that once you add scrollbars, people will want to bind their
own functions to do interesting things.

I just want to say to keep up the good work. I've just tried emacs 19.12,
and I must say that lemacs 19.6 is quite superior!
Especially the menubars. I am hopefull that 19.7 will have scroll bars.
I also like the way the middle mouse button is used in completions, etc.
Thanks to Jamie et. al.

I'll second (and third) the kudos. Since I brought up Lucid emacs, I
haven't even bothered to bring up GNU 19.<todays release>, and the 18.xx
version that I left lying around gets used when I dial in [no PPP/X yet :-( ]

Have you decided whether you'd like to work with me to merge the two
versions?

I have been wanting to ask for volunteers to merge the Lucid Xt
interface code into Emacs 19. I have put it off because,
if you want to merge the versions, that is better than finding
someone else. But it is getting to the point where I ought to
ask for someone else if you don't want to do it.

We got together and discussed the idea of completing the merge of our
code into your version decided against it. The primary reason is that
the structure of GNU Emacs 19 and its lack of clean internal
interfaces are fundamentally unmaintainable, and we believe that
trying to maintain it along with the additions to Emacs we need and
adding the things we need would be too expensive for us.

Unfortunately, GNU Emacs 19 is not constructed following modern
programming practice - even though there are procedural interfaces to
some functionality groups, one could not say that abstraction
boundaries are maintained. The mish-mash of data structures is just
too clumsy and incoherent to make life for the maintainer easy. This
is proven by the fact that Arceneaux was unable to add the extensions
we needed to an early version of GNU Emacs 19 given somewhat over a
year's time to do it and by the fact that it has taken you and Blandy
so long to get GNU Emacs 19 out.

We believe that the merge in the other direction is the proper way to
do things, and this is the course we shall pursue. There are things
that you have done to GNU Emacs 19 that are good, and we believe that
the best way to proceed is to clean them up and put them in Lucid
Emacs 19.

Maintainability is important to us because even though GNU Emacs 19
has no warranty, we must support a warranty on our commercial product
which includes Lemacs as a component.

I think this is not a good outcome for the Emacs user community, and I
hate to have to criticize the implementation of GNU Emacs 19, but at
some point clean code and good structure counts more than a false
sense of parsimony, and that point has been reached.