Background

I’m a long-time Emacs user who is new to the under-the-hood side of things. I recently started learning EmacsLisp and writing my own Emacs extensions. I like sitting in IRC channels devoted to technical subjects I’m interested in, so I found #emacs and started hanging out there. It was not long before I discovered the Terrible Secrets of the Emacs community: EmacsLisp Sucks and Emacs Is Dying.

This caused me a great deal of consternation and confusion, because I love Emacs and had recently become quite fond of EmacsLisp, so much so that I was toying with learning LISP as a standalone language. I tried to get more information, but “What do you mean, ‘EmacsLisp sucks’?” was more often than not met with a reply which seemed either elliptical, tautological, or a non sequiteur, at least to someone with my level of understanding of the issues (none)[1].

EmacsLisp Isn't Scheme

This, from my highly unscientific sample, is far and away the most popular reason EmacsLisp sucks. Well, that’s good to know. EmacsLisp is also not Perl, or COBOL, or IBM 1130 assembler, or a bicycle, or an orange. Thanks for the help!

I didn’t know Scheme – and in fact do not now claim to know Scheme, though I have begun working through SICP – and to my untrained eye, the main difference between EmacsLisp and Scheme was that Scheme required hitting the shift key a lot more (#t, #f, set!, predicate?, etc). Further inquiry eventually yielded the conclusion that this reason is really just a restatement of “EmacsLisp doesn’t have lexical scoping”[2].

That conclusion is not justified. There’s a great deal of difference between Scheme and EmacsLisp beyond just the question of lexical scoping. As the OP mentions in passing, but then ignores, there’s the fact that nil isn’t false in Scheme, but is in CommonLisp and EmacsLisp. Scheme has a stylistic convention of marking destructive operations with an exclamation point and predicates (boolean functions) with a question mark; again noted in the OP. I’m not sure the latter is so significant, but the former indicates a stigmatization of side-effecting in the Scheme community, more closely wedded to function programming concepts, that we certainly don’t see in EmacsLisp. Scheme is a Lisp-1, sharing the same namespace for variable and function definitions, so that the name list must be bound (at least locally) either to a function, or a variable, but not both. In a Lisp-2, you can use the same name for both a function and a variable.

EmacsLisp Isn't CommonLisp/Haskell/Other

I don’t know enough about any of these to offer any informed commentary, except to note that aside from Common Lisp, they all seems to be edge cases that either no one supports seriously, or only one or two people in the universe actually want.

Comment: That’s blowing Haskell’s uncommonness way out of proportion. It may not be very widespread in industry yet, but you could just have a look at the IRC channel, the mailing lists or hackage to debunk “one or two people in the universe”, even taking hyperbole into account 😊

The Emacs APIs Suck

I heard this a lot as well, but I have no idea what exactly is being referred to. Elucidation greatly appreciated.

My wild guess would be “because they’re not OO”, which doesn’t really bother me, but can be seen as a problem given the typical Java-centric way people are taught CS in the US.

The APIs related to match data are an example. Instead of returning a data structure of the match results (eg from a call to match-string), the match results are mutated in global memory and accessed in separate function calls. This is both less functional in style and more error prone. For example, if save-match-data is not used appropriately, then library functions can trample on the match data which a higher level function is in the middle of using.

Do You Mean EmacsLisp (Language) Or EmacsLisp (Library)?

Most people seem to hate EmacsLisp-the-language, but the primary obstacle to replacing EmacsLisp is, of course, EmacsLisp-the-library. 30+ years and 80+ megabytes of Emacs functionality, all in EmacsLisp. Redoing even the core editing functions is a terrifying thought to me[4].

Who Is EmacsLisp For? Who Wants EmacsLisp Dead?

This is the larger question I referred to in the Scheme section. I can’t say for sure, but my gut feeling is that most of the people who are unhappy with EmacsLisp are the people who are using Emacs as an application development platform rather than as an extensible editor.

I am, of course, aware of the kitchen-sink nature of Emacs, and revel in it. I know that the design of Emacs evolved in the time when a programmer only had access to a single terminal session to a computer, thus becoming in effect the world’s first Integrated Development Environment so a user could meet all their needs without dropping in and out of their primary application: the editor.

But I believe there is a difference between making Emacs read mail or news or have a shell because you needed to in the 1970s and making Emacs browse web sites or act as an httpd because you can in the 2000s. I wonder if perhaps the people who want to fix all of EmacsLisp’s problems want to do so more for their own benefit – so it will be a better general-purpose development base and toolkit – than to make Emacs better for everyone. I wonder if they have lost sight of the primary application: the editor[5].

Conclusion

This is meant to be polemic but not dogmatic[6]. I have stated my observations, opinions, and conclusions, but I have done this to show my point of view and lack of full understanding rather than to try and convince others that I am correct. I am not against fixing or replacing EmacsLisp, but I have yet to see any logical, thorough, and cogent arguments for it. Therefore: get to arguing!

Other commonly-named problems

No namespaces/modules/packages. This leads to dozens of functions that all start with e.g. “dired-”, even inside the library itself.

No object system (CLOS). This remark isn’t really true - there’s eieio - but the basic point is that the whole design isn’t object oriented (it’s debateable wether that’s a worthwhile goal)

Related to this, there’s no way to create a new abstract type that’s disjoint from other types (e.g. vectorp doesn’t return t on those, for example).

Other than for “conceptual purity” or something, why is this relevant? i.e. why would you pass an object to code that expects an ordinary vector, and expect that code to distinguish this case and treat it correctly?

Pipeline's Commentary

[2] Scheme is a standard lisp dialect, easier to implement than CL, partly because it’s smaller, partly because the standard was written with optimization in mind. In contrast, CL was written with commonality in mind. Important differences include a focus on tail recursion and CPS optimization. I think Scheme used to not have keyword arguments, but I’m no longer sure about that one.

Some Scheme implementations, notably my favorite Gambit-C, have keywords. It’s not specified in R5RS or IEEE. -JoelAdamson I think one may sum up the “EmacsLisp isn’t <foo>” problems pretty simply. EmacsLisp appears quite similar to other lisp dialects, but it has enough “gotchas” and differences to frustrate the beginning EmacsLisp programmer very badly. There’s tons of people with some understanding of scheme or lisp, but relatively few who really get into EmacsLisp. I expect that many EmacsLisp programmers are like me, desperately trying to apply knowledge of another language. Part of this problem is that twenty years ago, lisp programmers were accustomed to widely varying dialects. Today, most Lisp dialects are quite similar. EmacsLisp looks like an aberration if one does not consider its history.

[3] A tremendous number of emacs extensions are both large and complex, even untenably so.

[4] This is partly because, as you pointed out, dynamic scoping makes large projects “interesting.” Is it not a sign of EmacsLisp’s flaws that we’re all desperately afraid of an attempt to rewrite EmacsLisp libraries?

[5] As you pointed out, Emacs is not primarily an editor, it’s known first and foremost as a “kitchen sink.” There’s a reason that the first word in its name is “Extensible.” I feel I should also mention a popular Microsoft anecdote. Microsoft, at some point in its office research, discovered that any given feature is really only used by a insignificant fraction of users – but if it’s taken out, that individually insignificant fraction becomes irate. By the time you trim Word down to its “primary application,” you’ve lost 99% of its users.

I just wish to clarify that I was not pointing out that Emacs is not primarily an editor when mentioning the kitchen sink. My point is that Emacs should be regarded as an editor first and foremost. --mdxi

Utis' Commentary

ad Pipeline's Commentary [2]

Scheme is probably easier to implement than CL, because it is much, much smaller. But I definitely wouldn’t say that its standard has been written with optimization in mind. Following the book “Lisp in Small Pieces”, the existence of continuations as first class objects alone implies a performance penalty; likewise the fact that the first element of a form has to be evaluated, possibly at runtime; likewise the fact that scheme programs may overwrite built in functions. I guess that modern compiler techniques may be able to deal with that, I don’t know. But it is definitely not easy.

ad the OP's topic "EmacsLisp Doesn't Have Lexical Scoping"

EmacsLisp’s lack of lexical scoping isn’t so much a matter of writing and maintaining large and complex programs. The byte compiler does warn you, if you access a dynamically bound variable outside of its let-form. You have to add a variable declaration in order to get rid of that warning.

There are two other reasons why I, at least, want lexical scoping: 1) it allows faster variable bindings and better optimization techniques. Introducing lexical scoping could vastly improve EmacsLisp’s speed. In fact the slowness of creating a dynamical environment urges people (myself included) to use such ugly constructs as (let (var) (while <condition> (setq var …))). This won’t be necessary with lexical scoping.

2) Lexical scoping is a requirement for closures. Closures would allow for a lot of very useful programming techniques.

In that context: ad Pipeline’s comment [4]: I explained above, why I fail to see that EmacsLisp’s dynamical scoping “makes large projects ‘interesting’”. I hardly recall any problem comming up in emacs-devel or gnu.emacs.help that could have been attributed to dynamical scoping. So this point would require some extra arguments. After all, Emacs is a very large project. Moreover, I don’t understand why that would anybody make afraid of rewriting an EmacsLisp library in a different language – presumably one that does provide lexical scoping.

ad the OP's topic "Who Is EmacsLisp For? Who Wants EmacsLisp Dead?"

I strongly agree with Pipeline here. What does “editor” mean for the OP? Just writing ASCII text? Just writing text/plain in general? Just writing text/plain and offering special support for programming source code? Does “just an editor” include functionality like Tramp (for remotely accessing files)? If yes, why doesn’t it include a newsreader?

For me most of the so-called “kitchen-sink” stuff is simply part of an editor. There is a big benefit in having everything in a unified environment with a common user interfacec. If my editor would not provide functionality to read mail and news, then I’d drop it and look for a mail reader that let’s me edit programming text. In fact what the OP is attacking here is that Emacs is an Emacs, and not just an Emacs-like editor.

ad EmacsLisp in general

I didn’t know that “Emacs is dying” is a Terrible Secret of the Emacs community. If so then they haven’t told me in all those years. Well, yes, EmacsLisp sucks. And at the same time: EmacsLisp is a very nice language. A contradiction? Well, EmacsLisp is just lacking a few features and I’d wish by heart that they were introduced to that language: a few problems are mentioned above. But the most important problem is: speed. EmacsLisp (or rather its implementation, but that amounts to the same here) is too slow. Closures are lacking, too. Yes, a deeply integrated, efficient object-system would be necessary, too. Oh, and modules or a package system. And, as a personal wish of mine: tail call reduction.

So, I agree: EmacsLisp could and should be improved. But that doesn’t mean that we need to replace it with something else. --OliverScholz

Mathrick's plug-in

Why Emacs Should Die (a.k.a Death Rebirth Revolution [1])

Second of Terrible Secrets is discussed so often I doubt it’s a secret any longer (maybe it’s fact it’s secret that’s Terrible Secret 😊). So here are my reasons why I want to see current Emacs code thrown out the window and rewritten (and most importantly, redesigned) from scratch. What we want to retain is Emacs-the-project, Emacs-the-team, Emacs-the-community, Emacs-the-spirit, and also, Emacs-the-name 😊. But actual source code should be redone, because:

Emacs is extensible, yet its core is horrible example of one of most inflexible designs I have ever seen in use. Let just one word suffice: fringes. Options allowing to influence them in any way were introduced only in CVS, and still all you can do is use them for predefined you’re-debugging-this-line indicator.

Another example is that you can’t advice built-in function (you partly can, but only in EmacsLisp invocations, core will continue to use unadviced one). In general, your ability to do anything with builtins is most often void or negligible.

After all these years, you still have to do full edit .c file – save – recompile – try cycle to change core even in slightest detail. You cannot change it, you cannot extend it. No dynamic modules, no FFI, and (AFAICU) if you dump image with particular set of EmacsLisp preloaded, you’re stuck with that EmacsLisp in effect forever

You can’t use anything besides C for core. This, in advent of .NET-like runtimes, and era of omnipresent bindings, is Not Nice(TM). I’d really like to be able to use python for that, and maybe one day seamlessly interface Emacs with .NET runtime (if nothing else, just for introspection data you can get for free in .NET / Java, it’s really kinda necessary when writing in environment that consist almost 100% of library functions and classes)

EmacsLisp (as it is today) really sucks. It has loads of small annoyances, my personal pet peeve is lack of anything near concurrency. And I don’t even use Gnus (if I did, I suppose I’d kill for any threads in EmacsLisp ;). I don’t know how that’s usually done in “big” Lisps, but I presume CL has to have threads or equivalent primitive.

What I envision is ultra small and lean core, that does only one thing - allows extensions to interface each other, binary objects and Lisp-written alike. Ideally, there shouldn’t be any difference between C and Lisp code, each should be overridable and changable by outside. This way, you could plug in different display engines on console and X, making it possible to introduce for example structured graphics manipulation primitives (another long-time wish of mine). That means to see and manipulate pretty UML figures, I don’t need to leave Emacs, and I can still use all the graphics capabilities offered by X I use, without making life hard for console users. And plugging Emacs into .NET would be matter of one extension that bridges between Emacs binary and runtime. Incidentally, that seems to be design employed by MonoDevelop/SharpDevelop IDE running on .NET, where core has astonishing size of 23KB, and everything else (including GUI) is a plugin. Yes, I know it’s not small undertaking. But I can’t see how could Emacs survive and grow in its current incarnation (unless we are content with I’m-too-hardcore-to-move-from-console users only and no new souls coming)

That’s my 3 cents, hope I was clear enough and managed not to offend anyone 😊. Oh, and I don’t say everyone and their dog have to use .NET for everything, I just say that modern programmer’s editor needs to be able to do that, or will get forgotten in world of runtimes everywhere.

Mornfall's comment

I think Mathrick is on crack. [1]

[1] Really.

Spirit's comment

Mathrick, you will want Eclipse. 😊

Seriously, I tried that a few months ago. Still have the core but not much functionality. I’m not sure where such elementary things, like buffers, are supposed to be in that design. There’s one single static plugin, which can hold scripting engines, which in turn use the core library to communicate with each other. Different window systems are no problem at all, since you can just plug them in (need a streamlined interface though).

But then…there’s Eclipse and I think it does all of the above minus console modes (and it’s slow).

--Spirit

blandest's comment

I’v been using Gnus for a short period of time (less than 2 months) and found the lack of threads quite annoying. The only hack that I could come up with was to make an idle timer that will scan for new mail once every 30 seconds or so. There is also a 3 minutes delay between consecutive scans to avoid useless request to the mail server. How hard would it be to add only one background thread that runs these idle tasks ? (We don’t need Emacs to be a scalable, multi-threaded web server 😊) I guess it’s the same thing as adding support for any number of threads …

-- Blandest

pro-emacs comment

I think it’s quite funny that people never take into account the empirically observable reality. It’s like falling and staying in love with comunism besides real history.

Almost any kind of application I can think of, when there is an emacs mode for it, its always best in class. Going back from Emacs to other editors /IDE’s is like going back to the stoneage. If Scheme is so great - why is there no proof of it? Why is it possible to realize really complex extensions like org-mode and gnus in EmacsLisp, if it sucks so much? Where are better extensible programs in Java, Phyton, whatever?

I think it’s fun to program in Emacs Lisp, a huge project with a long history in this language yielded an unsurpassed piece of software (Emacs) - it can’t be bad, it must be very suitable for the task at hand.

-- pro-emacs

someone in the future comments

I am in the future, and here we first have Sublime Text become the absolute most popular editor with a Python interface, and then came Atom by Github in CoffeeScript / JavaScript, and VS Code by Microsoft in TypeScript. This is a future where JavaScript dominates. Lispy people can still extend Atom by ClojureScript, and some proclaim it be the Emacs of the new generation. The new trend is to do all UI stuff in Web terms, so that Atom’s CSV extension comes with a full fledged table editor.

-- One from the future

EmacsLisp is not a general purpose language

Get over it. Emacs lisp is designed to be an extension language for Emacs editor. It is ridiculous to compare EmacsLisp to Haskel, Scheme and Common Lisp.

-- unlocksmith

What about Emacs Lisp 2?

There are several common misconceptions voiced here

“Better” Lisps have concurrency. Nope, not really. They do in practice (because you can’t really not have it in 21’st century), but the standard has nothing to say about it. This is really up to the implementation to provide this feature (for now at least).

Scheme is a lot simpler than Common Lisp. Even if true, not by much. In the end of the day, you will still find yourself needing the same tools. In one instance those will come from the standard library, in another instance those will come from a library developed by community.

There are more developers in Scheme or Common Lisp than there are people writing Emacs Lisp. Not according to Github statistics at least. If anything, of all Lisps (unless you count JavaScript), Emacs Lisp seems to be the most popular one. Ahem. It’s not popular due to its features, but because it serves a purpose no other language can (being the scripting language of Emacs).

I think I read somewhere about the history of Emacs Lisp, and I think that Stallman said so himself that many of the language-related decisions were motivated by the simplicity and the implementation and the time it would take to come up with one as well as the presence of the practical problems the language had to address. In other words, in a sense, Emacs Lisp was (and still is) a prototype language, and as such it did an unthinkably good job. But there are some roadblocks, where the development of the language seems to have come to a stand-still because the change / update will require things to be thrown away.

In my view, these problems are

Performance. There’s really so much you can do with an interpreted language. In order to manage large projects on par with other modern editors the language needs to do better. Emacs Lisp needs an optimizing compiler. I think that LLVM might be a good target to compile to.

Concurrency model. Not just threads and other low-level tools needed for synchronization or mutual exclusion (I’d even prefer that those be hidden from sight), but a thought-through system based on either Actors model, or Concurrent C / Ada, or maybe something else, genuinely new?

Collections. Good programs must have good tools to work with data. This, again, means that there needs to be a system, a good design: perhaps, like iterators in C++, or perhaps such as Java collections, or maybe look into functional data types? Whichever way it will be, it’ll be still better than the current state, where one can’t even iterate over two hash-tables at once.

Interface with the outside world. If CFFI seems like a threat, than, perhaps, a different mechanism?

Security? It’s not always appropriate to trust the code written by other people blindly. Some concept of sandboxes would be nice for loading semi/untrusted code.

Modules (information hiding in general).

Types? On one hand these make programs more difficult to write, on the other hand, these make them compile better.

This work is licensed to you under version 2 of the
GNUGeneral Public License.
Alternatively, you may choose to receive this work under any other
license that grants the right to use, copy, modify, and/or distribute
the work, as long as that license imposes the restriction that
derivative works have to grant the same rights and impose the same
restriction. For example, you may choose to receive this work under
the
GNUFree Documentation License, the
CreativeCommonsShareAlike
License, the XEmacs manual license, or
similar licenses.