Critique of Where Perl 6 is Heading

The purpose of this essay is to explain why I believe Perl 6, the way it
currently seems to progress, is the wrong thing at the wrong time, and
why I predict (with all the expected caveats of predicting something)
that it won't be successful. I will also suggest a better alternative
for the future of Perl which makes more sense at this point.

Overview of the Current Design of Perl 6

The design of Perl 6 is progressively laid out in the Perl 6 Apocalypses
which are written by Larry Wall (the original creator and designer of
Perl versions 1 through 5), and exemplified in the Exegeses by Damian Conway
and others. There's a more progressive design which has been shared
among the members of the perl6-language
Mailing List, but hasn't been formulated into a specification yet,
and/or has not finally been decided upon by Larry.

It is evident from the Apocalypses and Exegeses that Perl 6 will break
compatibility with Perl 5 in many ways (usually deliberately).
Furthermore, it incorporates a lot of features and paradigms from
different languages that are not common in most languages in common use
today.

Why Perl 5 is Good

Perl 5 has seen incredible popularity. There are now millions of lines
of code written in Perl by thousands of programmers. (Possibly even
milliards of lines of code, written by hundreds of thousands of
programmers.) The so-called Perl 5 Man Pages
are (or at least used to be) intended for experienced Unix system
administrators and programmers who are already familiar with either
Perl 4 or with the Unix toolset of shells, awk, and sed. The Web
Revolution wanted something else. Many programmers who were new to Unix
(or even to programming in general) learned Perl because it was the most
suitable tool for the job, often using it in situations in which writing
a shell script would have been better. Thousands of CGI scripts have
already been written in Perl, and new ones are constantly being written.
Perl has also seen wide deployment in other areas -- bio-informatics,
for example, and programming UIs. MandrakeSoft has written the
Mandrake Linux installer in Perl, and also many of the distribution
system tools. And Mandrake is a very popular distribution.

CPAN has provided programmers with a great number of modules that can be
easily installed and used to solve most tasks much more easily. CPAN's
convenience and coverage is the envy of all other languages, and even
accounts for some people using Perl despite liking a different language
better.

One thing that has to be remembered is that Perl exhibits the Iceberg
Effect: for every piece of code that is distributed (either on CPAN or
otherwise), there's much more code that is kept in-house or in private.
Many CPAN modules have many equivalent ad-hoc routines doing similar
things written in Perl that are actively in use in many software shops
and were never released to the public, nor are going to be released. (I
know I wrote such things when I began programming in Perl)

Perl 5 is a modern, powerful language which can be used to write code
that either incorporates many different paradigms or even emulates many
other languages. As Larry Wall once noted:
"You can write Perl programs that resemble sed, or awk, or C, or Lisp,
or Python." (Or BASIC/Visual Basic for Applications, Pascal, a great
deal of Haskell code, PHP, etc.) Most people can write a lot of Perl 5
code and won't encounter a place where they think that it is inadequate.
Thus, Perl 5 is good enough not only for today, but also for the future
ahead. ANSI C is a much more limited language than Perl 5, and still,
many people are actively using it and writing new code in it. They also
often like doing so. If ANSI C is good enough, there's no reason to
assume Perl 5 isn't.

Why Perl 6 is Bad?

No one Understands Perl 6

Having read this title, you are probably thinking to yourself: "So you
don't understand Perl 6. What makes you think everyone else doesn't? How
do you justify this inductive thinking?" Let me tell you a little
story:

One day, I met with a good friend of mine (whom I highly appreciate both
as a person and as a software engineer), and I asked him if he reads the
Apocalypses. He said he does, but that he doesn't understand them. This
eventually made me realize that I also read them, and also did not
understand many things. And neither he nor I are particularly stupid
people. And here's an interesting quote from the famous
Weblog "Joel on Software": "Whenever somebody gives you a spec for
some new technology, if you can't understand the spec, don't worry too
much. Nobody else is going to understand it, either, and it's probably
not going to be important." (Read more at the link).

Granted, my editor told me that the Apocalypses are sort of Larry Wall's
braindumps, notes on design notes using a lot of jargon from the Perl
mailing lists and summarizing a lot of discussion that took place there
in one condensed article. However, they do indicate that a lot of new
things will be added that most people are not familiar with and will
need a lot of time to understand because they haven't encountered them
before.

It even takes Damian Conway, Allison Randal, et al. a lot of time to
prepare the Exegesis after the latest Apocalypse is published. Perhaps a
different, more accessible format of the Apocalypses is needed. Perhaps,
eventually, such accessible documentation will be made available,
teaching people new to Perl 6 or even Perl in general how to learn Perl
6. (I do hope it will fare better than the original Perl 5 man pages,
which were a positively awful resource for beginning Perl programmers.)

I'm pretty sure my friend and I are not dumb people. I know people who
will have even more difficulty, and they're not dumb either. Who is
going to take all of these people by the hand and teach them Perl 6?

Perl 6 is not Backwards-Compatible with Perl 5

You probably think: "Ha! Backwards-Compatibility is the curse of
software design. Let's break it now so we'll get something
manageable." That's very nice, but backwards-compatibility is also
something you need to make sure people can have some amount of
confidence in writing code that makes use of the language or API.
Breaking it on purpose, without a good enough reason, is bound to
cause resentment among the users, who may opt to convert to something
that is known to be more backwards-compatible or stay with the old
version.

Perl 6 breaks a lot of compatibility on purpose. In fact, most of the
Perl 5 code out there won't compile with it (unless Perl 6 will run in
some backwards-compatibility mode.) Naturally and hopefully, this Perl 5
code can be re-used from within Perl 6. That's better than nothing.

As an example of breaking things on purpose, here's some Perl 5 code I
wrote:

Recognizable, but completely different! Furthermore, the dereferencing
and method invocation in Perl 6 changes from "->" to
"." (just because that's the way it's done in languages such as
Java and Python), so the good old string concatenation operator is now
"_".

So imagine you have a large body of Perl 5 code which you need to
extend with more code. Are you going to write the new code in Perl 6?
Well, if you want other Perl 5 programmers (of which there are now
several hundreds of thousands) to be able to understand it without
getting on their nerves, forcing them to learn a new language,
forcing them to understand Perl 6, etc., then Perl 5 is the only
option. I predict that most software shops are going to stay a clear
mile away from Perl 6, and mandate all of their programmers to use Perl
5 exclusively.

Writing new code in Perl 6 while having to maintain old code in Perl
5 is going to be confusing, and will require the developers working for
your project to know both and to actively use both. Most Unix shops
just settle on either Perl, Python, or a similar Agile language, and
may even have strict style and use guidelines on how to write the code
in these languages. They're not going to like having a dual Perl
6/Perl 5 codebase.

While software enthusiasts, who like to write code for fun, may learn
Perl 6 because many of them like to learn new languages, they will
probably be told to use Perl 5 (or Python, or PHP) at their workplace.
Perl (1 to 4) grew as a language to get some Unix jobs done better. Perl
5 was intended as a revamp to eliminate many of the Perl 4 shortcomings,
and even to direct Perl for writing large applications. Both of these
goals were achieved, but what should still be remembered is that most
people use Perl for in-house use, and this is what "pays the Perl
developers' rent", so to speak.

Perl 6 may be a good language to design 10 or 20 years from now, but
it's certainly not one for this moment. Perl 5 is a good language today
already, and if we want people to adopt a new dialect of Perl, we need
to start from it, not from a completely new page.

Perl 6 is too Complex

Before we go on, I ask you to make sure you've read what
Joel Spolsky has to say about the growing complexity of programming.
Yes, programming is becoming more complex, and IT workers find
themselves having to (at least temporarily) specialize in a certain
field or limited set of fields. However, it is still important that the
framework designers try to reduce unnecessary complexity.

A good example are the Unix mail servers. Sendmail used to be the most
popular solution, but its configuration is incredibly complex (which is
just one of its many shortcomings). As a result, a new crop of mail
servers came about: qmail, Postfix, and others.
These all sported an easier configuration that even mere mortals could
do.

Another good example are languages like Java or PHP. These are highly
successful languages, in part because the rules, grammar, and syntax of
them were kept as simple as possible. While their standard libraries
are very comprehensive and complex, the language as a language is
very simple. (I don't suggest that Perl 5 follow this lead -- a
complicated core language is one of the things that makes it sexy -- I'm
just noting this fact).

The Perl 5 core language is already very complex. Part of this
complexity is justified, and the rest is a result of either legacy
features or things that some programmers like to use. Nevertheless,
Perl 6 will be much more complex than that, too complex for the present
level of Perl programmers.

When Perl 5 was created, people expected its programmers to be
experienced Unix sysadmins or programmers, probably with some working
knowledge of Perl 4 (the Perl 5 man pages originally reflected this).
The WWW boom wanted something else, and a great many Web programmers
learned it because that's what their job required, sometimes even using
it for writing "shell" scripts because Perl was all they knew at the
time. Further developments, like the growing popularity of Linux and the
bio-informatics boom, have made Perl even more popular. Some people
even learn Perl as their first real programming language.

So, Perl 6 will be too complex for them. Much too complex. If I and my
friend (who now eats hardcore scientific documents for lunch, and got a
reputation for himself as the local Linux kernel guru) could not
understand the Perl 6 Apocalypses, I expect much less of the many
everyday Perl workers, who are just using Perl to get their jobs done
and could not care less about all the extraneous Perl 6 features.

A lot of what Perl 6 integrates will probably turn out to be linguistic
fads that no one will want to use. Integrating everything at this point
is not going to be beneficial, especially for a language like Perl 5
that aims to be a no-nonsense language for everyday use.

Analogy with the Perl 4 to Perl 5 Conversion

A question that needs to be asked is whether this is analogous to the
Perl 4 to Perl 5 conversion that took place circa 1993 and also broke a
lot of compatibility, and required people to adapt or even rewrite a lot
of their code. There is a place for comparison, but nevertheless, the
situation today is very different. Perl 4 was heavily lacking in many
respects and justified a good redesign that would make it more up-to-par
with the powerful languages of then and now like Common LISP, Smalltalk,
C++, and Python.

Perl 5, on the other hand, can serve us well in the foreseeable future.
Another thing to note is that Perl 4 was a niche language that was used
by various Unix workers to write support scripts and the occasional
large-scale application, but not much else. Perl 5, on the other hand,
is much more commonly used. There are many more Perl 5 programmers, much
more Perl 5 code out there, and much more collective knowledge and love
of Perl 5. Also, the Perl "market" of programmers that can learn it is
much more saturated today than it was when Perl 4 started.

The Language Formerly Known as Perl 5

Here's a better idea: Let's continue to develop Perl 5 (either using the
Perl 5 backend, PONIE, or something else), making incremental
improvements, and so make sure it keeps with the pace of technology.
Sometimes, compatibility can be broken, or features removed, but not in
one big swoop, and always informing people about it first.

We need a new name, because Perl 6 bounds us with its upper digit,
leading us to eventually have Perl 5.666. I'll use the name Fifer here,
simply because it is an awful name that I don't intend to use. (I'll
let the Perl community pick a better name from the list of new names,
one of which I'm going to advocate). Fifer would be the language
formerly known as Perl 5. It would belong to the Perl Family of
Languages, which includes such languages as Perl 4, Perl 5, Perl 6,
Ruby, and even PHP.

10 years from now, Fifer will probably be more similar to what Perl 6
is today. However, at that point, there would be plenty of Rindolf code
that is perfectly working and modern, as people will have plenty of time
to gradually adapt their code.

In a way, I think this is what will happen regardless of whether people
read this article or not, but I like to make this prediction clear
because I think a programmer eventually needs to make such predictions
if he wants to make sure his work lasts for a long time. I see Perl 6 as
a dead end, and so don't intend to try to understand it further, and
don't intend to learn how to program in it, unless I actually have to
maintain some Perl 6 code. (which I don't expect to happen). And I
suggest you do the same.

The standing issues in Perl 5 (some of the major ones will be covered
shortly) can be resolved without a complete redesign of the language.
While a backend change may be in order (like switching to PONIE instead
of perl5), or at least a heavy revamp of the original perl5 one,
overhauling the language is not needed. The time calls for a gradual
evolution instead of a complete revolution.

Standing Problems in Perl 5 and their Solutions

Perl 5 is a good language, all in all. Nevertheless, it has some
standing problems which should be resolved. Perl 6 is the wrong way to
resolve them. Here is the list of the most major ones as I see them,
with suggestions on how to resolve them. Other people may have different
views.

Lack of Good Accessible Reference Documentation

The Perl 5 Man Pages are the official reference documentation for Perl
5. Nevertheless, they:

Are quite disorganized. Many things are documented in very obscure
places.

Rely on a lot of knowledge of other technologies (many of them
pretty obscure or largely superseded by Perl 5 now).

Give many examples that give a common use, instead of trying to
illustrate the point. Usually, they cannot be fluently read.

Are generally very hard for a person without a good background in
what they talk about to understand.

The Camel Book is
much better in this regard, but it is not freely redistributable or
modifiable, and not even available online except as part of the
pay-per-month Safari, or in various illegitimate online copies. It is,
thus, also inadequate.

A good core reference documentation needs to be written, especially
taking into account the fact that the ones for Python or PHP are
excellent. The easiest way for the community to resolve this is that the
authors of the Camel Book make its text available online, preferably
with its POD sources. This will not hurt the book sales much, as many
people buy it to read before going to bed, or to give as a present to
their co-workers, or to keep at their workplace for everyone to enjoy.

If this is not going to be done, then either the Perl 5 Core
Documentation will need to be heavily revamped, or an entirely new
documentation ought to be written. This will mean much more work on the
part of the contributors volunteering to do it, but it is quite
necessary.

Perl 6 would be a non-solution to this problem because it gives an
entirely new language for which new core documentation has to be
written.

(Note that I'm not referring to Perl 5 tutorials,
which help a beginner get started. There are quite a lot available, and
some of them are reportedly good enough.)

A More Unified Object System

The Perl 5 Object-Oriented Programming system is quite powerful, and
adequate for most needs. Nevertheless, to extend it whenever possible,
developers wrote a great deal of userland Perl modules on CPAN. However,
it is not perfectly clear whether it is possible to make active use of
several of these modules or all of them in one applications.

A good idea may be to create a new OOP system which features everything
these modules have to offer. This may be possible to do in userland,
while requiring people to choose between this or the original Perl 5 way
of doing things. It can actually model the Perl 6 OOP
framework, without requiring people to adopt Perl 6 entirely. Or it
can be something completely new that attempts to be more Perl 5-ish.

Regardless, it can be done in Perl 5.

Better Garbage Collection

The current perl5 garbage collection, which doesn't work well if there
are circular references, is incredibly limiting in the choice of the
application design. We need a better one. I believe Parrot and PONIE
will give it to us for free, but I may be wrong. In any case, perl5 can
probably be tweaked to adopt such a system.

Multi-Threading

The current iThreads threading model is very resource
intensive, and goes against most of the multi-threading tradition
used before Perl 5 adopted it. We need a better threading model in which
each thread simply shares the same memory space and variables as the
other threads. This is similar to what happens with POSIX Threads on
Unix, Win32 Threads on Windows, and the threading model of Java or
Python. It also was the old Perl 5.005 threading model.

It is possible that the perl5 backend cannot support this very well. If
Parrot (or an entirely different backend) does, then a change of backend
may be in order.

Other Issues

I have outlined some of the things I found pressing in Perl 5 in the Rindolf
Spec (version 0.3.5). Since then, I neglected working on the
specification (much less started working on a suitable implementation),
partly because I concluded that Perl 5 was already good enough for me.
Nevertheless, the specification may be studied to search for nice
additions to the language, all of them possible without breaking
compatibility.

Other people may have other things that bother them. I am happy to be
informed of any things that they feel are missing.

Conclusion

Perl 6 will probably not be widely adopted. The future of Perl lies
within Perl 5 and Fifer, i.e., the language powered by perl5, PONIE, or
perhaps a different backend. Any time invested in working on Perl
6 or learning about it can probably give you new insights about
programming, but don't expect your efforts to have any day-to-day useful
value.

People want a language that gets the job done, is fun to program in, and
which they love. Different people have different ideas of what this
language is, and they also use different languages for different tasks.
To quote my good friend (a different one), "How can you make a
programming language that is good for every purpose, if you can't even
make such a screwdriver?"

Perl 6 is the wrong thing at the wrong time. Perl 5 is already good
enough and will be so for the foreseeable future. We can evolve Perl
5 and gradually make it even better and more suitable for our needs, as
we see these needs coming. There is no reason to break every last bit of
compatibility, create an incredibly complex language which even experts
find difficult to understand, and integrate every last feature from
every other language in existence today, just because it is conjectured
that some people may find it useful.

A gradual evolution of Perl 5 would be a better idea. Fifer will be a
language of the present, instead of a very crude prediction of what new
languages will look like 10 or 20 years from now. In twenty years, Fifer
will probably be more similar to Perl 6. However, it will still be very
different, as things in the software world don't always develop the way
people expect them to. And it will also be able to run most of the Perl
5/Fifer code that has already been written (and possibly adapted to the
new changes).

Let's continue to work on Perl 5 and just make it an even better
language (and technology and culture) than it already is. What is for
certain is that Perl 6 isn't better.

in the meantime, perl6/parrot have basically dropped all of their original goals. new goals: get the mark&sweep stop-the-world garbage collector working. make perl6 rpograms not crash after 100 lines. announce win when perl6 reaches the just-100-times-slower-than-perl5 mark and so on.

Erm... subjective update: 2009 now, in my humble experience there is still not much of a Perl6 that I call tangible. I'm inclined to start saying... vaporware. (Debian doesn't know anything about pugs, rakudo or other scary vegetables. Only parrot has certainly gained some audience!)

9 years in the making... i'm still happily (ab)using perl5 when the need arises (seldom).

Re: My feeling
Yes, it does have a long way to go but consider where it's heading. Some of the things that will be incorporated into the language are still being researched and developed. Make no mistake, when it's done it'll be the most advanced language ever seen.

"Will it be complex?"

Sure, some things will be different. But as many of you have pointed out, writing some real apps with a language straightens out all the kinks in your brain, or at least, goes a long way.

Take Haskell, a very powerful language with which the Pugs perl6 interpreter is being built. Oooh, monads, type classes ... Believe me, that stuff is scary at first, especially with all the people going around whining, "What are monads? What's category theory? Do I need to study math to code?". Just write some code. No math required.
Monads: no big deal. The let you keep track of what's going on and when. Type classes: you'll love these, they're a nice generics mechanism. Believe me, you'll get it and you'll love it.

The clarity and coherency of Haskell's design will shine through after an indeterminate number of lines of code.
I suspect the same will happen to those choosing to move to perl6. Reading a spec 20 times won't make you grok any language. It's mileage on the keyboard that really counts.

"So, uh, will it be complex?"

No.

The whole point of the new language is to take what was perl5 and make it cleaner, more powerful and beautiful.

"Should I wait for perl6?"

What the Hell? What kind of self-respecting hacker waits for anything?

You've got perl5, right? If you like it, use it. If you think it should be better, contribute.

As for perl6: If it's taking too long for you, go to the pugs website and lend a hand.

Simple. If you don't contribute, don't complain. Perl can be what you want it to be.