Mark Jason Dominus: My main reasons were as I explained in the
preface of the book: It had become increasingly clear to me that most
Perl programmers seemed to be writing C programs in Perl, and that it
was possible to do a lot better than that. We have a really good
tool, and it's a shame that we're using it the same way that we have
to use the old crappy tool.

But I had a few hidden agenda items as well. The book is about how to
use closures, and I think there are too many languages around that have
no easy way to use closures. Java, in particular, seems to me to be a
giant step back towards the languages of the 1970s. My first draft of
the the preface had a long screed about how back in the 1950s,
recursion was unavailable in common industrial programming languages,
such as Fortran, and if programmers understood recursion at all, they
thought that it was an unnecessary, inefficient, ivory-tower
technique. But eventually languages like C and Pascal appeared, and
if you tried now to popularize a recursionless language, people would
laugh at you.

I think the same process happened with automatic garbage collection.
There, Java was the big turning point, and I think that in another ten
or fifteen years nobody will bother much with languages that have no
automatic storage management.

I think the same thing is happening with closures, but we haven't
gotten as far. It's still possible to promulgate a language like Java
that doesn't have practical closures. At the time I started writing,
Python didn't have closures at all and I heard Guido van Rossum say
that they weren't important. I think that's wrong, and that in
another thirty years people will laugh at anyone who tries to invent a
language without closures, just as they'll laugh now at anyone who
tries to invent a language without recursion.

TPR:At the start, you compare Perl, Lisp, and C and say that
Perl is more like Lisp than C. Do you think Perl would be as
powerful if it was closer to C? Could it be more powerful if
it was closer to Lisp?

MJD: I don't think anyone thinks that Perl would be more
powerful if it were closer to C. In the 1980s, Unix sysadmins wrote
in C and in Bourne Shell, and Perl has largely replaced both of those
for those sorts of tasks, leaving C a fairly small niche.

In the preface of Higher-Order Perl, I said that Perl shares
six of Norvig's "seven
features that make Lisp Different". The seventh one is the one
that Norvig calls "uniform syntax". That's the one that led Larry to
say that Lisp looks like oatmeal with fingernail clippings mixed in.
But it enables language features that are so powerful that they're
almost unimaginable to people who haven't studied Lisp. It's really
hard to do justice to the huge benefit that Lisp gets from its simple
and uniform syntax, which Perl just doesn't have at all. Any attempt
to construct Perl code dynamically is fraught with error, and we take
it for granted that it won't work. For example, in the documentation
of the Perl6::Subs module
that was recently released, Chip Salzenberg says "This module is a
source filter. Source filters always break." And I think that's
true. But source filters in Lisp are easy to write, and they never
break. Lisp's main assignment operator, "setf", is implemented with a
source filter. So if Perl were more like Lisp in that sense, I think
it might be more powerful. On the other hand, I don't think it would
be very Perl-like.

TPR:What's your relationship to Lisp? Do you still use it?

MJD: Occasionally. But I find Haskell and SML more attractive,
for various reasons. One is that the Lisp user community is severely
dysfunctional. Another is that I find the strong typing of Haskell
and SML fascinating.

TPR:Do you wish Perl was more Lispy or Lisp more Perly?

MJD: Neither, really. I like having more than one language.
Sometimes I wish Perl's syntax were less verbose.

TPR:Did you learn more about Lisp as you worked on this book? Did
you learn more about Perl?

MJD: The book took five years to write, and I learned a lot
about a lot of things in five years.

TPR:Readers learn a lot about functional programming from Higher
Order Perl. Do you think programmers should still learn Lisp
or some other functional language?

MJD: First I need to put in a disclaimer that I don't think
Lisp is a good example of a functional language and I don't think most
Lisp programmers would consider it one. "Functional language"
suggests a firm philosophical stance against side effects. You can
see this stance in true functional languages, like Haskell, where all
side effects must be quarantined in a special kind of data structure
called a "monad". I think most of the Lisp world doesn't care about
that stance.

That said, I do think that all programmers should learn as many
different languages as they can, and I'm puzzled when they don't want
to. It's like being a carpenter and saying that the hammer and the
chisel are good enough for you, you don't need to learn to use a saw
or a screwdriver.

The best thing about going to learn something new is that you never
know what you're going to find out. HOP came about because I
discovered all these techniques that work well in Perl that I had
never seen done in Perl before. It's hard to think these things up by
yourself; it's much easier to borrow them from someone else.

Programming communities tend to be isolated. We all know the same
languages, we all read the same books, and we all learn how to solve
the same problems in the same ways. If you want to be able to solve
the problems that have stumped the people around you, you need to know
something they don't know. One way to do that it by knowing how
things are done in a different community.

If you're going to learn a new language, Common Lisp would be a good
choice. So would Haskell. They're both powerful and useful, and
they'll get you thinking about things in a different way than you're
used to.

TPR:Are you surprised that after a couple of years of working on
a book about functional programming in Perl, Autrijus Tang
has lit a fire under the community by using Haskell to write
a Perl 6 compiler?

MJD: Everything Autrijus says and does astonishes me.

TPR:A lot of the techniques in the book seemed like it was
geared to programmers writing things for other programmers,
such as tools or modules. Was that your intended audience?

MJD: Yes, and I think I said as much somewhere early in the
first chapter. Part of that is my own bias, since I'm just not very
interested in user interfaces. But I think most programmers are
involved in writing code for other programmers, rather than for end
users. And almost all programmers end up having to put programmatic
interfaces on most of their code, because that's the only way you can
write regression tests for it.

TPR:When you discuss memoization and caching, you say that the
ability to do this easily (as with your Memoization module)
should be a major point of Perl evangelism. What other major
points of advocacy can we get out of Higher Order Perl?

MJD: Memoization is a great example because everyone anyone can
understand it in five minutes. If you have more time to explain, I
think the overloaded parser stuff from chapter 8 might be a good thing
to try. Writing functions and operators that combine simple parsers
into complex ones is a really useful technique that should appeal to
almost anyone who has ever had to write a parser.

TPR:Why did you choose Morgan Kaufmann as a publisher? As far as I
can tell, this is their only Perl book.

MJD: I think it is their first Perl book. I chose to publish with MKP
because I felt that there was a really good personality match between
myself and the people with whom I would be working. I'm really glad I
did; it worked out very well.

TPR:Did you design Higher Order Perl as a possible textbook?

MJD: No, just the opposite. I was terrified that people would
see it as an impractical academic treatise with no relevance to
professional programmers. I tried hard to rid it of any academic
aroma. So, for example, there are no exercises at the end of the
chapters.

The publicity people at MKP have asked me if I think it could be
marketed as a textbook, but I have a lot of trouble imagining what
class it could be used for.

TPR:Did your idea of the book change a lot since you first
started? What happened along the way to get the book to its
final form?

MJD: No, it came out pretty much the way I planned it. Even the table of
contents isn't too far off from what I originally proposed back in
late 1999.

The only big surprise was that in several places I found that there
was a lot more to say than I had anticipated. For example, the
chapter on iterators was at one point about 160 pages long. I just
kept digging deeper and deeper. I cut some of it out before the book
went into production, but most of it is still there.

TPR:As you wrote this book, you gave out chapters on a
semi-public mailing list. How did that work out for you?
Would you do it again?

MJD: That appears to have been a really outstanding success,
and I will absolutely do it again. When you circulate the drafts to
six hundred people, some of them read those drafts really carefully,
and you get excellent early feedback. For example, there's a section
in chapter 6 about sorting a log file into a different order while
it's still being written. That's there because Dave Turner said he
found my other application of cutsorting unconvincing and abstract. I
decided he was right. There's a big chunk of chapter 9 that I rewrote
because Greg Bacon sent me a long letter asking for clarification, and
it's much better as a result. That's the sort of thing that you hope
you'll get from the official technical reviewers, but you hardly ever
do.

TPR:Higher Order Perl is (or will be) free on your website. Will
it always be the published version, or will you update and
add to the content?

MJD: I'm not sure yet. One idea I've had was to turn the web version into
a wiki, with a few paragraphs on each wiki page, and let people
discuss the book, and add clarifications and criticisms on the web
site.

TPR:Giving away your book is a huge gift for the community and
represents several years of your personal time. How do you
hope people give back? Tipjar? Speaking engagements? Free
dinners at conferences?

MJD: I hope people will buy copies of the book to keep. MKP is
nervous that having the book on the web site will hurt sales. I don't
think it will, but I'd sure hate to be wrong.

Since you asked, though, I would like to put in a plug: I used to earn
my living by traveling around and teaching Perl
programming classes. I couldn't have written the book if I had had
to hold down a regular job. And not having a regular job also made it
possible for me to visit conferences and user groups and give talks
for free. So if people want to support me, they might consider
getting their company to bring me in to do some classes.

TPR:In the past, you have expressed strong feelings about
Amazon.com. Where would you like people to buy your book?
Do you have an affiliate link?

MJD: I'm still upset about Amazon's ridiculous "one-click"
patent, and I'm still boycotting them. Software patents like Amazon's
threaten the livelihoods of all programmers, including mine, and I
don't want to give my support to a company that means me ill. They
dropped their suit against Barnes & Noble, but the terms of the
settlement were never disclosed, so we don't know what happened, and
Amazon could turn around at any moment and sue anyone for violating
their patent. So the issue is unresolved.

I'd like people to buy my book from anyone but Amazon. My web site has an
affiliate link to either to Barnes and Noble or to Powell's Books,
depending on who isn't out of stock at the moment. I get a small
kickback from this, but I really don't feel strongly about it. I do
feel strongly about not supporting Amazon. There are a lot of places
to get books online; it's not hard to buy books from a company with
less abusive business practices than Amazon.

A review of Higher Order Perl will appear in the Summer 2005
issue of The Perl Review.