While I was at OSCON I found myself chatting with various non-perl and ex-perl folks and many of them had the same impression of Perl as a great language for hacking out short scripts to get stuff done, right now and possibly under severe time pressure. For these people, those scripts would very rapidly become unmaintainable. And it’s easy to see why they came to that conclusion. Perl was originally designed and implemented by a systems administrator as a tool to make his every day tasks easier so there are a bunch of shortcuts and defaults that do exactly that – make Larry’s life in 1987 easier. Which is great when you’re whipping up a script to meet an immediate demand, but horrendously cryptic when you look at it later.

This is a perfectly valid way of looking at a language that I’ll call ‘desperate perl’. Desperate Perl scripts tend to be written by folks who don’t necessarily know all of Perl and who certainly don’t know all of its ecosystem. The scripts are often written to a tight deadline and to handle a specific task. They certainly weren’t written with even half an eye to maintenance. And they live on. Boy, do they live on. There are CGI scripts written in 1995 that are still in every day use, never mind that they’re buggy, insecure and well nigh impossible to maintain, still these zombies shamble on, colouring the opinion of everyone who ever has to deal with them.

If there’s one thing we’ve learned from this, it’s that code is forever. When we were young, we did stupid things, but we grew up and everyone but our nearest and dearest (who have the good grace never to mention it) forgets that, once upon a time, we set fire to the living room rug while playing with matches/ate cornflakes with HP sauce/had a brief flirtation with the young Conservatives/had a brief flirtation with a young Conservative. We are forgiven.

Not Perl.

Perl’s reputation as a language that’s fine for writing short scripts, but which rapidly becomes unmaintainable once you try and write larger systems is bestowed upon it by people who looked at formmail.pl once and ran screaming into the night (a not unreasonable response).

But Desperate Perl isn’t the only language in the binary. Large scale programming with Perl uses, or should use, a different subset of the language. A key indicator that you’re looking at large scale Perl is when the script or library starts with something along the lines of:

#!/usr/bin/perl
use strict;
use warnings;

or, if it’s a library:

package Whatever;
use Moose;

There are other signifiers, but the important thing is that the code is written in strict compliant Perl. use strict crops up so often that the recent crop of modern (or enlightened if you prefer) perl modules, like Moose, turn it on by default. Indeed, if you’re using Perl 5, version 12 and you begin your code with use 5.12.0, Perl turns on strictures as well as enabling all the new features of the language.

We’ve had strict and warnings since forever and they’ve been strongly recommended for everything longer than about two lines since forever too. Those of us who program in Perl for a living get rather annoyed by the ‘Desperate Perl’ blinkers worn, however unwittingly, by so many outside our community. We write our code in a subtly different language. We write tests first, we have done for years. We write clean, maintainable, object oriented code. Nowadays we use Moose to help us with that; Perl’s out of the box OO features are, let us say, idiosyncratic – Moose makes our lives better.

Large Scale Perl is about more than just what’s in the distribution. Large Scale Perl has a wider culture than just the contents of the distribution. It’s about learning to use the CPAN and the CPAN toolchain. When I start to work on anything perl related that I expect to live for more than I couple of minutes, I start by doing:

Because I’m writing large scale perl, my modules go under lib, my tests go under t, any scripts I expect to install in my path go in script and they don’t have a .pl suffix. I lay my code out this way because, well, that’s just how it’s done. I run my tests from the project directory by doing prove -l ./t and, when I realise I have something that’s worth distributing (or deploying to the staging server), I add a Module::Install based Makefile.PL that lists my prerequisites, the scripts to install and various bits of metadata. By now, this is second nature – many clever people over the years have come to a collective decision about what a good Perl project looks like and they’ve written a fabulous toolchain for me as well.

If I distribute my work through the CPAN, there’s a large community of CPANTesters who will attempt to build my module and run its tests on a bewildering variety of platforms, and they’ll send me bug reports when those tests fail (bug reports which I can opt out of as well). Any documentation I write will be nicely formatted and accessible through search.cpan.org along with links to sites that’ll draw me a tree of all the modules mine depends on and, if it becomes adopted by the community, nice lists of all the modules that depend on mine as well. My module will get its own section in rt.cpan.org so there’s a standard place for people to report bugs.

Large Scale Perl comes with an entourage. Tools like Perl::Critic can check my code for bad style, Try::Tiny helps me get exception handling (which is annoyingly tricky when you attempt to get it right barehanded) right, Devel::NYTProf helps me work out why it’s running slower than I expected. The list goes on.

But, all the world sees is Desperate Perl. Search for “Perl Tutorial” on google and the first few hits are horrendously out of date – when I checked, the number one hit was all about perl 4. Apparently, O’Reilly’s biggest selling Perl title is the new edition of Learning Perl, which was published in 2008 and is a cracking tutorial. But the second biggest Perl title is Programming Perl which is ten years old this month. It covers Perl 5 version 6.0. I’m not even sure you can still build perl 5.6.2 – released more than 6 years ago – on modern systems. I certainly can’t imagine why you might want to. Programming Perl is a great book, but there are things it discusses that aren’t just deprecated, they’ve been recognized for the clusterfuck they were and excised from the language with extreme prejudice. As a community, we could really use a new edition. It would seem to make sense for someone to take the job on. A computer book that’s still selling 10 years after its first publication, 8 years after the language it documents, 5.6 was superceded by 5.8 looks like an ideal candidate for a new edition to me.

But then, a glance at the release dates of major Perl versions shows us why O’Reilly might hesitate to revise the book. 5.6 got released in March 2000, 5.8 in July 2002, 5.10 in… December 2007, 5.10.1 (which might have been better named as 5.12.0, given how much got changed/fixed) in August 2009, 5.12 in April 2010. That 5 year gap is the sort of thing that might give any publisher pause. Still, the new strategy of releasing regularly and often is starting to bed in. It’s getting easier to write perldelta.pod for each release and Jesse Vincent seems to be doing an excellent job of getting new release managers trained up and generally destressing the business of releasing a new perl. Who knows, by this time next year the perl release process may be no more difficult than releasing a module to CPAN. We do write good tools after all.

Reliable, regular releases with no big surprises are the sort of thing that gives programmers and organizations confidence that we’re here for the long haul. We perl programmers know that all sorts of good things were happening on the CPAN in the long gap between 5.8 and 5.10, but as good as it is, CPAN’s not as visible as the version number on the binary.

Hmm… I appear to have rambled some way from my starting point. Which probably means its time to stop. If I’m spared, I might manage to rustle up some kind of conclusion in a later post. Just remember, there are at least two perls: Desperate Perl, and Large Scale Perl. Both are excellent languages in their spaces, but you can’t (or shouldn’t) draw conclusions about one based on observations of the other.

I’m the author of that tutorial that appears in Google’s number 1 spot. And, yes, it’s a dubious accolade these days, though still one I have some pride in.

I think you may have spotted a genuine gap in the literature market. There’s a lot of legacy Perl out there, and the host organisations are naturally afraid to touch it. So if there was a guide to “Delegacy your Perl” or “Bring your legacy Perl into the light” it may be highly valued. That’s not the same as writing good Perl: it’s about understanding and incrementally saving code that’s already there. That book has not yet been written. It could help save Perl’s reputation.

By Sumeet Grover
Mon, 26 Jul 2010 03:33:18 GMT

Your thoughts very much echo the theories and principles of Software Engineering. Any piece of code that follows bad software design or no design principles at all is bound to be hard to maintain. In that sense, it’s about bad programming practices than blaming the language itself.

Excellent rebuttal, as Dave mentioned, and a nice redirection towards some key signs of well-written and modern Perl source. As you mention, the tools are getting better and better, so it’s much easier for beginners to write scalable and maintainable code these days without taxing their patience too much.

Maybe it’s time we added something to Perl’s name to make it clear that we’re dealing with a different paradigm to Perl 4 (and earlier, “desperate” 5.x variants). The “5” and radically different “6” seem to only make sense to the initiated and may be doing a disservice in terms of public awareness. (Let’s avoid Perl++, though.)

@Caleb – Such is the double edged sword of enterprise distributions and their long lived lives. As a developer, I find it somewhere between irksome and annoying. As a sys-admin, I find it extremely comforting.

It’s lamentable that it’s not MORE of a problem to me in my sys-admin role. That would indicate I needed to install packages/programs that required the newer version, which sadly is not often the case.

By Mallory
Tue, 27 Jul 2010 03:42:08 GMT

Dunno how this trackback thing works… posted a link to your article on SitePoint forums: sitepoint.com/forums/showthread.php?p=4657297 in the Perl, Python and Other Languages section.

By AJ
Tue, 27 Jul 2010 09:30:30 GMT

A later edition of Programming Perl includes updates for Perl 5.8.

Also, I challenge your notion that large-scale Perl programmers use a specific set of libraries. Most of the time we’re limited to whatever interpreter is installed, which is usually 5.8 or 5.10. If you are trying to limit dependencies as much as possible, things like Moose cause problems. Module::Install may be great, but ExtUtils::MakeMaker is what works when you need to make deb packages or RPMs. I agree there are clues to whether something was coded by a real Perl developer or not, but listing a raft of modules and then making swipes at Larry for what was the correct, accepted practice for the language a long time ago does not convince me you have truly identified the difference between “desperate” and “large scale” Perl.

And if you’re stuck programming with the system perl and what came with it then you’re as near to desperate as makes no odds. I certainly would be in your shoes.

But then, I’m building RPMs for installation on our production without having to deal directly with ExtUtils::MakeMaker or worry about clashing library versions with other applications on the platform and, once I’ve got the wrinkles ironed out and clearance from management I’ll be writing that process up too.

By thickas
Wed, 28 Jul 2010 03:23:11 GMT

Surely any poorly written code is poorly written no matter
what language the writer chooses.

I’d suggest there are other factors behind the criticsm of Perl: disgruntlement with the pace of development, that it remains a cathedral and that appealing features are not in it.

Why not criticise C or assembler programs because some programs don’t create abstract data structures to make the code easier to maintain ?

@thickas: If the tenor of the criticism had changed much over the years, I might be inclined to agree that it stems from issues with the pace of development, but it hasn’t.

Perl is a cathedral? Not from where I’m standing – Perl 5 certainly isn’t. The last Perl 5 developer release was the work of at least 105 people. 12.0’s AUTHORS file has over 1000 entries. The Rakudo/Perl 6 dev team is rather smaller, admittedly, but it’s always open to volunteers and does its work in public. @Larry is a smaller group certainly, but that’s the nature of language design – it’s not really something you want done by a large committee.

I think that the effort that Ruby and Python have gone through to create platforms similar to CPAN and tools & package layout conventions similar to those described above shows that most everyone thinks these are great. CPAN predates stuff like rubygems and PyPi, and is still superior to both platforms in a lot of key ways (like the testing integration).

However, I think Perl hackers have to come to terms with the fact that, for most people who aren’t Perl hackers (like myself), the part of the Perl ecosystem that bothers us the most is Perl.

For a Perl hacker, the fact that Moose exists at all is a virtue of Perl’s meta-programming facilities; for non-Perl hackers, the fact that it exists is a weakness in Perl. When a non-Perl-literate hacker comes across ‘$$’ in code, or ‘$<’ or any of its brothers in perlvar, they (rightly, I think) get the idea that Perl was designed for “Desperate Perl”, and that the subset you describe was a reaction to the myriad deficiencies endemic to the language itself.

By Mark Bradshaw
Fri, 30 Jul 2010 10:28:43 GMT

Thanks so much for your great work at OSCON this last week. I came in snickering at the Perl tracks (“Oh my. That’s still around?”), and left saying, “Good God I need to get me some of that!” I left Perl for the greener pastures of php and .net back in 2001. I haven’t noticed a lack, per-se, but in retrospect perhaps I’ve missed some really good things.

I wonder if there’s a good way for someone who’s very perl rusty to get up to speed on latest/greatest perl thoughts? It seems that there’s not a good book to recommend…

You can write beautiful, clean, maintainable Perl code, but I think what’s killing Perl is its learning curve – so many people write crap Perl because after 1 year of Perl they’re still language novices.

Compare this with the alternatives, where a year of experience is plenty.

By Homer
Sun, 08 Aug 2010 07:43:50 GMT

the jump from bad tutorials and being a desperate perl hacker to code examples that are better is so big it’s quite hard to progress…

By David Landgren
Tue, 31 Aug 2010 10:05:09 GMT

@Ori : language novices writing crap, this is so true. A few years ago the decision was made at $work to switch to PHP, the market for PHP programmers being (perceived to be) much deeper than that for Perl. But experience shows that the quality of the code they produce is, by and large, dismal. I think much of reason stems from the language itself.

I’ve managed to alleviate some of the misery of dealing with the intellectual impoverishment that PHP embodies by applying idioms I picked up programming in Perl. On the whole, PHP-only programmers look at my code and say “Wow, that’s a nice way of doing it”

So you take your junior programmers and do code walkthroughs and coax them into engaging their brain and thinking about ways to improve the quality of what they write. But even so, at the end of the day, you cannot write truly, elegant PHP. It just doesn’t exist.

No slices, list flattening, expression modifiers, a statement is an expression, maps, regular expressions (other than via a library API); you keep winding up writing make-work code that the Perl interpreter would otherwise do for free.

Another aspect is that many programmers seem to be unsure about what the exact terms of the agreement between them and the PHP interpreter, so you encounter all sorts of cargo cult paranoia such as

if (isset($var) && !empty($var) && is_array($var) && …)

This kind of confusion really doesn’t exist in Perl. You just test the variable for truth and get on with it.

You can work around some of the problems, but in the end, you cannot really lift PHP programmers into a higher plane: the language lacks the support to keep them up there. Scoping follows a weird BASIC model, lists don’t flatten and cannot be sliced, no anonymous code blocks… the list goes on.

In Perl, you can continue to write Baby, or Desperate Perl, but should you choose to improve your fluency, it will reward you many times over.

Comment A tale of two languages

Trackbacks are disabled

About Piers

Piers is a programmer, photographer, singer, cook and all round geek. He's based in Cornwall and, if you need to get in touch with him, you should try one of these: