February 2011 Archives

While I don't necessarily recommend the book in general, butterflies can be scary, so what appears to be a harmless lepidoptera might eat your brain. The bright colors are warnings against potential predators. Tread carefully. For all of the beauty, you have no idea what the butterfly might do to you.

Of course, Perl 5 is smarter and more dangerous than you think. If you fail to respect it, it will eat the competition.

Imagine if they teamed up, perhaps with a Perl 6 with access to most of the CPAN. Talk about a velociraptor with butterfly wings, or at least warning spots.

Take off your "My primary role as a software developer is the creation of
free software for others to use" hat. Replace it for a few moments with your
hat of "My primary role as a software developer is the creation of software
which provides a good or service to customers." (I assume you don't sell or
license this software; the software is a mechanism of producing your product or
service.)

Yes—but what matters more? Code you can deploy today—assuming
you can generate revenue from it—is more valuable than code you can
deploy next week. Code that exists today—and does what it needs to
do—is more valuable than code which may exist.

With that said, quality is important. Code you can deploy today is valuable for today, but if you can't maintain and modify and deploy a new version next week, it's less valuable than code that you can maintain and modify and deploy again and again and again. I've known projects which started out in one language because of the ease of prototyping, then switched from language to language as the project's scope expanded.

A good team with good programmers and good discipline and the
freedom and flexibility to refactor and revise the design of the project to
make the current version sufficiently easy to maintain and modify while
allowing future expansion is in a great position to continue to deliver
business value.

The technical requirements are a language with libraries and tools of sufficient quality to allow development, maintainability, and deployment.

"Sufficient", of course, is a management weasel word which hides a great
deal of complexity and other assumptions—but that complexity and those
assumptions depend on your business needs and your available programmers
and....

Even so, some of us have a secret weapon. Given an additional resource, such
as the ability to manage deployment yourself (without the limitation of, for
example, a $3.95/month shared hosting plan, an internal IT system which
mandates the use of WebSphere, or the Silicon Valley requirement that all new
startups must make use of monkeypatching), sometimes you can find a dramatic
improvement in quality or deployability or deliverability or ease of
implementation or clarity of code or not having to write much code at all.

For me, that's Perl. That's why I care about improvements to Perl
5. That's why I care about a Perl 6 in which I can write and with
which I can deploy great applications.

While explaining the meaning of the phrase "Modern Perl" the other day, I realized that there are two categories of users of Perl 5 when we consider the use of Perl 5.

One category uses Perl 5 because it's ubiquitous. It's already installed and configured (whether as part of the default OS installation or a shared hosting package or a helpful system administrator). It's easy to begin. Copious tutorials abound, the core libraries are sufficient, and Perl's lack of ceremony means that they can write baby Perl and get their jobs done and that's as far as it goes.

You're in the other category, almost assuredly. I am. We use the CPAN. We
look forward to upgrading to Perl 5.14 and Perl 5.16. We see something like MooseX::Declare or Plack::Middleware::Auth::Basic
and can't wait to replace custom code in our applications with a component
other people have written, documented, tested, and maintained. We're
comfortable with a community built around the simple structures that have
arisen from community consensus regarding the CPAN.

Modern Perl, as I see it, is an invitation for the people in the first group to join us in the second group.

With that said, we must keep in mind the concerns of the first group:

I'm not necessarily a programmer.

I'm not necessarily a system administrator.

I don't understand how to manage dependencies effectively.

I don't know which modules to use.

I'm stuck with an ancient version of Perl 5.

I don't know which book or tutorial to read.

... while removing technical and social barriers to encourage them to join the second:

Improving the development process and release schedule of Perl 5 to help supported (read "Modern") versions of Perl 5 spread

Not everyone in the first group will migrate to the second, but given that
the evolution of Perl 5 is primarily in the hands of the CPAN these days, we
can only do novices a service by making the CPAN ecosystem—code
andtribal knowledge—available.

The most recent time I did any significant GUI programming in Java was over a decade ago. Even though Swing had just come out, I used AWT because Swing wasn't available for the GNU/Linux platform to which I needed to deploy my code.

Since those days, Java's had a few releases and a few updates and a couple
of owners, and if I were doing GUI development in Java right now, I'd at least
consider whatever toolkit Eclipse uses—but my opinion on GUI development
in Java is completely out of date and largely irrelevant. If you were to start
a new GUI project in Java right now, you would do well not to listen
to me for specific details. I may or may not be able even to give a general
overfiew of the field because my knowledge of the state of the art is far, far
out of date.

Unlike a fine wine, my technical knowledge of Java GUI programming has not improved over time.

I could say the same thing about PHP or Rails or ASP.

Perhaps if we were better at acknowledging the fact that our experience bitrots as the state of the art changes we could have better discussions of the quality and utility of technologies.

(In other words, your experience installing Red Hat 5.2 and using it for 20
minutes is rather irrelevant these days, as is your use of Perl 4 in 1993 or,
these days, even Pugs.)

C. S. Lewis wrote something in the Screwtape Letters to the effect that
every vice is a virtue misapplied. Doesn't that describe software
development?

CPAN is valuable because of the semi-formal guidelines which have grown up around it:

a culture of sharing

a first-come, first-served namespace policy

a culture of testing and quality

good tools to produce, bundle, install, and verify distributions

optional but free bug reporting

a standard documentation format and structure

The common Perl 5 module documentation structure—a name, a synopsis
with code examples, a description, and then an API listing—is good for
describing a single, standalone module. You can extend it to explain the
philosophy behind your code and its use, as something like Test::More does, or you
can provide a very simple explanation of a very simple API, such as strict does.

Like many of the good features of Perl 5, the CPAN documentation format
borrows liberally from a feature of Unix: specifically, the format of man
pages.

Like many of the not-quite-perfect features of Perl 5, features borrowed heavily from Unix show their limitations. In particular, the biggest flaw of the API-heavy documentation structure is that it's only good at documenting APIs and its passable at best at describing the important relationships between components of the distribution which provide APIs.

In other words, if you want to understand DBIC and use it
effectively, you must understand the relationship between result
objects and resultsets as well as the separation of responsibilities between
them. (DBIC deserves credit for recognizing and addressing this documentation
gap by providing multiple pieces of documentation beyond API explanations, but
the problem is difficult to solve.)

You can see the problem more dramatically by exploring something like HTTP::Response, which is part of a well-factored and robust distribution but which has the relevant information spread through several separate pages of documentation because the structure of the documentation reflects the inheritance structure of the code.

Again, you can't fault Gisle for factoring his code well and documenting it effectively in the dominant style of the time (and Gisle deserves much credit for helping to spread the dominant style of documentation)—but isn't it ironic, a little bit, that where objects should be well-encapsulated black boxes that their internal design decisions are so apparent in the documentation?

Some will suggest that we need better ways to explore the documentation, but
I believe there's a bigger question we need to answer. When I'm flipping
through all of the documentation for DBIC to figure out the best place to put
validation code to ensure that my objects always get created in a known-valid
state, I need something far different than a list of methods used to search for
objects that already exist. I know what I want to do, but I don't know
which module's documentation holds the right answer.

Update: Your author disclaims the contents of this post and leaves it here only as a historical curiosity.

Last July, I wrote An
Accurate Comparison of Perl 5 and Rakudo Star. Despite almost seventeen
years of tuning and optimization, there's no huge difference between Perl 5's
startup time and memory consumption when compared feature-by-feature to Rakudo
Star.

The same holds true today. In fact, Rakudo's startup time in my most recent measurements is better than Perl 5.12.2—and bigger performance improvements are coming to Parrot in the next few months.

Perl 5.12.2's startup time, loading all of those modules exceptParrot::Embed is 0.627 seconds at best. Rakudo's startup time is 0.441 seconds at best. That means Rakudo starts 30% faster than a similarly configured Perl 5.

Rakudo's memory use is still higher; from the REPL, it uses 176,780/130,968
KB virtual and resident memory. Perl 5's usage is 141,204/47,884 at the
debugger. Memory parsimony is probably the next area of optimization for Parrot
and Rakudo.

Of course the Rakudo and Parrot developers can't optimize for your
needs without knowing your needs. Improving synthetic benchmarks may or may not
improve real programs. You can help these efforts by writing real programs, by
testing real programs, and by finding areas where real programs seem slower or
more memory hungry than they should. In particular, I believe the
parsing component of the Rakudo ecosystem has several undiscovered
bottlenecks which could yield notable improvements.

Modern Perl, the book, is now available in ePub format as well as A4- and letter-sized PDFs.

There is no DRM and there are no special licensing terms. Please
redistribute this book far and wide and tell everyone that they too can write
great Perl 5 code that takes advantage of all of the hard-earned wisdom and
effort we've put into improving the language in the past several years.

Of course, you can always tell people to buy the printed copy—I'm a
fan of paper—but the important part is spreading the word, whether
through reviews on Perl Monger lists, web sites like Slashdot and Ars Technica
and LWN, and of course booksellers. You're also very welcome to upload and seed
torrents, post it on ebook sites, or just hand around USB sticks whenever you
get together.

Bilingual Perl hackers have already begun translations into other languages, and we're very happy to see this and help in any way possible.

If we're going to show off how great Perl programming can be, let's make
sure everyone has a chance to read a good book about it.

What's the future of Perl? Let's look ahead five years to the possibilities.

(Note: like all intelligent, attractive, and good-hearted people, I use
the term "Perl" to refer to the entire Perl ecosystem and all versions of Perl.
If you want to argue that "Perl" has, now and forever, always meant only "Perl
5", please set your time machine to late 2000 when that argument was relevant
and, albeit barely, worth having. If you don't have a time machine, consider
that a decade is long enough to have learned some coping skills and rethink
your priorities.)

Who Uses What

No one uses Perl, any version, for new projects.

Some people use Perl 5 and some people use Perl 6.

Most people use Perl 5 and almost no one uses Perl 6.

Most people use Perl 6 and almost no one uses Perl 5.

What do they use it for?

New development.

Maintenance of existing projects.

Porting from other languages.

What's the relationship of Perl 5 to Perl 6?

One has subsumed the other.

One has failed and the other has survived.

Perl 5 is becoming Perl 6 as much as possible.

Perl 6 allows you to use Perl 5 in process.

What New Features Does Perl 5 Have?

A foreign function interface

A real object system with syntax and everything

An improved exception system

An improved concurrency system

Function signatures

Grammars

Junctions

Strictures by default

None of these

All of these

Do You Still Use Perl?

Yes

No

Is Perl Healthier Then Than Now?

Yes

No

With all of your answers, you have a decision to make. Do you like this future? If so, great!

If not (and here I admit to some concern myself), how can we make the future more appealing?

Do you know how to access the symbol table yourself, without resorting to string eval? Do you know how to find all of the variables within a package or all functions within a package?

This metaprogramming—or reflection—is a fundamental part of
higher programming languages. While all symbolic languages intend to operate on
parametric symbols instead of raw data (that's why you can write equations in
programming languages instead of using your computer as a glorified adding
machine), higher languages operate on language constructs as well.

You can see this in lower languages such as Java, where the only good way to make flexible and extensible programs is to generate big wads of XML which control your AbstractFactoryInterface instantiations such that the classloader can inject the right code to make the language do what you need without forcing you to write even bigger wads of repetitive boilerplate Java code.

(During a consulting job several years ago I had to generate 60+ unique reports with Crystal Reports. The reports differed by way of two column names, but Crystal Reports wasn't flexible enough to let me define one report once, then pass in the two parameters I needed. This is why 4GLs failed.)

Metaprogramming and parameterization and flexibility beyond "This is how you write programs in this language, darn it, and you shall never think of doing anything different that the language designer didn't have in mind." may be difficult to implement well and it may not completely satisfy everyone, but it's useful and necessary.

Consider Package::Stash. If you
couldn't remember all of the answers to all of the questions in the first
paragraph, you'll probably find P::S much easier. (Certainly it's less buggy
and much less annoying than munging the symbol table by hand.) Use it when you
need it. It's nice. It's Perlish. It doesn't look like the dog's breakfast of
random punctuation that you need to manipulate symbol tables (which are even
uglier than the standard dereferencing syntax, because they're special
hashes).

Now that I've explaind a bit of programming language design philosophy,
explained a dead end in programming language design, and pointed you to a
useful tool from the CPAN, let me get to the point:

There's no excuse for something like Package::Stash not
being core behavior of Perl 5.

Think about that for a moment.

How can you defend Perl 5 as a practical language, one which makes easy and hard things possible, when standard higher language behaviors such as symbol manipulation are so eye-gougingly ugly and esoteric by syntax that they are out of reach of even adept Perl hackers? These are not black magic concepts (else MIT wouldn't have taught freshmen Scheme as a first language). They are fundamental to a proper understanding of symbolic computing and practical programming.

I know the arguments against subsuming something like P::S in the core
though, and they're all nonsense.

You can already do all of these things with the existing syntax. You can also store relational data in flat files instead of an RDBMS. That doesn't justify anything.

Not everyone will like this syntax. The core should come up with something everybody like. Nonsense; the core should provide one really good way to do something (and the "They're just hash references with special keys and some other bits glued on!" approach isn't it) without precluding alternate approaches. Refusing to do something without complete consensus is as much a recipe for failure as adding every feature proposed without considering its necessity, utility, implementation, and coherence in the gestalt.

What about all of the existing code that doesn't use it?
Irrelevant; it still works. That code's already written. The cost of writing
that code is a sunk cost; you'll never get it back, no matter how much you
penalize code under current maintenance and new code written from this point
forward.

There are plenty of dependencies of popular CPAN distributions which do not
represent failures of the Perl 5 core to provide useful features in a usable
way, but consider this: every time you have to install or load one of several
competing exception or function signature or list handling or library
management or build systems or FFI workalikes or object system modules in a large chain of dependencies (of which a transient bug in any costs you debugging time), you're
paying a tax the core developers have levied on all users in lieu of subsuming
useful and necessary language features into the core of Perl 5 itself.

Things have improved in the past ten years (and especially the last 18
months), but this is why non-Perl programmers look at Perl 5 and say "Modern?
That's what you call modern?" All of the rounded corners and hip slogans and fauxhawks and body spray in the world won't change that.

Of course, it's difficult to criticize p5p for its conservatism: no one has stepped forward with a coherent and singular vision for the continued and managed improvement of the design and implementation of Perl 5 as Perl 5—nor is it clear that p5p would accept anyone other than Larry doing so.

If you're an old hand at patching and building Perl 5, you already know what to do. That leaves about a million of the rest of us.

First, check out a recent bleadperl from Git. Within that directory, check out revision 8bdb331 (as it's the basis of the patch). You can use later revisions, but you may have to massage the patch yourself:

If you don't use perlbrew, you should.
With this configuration you can install bleadperl with this patch as if it were
a released Perl 5, at least as perlbrew sees it. Change the value of
-Dprefix to match your preferred installation directory. After a
few moments, you can regenerate the parser and other generated files, then
build, test, and install the new Perl yourself:

$ make regen regen_perly
$ make test
$ make install

If you have trouble building or installing bleadperl, read the included
INSTALL file. I cannot provide technical support here; the usual Perl
5 support channels apply (especially if you also have trouble building,
testing, or installing bleadperl without this patch).

Then you can use perlbrew to switch to the new Perl 5 (perlbrew use
perl-5.13-method or perlbrew switch perl-5.13-method) and
test it with your favorite code.

Thanks to IBM dW and Seda Özses for fixing the code to which I linked
in CGI
is Okay, but Bad Code is Irresponsible. To their credit, the security flaw
and the Perl 4 style code in the article have become much, much
better—now it's good example code, which is what everyone in this process
wants.

This sort of thing will happen again. Perhaps we can learn from this experience and handle it better next time.

What should you do the next time you see example code with a security flaw and/or clunky style?

Contact the author directly and privately.

Explain the security flaw and demonstrate how to fix it and it alone.

Expand further on stylistic issues (explaining that, although there is not complete agreement in the entire Perl 5 community over some stylistic issues, the dominant trends are fairly widely understood as something we can all call modern or enlightened Perl), revising the code as necessary.

Suggest that the Perl 5 community has many resources such as PerlMonks, Perl Mongers, and mailing lists full of people who can and will perform technical review on code.

Be polite.

... if that does not work, publish an explanation and corrected code elsewhere.

Where possible, stick with the constraints of the article. For example, Ms. Özses's code has good reasons to use CGI instead of a Plack-capable module. Respect that. (Though where it's obvious that a CPAN module is far superior solution, such as in the case of CSV parsing, do recommend the module.)

We share a goal, that useful and usable and correct and secure and sufficiently elegant code is available. We do not have to write the same code nor the best possible code to achieve that goal, but...

... people who publish code examples have a responsibility to write good code.

... people who review and comment on code have a responsibility to
be both accurate and polite.

To that end, I owe Ms. Özses an apology. However I intend neither
personal attack nor insult in my writings here, I failed to live up to my
personal standards of kindness and politeness in this case. She deserves much
credit for taking the suggestions of various Perl 5 programmers despite
unwarranted criticisms.