Optimizing abc2ly.pl

Shortly after I wrote my last blog post, I started actively working on the ABC module again. First I had to enhance a few features so it could handle some whistle parts for a little musical I was asked to play for. Next I needed some enhancements to handle whistle / fiddle parts for the standard wedding music pieces for an upcoming gig. Finally getting obsessed with ABC updates, I decided to throw the code at a collection of 159 tunes I put together a few years ago.

Suddenly I was running into hard limitations of the ABC module. It was always slow, but taking 40 seconds to process my file of three tunes wasn’t really a problem for me. I did add some progress notification messages, and was shocked to learn it wasn’t the parsing of the ABC files that was slow, it was processing the parsed result. But when I got things working well enough it was actually trying to process all 159 tunes, it would run for 15 or 20 minutes and then crash. (This is under Niecza, BTW.)

I spent Friday and Saturday trying to optimize bits of code in there that I knew were terrible, with no noticeable improvement. Finally I did a profile run under mono. I found the results didn’t really mean anything to me, but I sent it to sorear++ to see what he thought. And he said, “Why do you need to use .eval in abc2ly.pl?”

Well, I quickly checked, and found all the uses were intended to convert strings like "3/4" to Rats. You may say “Why were you doing that? Shouldn’t .Numeric work just as well?” And the answer is yes, it does. (In my defense, I’m pretty sure that didn’t work when I started working on the ABC module.)

But now here is the tricky bit. Here’s a simple script to demonstrate the issue:

my $a = 0;
for 1..400 {
$a += "1/$_";
}
say $a;

On my machine, that takes 1.3 seconds to run under Niecza. Now let’s make a simple change:

my $a = 0;
for 1..400 {
$a += "1/$_".eval;
}
say $a;

This version, with just the .eval added, takes 39.0 seconds to run. Huge difference, eh?

It turned out that even though the evals were happening per measure of music and not per note, they were so slow they were completely swamping the performance of my script. With the evals, processing the wedding.abc file took about 28 seconds. Without them, 3.7 seconds. And remember the big ABC file that would run for 15 minutes and then croak? Without the evals, the entire file processes in 31.8 seconds. That’s easily 30 times faster.

Lesson? Don’t use eval unless you really, really need to.

This advice holds for Rakudo as well: the above fraction-summing scripts took .3 and 8.5 seconds to run, respectively. (Yes, Rakudo is doing a lot better performance-wise on those scripts than Niecza did!)