PSA: There is no such thing as a “RESTful URL”

use.perl, now with even more brokenness :-(

Requesting http://use.perl.org/search.pl?op=journals;content_type=atom now comes back with a 500 error and an empty body… and I can’t find any other way to read use.perl’s full journal section with an aggregator. Or even with a browser – if you leave off the content_type parameter, you’ll see the page comes back empty.

Sudden involuntary cosmopolitan

[I wrote the following over the course of 2 days, about two weeks ago.]

For want of a nail the shoe was lost.For want of a shoe the horse was lost.For want of a horse the rider was lost.For want of a rider the battle was lost.For want of a battle the kingdom was lost.And all for the want of a horseshoe nail.

Two days ago [on Aug 6], I checked out from the hotel where I’d been staying for YAPC. Unfortunately, I tried to to pay with my credit card.

Well, it has all the trappings of a regular credit card (or almost…), but it’s actually a debit card. I have no interest in taking out loans, but I wanted to be able to order things online from outside the German web, and Germany-typical payment modalidities are unknown elsewhere (whereas credit cards are somewhat unusual in Germany, in turn). So when my bank sent me an offer for one of those cards, I thought it was just perfect. I have since used this card some two dozen times, and with a single exception it always worked, both online and off. As a matter of fact, I used it on the day of my Lisbon-bound air journey.

That’s how I came to have a have a credit card that’s not a credit card.

Now, since I was travelling to a foreign country, I figured that having the hotel money on a credit card would be the safest bet for paying. Unfortunately it appears that unbeknownst to me, sometimes you need a PIN to pay with this sort of card. I’d received the PIN in the mail when I got the card, but I’d never needed it, so I hadn’t bothered to memorise the PIN.

And of course, it came up at the hotel. We tried a bunch of things, including me phoning home and asking the folks if they could scare up the slip with the PIN. They found one, except that number didn’t work.

Ironically, the MasterCard for my regular German debit account would have worked just fine (of course I do know the PIN for that) – so if I’d simply done nothing, I wouldn’t have had any trouble. But of course I didn’t have enough money in that account: it was all on the credit card account.

Heck, if I’d carried it all as cash that would have worked brilliantly.

Now, I went to check out at 15:00 – 50 minutes before the check-in counter at the airport would close, in turn the usual 30 minutes before the flight, which was at 16:20. The airport is barely over 5 taxi minutes away from the hotel, so 50 minutes seemed to be plenty early. In the end, when I started to get antsy and mentioned my flight, the hotel manager let me go without paying then and there.

But you know what’s coming, or else the story wouldn’t be so freakishly long: the credit card hold-up took long enough that I missed the check-in by 3 minutes. I should have called the airline to tell them I’d be in late – they could have held the check-in open a bit longer if I’d done that, and none of this would have happened. But that didn’t cross my mind; besides, I’ve been late a minute or two late for check-in at my home airport, and they always managed to get me on the flight anyway, so I thought that as long as I was close enough, I’d make it.

So I arrive at the airport and the clerks at the check-in counter tell me I’m late, and that there is absolutely no way that they can re-open check-in for me. I need to run over to the Air Berlin ticket office. Now I’m seriously worried. The flight is taking off in 26 minutes. Maybe I can make some kind of luggage arrangement so I can get on the plane and have the luggage trail me later; or something. I run over to the office… to be greeted by a sign that says “be back soon”.

Two minutes pass. Three minutes. Five. Eight. Twelve. Twenty now, the flight is going in 10 minutes – and no one’s at the office! Finally, some 30 minutes later, a lady shows up. I tell her what’s going on and she tells me there’s nothing to be done. My only option is to re-book the flight for next day. It’ll be a bunch extra, of course. Just great. Still, what can I do? So I tell her to book it; she asks for my transaction number and sets to work.

Then I think to myself, wait a minute, I shouldn’t be so impulsive. Let’s see if another airline has a flight out sooner and/or cheaper. So I tell her to abort the procedure. She asks, is that really what I want? Am I sure? Well, so I say, yes I am sure. OK, she says, and she does it. Now this transaction number is void, she announces; if I decide to take tomorrow’s flight, I’ll have to book a completely new ticket. How much? Nearly twice as much. This is the point in our program where I barely hold myself back from cursing – couldn’t she have told me that 30 seconds earlier?!?!?

Because now I have a serious problem: the credit card does not have that much money on it. I do have the money, but it’s in a regular German debit account – and since it takes 2–4 days for withdrawals from such accounts to go through, that money is non-existent for the purposes of booking a flight on the spot.

What to do?!

I do what I was going to: look around for the prices at other airlines.

Among the many other things Lisbon airport appears to lack is a central ticket office.

That means I hit up half a dozen airline ticket offices, one by one, asking for flights to my airport. It’s 16:30 so I’m under time pressure: the offices all close soon, including the Air Berlin one, which closes at 17:30. I get the same picture everywhere: significantly more expensive than the ticket offering I already have, or no flights to my airport, or all seats booked solid for a week, or whatever.

And now?!

Ah! I paid a small cash advance at the hotel. If they’ll let me have it back, that money plus the credit card would cover the flight – with a 20€ allowance left over. I won’t have enough cash on me to pay the long-term parking lot ticket at the airport at home (which comes to 40€), but heck, even if I can’t get at my car right away, at least I’ll be back in my town. So I call and ask. The poor manager sighs and says OK.

The taxi to the hotel and back eats 7€ out of my allowance. Ugh. If I run out of money, I’ll be screwed… so far it looks like it’ll suffice, and I can only hope.

Triumphantly I return to the Air Berlin office 5 minutes before it closes. I smile at the lady and tell her the good news. She smiles back and says she can only take either all cash, or charge it all against a credit card, but not both. That’s even though the Air Berlin office at my home airport could and did book a flight in that exact way just days before.

Of course, without the PIN, the airport ATM won’t let me convert the credit card allowance into cash either.

At this point, I’m feeling pretty defeated.

At least I am left with some 90€ in cash, so I’m not completely stranded in this airport. Unlike the story of another time where I came very close – an altogether separate level of unpleasantness –, I have enough fuel to communicate and get around town a bit, which gives me the leeway I need to hatch a plan. And I should probably have enough left at the end that paying the parking lot fee back home shouldn’t be a problem either.

So I relax a lot. Of course I’m still worried, since I don’t have an actual solution for how I’ll book my flight, just the means to implement one. I assume I’ll hit the World.pm social meeting, where I will hopefully find people who will be able to help me in whatever way they will. I was previously disappointed that I was going to have to miss the meet since it was announced so late and my return flight was long booked at that point. “I guess that solves that”, I think to myself.

I don’t know where the meeting will be, though. So the plan will have to involve going online somehow, to check the conference wiki. Since there is no free wireless at the airport, this implies going back to the venue where I’ll be able to use the conference WLAN, which is still operative because there are tutorial sessions still running over the next few days. So the venue is my next destination.

With the drop of my stress levels, exhaustion kicks in, so I look for seats in a quiet spot in the airport. I want to sit for a while to unwind a little and collect myself before launching into the next attack.

Lisbon airport does not seem to have any quiet spots.

I wander aimlessly… and run into Roz, who is at the airport way early for her flight. The hotel wouldn’t have her any more. So I sit down and tell her the story so far. We chat for a while, then her check-in counter opens. I tag along as she goes to checks in her luggage.

Near the counter, we run into evdb, whom I didn’t know personally, and probably would have missed if I wasn’t in Roz’s company. We tell him the story again, and to my astonishment he immediately asks in what way he can help – lend me cash, book my flight, whatever. I’m speechless at first, and of course immensely relieved and grateful. Maybe things will be OK after all.

The first thing we try is that evdb asks the counter, then some ticket offices, for flights; he has a few tacks I wasn’t aware of. Unfortunately, covering more airlines does not change the results: no tickets, booked solid, etc.

Next angle of attack: having him book my flight online. However, try as we might we cannot go online at the airport: more than half a dozen wireless ISPs offer connectivity, but none of their payment procedures work.

At that point, the two of them are running out of time. As a last resort gesture, evdb withdraws a fistful of cash and hands it to me – we’ll be in touch by email for paying it back –, before they leave to board their flight. I thank him and we say goodbyes.

Now I am really good cash-wise. I expect to need very little of his money, but I’ll be able to absorb surprise expenses without breaking a sweat. So I calm down a little more at the same time as my worry also increases back due to the fact that my situation is not yet sorted out after all.

After a short rest, I re-embark on my initial plan to find the World.pm meet. The part that follows here is largely boring – in the really good sense of the word. I take a taxi to the conference venue, connect to the wireless there, check the wiki, take notes, hit the metro, make the trip to the destination as planned, and find the pub where the World.pm meeting is taking place, all without a hitch. This stretch of the story passes completely uneventfully – what a welcome change that is.

So I arrive at the meet-up, and of course conversation turns to my story immediately, despite my not wanting to turn it that way myself, at least not right away – I want to ask for help, not beg for it or demand it. As soon as I’ve explained the situation, Nobull offers to help me out with cash or a flight booking.

Now the problem is, how do we book the flight? There is no free wireless around, only several closed networks. José mentions that one of the nearby cafés has an open access point, so we wander over to mooch some access. Since the signal is weak, we wander a little closer, but soon, the network disappears entirely. Our guess is the bartender or someone else in the shop saw us milling about with a laptop and turned off the access point. Bummer. So we walk back to the Irish Pub, where there is a strong PT-WIFI signal, and eventually we find one of the Portuguese mongers has an account with them, though no laptop. Another local monger produces his laptop, and a connection is eventually established, so I can finally get online to book the flight, leaving the payment details for Nobull to enter – who uses his wife’s credit card.

That “booking confirmed” screen is a truly international effort.

And with all that, my journey back is finally secured and I relax.

After socialising for a few hours, I take a taxi to the airport to sit out the rest night and the following noon. It’s nearly 2am and I don’t care where I’ll stay anyway, just as long as I know I’m going to get out of this hell hole OK. I’ll deal just fine with any inconvenience in the meantime.

I only manage to get a few hours of fitful sleep of course, and those are while sat in a hard chair… but I end up less exhausted than I should nominally be. In the morning, I hit the pay phones and call home to tell some very worried folks the good news.

There are wall plugs where I’m spending the night, so I can mooch electricity off the airport to feed my laptop battery. I won’t have to worry about conserving battery charge – more or less the only bright spot in this entire ordeal.

After wiling away the hours, the time has finally come: the check-in counter opens. Inwardly jubilant, I stroll over and hand the clerk my passport.

… Who looks up at me and says there is a problem, and would I please head back to the ticket office to sort it out? I curse. I was hoping never to have to see that lady’s friendly smile and her maddeningly intact throat again.

Turns out it’s in fact two problems.

Nr. 1: originally, the ticket payment had been refused by my bank because it fell into an untimely moment. Since I booked my flight relatively late, I had called Air Berlin and asked them whether I could send the money by wire transfer, because if they tried to re-withdraw, it might get there too late. (As previously mentioned, German wire transfers take several days to complete, because the banks like to sit on that money so they can keep it warm.) The lady on the phone OKed this. But it appears that even though the ticket was then considered covered – after all, they let me take my inbound flight! –, somehow they subsequently lost track of the money. Don’t ask me how they managed that.

Nr. 2: there was a problem with Nobull’s credit card – so he’ll have to provide written and signed confirmation of the booking in some form (fax acceptable).

Without these problems being fixed, they’re not going to let me on the plane – but of course, they very well will ding me for the full price of the flight I booked and which I am not being allowed onto.

Unfortunately I don’t have Nobull’s cell number. He had put his home phone on file when we booked the flight, so I ask the Air Berlin clerk on the phone to try that one. No luck: no one is home, only a mailbox answers.

I have 15 minutes to sort out the situation before the check-in closes – if I don’t get it fixed in that time, I miss the flight.

Some options, those are. I know what it will come down to, and surely so do you as you’re reading this.

I head back to the pay phones and call my folks to tell them the – decidedly worse this time around – news. They mention that they’ve called the colleague and told him what happened, and that he offered to sort out any situation I might need help with. Which is something I was hoping to avoid – he’s bailed me out of more than enough situations already. But it’s no longer a matter of choice.

So I call him and give him the run-down. He tells me to call back in half an hour. I do. He tells me he has called Air Berlin and tried the entire battery of approaches in the book – all unsuccessfully. They simply will. Not. Budge. He tells me to call back again later.

The next time I call, he is at the airport back home, and speaking to the Air Berlin clerks there. He booked another flight with Air Berlin – unfortunately! They seem to be the only option for getting out of Lisbon right now. They’ve offered him to rebook today’s missed flight, which at least means it won’t be much extra. When the clerk asks him to pay the difference in cash, he has the presence of mind to ask whether that will really suffice – is the ticket actually covered? Good thing too, because she calls the headquarters in Berlin and finds out that no, the rest of the money was still pending. In other words, if he had relied on the legendary diligence and help of the Air Berlin staff, I would have been slated for another day stuck in Lisbon airport.

But by all appearances, at this point, things are finally sorted.

I sit at the airport a while longer and notice that I don’t manage to make much progress through my reading material: I keep dozing. So I throw fiscal caution to the wind and decide to see if I can find a room in a hotel. It would also be nice to have an opportunity to shower and brush my teeth. But after calling several hotels I know to be cheap, I find they are all booked solid. So for another night, sleeping in the airport like a bum it is.

This time, I’m not particularly relaxed. I can only imagine what twist of fate is going to await me tomorrow when I try to board my flight. The hours pass a lot more slowly this time. By noon, the sleep deprivation is catching up with me; I fall asleep whenever I sit down. I discover a restaurant hidden away on a third floor in the airport, where there are soft chairs – even couches! The tendons in my neck are eternally grateful.

I don’t sleep all that much or all that well anyway, of course.

Which is just as well, because it means this time I am at the check-in counter very early. With quite a bit of trepidation, I hand my passport to the clerk… who checks me in without any hesitation. I can barely believe it… I am going home! It’s all I can do not to do a few fist-pumps.

The rest of the journey is, finally and thankfully, completely uneventful.

Thanks, Air Berlin. I made my own mistakes to be sure, but it was your being deeply true to German customer service tradition that really granted me an opportunity to properly enjoy the hell out of this bloody airport – to the tune of half a thousand bucks up in air (quite literally). Double the trip price for double the hell – what more could one ask for?

Saturday August 15, 2009

04:00 PM

On Delight

I often see utterances along the lines that we in the Perl community need to recognise how bad we are at design and that we should get in touch with willing real designers who can do real design for us. I think that this precise utterance points to the reason why Perl applications don’t have good design.

Why would real designers even feel motivated to participate in a community that has no specific appreciation for what they are doing, nor, therefore, any ability to provide useful feedback? Anyone with highly specialised skills has this experience, and we as programmers do too: it’s frustrating to develop software for people who cannot appreciate the coolness of particular interesting bits, have no idea what feedback would be useful, and don’t know what they want, or what they need, or how to communicate it. “Real designers” would feel alien in the Perl community.

Paul Graham once wrote about American car companies that because the executives had no taste themselves, the cars produced by their companies ended up ugly. It’s not that the executives designed the cars, but they didn’t know how to tell who to hire to make tastefully designed cars nor did they know to appreciate the work any designer might have done, or how to build the culture and structure to foster design and designers. Microsoft has the same problem.

Perl does not have a culture of appreciating design. This “we need real designers so they can put a good coat of painting on our crud” mentality is perfectly symptomatic of that.

I appear to be one of the few Perl programmers who have any direct appreciation of (if not particularly great skill at) design; it sure feels lonely out here sometimes. In contrast, both PHP and Rails are infested with these people. I have seen reams of postings about typography out of the Ruby people I follow, f.ex., and not a single thing about it from a Perl person, if memory serves. Even the Python community seems to have some awareness, even if less than the others.

That means whatever full-fledged web apps these people write, they’re going to look at least passable, out of the box; even if the developers themselves are not great designers, they’ll know to build their apps from things that give them good defaults, and they’ll be able to recognise better designs or truly good detail tweaks when they come along (and good design is all about the details).

In his essay about Why software sucks, Scott Berkun includes a translation table for what people mean when they say something sucks, and the entry of interest there is “It’s so ugly I want to vomit just so I have something prettier to look at”. I think a huge reason for the marketing problems that Perl is having is the simple fact that a lot of Perl stuff looks ugly.

The few widely-known Perl things that don’t, don’t advertise Perl as a primary ingredient of theirs. The only exception I can readily think of are the SixApart applications, but I bet that even then, only MovableType is widely associated with Perl – and MT’s sun has long set.

Perl is associated with ugliness. I seriously think that this sets an expectation that is part of why the ridiculous “executable line noise LOL” meme persists across the Reddits and the Twitters of the world. There is a huge number of people who write stuff like that even though they have never actually written any Perl code, or even seen any – except maybe for the various iconic JAPHs, quines, golfs and monstrous regular expressions. And their direct experience of Perl apps and sites is that they tend to look nauseatingly ugly. Is it any wonder that they’ll readily pick up and propagate the “random punctuation LOL” meme? Bad design is definitely an inherent aspect of Perl’s marketing problem.

Frankly, I feel pessimistic about the prospects of the Perl community in this respect. You can’t suddenly imbue a culture with a sense for design when it has never known the first thing about design nor had any appreciation for it.

I think I heard something like this from Nick

Quite frankly, we’re dying off because the companies who use our software do not give back. The irony of the situation is that, in order to improve my motivation to do open source, I have to charge for it.

Polite fiction and the elephant in the room

Who’s trying to be realistic by attempting to release software with some quality expectations, notably by making the upgrade process seamless and introducing as little bugs as possible?

The problem with that argument is that 5.10.0 did contain regressions.

That the rarity of releases is justified by the high quality standards being adhered to is, unfortunately, merely polite fiction. What it all boils down to is simple:

Shit happens.

No matter how much we’d like for it to be otherwise, shit happens. The release process needs to be reality-based: it needs to deal with the very real fact that shit happens and has in fact already happened. The only realistic way to deal with that is for releases to be easy enough that the shit that does invariably happen can be dealt with soon enough, so that it won’t cause too many people too much trouble.

And the best way to make releases easy is to make them frequently.

And the simplest (though not only) way to ensure that is to put them on a schedule.

Whenever I hear someone saying we should have regular releases, I hear we should release when Venus enters Pisces.

There is your misunderstanding, then: what they are really saying is we need a reliable plan for dealing with accidents. If you can make that happen without sticking to scheduled releases, that’s fine too.

Yes: releasing often does not affect the quality of software by itself. But releasing it rarely almost certainly will – negatively. Because shit happens.

I don’t like it either! But that’s reality for you. I doesn’t care what you like. It just is.

Wednesday June 17, 2009

08:25 PM

Anyone want to adopt Getopt::Auto?

I took over this module from Simon Cozens nearly 5 years ago when he gave away his modules, but never got around to any of my ideas for it. What was I thinking? In hindsight, I don’t get around to my own ideas for comparable lengths of time; I should never have applied for it.

I did start a refactor at some point that’s still sitting in my projects directory, but in all honesty I didn’t get very far before losing steam. I even delayed writing this note, thinking I’d get around to making my changes available, but they’re insubstantial and unfinished and it really doesn’t matter any more. There’s a patch in the RT queue that’s been sitting there for years. I didn’t think the patch too great, but it would have been better to apply it and release than leave it in limbo waiting for a reformulation that I’d never get around to.

It is time to admit defeat. I’ve been a bad steward.

Is there anyone reading this who cares about the module? Want to take it over?

(Rationale: setting up complex pipes manually with IO::Pipe/socketpair + fork results in brain-melting code. Shell shows, however, that pipes can easily be described declaratively. Why is there, 20 years after the inception of Perl, no module that provides a piping API congruent with shell syntax?)

Saturday January 31, 2009

10:47 AM

A better quoting-aware word-splitting regex

Anyone who has read Jeff Friedl’s book can write a decent basic word splitter regex off the cuff:

m{ \G [ ]* ( " [^"\\]* (?: \\. [^"\\]* )* " | [^ ]+) }gsx

I have written and used this many times. What has always bugged me about it, however, is that it captures the delimiters along with the content, so afterwards you have to do something like this to the captured value:

s!\A"(.*)"\z!$1!;

This is… not pretty.

Of course, you could use two captures:

m{ \G [ ]* (?: " ( [^"\\]* (?: \\. [^"\\]* )* ) " | ( [^ ]+ )) }gsx

But then you need to check which of the two captures has the value – is it in $1 or $2? So this is still inelegant. The pattern has already done all the work of examining the string – why can’t it provide its results in an invariant form?

The problem is that the presence of the trailing quote must be dependent on the presence of a leading quote, so you must keep the quotes inside the alternation, so there is no way to avoid having either two distinct captures that exclude the quotes or a single broad capture that includes them.

Except, of course, you don’t have to and there is. True enough: when you rely on the matcher to pick an alternation implicitly, the quotes must be included in the alternation. But by using an extended regular expression feature (that has been marked experimental for a decade – what’s up with that?), namely conditional matches, you can make the match of the trailing quote conditional on the leading quote independently of an alternation.

And of course you can (and must, in this case) use a conditional match in the middle to explicitly specify which of the cases to pick, depending on the presence of the leading quote.

This way, you can match surrounding delimiters in a captured alternation for some of its cases but not others, without having to include the delimiters in the capture.

Note that the interesting capture is now $2, not $1 – we need the first capture for the quote, since conditional matches can only use captured groups as conditionals. However, the matched word is always in $2, regardless of whether quotes were involved. Furthermore, $1 has now turned into a true boolean flag, whereas previously this information had to be inferred (however easily).