Madlib-style forms increase conversion by 30%? Well, maybe …

A while back, while writing about Google’s Social Graph API, I mentioned Jeremy Keith’s audio-sharing web app, Huffduffer, and its unconventional signup form. If you haven’t seen it, here’s what it looks like:

The link is to a fantastic piece by Cennyd Bowles on the fact that basic statistics are so often overlooked when conducting A/B tests. A quick excerpt from that article (though I strongly recommend you read the whole thing):

Last week I tossed a coin a hundred times. 49 heads. Then I changed into a red t-shirt and tossed the same coin another hundred times. 51 heads. From this, I conclude that wearing a red shirt gives a 4.1% increase in conversion in throwing heads.

This example is only partly in jest: many A/B tests make similar conclusions from similar sample sizes. What I don’t see, however, is what Keith sees as this being “accounted for”: Luke’s post makes no mention of sample size, or the duration over which the test was run, or whether the number stated is an absolute percentage or a relative one.

So, for the moment, what can we take from this? It’s not certain that this kind of form will work wonders for every kind of site, and it’s also not certain whether or not its performance might be due solely to its novelty (in which case the advantage would dwindle as more sites began employing similar forms). On the other hand, looking at the side-by-side screenshots in Luke’s post, the new-style form does feel more personable and appealing, less cold and machine-like, so maybe there’s something to it.

What do you think? Will we start seeing an abundance of story-style forms on the Web? Outside of signup forms, what else can you see them being used for?