X Factor 2010 Review: Reading the Runes of the Running Order

Posted by Andrew

The running order clearly has a big effect on X Factor voting, but even with the voting statistics to hand, quantifying its effect isn’t a straightforward matter.

We can say, for example, that in the first seven weeks of 2010’s competition, acts performing in the pimp slot (last of all in the running order) outperformed their series average vote by 67%. But such raw statistics can’t take us very far: the pimp slots in week 8 (Cher) and 9 (One Direction) actually saw those acts underperform their series average. You have to view it in context – in this case, the context of two acts on a downward trajectory needing a strategic boost.

Similarly, the acts performing first in the first two weeks – FYD and Storm Lee – were eliminated. That’s not surprising. The “death slot” is most potent in the early weeks, before acts have had much chance to build up a loyal base, and especially (in Storm’s case) when Strictly is still on the other side. Singing first can still work against you later on – Matt Cardle’s two weakest performances of the series, in terms of his vote proportion relative to the mean, came when he opened the shows in weeks 7 and 10.

But if you looked only at these raw stats, you would be confused by the fact that Wagner outperformed his series average vote by 38% when he opened the show (and was eliminated) in week 8. You have to know the context – that having done all they could to bolster Wagner’s vote in earlier weeks when his support was weak, producers were now faced with Wagner’s support trending upwards too quickly for comfort.

So we think the most meaningful way to judge the effect of the running order is act-by-act. Let’s start with Matt:

[Some notes on these graphs, first. The grey bar shows how far through the show the act performed. Pimp slots are shown in dark grey, and performing first is shown by no grey bar at all. The vote trendline tracks performance relative to the mean vote given the number of acts each week. In later weeks when acts sang two songs, we’ve gone with the running order from the second set of songs.]

Matt’s vote tracks his running order pretty closely, with the three peaks in his vote coinciding with his three latest slots in the order, including week 2’s pimp slot. It has to be an open question, though, whether he’d have done just as well in weeks 5 and 8 with an earlier slot, given that these were arguably his two best songs of the live shows – ‘First Time’ and ‘Nights in White Satin’.

The story here is one of steady decline only partially arrested by favours from the running order. Their week 5 pimp slot seemed poor at the time and actually saw them underperform both the previous and successive weeks (although Matt’s storming vote for ‘First Time’ will have depressed the others’ relative figures in week 5). Their second pimp slot, in week 9, bucked their downward trend, but it was too little too late.

What about Cher?

No surprises here. She peaked with her pimp slot rendition of ‘Stay’ in week 4 and crashed back to earth when put on first in week 5. Her vote was on a steady (but very slightly downward) trajectory since then, with a combination of sympathy bounce and pimp slot giving her a fillip in week 8, and singing fourth of four in the Saturday of the final helping her to a small uptick – though not enough to get her into the final Sunday.

There is, we should acknowledge, a cause-and-effect question here – that is, Cher will have been put in the pimp slot in week 4 in part because producers knew she was singing a strong song. But was her ‘Stay’ really that much better than her week 5 show-opening ‘Empire State of Mind’? We didn’t think so. It must be the running order that largely explains that dramatic difference in vote, allied to the superior judges’ comments for ‘Stay’ (they don’t call it the pimp slot for nothing).

Mary Byrne’s chart doesn’t make for pretty viewing:

After a storming start she was already falling off a cliff before week 4 began a series of early slots in the running order for her. Perhaps a late slot for her might have arrested the decline – producers made sure that we never found out.

In sixth place was Wagner:

We’ve discussed his voting trajectory plenty in other articles. Suffice to say here that the big block of grey shows how producers wanted to keep him bouyant through the first seven weeks, and that putting him on first as part of the week 8 assassination successfully chopped off a trendline that was then growing too quickly for producers’ tastes.

Another whose trajectory we’re familiar with is Katie:

Her first pimp slot prevented a sympathy bounce comedown, but a decent slot in week 4 couldn’t save her from the singoff. Poor slots in weeks 5 and 6 dampened her chances of a second bounce, while her second pimp slot in week 7 accentuated that second bounce when it came.

Eighth-placed Paije has an interesting chart:

Twice in his seven weeks, Paije was sent out to open the show (‘Ain’t Nobody’ in week 3 and ‘Crocodile Rock’ in week 7), and in both cases he did significantly better than the previous week. This could be just noise, although it perhaps reflects a degree of sympathy from his fans – in week 3 it was his second poor slot in a row, while in week 6 his vote may have been boosted by his fans reacting to a widely-publicised alleged voting leak that had him running bottom.

That week 6 spike was bad news for Aiden:

What we see here is probably a vicious circle, with progressively poorer performances after his, er, memorable ‘Mad World’ in week 1 being reflected in both lower votes and worse slots in the running order as producers gave up on him.

Also peaking in week 1 was Treyc:

Although it’s a pretty poor show that she managed only 77% of the mean vote from the pimp slot in the opening show. That may explain why producers didn’t show too much interest in her thereafter. She got a surprisingly great slot in the running order in week 5, but coming down from a sympathy bounce it wasn’t enough to see her clear of the elimination zone. She just wasn’t that popular, it seems.

The last act worth a look is Belle Amie:

Although here it’s hard to tease out the effects of the running order from the sympathy bounce. Following their week 2 singoff survival, their decent slot in week 3 worked with the grain of their bounce and their poor slot in week 4 worked with the grain of their comedown.

Nothing much of interest can be learned from the other four. Poor old John Adeleye, week 3’s departure, never had a good slot in the running order so there’s nothing to compare. Storm Lee’s opening of the show lopped a third off his vote compared to singing fifth in week one, while Diva Fever had slots midway through on both occasions.

A final word about Nicolo Festa, eliminated first in the first show. On the face of it, he had a decent-looking slot in 12th – but was sandwiched between two high-performers of that week in the vote-topping Mary Byrne and fourth-placed One Direction, and he was probably forgotten quite quickly in consequence. It goes to show why the running order is more of an art than a science: other things being equal, it’s better to sing later – but other things are rarely equal.