Speaking of "flip-flops" under ordinal judging, I really cannot see what the fuss is about or why this is considered an undesirable feature of a judging system -- so much so that we had to change the judging system, twice, because of it.

This is how flip-flops occur.

After five skaters, with one to go, here is the provisional ranking.

A
B
C
D
E

Now skater F performs. Although he did not skate well enough to escape last place, he did skate well enough to steal a couple of ordinals from D.

So it changed places that weren't adjacent to F's placement and that were medal positions, and also along with the factored placement changes it changed the overall results from something like

B
C
A
D
E
F

to

A
D
B
C
E
F

B went from gold medal position before F skated to bronze in the final results, without F himself getting anywhere near medal position (from most judges -- but it was the one or two outliers who made the difference).

That kind of thing didn't happen very often, but when it did, it got people upset.

The more benign kind of place switching that you mention happened all the time but no one really cared except the skaters who may have left the arena after they skated and came back later to find themselves in an unexpected final position.

After the Compulsory and Original Dances, both Usova/Zhulin (won both CDs) and Torvill/Dean (won the OD) were in control of their own destiny; Grishuk/Platov were behind on factored placements so they would have needed help to win.

After Usova/Zhulin and Torvill/Dean skated their Free Dances, U/Zh were ahead in the free dance as well as overall. Since the last team G/P couldn't catch them just by beating them in the FD, they seemed assured of gold.

Well, the ordinals got mixed up so that U/Zh and T/D flipped places in the free dance, and that mixed up the overall standings as well.

Before G/P skated, the FD standings were
U
T
and so were the overall standings.

After G/P skated, the FD standings were
G
T
U

and the overall standings were
T
G
U

You can see the source of Zhulin's consternation.

The Britons, who skated to "Face the Music and Dance" by Irving Berlin, entered the evening with only the slimmest advantage over the defending world champions, Maya Usova and Aleksandr Zhulin of Russia. Before the evening of free dancing ended, both couples would be upstaged by the second Russian pair of Oksana Gritschuk and Yevgeny Platov, whose exuberant performance earned them the highest scores of the competition, including three 6.0's for artistic impression.
Ads by Google

"We thought we were the winners when we saw those 6's," said Platov, who skated after the other two couples had finished. Mathematics Competition

Instead, those 6's ended up benefiting Torvill and Dean by knocking Usova and Zhulin out of first place in the free dance. By the time the mathematical smoke had cleared, Torvill and Dean were declared the overall winners because six of the nine judges placed them in second while five put Usova and Zhulin in first or second place.

"It was too algebraic," said Zhulin, who ended up settling for the bronze medal with Usova, behind Gritschuk and Platov.

I’ve mentioned before that when my daughter was skating (under 6.0), there was a competition where all of the judges placed her 3rd, but her final placement was 4th. It’s very difficult to explain to a 9 year old who thinks she’s won a medal, that no, in fact you did not win a medal. And while I understand how the ordinal grid works, it still seems wrong to me that she didn’t get a medal. If I feel that way, imagine the reaction of parents who DON'T understand how the grid works.

These kinds of flip flops happened far more frequently that Mathman would have us believe and it made the results appear to be off every time it happened. While people on discussion boards may only be interested in elite skating, there is much more to the figure skating world that the handful of skaters we see on TV and nobody liked flip flops.

If parents think their kids are getting screwed over by the scoring, they won’t waste further money on the sport.

To me, the only reason Zhulin is upset is because of how the sequence unfolded in time, not because of how the voting system works.

Suppose that all three teams skated the same, that all of the judges voted the same way, but G&P had skated first instead of last. Then all is well

G
T

G
T
U

Factor in the OD:

T
G
U

The "flip-flop" has magically disappeared. That's because it wasn't there is the first place, partial rankings in the middle of the event being irrelevant if not misleading.

Here is 2002 Olympics.

After Sarah and Michelle had skated, the overall rankings were

M
S

But Michelle knew she was out of luck. Sure enough, Slutskaya inserted herself and it ended up.

S
I
M

Michelle has no reason to be puzzled or outraged. This is exactly the same as if the skate order had been I, M, S.

I
M

S
I
M

Or if the order had been S, I, M.

S
I

S
I
M

Moral of the story -- don't count your chickens until they hatch.

Originally Posted by Dragonlady

I’ve mentioned before that when my daughter was skating (under 6.0), there was a competition where all of the judges placed her 3rd, but her final placement was 4th. It’s very difficult to explain to a 9 year old who thinks she’s won a medal, that no, in fact you did not win a medal.

I can see where it would be difficult to explain to a nine-year-old that you have to look at all the ordinals for all the skaters, not just your own.

I think most adults can understand how a skater can get a couple of firsts, a couple of seconds and a couple of fourths, and end up ahead of a skater with all thirds.

To me, the only reason Zhulin is upset is because of how the sequence unfolded in time, not because of how the voting system works....
The "flip-flop" has magically disappeared. That's because it wasn't there is the first place, partial rankings in the middle of the event being irrelevant if not misleading.

Yes, but the partial rankings in the middle of the event are announced, which means that the way the event is presented to the public and to the participants is misleading by design (not by intention, but by the system itself).

Here is 2002 Olympics.

That's purely a factored placements flipflop, which were more common and more predictable. Kwan would know once she lost the LP to Hughes that she would lose the gold if someone else beat her in the LP.

The situation would have been comparable if Kwan had been ahead in the freeskate as well until Slutskaya skated.

But yes, since factored placement flipflops were very common, the skaters surely learned to expect them even if the less experienced fans did not.

Ant, my man! Is this your example of something that is hard to understand? Let's go over it again. 3+1 = 4, ...

No it was more the working out of the majority ordinals etc detailed in the Sandra Loosemore link that gkelly provided. That and the icecalc matrices that were generated for orindal judging that i never knew how to read or understand. Nowadays, the protocls and totals make it easier to understand! Really the only point I was making was when people say that COP is soooooooo difficult to understand, 6.0 was waaaaaaaaaaaaaay easier. I don't think it was.

Re the copyright query, i'm not sure what happens for shows but for competitions I believe the skaters have to submit the names and cuts of the music along with their application form, and I always assumed it was to clear it from a public playing point of view for copyright. Way back when i did the adult nationals I even had to write down the code from the CD....i wonder howyou do that nowadays with downloads?

No it was more the working out of the majority ordinals etc detailed in the Sandra Loosemore link that gkelly provided. That and the icecalc matrices that were generated for orindal judging that i never knew how to read or understand...

But it is very interesting!

One salient feature of majority of ordinals is that it allows a compromise candidate to win who is no one's first choice but at least none of the voters despise him.

Let's say you are trying to hold your fragile democracy together and it is being pulled apart by warring factions A and C. Here are the preferences of four voters (the first column is the preferential ordering of the first voter, etc.)

A.....A.....C......C
B.....B.....B......B
C.....C.....A......A

Neither A nor C has a majority of first-place votes. But B has a majority of first and second place votes. So the compromise candidate, B, wins by the majority of ordinals method, thus saving the nation from civil war.

The problem with OBO is that it invites rock, paper, scissors problems. I put out rock, you put out paper, gkelly puts out scissors. Who wins? (This is less likely to cause difficulties if there are more judges than skaters, however.)

Speaking of nine-year-old kids, I wish I had been a figure skater when I was nine. I would have had a field day with all this. I could have explained in great detail to all my friends exactly why they lost. Imagine how popular I would have been!

In my daughter's case, there were 3 judges, all of whom placed Tink 3 giving her a total of 9 points. Another girl had two 4th place ordinals and one 1st placed ordinal. Statistically they were tied at 9 but the other girl had a higher ordinal than Tink, giving her the medal. I understood this even if my daughter didn't. She looked at the row of 3's beside her name and wanted to go get her medal.

There is a saying "Not only should the result be fair, but it must be SEEN as fair" and the problem with flip flops and they don't seem to be fair. In my daughter's case, a majority of the judges said my daughter beat this girl. One judge saw it differently, so it wasn't even a majority opinion to give it to the other girl. That's the part that I have difficulty with.

I guess for an event for children with only three judges, they figured this was an easy way to do it. (This is a called a "Borda count" method, by the way.) But under either of the two IJS 6.0 scoring methods, majority of ordinals or OBO, your daughter would have won (depending on other skaters, of course). She had the "majority of 1st, 2nd, and 3rd place ordinals" over the other girl (majority of ordinals) and also head to head (OBO).

But you can't blame the 6.0 systems for that. In fact, adding up the ordinals is more like CoP scoring (add up the points) than it is like true ranking systems. Under CoP two judges might have given your daughter (in PCSs, let's say), 30 points and one judge who didn't like her 10 points. All judges gave the other skater 25 points. Same thing. Two of the three judges thought your daughter was better, but in an add-up-the-points system like the CoP, the other girl wins.

Edited to add Let me put this more emphatically. There is no purely ordinal voting system in which your daughter could have lost to the other girl, holding two ordinals out of three.

In an add up the points (or -- yikes! -- add up the ordinals) system, yes, weird things like that are not rare at all.

Speaking of "flip-flops" under ordinal judging, I really cannot see what the fuss is about or why this is considered an undesirable feature of a judging system -- so much so that we had to change the judging system, twice, because of it.

The majority criterion, which ordinal voting systems aim to satisfy, is NOT essential in sport. Medal winners shall be determined, not by the consensus of a "society", but by their actual performances. Sport judges are expected to give absolute measures in terms of, for instance, the height of a bar cleared and the number of goals scored, rather than the relative rank distance between competitors. If Jumper A clears 2.02m and Jumper B fails it, then their relative placement in that high jump competition should remain constant regardless of the performances of others. If Dai was truly better than Hanyu in the Short Program at the last Japan Nationals, it should hold true even if the judges re-watch the footage later after they have seen an indefinite number of performances during the time interval. In other words, it should be the absolute values (point) not the relative values (ordinal) that are being compared in the decision of the rank order of performances, and so indispensable are such criteria in sport ranking as independence of irrelevant alternatives, independence of clones, and the consistency criterion that any violation will be deemed suspicious and scandalous by the mass of the sport fans. As a consequence, the 6.0 One-by-One system (a Condorcet method) or any voting system that fails to escape Arrow's impossibility theorem is a dead-end approach. Cardinal voting systems (e.g., range voting) and interval-scale measurements are the only options that seem fair, logical and relevant in sport.

Can you name any sport competition (other than the old 6.0 in figure skating) that uses an ordinal voting system in judging?

I think we should abandon the application of traditional social choice theory to sport judging. Sport is not an election. It is more about performance evaluation than about preferential voting.

Sorry. Let me paraphrase my post in plain language: Flip-flopping is bad in sport because most fans will not understand how it can happen and thus will suspect something wrong with the judging system. If to earn credibility from the fans is one of the goals of a sport judging system, any ordinal system that permits flip-flopping will not work. Even if someone is familiar with the mathematical aspect of it and understands how it can happen, it does not mean that it should happen or it is a logical way of judging. If A performs better than B, then A performs better than B. It should have nothing to do with how well other competitors perform. When this principle is violated, the spirit or the perceived fairness of the sport is also violated. The old 6.0 system (an ordinal voting system) was a rarity in sport for it went against the conventional logic of sport. Its demise was for a good reason.

It is not possible to determine whether this is true or not. (Note the undefined term "better.") Different observers have different opinions, and there is no external standard, like a tape measure or stop-watch, that can settle the matter independent of human observation and interpretation.

If A performs better than B, then A performs better than B.

It is not possible to determine this with certainty or objectivity. The best we can hope for is some kind of consensus among expert observers. Lacking unanimity, there are various schemes, none perfect, to resolve the conflict.

Figure skating is not like other sports, nor should it wish to be. A well-crafted figure skating performance makes my heart sing. To say that it sings 168.31 decibels loud reduces figure skating to -- well, to a mere sport. (YMMD )

It is not possible to determine whether this is true or not. (Note the undefined term "better.") Different observers have different opinions, and there is no external standard, like a tape measure or stop-watch, that can settle the matter independent of human observation and interpretation.

It is possible to determine which skater is "better" as long as a standard is established by a well-sampled interest groups.
Though subjectivity is involved, the system (FINA rules) used for judging an Olympic diving contest is "transparent and easy to understand" according to Michel Balinski and Rida Laraki's Majority Judgment: Measuring, Ranking and Electing. "Clear-cut, absolute meanings are ascribed to each of the numbers...Scores determine rankings, so there can be no flip-flops...The simplicity, in contrast with the rules for skaters and gymnasts, is striking." (http://books.google.ca/books?id=sSXxXBxuorIC&pg p. 147-148).
Indeed, differences between judge ratings are inevitable. It however can be estimated, accounted for, and hence controlled in a measurement model.

Originally Posted by Mathman

Figure skating is not like other sports, nor should it wish to be.

Let me put it this way: Do you think figure skating is more of a sport or more of a piano contest or more of a dance competition or more of a beauty pageant? Sport ==> Measurement Theory. Piano Contest ==> Measurement Theory. Dance competition ==> Measurement Theory. All three types of international competitions currently employ criterion-referenced measurements rather than preferential voting. Beauty pageants sometimes involve preferential voting. For instance, the top five finishers in Miss America are ranked by the judges and the winner is determined by the Borda count method. Electronic voting by the TV viewers is incorporated from time to time in Miss Universe competition.