Was That Jump A 6? Subjectivity In Olympic Judging

And it's time to talk sports, specifically the Olympics; the skill, the precision, the pure athleticism. And yes, the style. And we're talking specifically about ice skating, because our own Mike Pesca has some thoughts on that sport.

Good morning, Mike.

MIKE PESCA, BYLINE: Hello. How are you?

MARTIN: Hello. I am well. So, I understand Olympic ice skating has undergone some reforms in recent years. Do tell? What changed?

PESCA: Yes. Well, after those scandals of vote trading, they decided that they had to up haul the system - some upheaval, overhaul...

(LAUGHTER)

MARTIN: Or overhaul. Yeah.

PESCA: Overhaul. There's the word I'm looking for.

MARTIN: Up haul, though.

PESCA: All right. Moving on.

MARTIN: OK. Yeah. Moving on.

PESCA: Yeah, they U-Hauled the system.

(LAUGHTER)

PESCA: And they instituted a new series of changes. And among those changes, the biggest changes were they imposed anonymity on the judges. And the idea was that if there was to be vote trading - I'll vote for your country if you vote for mine - if it was anonymous what judges gave what marks, then no one could be sure if anyone followed through on vote trading schemes.

I guess it was an idea that has some logic to it, but how it's worked out has not been what they wanted or what the Ice Skating Federation have said they wanted. And I talked with Dartmouth's Eric Itzkowitz, who is a great economist, who wrote in the Journal of Sports Economics. And he said studied all the numbers from before the changes and after the changes and he found because of anonymity, there's now worse vote trading and that judging bias is 20 percent higher than it was under the old system.

MARTIN: So this is not a good thing. And this is something you say is permeating some Olympic sports - this idea of subjectivity and it gets a little mushy, right?

PESCA: Well, OK. So yes, so many of the Olympic sports are I know, higher, faster, stronger is the ideal, but so many of them are based on style and based on how cool the trick was, not just on how far you went or how fast you went or how many goals were scored.

MARTIN: Mm-hmm.

PESCA: So, of course, subjectivity has to be a part of it. But let's try to figure out the best way to the subjective. And what Itzkowitz points out is that it seems like the Ice Skating Federation maybe had an idea. You know, based on let's look at parliaments, right? Sometimes there's a thought that if you do a roll call vote everyone is accountable. But there's another thought that if you do a voice vote then lobbyists won't know who to target.

So maybe, you know, that's good enough and in different systems anonymity might work. But it's pretty clear to him that in ice skating transparency works. And, in fact, the United States discloses all its judges' scores and they're found to be less biased than all of these other countries.

I asked him, well, was this just a dumb mistake or was this, do think, you know, disingenuous or purposeful? And he says that could go either way but there was another aspect of what they've done with overhauling the system. There are 12 judges and they throw out randomly three of the votes. There used to be a throw out the highest or the lowest.

Throwing out randomly three of the votes makes no sense. Like, no economist or statistician would say that worked. And he went back and found all these examples where if they counted all the votes you'd have different winners, and there is no evidence that there shouldn't have been different winners.

MARTIN: But you're not saying that subjectivity shouldn't be part of the judging. I mean...

PESCA: It has to be. You just try to find the system that gets the subjectivity right.

MARTIN: We have to point out the guy Sage Kotsenburg - I mean he won the first Gold Medal of the Games. And his own coach says to be that stylish is actually to be technical.

PESCA: Well, of course, yes. I understand what that means. And in the sport of snowboarding the ethic is, look, we can say, you know, if you rotate at 1080 you get this many points - you know, in skating it's a little more precise. But snowboarding is, like, hey, that guy just looks the coolest.

(LAUGHTER)

PESCA: And so they let the judges vote that guy a gold. And most of the snowboarders say that's a good thing.