Judging judges: New method of determining consistency

Matt Podgorski, a boxing judge for 14 years in Florida and Illinois, is a numbers guy. He makes a living doing statistical analyses for a food service company outside Chicago.

Thus, it’s no surprise that he has long contemplated whether there’s a way to quantify the performance of boxing and MMA judges, who are both praised and criticized with limited statistical data to back up those opinions.

“I started officiating as a professional boxing judge in 1998,” said Podgorski, who was only 18 when he judged his first fight. “It has always bothered me how subjective judging is, and more specifically, how subjective the perception of judges has always been.

“Boxing and MMA judges are often evaluated based on whether or not they have had a controversial decision. This is a poor way to assign and regard professional judges.”

With that in mind, Podgorski came up with a method to evaluate the performance of judges worldwide by determining the percentage of instances his or her scores are consistent with the other two judges working the same fights. He calls it the Pod Index.

Podgorski keeps round-by-round data on judges in all world title fights since 2009. RingTV.com will post the results of his analysis each week, beginning today. (See the accompanying charts.)

“The overall goal of the Pod Index is to bring an objective metric to a subjective activity,” he said. “The Pod Index will allow anyone to determine how consistent a particular judge’s scoring is to their counterparts.

“The tool that produces the Pod Index metric also is built to identify trends in judges’ scoring. The tool can identify what country’s matches produce the most consistent scoring. It can tell you which weight classes have the most consistent scoring. It can even identify a fatigue factor, whereby judging in the later rounds tends to produce less consistent scoring.”

Podgorski acknowledged that no one can determine whether any judge has scored a round correctly; that’s purely subjective.

Thus, he was careful to point out that judges with low scores in his Pod Index aren’t necessarily bad judges.

“A low score doesn’t imply that a judge is ‘bad,’” he said. “It simply means that the judge doesn’t score rounds the same as his or her counterparts. That may be due to style preferences more than quality of judging. But it’s ultimately up to the fans to decide what the Pod Index says about boxing and MMA judges.

“I also want to point out that the Pod Index only works with large sample sizes. One cannot take the Pod Index for a judge for one fight and say he or she is inconsistent. A minimum of 60 rounds is required before evaluating a judge’s performance.”

Podgorski said that his evaluation tool is comparable in some ways to CompuBox statistics, which tracks punches in major fights.

The CompuBox numbers are by no means officials; they’re simply another layer of information with which to evaluate what boxing fans and those who work in the sport see. The same applies to Podgorski’s Pod Index.

Podgorski also believes the Pod Index will have a practical purpose: Sanctioning bodies already have expressed some interest in using his data to improve judging consistency.

“With this judging evaluation system and the Pod Index metric that it produces, I hope to give something back to two sports that I truly love as both a fan and professional official,” he said. “I hope that boxing and MMA fans will enjoy it and find it useful information.”

RANKING OF JUDGES BY PERCENTAGE OF INSTANCES THEIR ROUND-BY-ROUND SCORES ARE THE SAME AS THEIR COUNTERPARTS AT RINGSIDE IN WORLD TITLE FIGHTS