View Single Post

These articles are a series mmaweekly did that argues who's system if judging is better. you should really check out.

Part 1

Quote:

DECISIONS, DECISIONS –
Part I: Nobody’s Perfect

Like all sports, MMA is a topic that lends itself well to debates. Which fighter is the best? What was the biggest upset of all time? Would Art Jimmerson have done any better without that one glove? But no matter the topic, most debates come back to the one big question: UFC vs. Pride.

There are many aspects to this question, from fighters to foot stomps to Fedor. However, there is one difference that has the most direct effect on a fighter’s record: the method by which judges award one fighter a win and the other a loss. And unlike most debates, the question of which organization has a better system of judgment might actually have a scientific answer.

What follows is part one of an examination of MMA judging criteria, the rules governing Pride decisions for events held in Japan and the Unified Rules of Mixed Martial Arts, first instituted by the New Jersey State Athletic Commission and most prominently associated with the UFC.

At its essence, the difference between the two systems is that of qualitative versus quantitative judgment – of Pride’s demand that judges decide who has fought to greater effect and the UFC’s attempt at requiring judges to assign points to fighters and therefore make a quantitative judgment of the fight. Admittedly, neither system is perfect, each with its own advantages and flaws. By subjecting the two to critical examination, we can see which system’s flaws are a greater hindrance to providing consistent, accurate, and objectively correct decisions.

Pride’s criteria are simple: Judges are to consider the fight in its entirety and decide the winner based on the following aspects, in order of priority:

1. Effort made to finish the fight via KO or submission
2. Damage done to the opponent
3. Standing combinations and ground control
4. Takedowns and takedown defense
5. Aggressiveness
6. Weight (in the case that the weight difference is 10 kg/22 lb or more)

All three judges must name a winner, so that, except in matches conducted under special rules, there can only be unanimous and split decisions, but no draws.

The problem with Pride’s judging rules is that of subjectivity. Even though Pride’s rules give a clear ranking of what aspects are more important than others, there is no mandate as to how much more important one category is than the other. Is damage worth twice as much as ground control? More than that? Who knows? This makes Pride’s criteria decidedly unscientific and subjective. Without concrete numerical values with which to measure one fighter’s performance against the other fighter’s, judging becomes a battle of differing interpretations of the criteria and differing opinion as to which particular minutes or seconds mattered most. One judge might give weight to a nearly successful submission attempt by one fighter while another judge will choose to favor the other fighter based on his accumulation of damage and combinations while on the feet. These two judges will arrive at different decisions, even though they both agree in principle on who won each aspect of the fight. In addition, by considering the entire fight, judges are often unduly influenced by the closing minutes of the fight, which leave a lasting impression more so than the beginning.

The problem of subjectivity is simple but its effects are profound. When judges are left to decide a winner based on their own subjective opinions, you inevitably end up with more disagreement, more controversy, and poorer decisions. Without an objective standard, fighters can’t form a strategy; not knowing if the judges they’re fighting in front of will weigh one aspect of the fight more than another. And when decisions are rooted in opinion, fans with a different opinion as to the winner will lose respect for the judging system and distrust the organization.

The criteria used by the UFC are more detailed and therefore engender different problems. Firstly, the “ten-point must” system is used, whereby the winner of each round is granted 10 points with the loser of the round receiving nine points or less. At the end of the fight, points are tallied and the fighter with the most points wins, either by unanimous, split, or majority decision. If both fighters have the same number of points, the match is ruled a draw. It’s the “nine points or less” part of the equation where things get dicey. Unlike in boxing where points are automatically deducted because of knockdowns, judges in MMA are noticeably reticent to mark 10-8 rounds, while 10-7 rounds are almost unheard of. This means that a fighter can overwhelmingly win a round and receive a 10-9 score just the same a fighter who held only a modest advantage.

This is likely the result of just how complicated it is to determine the winner of a round. Though Mike Goldberg breezes through the standards, saying that UFC bouts are judged based on effective striking, grappling, aggression, and octagon control, the official guidelines are not quite so clear.

To begin with, the priority of those categories (striking, grappling, etc.) is not a constant; judges must first take note of where the majority of action takes place during a round. If the majority of the round is spent on the mat, then grappling is worth more, while striking would be worth more should the majority of action take place on the feet. If a round contains an equal amount of stand-up and ground fighting, then grappling and striking would be weighed equally. Next, judges must keep count of “the total number of legal heavy strikes landed” to determine who wins the striking component of the round. While Pride’s rules use damage as the qualitative arbiter of striking effectiveness, the UFC’s rules require a quantitative numerical advantage for one contestant. The merit of each system is debatable, but it would seem certain that by using a clear quantitative standard, the UFC’s criteria would make for more objective decisions in the area of striking.

Quantitative definitions seemingly fall apart when judging grappling in MMA. The rules attempt to set a standard, saying, “effective grappling is judged by considering the amount of successful executions of a legal takedown and reversals.” However, takedowns and reversals are not the only grappling techniques that earn points. With submission attempts, guard passes, sweeps and other ground activity, not to mention Greco-Roman and other grappling action on the feet, judges have to keep track of a staggering number of factors and decide which of these to weigh more heavily than the others. The rules would like judges to count techniques and arrive at a concrete number corresponding to each fighter’s grappling performance. But based on the vagaries of the guidelines, is there any guarantee that judges will arrive at the same number? It is no wonder that takedowns (even into guard) and maintaining top position hold so much sway over UFC judges. These easily quantifiable factors can provide a simple number to judge grappling. But given the complexity of MMA, is that simple number good enough to encapsulate all the grappling in a round and thereby give one fighter a win and another a loss?

The idea of an objective standard is laudable, but as Pride and the UFC demonstrate, nearly impossible to implement. In Pride’s case, the simplicity of their criteria should indicate the same winner to anyone viewing the fight, so long as everyone grants the same weight to each category of performance. In the UFC, each judge would ideally count the exact same number of heavy blows landed, the number of successful grappling maneuvers, and submission attempts. Each judge would utilize the objective standard to decide the winner of each round and all three judges should arrive at the same decision. In both cases, proper standards should provide the same results no matter who the judges are. There should never be a split decision.

But there are plenty of split decisions. Viewed individually, split decisions can be blamed on a particular judge, simply accounting for inevitable variables in vantage point, concentration, and judging bias (e.g., giving more weight to striking rather than grappling or submissions). However, on a systemic level, a large number of split decisions would perhaps indicate a deficiency in the judging criteria itself. If judges frequently disagree with each other, it might be an indication that the judgment parameters are themselves unclear and inherently lend themselves to discord.

So getting back to the original question: Who’s judging criteria is better, Pride’s or the UFC’s? If split decisions are an indicator of poor judgment criteria, perhaps figuring out which system produces fewer split decisions will indicate which is the better one…

And Part 2

Quote:

The first part of this investigation into the judging criteria used by Pride and the UFC examined the methods used by each organization and laid out the inherent flaws in each system. To sum up quickly, Pride’s system, while much simpler, provides little concrete direction to its judges and leaves too much to their subjective opinion. The UFC’s criteria requires judges to keep track of a huge amount of information, watch for dozens of techniques, and arrive at a difficult-to-comprehend score for each round using the “ten point must” system.

The end of the article presented a hypothesis: If a judging system was perfect, then nearly all decisions would be unanimous, no matter who the judges were or how many of them judged the fight. If that is true, then looking at the number of split decisions each system produces could indicate which system of judgment is better.

To be clear, split decisions are not inherently bad. In fact, on an individual basis, split decisions are absolutely necessary to the judging process. Mixed Martial Arts is beloved for its dynamism; the notion that anything is possible and you never know what will happen next. No criteria should be so rigid that fighters lose their incentive to take the fight anywhere they wish. If a computer were used to judge fights, for example, fighters would know which techniques and strategies would always result in favorable judging, and we would start to see that exact same fight play out over and over. Using human judges inherently involves some degree of subjectivity, which naturally results in split decisions. That subjectivity keeps the fighters on their toes and keeps fights interesting.

But on a systemic level, split decisions are inarguably a bad thing. One judge who consistently disagrees with his peers means you’ve got a bad judge; when none of the judges can agree with each other, it means you’ve got a bad system. And when judges can’t consistently agree on a fight’s victor, it degrades the quality of their decisions, even the unanimous ones.

So which organization produces more split decisions, Pride or the UFC? To see, we must look at the outcome of every Pride and UFC fight from September 28, 2001 (when the UFC held its first show in Las Vegas, marking widespread acceptance of the Unified Rules) until the end of 2006, excluding the fights from Pride’s American show on October 21, 2006, because those fights were contested under NSAC rules.

Since September 28, 2001, Pride has held 425 total fights, 137 of which have gone to a decision. Of those 137 decisions, 33 of them were split decisions, for a split decision rate of 24 percent.

In the same time period, the UFC held 387 fights, of which 112 went to a decision. In all that time, only 10 of the UFC’s decisions, or 9 percent, were split decisions. This would seem to indicate clear superiority for the UFC, however that 9 percent number is somewhat deceiving. Because the UFC criteria allows for draws as well, there were also five majority decisions (where two judges pick one fighter and the other calls a draw) and two draws, both of which were split draws (one judge picks one fighter, one judge picks the other fighter, and the third judge calls a draw). Since these also involve judges coming to different conclusions, they must also be considered split decisions for our purposes.

Still, the UFC beats Pride by nine percentage points, with only 15 percent of its decisions not unanimous, compared to 24 percent for Pride. This should clearly indicate something wrong with Pride’s decision criteria. Nearly a quarter of its decisions have one judge saying that the wrong fighter got the victory. Utilizing human judging necessitates some split decisions, but a 24 percent rate signifies that the murkiness of Pride’s criteria leaves too much to the subjective opinions of its judges.

So the UFC’s criteria would seem to be better. With judges able to reach a consensus 85 percent of the time, the system would seem to be clearer than Pride’s and clear enough for judges to apply it uniformly the vast majority of the time. But is that really true? How uniformly are the judges seeing the fights? Or put another way; are these unanimous decisions really unanimous?

Thanks to the Nevada State Athletic Commission, we are able to take a deeper look at this question. The NSAC posts the results of every fight card held in Nevada on its website for anyone to access (something every state athletic commission should really do). They make public how each of their judges scored each fight, giving anyone who wants the opportunity to examine if a decision was truly unanimous or not.

A true unanimous decision would be one in which all three judges score all three rounds the same. For example, a bout that gets scored 30-27 by all three judges would be a truly unanimous decision. But what about fights that get scored 30-27, 29-28, 30-27? What happened in the one round that one judge scored for the other fighter? Even though all three gave the victory to the same fighter, making the outcome a unanimous decision, there was still one round that was a split decision. Using publicly available data from the NSAC website, we can analyze split decisions on a round-by-round basis. The same logic we used when looking at fight results should apply to the findings for round results; if judges consistently disagree about the scoring of individual rounds, this too could point to a problem in the judging criteria.

To look at this problem we must expand our scope a bit. To arrive at a total number of fights similar to the figures dealt with before, we will look at every fight conducted by the NSAC from its first event in 2001 until the end of 2006. In that period, NSAC judges watched 425 fights (mostly UFC and King of the Cage bouts), of which 96 went to a decision. Each of those 96 fights had three rounds, plus there were nine title fights that had an additional two rounds apiece, for a total of 309 rounds judged by the NSAC. Of those rounds, 235 of them were scored exactly the same by all three judges. But that means that 74 rounds were split, with one judge scoring the round differently than the other two judges. Divide 74 by 309 and what do you get? It equals 24 percent, exactly the same proportion of split decisions received in Pride.

In all, only 30 percent of decisions handed down by NSAC judges were truly unanimous decisions. The remainder of the fights – more than two thirds – contained some discrepancy between judges, whether calling the fight for a different fighter, calling a round for a different fighter, calling a draw, or calling a 10-8 round when others didn’t. And this was not the case where one bad judge was responsible for an undue share of disagreements. Each judge was responsible for his relatively fair share of discrepancies.

So in the end, it seems just as difficult for judges to implement the Unified Rules criteria as it is for Pride’s judges to use that system, with each producing a 24 percent disagreement rate. And in the end, with each system’s flaws laid bare, we are left with our own choice of which system is superior.

One can quibble with Pride’s criteria for its subjectivity, but there is a measure of respect due its judges. True, their decisions are rooted mainly in opinion, but they are entitled to and empowered by those opinions. Disagree with their choices if you will, but by choosing to use a qualitative standard, opinion is the best you’re ever going to get. Those decisions, even the split ones, are well within the spirit of the Pride Fighting Championship.

In the case of the UFC’s rules, practiced in this case by NSAC judges, we are forced to be more critical. The rules were enacted specifically to avoid the subjectivity found in Pride. By adhering to a quantitative standard, the goal was to produce better, more objective decisions. But that only works when the rules are enforced uniformly by every judge at every event. Over time, we’ve seen major inconsistencies in the enforcement of judging rules. For instance, the first three events ever judged by the NSAC contained two draws and one majority decision. In the five years since – a span of 49 events – there have been only two majority decisions and one draw in Nevada. We are left to believe that almost none of the fights in the past five years have been as close as those first ones or more likely, that judges have simply stopped marking 10-10 rounds. So while the UFC’s criteria certainly produce a greater number of “unanimous decisions” than Pride’s, one can’t help but question the quality of those decisions. Just because three judges agree on a winner doesn’t make that decision a good one if the judges can’t properly wield the judging criteria.

But no matter what the data says about current NSAC judging, the UFC’s criteria hold a significant advantage over Pride’s: it’s not the system that needs upgrading, it’s merely the application of that system that should be improved. The rules used by the UFC are solid and should provide excellent decisions once they are adequately clarified so that every judge is on the same page. Indeed, with the UFC expanding into an increasing number of states, it behooves them to make sure that their rules are easily understood and enforceable by judges from any state athletic commission, especially those new judges who may not be that familiar with the nuances of MMA. For the UFC, the best advice is what the NRA has been peddling for years. It’s not about making new rules; it’s about properly enforcing the rules that are already on the books.

So after all is said and done, which system of judgment is the better one? With time, clarification, and experienced judges, there is no reason why the UFC’s criteria wouldn’t be far superior to Pride’s. However, given the currently muddled state of judging in MMA, there is no clear winner for the moment. It unfortunately looks to be another split decision.