ESPN Unveils its New Total Quarterback Rating (QBR)

When you are a large corporation, like ESPN, you have the power and influence to change public perception. That doesn’t mean it will always work. Starting tonight, though, the network will try, when it unveils a new rating system, called Total Quarterback Rating, which it has dubbed QBR (TQR apparently made too much sense as the actual initials).

Innovation is often driven by smaller companies, and larger companies then pick up on it or by out the successful smaller innovators. As I read through the information being released about tonight, I can’t help but think of Football Outsiders’ DVOA rating for Quarterbacks, or Advanced NFL Stats use of Expected Points Added, and Win Probability Added, as many of the concepts described have already been employed by those with a voice not as large or broad as the television production side of ESPN.

In fact, Dean Oliver mentions both of these in his explanation. The idea behind this QBR involves many of the same concepts–not every situation is identical, a 6 yard completion on 3rd and 5 is more valuable than on 3rd and 15, etc., and every situation has an expected point improvement and improvement in win probability added. For example, a 50 yard touchdown on third down in the fourth quarter will result in a large swing in the probability that the team wins, if it was a tie game, but have little impact on win probability if it was a 25 point game at the time.

None of these concepts are new. ESPN may use slightly different weights, but the ideas have been around. By using the play by play information, everything a QB does–runs on 3rd down, sacks taken, throwing a pass away rather than take a sack–can be incorporated.

I have a few concerns about it, but I’m also not going to demonize it. Is ESPN going to promote it, and potentially make you sick? Probably. Might it have value? Yes. Will it be perfect? No, nothing is.

My first concern is this clutchness factor. If you use a win probability model, some plays will already have higher leverage. Also, as a predictive measure, I’m not sure making a big play in the first is any less notable than reversing the order in the fourth quarter, but it sounds like it will increase one quarterback’s rating more. I’m not sure how much this weighting affects it.

Second, the attempt to separate out teammate contribution. It’s a laudable goal. We know that sack cost the team and forced them to punt, or that touchdown pass had a big impact. We may also know that a longer pass is generally more the responsibility of the quarterback, while a shorter pass is more about the receiver (and blockers) turning a play into something. That’s not always true though. It sounds like ESPN will use video spotters to be making determinations on things like contested passes, passes thrown under pressure, drops, etc. These are still subjective determinations. This new stat will purport to rate quarterbacks by assigning a percentage responsibility for the point improvement or win probability improvement on various types of plays.

Finally, transparency and ability to reproduce the numbers. I understand the general concepts as explained in the various articles, but we don’t really get to see how the sausage is made? How can we check to know if it is accurate or doing the best job possible?

Ultimately, QBR will kick out a number on the scale of 0-100, with 50 being an average performance. It will be a number that the public can latch onto, just like Passer Rating, for its flaws, now kicks out a number and people generally know what constitutes a good game or bad game by being told a passer rating. Unlike passer rating (which I can reproduce if I have the underlying results) I can’t reproduce this QBR without being told what it is.

I welcome all new research. Ultimately, it’s success or failure will depend on how well it fits winning and scoring points, the things for which a quarterback should be judged. And it will be judged on its results over the next few years. It won’t automatically ascend to displacing other measures.

Jason Lisk should come up with his own rating system. I’m sure it will be quite swell. Dude knows his shit. And would love to see how it compares to ESPN’s throughout a season. Hell, we could all vote at the end of the season to see which is best.

pigskin28

Sounds promising. The rankings for qb’s from last year are pratty convoncing.

Rockstar

Read 3/4 of the post. A new record for a Lisk post. Good read, but no attention span has always been a crutch.

The caption is classic. Too bad somehow ESPN will come up with a way to put Sanchez near the top of the list. Ratings baby!

Chief

Too bad somehow ESPN will come up with a way to put Sanchez near the top of the list.

If I read on Twitter right, Kerry Collins’ 2010 was 3 spots better than Sanchize’s.

http://twitter.com/MantisToboggan5 Mantis Toboggan

Sounds promising. The rankings for qb’s from last year are pratty convoncing.

Eli was ranked 7th last year. A failed system, he blows.

/tbl

Gaseous Clay

Ultimately, QBR will kick out a number on the scale of 0-100, with 50 being an average performance.

Blasphemy. 80 is average, and 158.3 equals perfection. It is known.

jim

Statistically, who is more important to their team: a pitcher or a QB?

wilhelmreems

Statistically, who is more important to their team: a pitcher or a QB?

QB, more likely to play in 100% of games rather than the pitcher who would go every 5 days (at least for 1 starting pitcher)

wilhelmreems

damn, the one time i’m on for tblad no one is around.

http://twitter.com/BrawnyTom BrawnyTom

It lost me when it had Matty checkdown rated #3 QB. That’s about 6-8 spots too high for him. Also, what the hell is “clutchness factor”? Does a Qb get credit for a 6 yard dump off on 3rd and 12 if the back break a few tackles and runs 40 yards to get within easy FG range for a win? What if his WR “Stevie Johnsons” the game winning TD pass? Does he still get credit?

http://twitter.com/MantisToboggan5 Mantis Toboggan

Does a Qb get credit for a 6 yard dump off on 3rd and 12 if the back break a few tackles and runs 40 yards to get within easy FG range for a win?

/flacco’d

Tickkid

It lost me when it had Matty checkdown rated #3 QB. That’s about 6-8 spots too high for him. Also, what the hell is “clutchness factor”? Does a Qb get credit for a 6 yard dump off on 3rd and 12 if the back break a few tackles and runs 40 yards to get within easy FG range for a win? What if his WR “Stevie Johnsons” the game winning TD pass? Does he still get credit?

They act as if they will facter that stuff in when they do it – but as Lisk so valliantly stated – if the QB throws to Megatron and the dude flies through time and space to catch a TD and tap two feet in in the end zone, how much of that is QB and how much isn’t. And if a QB spots that Reggie Bush is going to have a clear path to the end zone if he throws the out and Bush catches a 4 yard out and scores a 50 yards TD – how are you going factor QB vision in/out of the process. The subjective parts make me nervous, especially if any of the people that review game film work/live near Cowherd or Bayless.

/yelledatthetopofmylung’d

squawkbox

The “clutchness” factor also assumes that success in the 4th quarter is somehow better than success in the 1st quarter. So:
two quarterbacks have essentially the same performance in a game,
away qb does his work through the first 3 qtrs and wins
home qb does most of his work in the 4th qtr and loses by a fg

Does the home qb get clutch points to put his performance past the away qb? I call bs on this – consistency and putting away a team early is more important than coming back.

sleepycarl

“ESPN Unveils its…”*

Sincerely,
That guy

traderjack21

So is this TQR as highly irrelevant and meaningless as the real QB Rating?

Quietgoesthedon

I’m confused. We can’t check the results ourselves because the formula is too complex (or too dependent on random subjectivity) for Joe Sixpack to check, or because ESPN won’t unveil the complete formula and how it’s used?

The problem with this formula, like every other formula that attempts to measure performance, is that the subjective elements, the “clutchness,” and the relativist nature of soft data all relies, in the end, on some guy assigning value to something based on how he feels about it. We can count certain things. Touchdowns. Yards per completion. Sacks. Hurries (those are more problematic). Completion percentage. When you start assigning values to interpret that data, as part of a larger formula, it always strikes me as so contrived as to be next to worthless.

vajayjay redick

We can count certain things. Touchdowns. Yards per completion. Sacks. Hurries (those are more problematic). Completion percentage. When you start assigning values to interpret that data, as part of a larger formula, it always strikes me as so contrived as to be next to worthless.

ANY/A is a perfectly good stat that already exists, and it’s easier to calculate than the existing passer rating (though i’m not sure about the weights). It’s not on a 0-100 scale though so i guess ESPN can’t market it. I get what they are trying to do, but there are too many flaws to overtake the established metric that’s been quoted for 40 years. They will beat us over the head with it this year that’s for sure.