Most Recent Extra Points

Week 11 DVOA Ratings

by Aaron Schatz

Some things just try my sanity. Take, for example, the Philadelphia Eagles. The good news this week is that the Eagles drop behind the undefeated Tennessee Titans. The bad news is that they don't drop any further than that, and they still stand out as a team where DVOA seems to bear no connection with actual wins and losses.

Or do they?

Two spots below Philadelphia, very close in DVOA, you will find Green Bay -- a team that has an even worse record than the Eagles do (5-5). Do these two teams have similar characteristics that might cause them to be overrated by DVOA? Actually, no.

The Eagles struggle to run the ball in short-yardage situations, ranking 31st on power runs. That's not a problem for the Packers, who rank ninth on power runs.

The Eagles have problems converting third downs in general. Their offense ranks 11th overall but 22nd on third and fourth down. The Packers, however, are actually better on third down. They rank ninth on offense overall, but fifth on third and fourth down.

The Eagles were struggling in the red zone earlier in the year, but they aren't anymore. Both the Packers and the Eagles have a red zone offensive DVOA within one percentage point of their total offensive DVOA.

The Packers are getting killed by the run -- 25th in defensive DVOA on first down, 32nd on second down -- which constantly gives their opponents nice third-and-short opportunities. Not an issue for the Eagles, who rank between fifth and eighth on every down whether you are measuring pass defense, run defense, or the combination of both.

Neither team has played a particularly hard schedule so far, and they rank ninth (Green Bay) and 10th (Philadelphia) in VARIANCE.

There's one particular DVOA split the two teams do have in common: both offenses struggle after halftime. The Eagles' offense ranks third in the first half of games and 20th in the second half. The Packers' offense ranks second in the first half of games and 18th in the second half. Other than that, it's hard to find a place where we could somehow tweak the DVOA formula based on a belief that the accuracy of our ratings for Philadelphia and Green Bay needs to be improved. (That's not meant to be obnoxious -- we're really trying to find a way to improve our ratings to fix the "Eagles problem," and it's damn hard.)

Here's what both teams really have in common: Their wins come by large margins and their losses by small ones. The Packers have lost three games by a field goal or less, and their largest loss is 11 points. All four Philadelphia losses have come by less than a touchdown. The Packers have outscored opponents by 65 points with +7 turnover margin. The Eagles have outscored opponents by 71 points with +3 turnover margin. Despite five losses, the Packers have only one game with DVOA below zero: their Week 4 loss to Tampa Bay. Despite four losses, the Eagles have only two games with DVOA below zero: their Week 2 loss to Dallas and this week's tie with Cincinnati.

The biggest difference between Philadelphia and Green Bay? Their chances of making the playoffs. Green Bay is part of a three-way tie for first place between three 5-5 teams, and ranks 25th in the NFL in future schedule strength. Our playoff odds simulation gives them a 67 percent chance of making the postseason. Philadelphia plays in a division where all four teams have winning records, and has the third-hardest remaining schedule. Our playoff odds simulation gives the Eagles a 41 percent chance of making the postseason -- actually, it gives them less chance than that, or will once we fix it. Apparently we forgot to give the playoff odds simulation the ability to recognize ties.

Speaking of the playoff odds, Tennessee's chances of going undefeated rose only slightly this week, from 4.8 percent to 7.6 percent. What makes this watch interesting is that their toughest opponents are the final two: Pittsburgh and (a now fairly healthy) Indianapolis.

Which team is the opposite of Philadelphia and Green Bay? Carolina, which is 8-2 but ranks just ninth in DVOA. The explanation here is pretty simple: schedule. The Panthers rank 29th in past schedule strength -- and first in future schedule strength. If you're looking for a test of how much the guys in Vegas read Football Outsiders, check the line when the Panthers go to Green Bay in Week 13.

* * * * *

NEW IN PREMIUM THIS WEEK: The Premium DVOA Database now includes Adjusted Line Yards, Adjusted Sack Rate, and all our other various offensive and defensive line stats going back to 1997. Right now, you can view each year; soon, we'll be adding views to see each individual team's history in these stats.

* * * * *

These are the Football Outsiders team efficiency ratings through 11 weeks of 2008, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted to consider all fumbles, kept or lost, as equal value. These ratings also include opponent adjustments. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. WEIGHTED DVOA is adjusted so that earlier games in the season become gradually less important. It better reflects how the team is playing right now.

As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.

To save people some time, please use the following format for all complaints:

NON-ADJUSTED TOTAL DVOA gives performance without adjustments for schedule strength, fumble recovery luck, and weather/altitude on special teams.

ESTIMATED WINSuses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.

PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.

FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.

I don't know - if I were a Lions fan, I would be rooting for 0-16 by now. It seems like the perfect capper for the Millen era (it's not like the balance of the season isn't his fault). And if you're not going to win, you might as well do something memorable in losing.

Fun fact: Dallas is rated #22 after beating the #3 (the Eagles), #5, #8, #11, #23, and #28 teams. They also lost to the #6 team in OT, and lost to the #1 and #32 teams when they had "Dallas Quarterback" at the helm.

Meanwhile, the Eagles are ranked #3 and have beaten #32, #27, #25, #13 and #7. They've also lost to #1, #11, #16, and #22 (Dallas), and tied #28.

I'd feel bad about it if DVOA could reasonably distinguish between "the Cowboys are playing with their backup quarterback and don't feel like trying to win since he sucks so let's get destroyed by a crappy opponent" (STL) and "the Cowboys are playing with their backup quarterback, so let's play pretty good defense so we can beat a good team without our starter" (TB). Would it also be churlish to point out the Eagles are 3rd in the NFL in Points For - Points Against, while the Cowboys are 18th?

No, clearly the Eagles are ranked higher than the Cowboys because Aaron hates Dallas because he departed from his Pats love growing up to root for the Orange Crush only to be disappointed when they got smacked around in the Super Bowl, and therefore makes DVOA hate the Cowboys.

Of course, there's always more to it than that, which I'm sure you're aware. The drawback to the "my team beat teams of the following ranks" perspective is that it ignores the character of the wins.

Dominating the Texans 40-10 speaks very well of you, even if they're a bad team. Barely beating the Packers due to fluketastic factors that are unlikely to be repeated speaks less well for you, even if it would be a victory over the #5 team, not the #26 team.

As Aaron explains quite openly, Philly either blows the snot out of the opposition or loses narrowly. When you blow out another team, you're effectively demonstrating that this win isn't random, that if you played this team 10 more times, you'd probably win 7-9 of them. When you win by three, on the other hand, you're effectively demonstrating that, while things may have come down on your side today, over a ten game stretch it's anyone's guess how things would go. If this were not true, pythagorean ratings would not be so effective.

So yeah, it's not whether you win or lose, it's about how you play the game :)

"When you blow out another team, you're effectively demonstrating that this win isn't random ... When you win by three, on the other hand, you're effectively demonstrating that, while things may have come down on your side today, over a ten game stretch it's anyone's guess how things would go. If this were not true, pythagorean ratings would not be so effective. So yeah, it's not whether you win or lose, it's about how you play the game."

Not so fast there pardner. The fact that the Eagles are not very good demonstrates the limits of this theory, as Aaron states above. Killing bad teams and losing to good teams is apparently a bit more complex. For example, maybe the bad team rolls over. Maybe (kinda like a bully) it's easy to have confidence to beat a bad team but harder to believe you can compete with a good team. In spite of what the numbers say, I think there's something to the previous poster's argument that cannot be easily dismissed.

It seems slightly tautological to use "The fact that the Eagles are not very good" as a defense for the position that the Eagles are overrated compared to the Cowboys. The very subject we are implicitly discussing is whether or not the eagles are good, it's slightly cheaty to use your conclusion as evidence of the truth of itself. Anyhow...

You appear to be making a separate argument from what the original poster proposed. His position appeared to be that you could deduce team quality from looking at the quality of team won and lost to. I countered by saying that this position, while true, overlooked the importance of the nature of the win/loss, which is the major reason why the disparity he, quite rightly pointed out.

Your position, as near as I can determine, is that the pythag isn't perfect because a team may have psychological factors corrupting the reliability of the data. This is certainly possible (though hard to prove, since the results are the same whether it is random or psychological.) However, you do not propose that this applies to all such teams, (which would defeat pythag with such teams were it true), you merely propose that it is true with the eagles, and that the possibility in general lends weight to the original poster's position.

Your first allegation may well be true, not being clairvoyant I cannot say what the cause of the Eagles' unusual pattern is. However, I feel as though your tag-team is misplaced, the original poster made no argument regarding causes of defeat/victory, no assertion that Dallas wins with a stout and robust psychology that plays each team equally.

As a rule, pythag tends to hold true, and it certainly has exceptions, as you have pointed out. That, however, doesn't take away from the fact that it is a rule that at least needs to be addressed, even if dismissed due to alleged exceptions to it.

Or it could be possible that there is a great deal more randomness in the game than can easily be accounted for, and that trying to determine which of two decent teams is better based on past performance is like trying to figure out whether a carpenter ant is heavier than a bumble bee using only a bathroom scale. It's possible, and some measurment is better than none, but there are a lot of fluky things that could mess with either reading...

And as to the psychological argument, I would counter by saying that some teams may roll over when they know they're bad and facing a good team, but other teams get their hackles up and play hard. Last year, for example, EVERYONE put up practically their best game of the season when they played the Patriots. And still lost.

Nice vocabulary Sartre. Never use the simpler word when a more impressive one will do, huh. You must have degrees. I am the Other that defines your argument. (A little philosophy for you.)

In regards to your first paragraph, you lopped off the final part of the sentence "as Aaron states above." To be exact, Aaron stated, "The bad news is that they don't drop any further than that", implying that he agrees that the Eagles aren't very good. I said nothing about the Cowboys. I am making the argument that the Eagles aren't very good. Numbers don't define my viewing experience; they support it. I watch the games, and from what I see, the Eagles aren't very good. That is all.

by Love is like a bottle of gin (not verified) :: Tue, 11/18/2008 - 11:26pm

Actually there is not another word which you can slip in to replace tautological, stop being an idiot. Criticizing someones perfectly sensible sentences is the last refuge of someone who has lost an argument.

"Redundant" means saying something, than saying it again. "Tautological" describes an argument whose conclusion is contained within it's premises (i.e. a circular argument). They're really not close in meaning, and his using "redundant" wouldn't have made any sense. His criticism wasn't that you were repeating yourself, it was that you accepted as a given that which you were trying to prove.

Yes Scott. Some are describing the logical meaning of "tautological". I am describing its rhetorical meaning. It appears we come from different worlds my friends. See, I wasn't trying to prove anything about the Eagles. I was repeating what the author had stated, as I stated above. Schatz seemingly agrees that the Eagles are rated too highly as he acknowledges the limitations of his own system.

Do you (that's plural you) not realize how seriously you take yourselves, and how much fun it is for me to stick responses in your craw? Just to watch you respond? DVOA is fun and interesting. It is also flawed. The most fun part of this site to me is to try to disagree and watch the logicerati emerge.

Parenthetically, I realise that this is feeding the troll, but nonetheless...

Do you (that's plural you) not realize how seriously you take yourselves, and how much fun it is for me to stick responses in your craw? Just to watch you respond? DVOA is fun and interesting. It is also flawed. The most fun part of this site to me is to try to disagree and watch the logicerati emerge.

There's nothing to realize, because there's nothing novel here. I see this all the time. Guy on an internet forum makes a bad argument, gets called out, realizes he's backed into a corner and decides he'd rather be a jerk than look like an idiot. But claiming you were trolling all along just makes you look like a jerk and an idiot.

Don't do that. You can concede that you're unable to come up with an adequate counterargument without conceding that your position is wrong. Most people will accept that. Trust me. I'm a terrible debater, so I should know.

Wouldn't it make more sense to focus on the "logical" meaning of a term when you are discussing the logic of an argument? Just asking.

Though I don't think that was class warfare. It was a claim that he comes from a world of rhetoric and literary genius far more shiny and happy than the dank and cold world of logic where everyoe else here lives.

I'm wondering if all these complaints would be occurring if the Cowboys had lost to the Redskins this week.

A lack of injury adjustments is DVOA's major weakness, but the Cowboys case is extreme. The Cowboys arguably went from the best quarterback in the league to the worst quarterback in the league. Meanwhile, their top corner, a starting lineman, the back-up running back, and a key linebacker in the rotation were also out of the lineup. Of course that's going to throw a system that doesn't adjust for injuries out of whack.

On an unrelated note, a lot of teams improved their overall DVOA this week. So hooray for everybody! Except the Hawks, who now suck even more.

While you do offer a good point in the limitations that DVOA has regarding injured starters, I tend to think of injuries in a different light. DVOA is measuring the performance of the playing members of a team, be it superstar QB or third string WR. DVOA shouldn't adjust for the talent level of the players on the field because that is exactly what it is measuring: how the talent performs on the field. If anything, I see that while Dallas has a potent lineup of first string talent, their injury replacements have been lacking when asked to perform and that is reflected in their ranking. That may seem unfair, but it can give a good perspective on the needed depth of the team.

The issue is whether you want DVOA to predict how good a team actually is.

Saying Dallas hasn't been very good without Romo is true. That doesn't mean that now, with him back, they are the team they were without him. If you take away the games without Romo, you're somewhat overrating the cowboys. If you leave them in, you're somewhat underrating them.

"I see that while Dallas has a potent lineup of first string talent, their injury replacements have been lacking when asked to perform and that is reflected in their ranking."

Thats utterly ridiculous. ANY team that loses a probowl quarterback is going to decline. NO ONE has that depth.

Eh, makes for a better argument. Besides, when you think about it, it's not that huge of a stretch. It ain't like Tom Brady's around to automatically get all the gravitas. Romo's 5th in DVOA despite his wide receiving corps (especially Owens) and the offensive line both having a terrible year. Two different quarterbacks (granted, neither were all that great to begin with) played absolutely abysmally in the exact same situation. There's also a lot of parity in QB DVOA right now.

The Cowboys are clearly ranked too low because they beat the #3 (the Eagles), #5, #8, #11, #23, and #28 teams. Ignoring that they had "Dallas Quarterback" at the helm is way better than this. Don't those number just seem a bit off to you?

I don't think this has been mentioned, but TEN is thus far on pace for a historically great pass defense DVOA. Looking at the previous years, the 2002 Bucs stand out by a country mile at -55.8%, then you have a clutch of teams in the -34-36% range. Unless I missed somebody, TEN would currently rank 6th best at -33.4% pass D DVOA. It's possible to run on them, particularly when Haynesworth isn't in the game, but they've really put the clamps on opposing passers this year.

for what it's worth - The Vikings were a far better offensive team in 2007 than they have been in 2008. The only changes - Berrian replaced Wade as number 1 receiver and Frerotte replaced Jackson at QB.

Looking back now I think it was a bad move. Jackson lost to GB on the road and he made one horrible pass but he kept them in the game with his scrambling to that point. Losing to GB on the road according to DVOA isn't exactly horrible. Then they lose to Indy which it turns out is a good team. If Shiancoe could catch that game probably turns out to be a win.

This offence can't afford sacks - Frerotte takes far more than Jackson and as it turns out Frerotte makes more mistakes (ints) as well.

I'm not sure I buy Frerotte has been better than what you could have expected from Jackson.

Hear, hear! DVOA is sometimes "obviously" wrong, and is by no means perfect. But it is often right about non-obvious things, too. If it just parrots back what we already know from win-loss record and scoring differential, then it is a colossal waste of time - but it doesn't.

The Eagles get a lot of their stats from blowing out the three horrible NFC West teams they've faced. Otherwise, their point differential is nothing special.

My point is this: The Eagles have not beaten very many good teams, and have lost to most good teams they've played. Dallas meanwhile, has beaten (when healthy) nearly every good team they've played (the Cardinals OT thing was partially healthy, partially not since the broken finger came in the beginning of OT). They lost to Washington in a very close game and now have evened the score with them. I just don't see what else Dallas has to do to climb the rankings or what else Philly has to do to go down. Being 0-3 in the Division should be a good indicator of what the Eagles are. They lose to good teams and blow out bad ones. End of story.

by Love is like a bottle of gin (not verified) :: Tue, 11/18/2008 - 7:27pm

DVOA is not a merit based system. It is a mathematical model. The cream does not always rise to the top. It is just a simulation with some well understood weaknesses (not knowing that the huge drop in DAL QB play was irrelevant (maybe)) is one of them).

Life isn't fair and football seasons with tiny samples are even less fair. Personally I have no idea what Dallas fans are upset about? Wouldn't you rather be 6-4 and have negative indicators from mathematical models than 5-4-1 and have positive ones?

It's not even really a simulation. It's an accounting method. It's not actual propegating anything forward according to as set of governing laws--it's just counting how many times certain very carefully chosen events happened for each team, and comparing them to how often they happen on average. It's a method for gathering data that could be used in a simulation (like the playoff odds report), but not a simulation in its own right.

But you are mistaking the principle purpose of ratings such as DVOA. DVOA does not tell you who won and who lost, the standings tell you that quite easily. The purpose of DVOA (and much of statistics) is to extract the predictive data, information that is reliable to draw conclusions from. To appreciate DVOA, you must fundamentally respect the fact that the final score is only partially related to what the game says about the teams. If a team wins by 3 because they recover all of the 6 fumbles that occur, that goes down as a win for them, but the fumble recoveries mean next to nothing about how good that team is (or how good the team that lost is.) DVOA will look at that game and say flat out that the winning team was worse than the losing team (probably) because there were huge luck factors that helped the winning team. To say that a team's DVOA is low simply because of mere wins and losses is to miss the purpose of the system in the first place.

As a matter of further principal, you probably ought not use three games as the bedrock for your argument "Being 0-3 in the Division should be a good indicator of what the Eagles are." I mean, the sample size is so small that it's not entirely reliable. Come back when they're 0-6 in the division.

Since we're having so much fun, I'd like to comment on this one too: "The purpose of DVOA (and much of statistics) is to extract the predictive data, information that is reliable to draw conclusions from." I've never seen a mission statement from FO on DVOA. Is this really true or your assumption? If true, it does a pretty poor job, as multiple times, DVOA has been shown to be about as consistent as Pete Prisco in its predictive qualities.

I'd venture it's main purpose is to make money for its creators. At that, it does a great job. Here we are.

Viny,
I'm in 2 office pools, one picking against the spread, one is a confidence pool. Each pool has over 200 people in it. I use DVOA and injury reports to make my picks. I'm first in both pools. From my perspective, DVOA is pretty damn useful since I don't have time to watch all 32 teams every week. So if you have a shred of intelligence you can use DVOA to make $ for yourself as well.

Having won two pools using DVOA, injury and weather reports last season.

Currently in third place in both pools this season having missed an entire week due to general drunkenness at a wedding. And thanks to KUBIAK, DVOA, injury and weather reports, having the best fantasy football team in a big league.

You're also falsifying excuses for the losses. The Cowboys were 10 points behind in the Arizona game, which they were barely able to cover to force the overtime (in which Romo's pinkie finger got hurt). They covered to get to OT, then lost on a special teams play.

And when we look at 'Special Teams DVOA'...oh, look at that! The only team worse than the Cowboys is my own Vikings!

Welcome to the wonderful world of losing on return TDs. Rather than talk up your Cowboys here, perhaps you should email Wade Phillips and let him know that 15-20% of a game (special teams) is a large chunk and needs some coaching.

In short - to 'climb the DVOA rankings', what Dallas needs to do is produce sustained drives that don't occur in garbage time. And beef up their special teams.

Philly's estimated wins according to DVOA is 6.7? Or just below seven wins, right?

That means that if they were 5-5, they'd be just two games lower than what we'd expect.

So, this is a case of the discrete nature of wins and losses working slightly against them.

Is this so different than the Denver Broncos who have 4.8 expected wins right now, but are 6-4?

And with so many teams being compared, don't you expect random chance to show a team or two to have different records than you'd expect even while most teams are about where you'd expect?

Maybe there is something to it happening to the Eagles more than other teams, but I think that the perception is less caused by a problem with the system as it is caused by the ramifications of a high profile team or two outperforming their rating all while another team underperforms theirs.

A slight underperformance by record combined with a slight overperformance combines to create a percieved substantial difference when the difference is really minute.

You could make this argument even stronger by saying a tie is "half a win", in which case the Eagles have 5.5/6.4 wins.

The reason people think the Eagles are overrated is because they have shown a knack for losing close games, and because they were just tied by the freakin' Bengals in a game where the Eagles generally looked like crap.

Perhaps the aspects of game-day coaching that Reid is terrible at (clock management, for example), simply are not measured by DVOA.

"When you blow out another team, you're effectively demonstrating that this win isn't random, that if you played this team 10 more times, you'd probably win 7-9 of them."

Is this really the case, and is this really what DVOA wants to hang its hat on? I don't have the data (translation: too lazy to do the study that wouldn't even be that hard), but we do have plenty of examples of teams playing each other twice in one season. It would be interesting to see how many teams blew out their opponents, then lost fairly convincingly to them later in the year (Wasn't there a Bills 31 Pats 0, followed by a Pats 31 Bills 0 a few years back or something like that? I know that's anecdotal evidence, but it does happen.) Blowouts aren't necessarily solid demonstrations of superiority, sometimes even a good team lets a game just get out of hand. ("Momentum, blah blah blah")

On the other hand, there are definitely emotional/attitudinal factors involved. I honestly believe that the Giants good showing against the Pats last year in the regular season made them more confident in the super bowl. It's an interesting question.

Yes, blowouts both ways do occasionally happen. But, you need to read the article on DOMS, GUTS, and SKATES, which shows that big margins of victories do tend to indicate something, especially in predicting teams winning the SB. However, that's not all DVOA stats hang their hat on. DVOA is designed to measure which teams win by consistently above average play and discounting plays that tend to be more unquantifiable, e.g. very long runs from scrimmage. The team gets credit for the long run, but not as much as the team would from a series of above average runs yielding the same result. The theory is that if you always do above average, you will continue to do above average. If you sometimes do something spectacular, you may do it again, but probably cannot expect it on a regular basis. Of course in the nature of a football game, a spectacular play may be all that needed in a close game to transform a loss to a victory. And with only 10 games having been played, two victories could turn a 4-6 team into a 6-4 one (or vice versa). Thus, if you want rankings that score teams based on wins/losses, there are places to find them (www.beatpaths.com for instance). Use DVOA for what it is good for. The authors of this site spend plenty of time trying to determine ways to make their scale better. However, it isn't likely to diverge much from what it is now, as it captures what it is intended to capture pretty well (an objective appraisal of a teams strength based upon available measurements, the play-by-play summaries).

"Yes, blowouts both ways do occasionally happen. But, you need to read the article on DOMS, GUTS, and SKATES, which shows that big margins of victories do tend to indicate something, especially in predicting teams winning the SB"

No, it really doesn't. Its on the right track, but its got some huge logical fallacies.

I would assume what the original poster meant when he said "if you played this team 10 more times...", he actually meant "if this exact game happened in 10 identical parallel universes, you'd win in 7-9 of them".

Obviously over the course of the season things change, which can shift the balance of power. The biggest example of this is, of course, injuries--e.g. playing the Colts without Bob Sanders or Jeff Saturday is infinitely easier than playing them with. The Pats got blown out by the Steelers the year they went 15-1, but dressed ONE RB that game, and were also low on CB's, due to injuries. When they met in the playoffs that year, they were healthier and the outcome was different. There's also schematic things that change...a team will come up with something new (e.g. the Wildcat, Kordell "Slash" Stewart, the Pats crazy spread offense from last year) that will take other teams a few games to figure out. In the Bills-Pats example that you mention, they played in Weeks 1 and 17. In Week 1, Laywer Milloy had been released by the Pats and gone to the Bills a few days before the game, taking all his knowledge of the Pats defensive formations, offensive preparations, and locker room presence with him, and hence the Bills blowout. That game could have been played in 10 parallel universes, and the Bills probably win in all 10 of them, despite the fact that the Pats were a deserved 14-2 that season and the Bills much weaker (don't remember at the moment).

This of course highlights a huge problem with DVOA, along with EVERY other objective ranking methodology: you MUST assume that the team you're ranking has some reasonably constant value of "goodness" that you're trying to measure. You can't take into acount changes over time, or at least you can only take into account changes over a long time (say, a significant portion of the season), because of sample size issues. In this sense, DVOA is actually better than methods that look at W/L, because, by looking at every play, DVOA has more samples per unit of time and gives a more meaningful picture of how a team is playing at any given moment.

<packer-homer>Along with the things the article mentions, the Packers have won 4 of their last 5 (my system gives less weight to earlier games). If they ever figure out how to play run defense, they could be very hard to beat.</packer-homer>

Um, before Sunday's game, the Packers had just lost two games in a row (Tennessee and Minnesota). They've won three of their last five, but the sequence went like this (from earliest to most recent): W, W, L, L, W.

You could just as easily say, "The Packers have lost two of their last three," which sort of goes against your point. Aren't multiple endpoints fun?

Does it make any sense (and is it possible) to replace the 0.0's with E's for teams that are mathematically eliminated from the catgory in question? That way it's easy to tell at a glance that Detroit isn't making the playoffs not as a matter of sucking so badly they'd need more than a 10,000-1 shot of luck (which has made it a foregone conclusion since week 3) but as a matter of there being no possible combination of games that can put them in. The same is true if my math is good of STL and SEA with regard to the bye (they can't catch the Giants, and I don't think they can catch the winner of CAR-TAM, since CAR would have nine wins and TAM would have eight including seven conference wins). By contrast, inferentially CIN has a zero chance because they're so bad they have virtually no shot of getting all that they need, but one time in a million they *could* make it.

Sadly, the Detroit Lions are not yet mathematically eliminated (much like GM, it would seem).

See, they may be 0-10, and they may have '0.0%' probability of reaching the playoffs because they (to quote Homer Simpson) are the 'suckiest sucks that ever sucked', but...

They could still be the NFC North representative. If they win out, and if Green Bay, Minny, and Chicago all lose their remaining games, then Detroit would end 6-10, and the other would end 5-11.

Sort of. Chicago has already swept Detroit, so they'd need Chicago to end a game behind; 5-11. Chicago plays Minny and Green Bay once each, and needs to lose both. Which means Minny and Green Bay each get one more win.

So really, you could, theoretically, have a 3-way tie at 6-10 atop the NFC North. And the Lions could be one of those three teams, having (theoretically) split the series with both the Vikes and the Pack.

The Lions' remaining schedule, listed by DVOA rank of opponent, in descending order: 2, 5, 8, 10, 15, 18. I don't believe they can win one of those games. Winning all six? We might as well discuss the value of replacing Fox' pregame show with talking Peeps.

Beware underestimating the probability of disjunctive events. Detroit has six games left... if the average probability of them winning one is 25%, they would win at least one game with (1-.75^6) = 82% probability.

In order to have just a 50% chance of winning one game this season, they'd have to average a 10% chance of winning each remaining game, and I think that's low.

Detroit needs a team with a bad offense to keep the score low and close, hoping to get a lucky field goal at the end to win it. As a UVA football fan, this was something I became very familiar with last year. Look at Minnesota. Detroit lost to them 10-12 @ Minnesota in October. They'll get Minnesota at home on Dec 7. Minnesota also has the worst special teams according to FO stats... and ST happens to be Detroit's strength.

I think people need to let go of the "DVOA doesn't reflect reality" canard and instead look at it as "the Eagles and the Packers are underperforming". They're underperforming DVOA and they're underperforming their pythagorean W-L. You can look into why you think they're underperforming, and whether you think that will continue or not - but it's not necessarily that DVOA is "wrong".

Raiders behind Bengals by one spot and behind Seahawks by two spots. This is not right. Raiders have better record than bengals and are just better than Bengals in any way you analize football.
Seahawks much crappier team than Raiders. Seahawks cant do anything right and is amazing they have two wins. They beat 49ers in Singletary pants game. Big dela. Then other win Seahakws have was vs Rams when win game in overrtime. Big whoop.
Raiders beat Jets. Very good win. Also beat Chiefs on road and that counts for soemthing. Also if noticed, Chiefs playing much better now. almost beat Jets and Chargers and some others. So beating Chiefs 23-8 has to be in Raiders favor.
Raiders going to beat Chagers in December 4 game and move on up to the teen side.

Raiders can still make playoffs. Must win divisiona. Only 4 games behind Broncos. Can get to only 3 games behind Broncos if win on Sunday. ?if broncos win thren Raiders eliminated from winning dikvidion. all thatwwould be left id for ZJ Ruissell to get ,more playing time and have D McFadden finish strong and get offensivre rookie of year award. Then Raiders have to bring in proven winner coach. Good chance Bill Cowher or Jim Fassel next Raiders head cach. Raiders to scary many teams in 2009. The third time is the charm and next year is third year for J Russell.

If you think Bill Cowher is coming to raiders anytime soon, you HAVE drunk to many sierra nevadas. I'll bet you a six pack the only two jobs he would consider (IF available) next year would be Cleveland or Carolina. Again, I applaud your loyalty and think the AFC west is likely to remain a weak division for the next several years. However, only a few short months ago you were banging your "raiders to the playoffs in 08" drum rather loudly, so excuse me if I doubt your claims that '09 is their year. Unless, of course, the shriveled shell of Al Davis is actually entombed in the ground. Then again, he might figure out a way for his corpse to run the team...

Hold on a minute. Sympathy for the football fans, sure, we can take that, but you can't possibly compare us to what other fans have to go through. Defending Cup champs (four titles in 11 seasons/12 years), NBA contenders (six straight final four appearances and one title), and a team two years removed from a World Series appearance. (There are also WNBA titles to consider ...) The NFL team is the only one that truly, thoroughly stinks right now.

If you're an NFL-only fan, sure, Detroit's not the place to be, but aside from that, we've been doing just fine, thanks ... in sports terms, that is.

And here we are again, a great team - statistically speaking - and a poor record to show for it. What's the old football saying - offense is skill and defense is scheme? Maybe the Iggles need to revise that - blowouts are skill and close losses are coaching?

I don't see what the big deal is about the Eagles. DVOA says they should be 7-3 and they are 5-4-1, Detroit has underperformed their estimated wins by 2 games, the Jets have outperfomed by almost 2 games. There is simply nothing exceptional about what the Eagles have done, it just looks worse since their DVOA rating is so high.

Also, it seems to me that Aaron gave us the answer. The Eagles woefully underperform on 3rd down. If I remember correctly, that is largely considered to be just luck(when 3rd down performance deviates dramatically from 1st and 2nd down performance), and is in fact a strong predictor of offensive improvement the following year(so you can all look forward to yet another very optimistic Eagles projection for 2009 !).

The problem is that since 3rd down is such a high leverage situation, bad luck on 3rd down can be disasterous to an offense. Mix in the high profile nature of some of the Eagles short yardage failures and you now have a situation that is out of control and has gotten into the heads of the coaching staff and the team. In other words, I think what was bad luck is quickly turning into a neurosis in Philly that is destroying the season, and DVOA can't possibly measure that.

I have a suggestion. Just add an extra column to the table (name it Call-in Radio Analysis Power Production Efficiency Rating, or CRAPPER for short), where you just rank the teams by W-L record, using DVOA to break ties. You could even add a special caveat where the Eagles are never allowed to be ranked higher than the Cowboys, no matter what.

This would make everyone happy. The Dallas homers could use the CRAPPER rankings (along with Jerry Jones' boldly objective predictions) as evidence of how great their team is and how much of a lock they are to make the playoffs, while those of us who understand the concepts of sample size and short-run statistical variance could still enjoy the DVOA rankings.

"Question to Jones: Obviously you see the positives of everyone coming back, can you see the Cowboys in the playoffs?

For the record, Jones was asked if he could “see the Cowboys in the playoffs.”

His answer, verbatim was this:

“Yes, absolutely. That’s not optimism. I just see that we were going to have to be a team that is playing well and won a lot of ballgames at the end of the year to be what we want to be anyway and this is as good a time to start as next week. I certainly do feel we’re going to be a team that plays well enough to be thinking about the playoffs.”

I understand the concepts. They just aren't put into use as correctly as they could be. Forgive me for being critical of a system that was wrong on the two teams last year and will likely be wrong on them again. Let's not fix it, let's just act like the system is perfect and brush off any criticism as "Dallas homer-ism." Man, you sure sound smart! Glad you set me straight.

One thing I'm curious is how "iterative" DVOA is. I recall a couple of years ago when the Colts broke the system because of their pathetic schedule, the solution was to iterate the DVOA a second time and make a "second order" DVOA...i.e. you used to use VOA to estimate opponent adjustments, and compute DVOA from that, but now you iterate a second time and use that first DVOA estimate to re-estimate opponent adjustments and then re-calculate an updated DVOA.

Do you iterate more than that (i.e. to convergence, or at least several times more)? I.e. you you then take the updated DVOA, re-calculate opponent adjustments, use that to re-calculate DVOA, and repeat n times until DVOA stops changing? Or are you still using just the "second order" DVOA? I can't find it conclusively explained in the "Methods" part of the site (but I confess..I haven't looked lately...since the major overhaul).

If you're not iterating, I would be curious to see if further iteration might "fix" the problem with the Eagles and Packers, or at least move things around a little...

"Life isn't fair and football seasons with tiny samples are even less fair. Personally I have no idea what Dallas fans are upset about? Wouldn't you rather be 6-4 and have negative indicators from mathematical models than 5-4-1 and have positive ones?"

According to PFP2008, I should be very discouraged by a team that is surpassing its expectations (aka the Cowboys) because they are scheduled to regress soon. It would be better to be the Eagles than the Cowboys right now according to the book.

Forgive me for being "homer-ish," but when your team continues to beat the formula and your rival continues to fall short of the formula, it's frustrating. How many years will the Eagles be projected to be the next coming of the Patriots and the Cowboys be projected to be a bottom-dweller in the NFC East? In the Week 3 Audibles, Aaron said something to the effect of "I don't know why DVOA doesn't like them, but the Cowboys are a really good team." Combine that with Philly, Washington, and now Green Bay being overrated (in my opinion) and I see a formula that needs work.

My point here is not to complain about how the Cowboys are better than the Eagles. My point here is that the formula needs to be worked on because it has some glaring flaws. For example:

I'd like to see something more tangible than a horrible variance when it comes to teams who have short-lived but majorly performance-altering injuries. An idea for this would be decreasing the weight of the games missed by the player(s) who clearly make the team move. For example, I don't think that the three game stretch without Tony Romo should have as much weight in assessing the Cowboy's offense on the whole as a three game stretch with him. It just messes up the grading system and throws off the opponent strength adjustments. For example: Let's say that Peyton Manning was on fire this season, but missed the three games leading up to the Ravens game. During those three games, the Colts offense dropped from 3rd in DVOA to 25th. Peyton comes back and torches the Ravens who were let's say #1 in defensive DVOA coming into the game. The Ravens' defensive ranking would plummet because they got demolished by what averaged out to be a bad offense due to the time with Jim Sorgi (or some "Colts Quarterback"). I see that as a flaw in the system. I understand why the Ravens plummeted, but that's not a correct assessment of who they are as a defense. That's why I feel that injury-stretches like that should be weighted less-heavily than normal games in terms of a team's total DVOA.

I'd also like to see something more in terms of field position when rating a defense. I've tried to research this around the site and in the book, but nothing really stands out to me as answering my question. Let's say, for example, that a defense continually starts drives in their own territory, whether it be from offensive turnovers or horrible special teams. DVOA is set up to grade different field positions more heavily than others (a defense that gives up 20 yards from their own 40 is graded worse than a defense than gives up 20 yards from their opponent's 5 yard line). While that sounds like a good idea, it doesn't always work right (as far as I understand how it works, which I am somewhat confused about). I think that something along the lines of a drive length grade should be applied. Let's say Tennessee's defense starts every drive on their own 40. They allow 10 yards every time and allow a field goal from their 30 yard line every time. Is that really a bad defense? No, they allow only 10 yards every drive. That's amazing! But DVOA would not like that (as far as I understand it) because they are constantly giving up valuable yards and points. If they put up the same yards allowed from their opponent's 10 yard line every time, they would be hands down the greatest defense ever, because their opponent would never score a single time. See my point? That's why I think defenses should not only be graded on where they allow yards, but how many yards they allow total. I don't know if DVOA takes this into account, but that's my idea. I can't find anything about that concept anywhere though.

"Also, it seems to me that Aaron gave us the answer. The Eagles woefully underperform on 3rd down. If I remember correctly, that is largely considered to be just luck(when 3rd down performance deviates dramatically from 1st and 2nd down performance), and is in fact a strong predictor of offensive improvement the following year(so you can all look forward to yet another very optimistic Eagles projection for 2009 !).

The problem is that since 3rd down is such a high leverage situation, bad luck on 3rd down can be disasterous to an offense. Mix in the high profile nature of some of the Eagles short yardage failures and you now have a situation that is out of control and has gotten into the heads of the coaching staff and the team. In other words, I think what was bad luck is quickly turning into a neurosis in Philly that is destroying the season, and DVOA can't possibly measure that."

I understand the concepts here. The fact that first and second down greatly determine third down performance is good logic. But if a team is just downright wretched on third down very often, there needs to be some compensation for that. Long third downs are low-percentage in terms of success. Short third downs are much higher in terms of success percent. But when you have a team like the Eagles that is so bad in short yardage, you have to adapt to their flaw and rethink the system. Maybe they need to have their own formula that stresses third down performance or something. I don't know. But the goal is to be as correct as possible, and the formula is not perfect. That's my entire point. If a team beats the system, it's the system's responsibility to beat the team back.

These are just a couple ideas I've had about things since I bought the book and have been coming to this site. I like the system, I really do. And I realize that it's not opinion-based in terms of ranking the teams. So I understand why Dallas is 22nd. My point is that Dallas is not really 22nd, and the Eagles are not really 3rd, so I think that would mean the formula needs to be re-evaluated and tweaked. The goal is to be as accurate as possible, is it not?

By the way, if anyone can clear up some of my questions regarding the formula and such, I'd appreciate it. It's impossible to accurately assess a formula you don't know everything about.

Thoughtful post Key. You articulate in a much more eloquent manner what I have responded to in this thread, which is to push back when I see blind allegiance to a system instead of what the Japanese call Kazan: a commitment to constant change and improvement.

"I'd also like to see something more in terms of field position when rating a defense. I've tried to research this around the site and in the book, but nothing really stands out to me as answering my question. Let's say, for example, that a defense continually starts drives in their own territory, whether it be from offensive turnovers or horrible special teams. DVOA is set up to grade different field positions more heavily than others (a defense that gives up 20 yards from their own 40 is graded worse than a defense than gives up 20 yards from their opponent's 5 yard line). While that sounds like a good idea, it doesn't always work right (as far as I understand how it works, which I am somewhat confused about). I think that something along the lines of a drive length grade should be applied. Let's say Tennessee's defense starts every drive on their own 40. They allow 10 yards every time and allow a field goal from their 30 yard line every time. Is that really a bad defense? No, they allow only 10 yards every drive. That's amazing! But DVOA would not like that (as far as I understand it) because they are constantly giving up valuable yards and points. If they put up the same yards allowed from their opponent's 10 yard line every time, they would be hands down the greatest defense ever, because their opponent would never score a single time. See my point? That's why I think defenses should not only be graded on where they allow yards, but how many yards they allow total. I don't know if DVOA takes this into account, but that's my idea. I can't find anything about that concept anywhere though."

You've misunderstood DVOA in this regard. In your example, the Ravens defense would be judged on how well it performed relative to what you would expect given it's context (caliber of opponent, time remaining, score, and field position). So, if a typical defense starting on its own 40 should expect to give up 25 yards while the Ravens only give up 10 every time, then Baltimore would have a great defensive DVOA. That's why it's called "Value Over Average."

This is a strength of DVOA, not a weakness. The formula's cognizance of field position means that a defense joined to a terrible offense and special teams unit doesn't get penalized for the higher resultant points per game allowed.

I see what you're saying. Thanks for bringing up those points. It changes my opinion a bit, but I still have some more questions:

I thought I read somewhere in the book that there were zones on the field in terms of making calculations based on where drives start, but I'm not sure. Is this true? I mean I know that they said that each yard is graded differently, but how many defensive drives start on their own 37 yard line? That's kind of hard to make league-wide comparisons with. In the zone case, defenses would be graded in comparison to others who start drives in the same zone. If so, I like that, but I still would say that short defensive drives should have slightly less weight than long ones, though. Or does the points system restart in each zone/remain cumulative throughout? For example: Let's say the hypothetical zones are 0-20, 21-40, 41-41, 40-21, and 20-0. If a drive starts on the offense's own 20, and they drive to their opponent's 40 and are stopped, they earned 40 yards and the defense allowed 40 yards. They drove from the edge of the first zone to the start of the fourth. Are they graded simply in terms of how far they gained from the 20, since that's where they started (for example, they earned x number of points on the drive starting from Zone 1), or do they get graded by each zone individually and cumulatively? Like, they passed through the 2nd and 3rd zones, so they would have good points in those zones, but bad in the 4th zone because they stopped. So overall, they would get the credit for going 40 yards, but would still have bad points in the 4th zone. Which system is in use here?

My main question (if the previous paragraph was too convoluted and confusing) is whether or not long drives are graded harder than short drives (defensively at least), and if they are, how does that work?

I also have a question about success %. I'm not sure how the percentages are distributed defensively. Let's say a MLB makes 80 tackles all season, but each of them came on 7 yard runs on first down. Does that player then have a 0% success rate? Or is the lack of success put on the defensive line? If so, does every defensive lineman get the blame? Or only the ones the play went between? Does the D-line and the LB get bad credit? Do the other LBs get bad credit as well? Are teh safeties and corners affected as well? This is probably the most confusing part of the entire system to me. If anyone can explain this, I would appreciate it.

The short answer is that DVOA is built around plays, not drives. First and ten from the 37 is compared to other first and tens in the 20-40 zone, while 3rd and 8 is compared to other 3rd and 8s. Where the drive (or even the previous play) started is irrelevant.

As far as the injury question you asked above, your concerns are entirely reasonable, but you're probably asking too much of the system. One big problem is that beside QB, the play-by-play doesn't say who's on the field for any given play (and game charters can't pull it off the video), so you can't compute DVOA with and without the starting left tackle. We all understand the difference between the Cowboys with Romo and the Cowboys without him, but both sets of plays took place, and DVOA reflects that.

When Aaron says:

"Other than that, it's hard to find a place where we could somehow tweak the DVOA formula based on a belief that the accuracy of our ratings for Philadelphia and Green Bay needs to be improved. (That's not meant to be obnoxious -- we're really trying to find a way to improve our ratings to fix the 'Eagles problem,' and it's damn hard.)"

he's serious. If he can find an adjustment that will make the Eagles more accurate without making numbers for the rest of the league less accurate, he'll be thrilled to do it.

(Disclaimer: I am an Eagles fan. I do not believe that they are playing like the third-best team in the league by any means.)

Regarding tweaking the formulas to better account for injuries: it's one thing to say "The dropoff from Romo to "Dallas Quarterback" is huge, so let's account for the difference when assessing their performance," but what about a team missing their Pro Bowl center and replacing him with a rookie free agent center? Not all positions are as easily quantified, despite the fact that they may be as important as any other on the field relative to their team's DVOA.

Also, DVOA is measuring an entire team, not just the players who would start if everyone were healthy. If your starters are all Pro-Bowl caliber, and your backups are all Loser League all-stars, injuries will lead to crappy players dragging down the team's DVOA, and I don't know that that's a bad thing. Anyone can say that the Cowboys are worse with Brad Bollinger starting instead of Romo; that's not something you need a complicated metric to tell you. DVOA is, in essence, penalizing the Cowboys for the games in which they started terrible quarterbacks since they had no better options, and I don't believe that's something that should be corrected for. (I do recognize that there are not that many quality quarterbacks around, and situations like the '06 Eagles [featuring Jeff Garcia as #2 QB, a guy who is currently starting in TB] are the exception rather than the rule. Maybe the Cowboys legitimately had no chance to sign a better backup QB, but that's not DVOA's fault.)

I also question the idea that the third-down performance metric should be reworked to account for the Eagles. It's been mentioned numerous times that third-down performance that isn't in line with first/second down performance (both on O and D, and either lesser or higher) tends to even out over time. Plenty of other teams have performed oddly on third down thus far (history-of-DVOA, not just this season), and until someone can demonstrate that the Eagles are somehow different from every other team in that category, I'm inclined to think that they will find their way back to the mean eventually. (It may well be by regressing on first and second rather than improving on third, but that would also sort out the situation in an acceptable manner.) Maybe the Eagles are possessed of an unprecedented ability not to convert third down, in a way that persists over multiple seasons, is specific to them only, and cannot be explained simply by variance or other random statistical noise. I'm not going to say that it isn't the case, but without a more analytical look at the data in that regard, I don't think the data merits modifying DVOA to account for one team.

Philly's #3 ranking is a little deceptive in that the gaps in DVOA aren't consistent. The Giants and the Titans are 1-2 by considerable margins, while the next four in the rankings are grouped pretty tightly.

The only reason that the Eagles didn't fall further than the third spot is that the previous week's 4-6 (BAL, ARI, PIT) all had disappointing outings. Even so, Reid's boys are just about as close to being 7th as they are to being 2nd. This jibes pretty well with my subjective impressions-- there's NYG and TEN, and then there is a muddle of "pretty good" teams filling out the top ten that could feasibly be ranked in lots of different orders.

If you're looking for a test of how much the guys in Vegas read Football Outsiders, check the line when the Panthers go to Green Bay in Week 13.

I don't think it's earth-shattering that the Week 13 line will likely reflect that Carolina isn't as good as its 8-2 record, and Green Bay is better than its 5-5 mark. Just looking at the point differentials of the two teams will tell you that.

Important distinction: Just because someone else comes to the same conclusion as you doesn't mean they copy off your paper or use the same methods.

More important than that is the fact that Vegas isn't in the business of predicting winners. They are in the business of getting as many people to gamble as possible. They make money when they can get enough people on each side to feel that the gamble is a good one and to put their money on the game.

So, even if Vegas thinks that the game should be close, they may set the spread high just to get a lot of people to place bets and to make more money.

if your logic is 'the Titans won't be able to up their level of play to beat a hot team in the playoffs' ... you also must acknowledge the corollary that the titans are unlikely to come out in the playoffs and turn in a stinker, beat themselves etc. I would suppose they're just as likely to not face a 'red-hot' as they are to get rolled by one. Although I can't recall any number-crunching re: teams upping their DVOA in playoff games... is it all a wash?

Actually, having one of the top DVOAs and a low variance is the formula for winning in the playoffs. You want to put up a DVOA close to your average every week, because when you play at that high level, almost no teams can beat you.

In this specific instance, the Titans have a DVOA of 30.0% and a variance of 3.2%. (I'm assuming that variance is the standard deviation, or else squaring it would result in way-too-high standard deviations for some teams. Someone please correct me if I'm wrong.) This means that 65% (I think) of the time, you would expect the Titans to have a single-game DVOA between 26.8% and 33.2%. It also means that over 80% of the time, the Titans will have at least a 26.8% DVOA. That's superb.

Conversely, look at a team like the 2006 Bears. They finished the season with a DVOA of 24.0% and a variance of 28.5%. That means that 65% of the time, you would expect their single-game DVOA to be between -4.5% and 52.5%. Likewise, roughly 20-25% of the time, you would expect a below-average performance from them.

So, 80% of the time, the Titans would have a 26.8% DVOA. Only one other team this year averages greater than 26.8% DVOA: the Giants. Only ten other teams have a DVOA within 17%, the home-field adjustment (though the Tians will almost assuredly have home-field advantage in the playoffs): the Giants, Eagles, Ravens, Packers, Cardinals, Steelers, Buccaneers, Panthers, Colts, and Redskins.

Now, of course, those teams all have variances as well. How many teams have a DVOA plus variance greater than the Titans' DVOA minus variance (26.8%)? Only eleven: the Giants, Eagles, Ravens, Packers, Cardinals, Buccaneers, Panthers, Colts, Jets, and Falcons.

What does all this mean? Glad you asked.
Let's say the Titans play a game at their lower bound of variance, 26.8% DVOA. There's roughly a 1/5 chance of this happening. The Giants, for simplicity's sake, have a 4/5 chance of playing better than this. The ten other teams listed above, for sipmlicity's sake, have roughly a 1/3 chance of playing better than this, cumulatively. Any other playoff contender (there are roughly ten) has a less than 1/5 chance of having a greater than 26.8% DVOA, so we'll say they have a cumulative 1/6 chance.

So if the Titans...
play at their lower bound, 26.8% (1/5 chance):
- 1 team has a 4/5 chance of beating them
- 10 teams have a 1/3 chance of beating them
- 10 teams have a 1/6 chance of beating them
overall, you get (1/21 * 4/5) + (10/21 * 1/3) + (10/21 * 1/6) = 27.6% chance they lose

play at their average DVOA, 30.0% (for simplicity, 3/5 chance):
- 1 team has a 3/5 chance of beating them
- 10 teams have a 1/4 chance of beating them
- 10 teams have a 1/6 chance of beating them
overall, you get (1/21 * 3/5) + (10/21 * 1/4) + (10/21 * 1/6) = 22.7% chance they lose.

play at their upper bound, 33.2% (1/5 chance):
- 1 team has a 3/5 chance of beating them
- 7 teams have a 1/4 chance of beating them (take away Packers, Jets, Falcons)
- 13 teams have a 1/7 chance of beating them
overall, you get (1/21 * 3/5) + (7/21 * 1/4) + (13/21 * 1/7) = 20.0% chance they lose.

Now, that's very, very, very, very crude. And when I say "lose to", I mean "have a lower DVOA than". But what you can see, is that no matter how well the Titans play, they have roughly the same chance of losing to a random team, at that chance is only ~25%.

Now, would they be likely to have a great game to outplay the Giants? Probably not. But they are also not likely to have a bad game to be outplayed by any other NFL team. They're unlikely to lose until they would meet the Giants in the Super Bowl, and the Giants have a relatively high variance and are therefore more likely to get upset along the way.

"They're unlikely to lose until they would meet the Giants in the Super Bowl, and the Giants have a relatively high variance and are therefore more likely to get upset along the way."

I don't know if I would use variance the way you did. For example, rounding to the nearest full percentage point, the Giants have had 2 games out of 10 below 30% total DVOA. The Titans have had just 4 games out of 10 greater than or equal to 30% DVOA (which implies 6 that did not).

The main reason for the high Variance for the G-men is their stink-fest vs. the Browns. That matters, but to what extent? I would argue that in the context of what you are arguing, not all that much. By my estimation, the Giants and Titans have both played 2 games where they could have lost to a playoff (caliber) team playing well-- the Giants' games vs. Cle and Cin, and the Titans' games vs. Bal and first vs. Jax. The main differences: the Giants would likely have lost to a playoff (caliber) team that was playing poorly if the Giants play like they did vs. Cleveland, and the Giants have had quite a number of very very high DVOA games where, if they performed like that, they wouldn't lose even if the other team is playing very well.

Take that together, I don't think I agree with you on the premise that the Giants are more likely to get upset due to their high variance. Especially given the very small sample sizes on which the variance is determined.

Yeah, I probably overstated the Giants' variance. I more intended my post as an exercise to show that a good team should prefer a low variance.

What's nice for the Giants is that their DVOA is so high that a medium-tier variance like they have is not very critical. There are still only twelve teams that have an upper bound greater than the Giants' lower bound, which is only one more than Tennessee.

Key19, I'm not sure what you are so worked up about. Remember that DVOA is only meant to be a statistical tool for people to use in forming their own subjective conclusions. The PHI/GB problem is something that the FO people openly admit is a problem, and they are trying to fix it.
As for Dallas being ranked so low, the team played three HORRIBLE games in a row. Of course DVOA is going to be down on them. That's why we humans get to use common sense and realize that the Cowboys will much better in the near future than their DVOA suggests. Dallas's low DVOA does not imply that the statistic is flawed. Rather, it implies that we should always consider statistics in context and that it takes subjective analysis of the data to interpret the hard data provided by statistics. Right now, there are very good reasons to assume that Dallas's DVOA will not be a good indicator of future performance. Any statistical measure will be wrong from time to time because no stat can take into account all the variables.
All that being said, if the FO writers wanted to indulge Cowboy fans, they could list Dallas' total DVOA (or at least the offensive DVOA) from the games in which Romo played in the commentary section. I suspect that that would give us a better idea of how the team will perform in the future. Then again, that might simply be too much work to be worth it for them. Nevertheless, I would love to see the "Healthy Romo DVOA" for this year.

Corollary to the third-downs argument, Eagles games also feature more plays and more drives than other top teams. Last week's drive stats show that Philly's defense has faced more drives than all teams besides three-and-out Chicago and three-and-TD Oakland. Its offense has run more plays than any other top-20 team. This might be attributable to its "pass-first" offense, as discussed in the Greg Cosell thread.

As a consequence, the Eagles' struggles in the red zone on offense are magnified, because their opponents have more opportunities to score.

First, pass defense. Only ten teams have above average (ie, negative) DVOA ... but nine of them are -11% or above. Typically as the season progresses, the numbers converge around zero. But this season the opposite seems to be happening. Do you have an explanation? Are interceptions being concentrated on an abnormally low number of teams?

Second, total offense. Some 21 teams are above average; 22 on weighted DVOA. There are as many teams over +10% as under zero. Do the base numbers need to be shifted? I remember reading this is an exceptional year for offense: do you have an explanation? Completion percentage? Third down efficiency?

Third, offensive balance. Of 32 NFL teams, fully 25 are either above or below average in both rushing and passing DVOA on offense. (Thanks, Ryan Grant!) I'm sure this is unprecedented. It suggests more strongly than ever that the two aspects can't be separated. This reminds me of the arguments about Indy and NE's 'overrated' rushing offense in previous years. For 2008, I'm disinclined to believe that Minnesota's run offense is inferior to its pass offense -- or, for that matter, to the Browns' run offense. Is there a way to untangle the two, to recognize that teams stack the box against Purple Jesus, to add an element of DPAR into DVOA?

If you look at history in the NFL, there is something remarkable about what the Eagles have done both this and last year.

Last year, the Eagles finished 8-8 with a +36 point differential. They are the only team since the 2002 realignment of divisions to end up in 4th place with a positive differential of points. This year, they are in 4th place with a +71 point differential.

They are also on track to score well over 400 points. No team has scored over 400 points and failed to go at least 7-9 (2004 Chiefs) with a positive point differential or 6-10 (2001 Colts) with a negative one. In fact, the Chiefs and Colts are the only 400+ point offense with a losing record since at least 1988. And the 400+ point offenses in recent years who ended up around 8-8 had the following totals and differentials in scoring:

2007 Cardinals at 402 points (+5 points) and an 8-8 record
2006 Cowboys at 425 points (+75 points) and a 9-7 record
2005 Chargers at 418 points (+106 points) and a 9-7 record
2004 Vikings at 405 points (+10 points) and an 8-8 record
2004 Chiefs at 483 points (+48 points) and a 7-9 record
2003 Vikings at 416 points (+63 points) and a 9-7 record
2002 Falcons at 402 points (+88 points) and a 9-6-1 record
2002 Saints at 432 points (+44 points) and a 9-7 record
2002 Chiefs at 467 points (+68 points) and a 8-8 record
2001 Colts at 413 points (-73 points) and a 6-10 record
1999 Panthers at 421 points (+40 points) and an 8-8 record
1995 Vikings at 412 points (+27 points) and an 8-8 record
1989 Bengals at 404 points (+119 points) and an 8-8 record

In order for the Eagles to fall into the middle of that range with, say, an 8-7-1 record, 420 points of offense, and 370 points of defense, they'd have to continue to score 26 points per game on offense, but start coughing up 30 points per game on defense over the next six games. However, they've been remarkably consistent in the first and second sets of 5 games so far. First 5 games, offense 25.4 points per game, defense 19.4 points per game. Last five games, offense 27.4 points per game, defense 19.2 points per game. It really doesn't seem like their defense is just suddenly going to collapse like that does it? So if they continue playing like they have been, they'll score about 160 more points and give up about 115 more points and end up with a scoring differential over +105 like the 1989 Bengals. But unlike the Bengals, they seem very likely to accomplish that without really big blow out wins (like 61-7) and no blow out losses (like 7-24).

This suggests that they might very well act like the 2006 Eagles. The 2006 Eagles were 5-4 with a +70 point differential, 4 close losses by 7 points or less, heading into a week 11 versus a perceived weak Titans team. After choking on the Titans and being rolled by the Colts the following week, they came back and ended up 10-6 and a +70 point differential. In other words, they finally played up to the supposed talent DVOA said they had when we got a full season's sample of games.

The other possibility is an epic scale collapse not seen for quite some time, like the 1994 Eagles. They were 7-2 with a +63 point differential entering week 11. After losing their last 7 games, they ended up 7-9, a 0 point differential, and looking for a new head coach, new owner, and new quarterback.

This team has the talent to be like 2006, and the neuroses to be like 1994.

Although you dropped the Worst DVOA Ever Watch feature, a record for futility may still be set in 2008. The largest number of teams with -30% DVOA or worse in one season is four, set in 1999. There have been no more than two since 2002. (Both of those years included expansion teams.)

Looking at the year-end figures for Rush Offense DVOA, the Giants are on track to be the third-best rushing team in the DVOA Era. Second is Kansas City in 2002 with 27.2, and first is St. Louis in 2000 with a mind-boggling 35.8.. The rest of the top ten is littered with more Chiefs and Broncos, as one might expect.

One thing to consider with the Eagles disparity is the "pass first and last" nature of Reid's game plan. Compare the Eagles approach (and notably the '07 Pats)with the Steelers under Cowher. In the 2nd half, the Eagles (and '07 Pats) would continue to throw regardless of the score. If the Steelers were ahead,Cowher would increase the ratio of run to pass. This difference in approach would tend to overvalue DVOA for Reid's philosophy vs Cowher's as there will be a larger point differential and a greater margin of victory. But the Cowher approach might lead to more victories. Cowher has a (well known) record of something like 100+-1-1 when leading by 10+ points. I wonder if this difference in philosophies is driving the difference between DVOA and record? Maybe a look at 1st half or 3 quarter DVOA's??

Second thought: on the Cowboys and the missing of Romo. DVOA, and other metrics, like final scores through the year, should, IMHO, be used to measure the performance of the entire 53+ man roster, not just the starters. Injuries count, and I recall someone (Hmmmm... I wonder who?) observed that the Cowboys had better than normal "injury luck" in recent years and speculated that might not continue.

"This difference in approach would tend to overvalue DVOA for Reid's philosophy vs Cowher's as there will be a larger point differential and a greater margin of victory."

I don't think so - DVOA doesn't deal in margin of victory. It deals in successful plays. So take an archetypical Cowher team that starts the second half up 14-0 getting the ball on its 20 yard line. It runs every play, gaining five yards on every first down, three on every second down, and two on every third down, and uses 40 seconds per play. It'll have a total of two drives in the second half, scoring a TD on the first (which will end at the beginning of the fourth quarter), and it'll likely win 21-7 -- and its DVOA will be through the roof, because it will have run something like 45 successful plays in the second half.

The 2007 Pats DVOA was through the roof not because they scored so many points; rather, the sky-high DVOA and the high point totals were both caused by the fact that the Pats continued with a very successful pass-first offense throughout the game.

Which still tells us very little about the Eagles. But I still claim that DVOA isn't "broken" - it's telling us that the Eagles and the Packers are both underperforming their "innate quality". And I think that the divergences between DVOA and end results are what makes things interesting. At least, as long as it's not my team.

"I don't think so - DVOA doesn't deal in margin of victory. It deals in successful plays. So take an archetypical Cowher team that starts the second half up 14-0 getting the ball on its 20 yard line. It runs every play, gaining five yards on every first down, three on every second down, and two on every third down, and uses 40 seconds per play. It'll have a total of two drives in the second half, scoring a TD on the first (which will end at the beginning of the fourth quarter), and it'll likely win 21-7 -- and its DVOA will be through the roof, because it will have run something like 45 successful plays in the second half."

I think the point is that sometimes there's a difference between a "successful play" according to DVOA and a successful play in real life. My understanding (and please correct me if I'm wrong) is that DVOA doesn't deal very well with game situations. To take one example, early in the 4th quarter the Ravens put together a sustained drive of runs and short passes. Which would have been great, except they were down by 17. Were those plays successful? Depends on how you look at it. So the archetypical Cowher team might be just fine with 2 and 3 yard runs that keep the clock moving, even if they don't go all the way down the field to score. When you're ahead, the most important thing is to avoid disaster plays. So "unsuccessful" plays which result in short gains for you or moderate gains for the other team might actually help you win.

DVOA doesn't deal in the margin of victory, but margin of victory is part of the argument being used as to why Philly's DVOA doesn't feel right.

Kurt is in the right direction of my thinking. But in addition to avoiding disaster plays, Cowher excelled at shortening the game when ahead. That is not part of the DVOA calculus.

The example that comes to my mind is: 2:40 left in the game, you are up by 2 at mid-field 3rd and 2. Opponent just used his last time out. If you get 3 yards running or passing, DVOA is happy and gives you an "attaboy." If you fail DVOA gives you nothing. But failing with run is better in real life than failing with an incomplete. If you gain 1 yard on a run, you punt after the 2 min warning and give them, oh, 1:50 to cover maybe 80 yards. An incomplete pass you punt at 2:30 they get the ball on the 20 with 2:20. 30 seconds and an official time out are not counted in DVOA land but are valuable.

ADDED IN EDIT: I guess what I mean is DVOA measures execution, not strategy?!

This is what i had projected since those games are close ones.
When Philly couldn't couldnt score tds, they made FGs as well as defensive stands to give their offense opps to score more points.

Now thinking ahead... say in Week 13. I would like to to know DVOA in terms of elite superstars.. say TO known for national TV scores like on TG and certain rivals (49ers & Eagles). May I propose a DVOA criteria for Thurs night, Sat, Sun Night, and MNF games? Seems like Thurs night games produce more points than SNF or MNF because the entire lineups require us to submit before 1st kickoff time of the week (Thurs night game). Since after 9/11. NFL been using Thurs night games in Week 11 to flex scheduling which aligns our divisional head to head matchups in Week 11 to 13.

"The short answer is that DVOA is built around plays, not drives. First and ten from the 37 is compared to other first and tens in the 20-40 zone, while 3rd and 8 is compared to other 3rd and 8s. Where the drive (or even the previous play) started is irrelevant."

We need a new formula for calculating the DVOA of the eagles. Possibly including variables such as understanding the rules of nfl football, or maybe even the fact that everyone watching the game knows they aren't going to do anything.

Team A, after 10 games:
5-5, with 4 losses coming by a TD or less, and all 5 wins by 14+ points. Ranked 3rd by DVOA, with a DVOA of 28.2%. By W-L record, Team A was third in its division. By DVOA, it was first in its division. [Embattled Head Coach] was accused of poor playcalling/time management, especially in close games. This alleged inability to win close games was frequently used to explain discrepancy between Team A's W-L record and DVOA.

2008 Eagles, after 10 games:
5-4-1, with 4 losses coming by a TD or less, and 3 wins by 14+ points. Ranked 3rd by DVOA, with a DVOA of 25.2%. By W-L record, Eagles are 4th in their division. By DVOA, they are second. Andy Reid is accused of poor playcalling/time management, especially in close games. This alleged inability to win close games is frequently used to explain discrepancy between the Eagles' W-L record and DVOA.

Team A, games 11-16:
5-1, with a 3-0 record in games decided by a TD or less. Had the division title wrapped up in time to rest starters for the final game. Turns out DVOA wasn't so crazy after all.

In case it's not painfully obvious yet, Team A is the 2006 Eagles. So, two years ago, when everybody said that DVOA needed to be fixed because it was overrating the Eagles so much, it turned out that no, DVOA was right, the Eagles really were the best team in their division. And now, in a virtually identical situation, everybody is saying that DVOA needs to be fixed because it's overrating the Eagles so much. But maybe DVOA is right this time, too, and the Eagles really are the second best team in their division.

So now we have the Eagles OC (whose name I can never spell right) disagreeing with the playcalling - the same OC who took over playcalling in '06 and ran off the 5-1 record - by indicating that the Eagles should run more.

Which we all seems to know, right? The Eagles should run more, which should improve all of their weaknesses. So, why don't they? Why does the Eagles Head Coach not pay attention to the information that all of us seem to know?

Oh, yeah ... He reads FO!

Hear me out on this. If you were to look at objective stats and find that you were the 3rd best out of 32, would you doubt yourself? Or would you believe that, no matter what the actual past outcome, your plan was correct for the future?

This really bothers me on a high level. When Chilly was around, and he complained about running the ball, Reid seemed to listen and follow his advice. Take the 2006 example noted by Alex51, and it's almost comical, right?

Let's get deeper, which as we all know, requires a Q & A - vesini style!

Q: Why does Reid think he does not needs a short yardage back?
A: The first rule of football is this - never get into a running game match with Jeff Fisher. But only slightly less know is this - FO says teams (in general) win with the pass and not the run!!! (except for short-yardage, but why quibble, right?)

Q: Why does Reid think he doesn't need a capable TE?
A: If only Reid looked at LJ Smith and said "You were this great colossus, and yet you cannot move the chains on 3rd down" ... And yet, I digress ... Reid doesn't need a TE when he can have an extra TE (or 2 or 3), 'cause throwing is the thing ... or don't you read FO?

I'm convinced this is the problem, so let's run an experiment. Change DVOA order next week to run as 32 to 1, then see if Reid gives up playcalling to Mornhinweg (thank you, eagles.com). It can't hurt to try, right?

The Eagles should run more, which should improve all of their weaknesses. So, why don't they? Why does the Eagles Head Coach not pay attention to the information that all of us seem to know?

Maybe it has something to do with the fact that their starting RB missed 2 games with injury (including one game that the team lost because they were stopped at the goal line after 4 running plays). They can't give Westbrook more carries when he's on the bench recovering from injury, and if they give his backups more carries, then they aren't exactly playing to the team's strengths anymore. Give them a healthy Westbrook in the Chicago game (or for that matter, for one series of the Chicago game), and they'd have a 6-3-1 record, and people wouldn't all be freaking out about how DVOA could have them so high. As it is, if Westbrook stays healthy, the Eagles should have no trouble running the ball, and they'll probably do so more often. Then, when they start winning more games, everyone will wonder why Reid didn't run more all season, ignoring the fact that his RB's injuries prevented him from doing that.