FO Mailbag: Philadelphia's ALY/ASR vs. DVOA

It is, in fact, unprecedented. Right now the Adjusted Line Yards and Adjusted Sack Rate numbers go back to 1995, and in that time there has never been a team as good on offense as the 2013 Eagles despite taking tons of sacks and getting running backs constantly stuffed at the line. If we look at the difference between rank in ALY and rank in rushing DVOA, and then the difference between rank in ASR and rank in passing DVOA, we end up with a combined difference of 51 places in the standings. That blows away the previous record, set by last year's Packers, who had a combined difference of 40 places. No other team has ever been above 33.

And here's what's interesting... this was a huge year for offenses that were strong despite being poor in both of our offensive line stats. That team with a combined difference of 33 is this year's San Francisco 49ers. Two other teams have combined differences over 20, and both of those are also NFC playoff teams. Oh, by the way, Green Bay has a combined difference of 15, which means five of the top six teams in this "stat" in 2013 are NFC playoff teams. (The sixth would be Miami.)

Year

Team

Adjusted Line Yards

Rank

Adjusted Sack Rate

Rank

Run DVOA Rank

Run Dif

Pass DVOA Rank

Pass Dif

Combined Difference

2013

PHI

3.72

26

9.4%

31

1

25

5

26

51

2012

GB

3.86

25

8.6%

31

13

12

3

28

40

2013

SF

3.57

29

7.8%

22

14

15

4

18

33

2011

GB

4.05

17

7.4%

23

7

10

1

22

32

1997

CIN

4.05

16

8.9%

23

4

12

4

19

31

2010

PIT

3.88

19

8.6%

29

14

5

3

26

31

2012

CAR

3.49

30

7.6%

21

8

22

12

9

31

1996

CIN

3.72

24

8.1%

23

6

18

11

12

30

1999

OAK

3.98

11

8.5%

24

4

7

2

22

29

2010

GB

3.82

23

7.2%

21

10

13

5

16

29

2009

GB

4.25

9

8.6%

30

2

7

9

21

28

2007

JAC

4.10

18

6.9%

17

5

13

2

15

28

2013

SEA

4.05

9

9.6%

32

7

2

8

24

26

2013

CAR

3.91

14

8.2%

25

4

10

14

11

21

What's the explanation for this? Well, you could tie these four NFC playoff teams together by pointing out that they all have mobile quarterbacks with good yards per carry numbers. Since rushing DVOA considers all runs, but ALY considers only running back carries, that is part of the difference between the two. That doesn't do a lot to explain the difference between ASR and passing DVOA, however, since sacks are included in passing DVOA. Shady McCoy has always been a guy who gets stuffed at the line a lot, but Frank Gore really hasn't been, and the San Francisco offensive line was first in ALY a year ago. Seattle probably ranked dead last in ASR because of the injuries on its line this year, but the other three lines in question were fairly healthy.

It's also interesting to look at the other teams that had a big difference between ASR/ALY and offensive DVOA, because a lot of them happen to be the recent Green Bay Packers. I did this list as a top 12 rather than top 10 to show that the Packers are on here for the last four straight years. (Note that Seattle and Carolina are just added to the table to show readers their numbers; they don't actually rank 13th and 14th.) Aaron Rodgers has always had a great rushing DVOA, of course, and he also has been great at making plays happen despite a porous offensive line. But that still doesn't explain why the Packers kept getting better rushing DVOA than Adjusted Line Yards with running backs like James Starks and Ryan Grant.

The 1996-1997 Bengals are not a team I know much about. Jeff Blake was their quarterback until the end of 1997, when he got hurt and Boomer Esiason ended his career with an absurd five-game hot streak (13 touchdowns, two interceptions, 58.0% DVOA).

The other interesting thing to note is that the record for the opposite side of this "stat" was also set this year. Houston ranked sixth in ALY and 11th in ASR but ranked 24th in run offense DVOA and 30th in pass offense DVOA, a combined difference of -37. That broke the record held by the 2001 Carolina Panthers. This list, unlike the other one, doesn't include a lot of recent teams except for this year's Texans and Lions.

Could it be that we are expecting a normal distribution, with the median closer to the 0-4 yard mark, but with someone like Shady McCoy, it's boom or bust: a lot of losses and a lot of 11+? Then maybe you have good DVOA rushing but a relatively low AYL.

There is another RB that we tend to think of as boom-or-bust -- Barry Sanders. I just took a look at Detroit in the mid-90's. ALY stats only exist for the end of Barry Sanders's career. For those years, though, ALY was middle of the pack and rushing DVOA was near the top. So there was a difference, but nowhere near as profound as Philly's this year. Therefore, a boom-or-bust RB is probably only part of the story. The rest of the story is probably the fact that the Philly QB was Vick for much of the year (plus Foles did pretty decently running the ball as well).

DVOA *loves* it some QBs who can effectively scramble. They flat out bust rushing DVOA -- Aaron has admitted this is a problem and probably an uncorrectable one.

The early-1990s Detroit teams didn't have the vast split in part because Sanders was paired with some amazingly immobile QBs (Mitchell was once compared unfavorably with an unbalanced washing machine), and because Detroit's penchant for throwing INTs brought down their passing DVOA.

That said, all of FO's rushing analyses seem suspect in sniff testing. I have a suspicion DVOA likes passing so much that the relative effect of rushing is lost in the noise. It just doesn't have much power, so it can vary a lot without any real notice.

Isn't your last point really more of a statement about where the NFL game has gone rather than an indictment of DVOA? We've had lots of offenses in the last decade that were great offenses without being highly effective rushing offenses. Seems like that supports DVOA's treatment of the rushing game.

To me an interesting test will be to see how DVOA continues to stand up as it is pushed back into the eighties and late seventies. Will the offensive metrics still produce smell test passing lists as the timelines moves backwards.

I'm not saying DVOA doesn't rightly weight passing higher, I'm saying it weights it so much higher that it effectively cannot judge rushing success at all.

It used to think the Rodgers GB teams were rushing juggernauts. They could rush via smoke and/or mirrors, but utterly could not force a yard like a vintage Seattle, KC, NYG, or SF team. A less glaring example was the Reid Eagles -- they at least could run competently on rushing downs, but similarly were a disaster in goal-to-go or 4th-short rushing. DVOA really likes teams that run draws and similar 'looks like a pass' runs.

I'll give a college example. DVOA would really like FSU, but they don't run because they can run, they run because the defense is defending against Winston. Wisconsin lets you know they're going to run. They call ahead and make sure you're forewarned. And they'll still run for 250 yards on you. DVOA doesn't separate out these two styles.

(As a glaring counter example -- DVOA would probably really like the GT passing offense. All 7 passes per game of it.)

I don't think there's one single explanation for the gap between Green Bay's offensive DVOA and line stats. The line has clearly been a weak link for some time — Rodgers, the receivers and now Lacy have been prolific in spite of it. Runs from Rodgers and Randall Cobb have boosted the Packers' DVOA. Above all, since 2009, the Packers' running backs have a total of 16 fumbles between them (by year: 2, 2, 4, 3, 5) in 1806 rushing attempts. DVOA heartily approves of that.

There's also the matter of the RB. From 2009 to 2012, the running game was anything but boom-and-bust: the Packers were poor at breaking long runs, but they were stuffed less often than average in three seasons out of the four. Their ALY average, mediocre as it has been, was still superior to their RB yards. But now that they have at last found NFL-quality running backs — James Starks, DuJuan Harris and now Eddie Lacy — the gap has started to close.

I agree. This does not exactly prove the ALY/ASR stats to be invalid or worthless. But it certainly points out that they are easily fooled, at best. And likely they would benefit from the addition of some other variable(s) to fill in the gap which leads to them being "fooled" repeatedly and badly.

I think the reason the Lions have a bad rushing DVOA but good ALY is at least partly due to higher than average fumbles. The team had 29 fumbles and the average is just under 21. Not sure about the breakdown between passing and rushing play fumbles but I assume most are due to Reggie Bush on running plays. Interception (and QB fumble) rates could be doing a similar thing on the passing side. I know fumbles, even if not turnovers, are very bad plays in DVOA and require many good plays to overcome.

The table bothers me... You're using sums of an ordinal ranking, which doesn't account for the spread of the rankings. Wouldn't it make more sense to show the data plotted on an x/y axis?

In regards to the differences between the metrics, I'd imagine it's at least partially because DVOA accounts for turnovers and "situational football". For instance, DVOA wouldn't give you much credit for an 4 yard run on 3rd and 5, but DVOA would love 2 yards on 4th and 1. I'm not as familiar with ALY, but I'd imagine it doesn't work on the same way.

This isn't that surprising a result if you're familiar with Oregon football. Oregon was very much a boom and bust kind of running style, and regularly led the NCAA in 20+ runs and explosive plays running the ball. The goal in the Oregon system is to get running backs quickly past the LoS and into open space early while spreading the field so that there aren't many secondary/linebackers that can make a play in time. Watching the Eagles that's basically their plan too, albeit with somewhat different execution.

So you'll get things like run for 0, run for 1, run for 15. That's 2 failures of three runs, making ALY look like shit - but it moves the chains.

Another issue, I bet, is that ALY adjusts based on things like running from the shotgun. Well, most of Philly's runs are from the shotgun, so that'll also skew the results in some way until everyone starts doing it.

I never really thought ALY was a great measure of run blocking, more of a measure of how many men tend to line up in the box. Teams like the Brady Pats of the Manning Colts were always high in run DVOA because they rarely saw a loaded front while a great runner like AP had to contend with a free hitter in the box, leading to more stuffs but fewer defenders if he breaks through the first level.

ASR is a measure of the pass protection (which is more than the line), the quarterback and the scheme. Young passers also tend to take more sacks, which could be influencing the Eagles' high ASR. The Eagles also tend to use a lot of spread formations which invite the blitz.

The four teams listed from this year are all run heavy, will see loaded fronts and so will be more boom and bust.

As got the niners' decline in ALY, two factors are pertinent. The previous year the NFL was still catching up to the power/trap scheme from multiple formations that Roman installed. I think the rest of the NFL has caught up to some extent, though there have been some indications that Roman has a new iteration of this up his sleeve. Secondly, most teams that the niners faced last year were still giving a big cushion to Randy Moss, which would boost the ALY even though Moss didn't really deserve it anymore.

similar to Karl's thoughts... ALY/ASR might be measuring "defensive sellouts guessing correctly". Loading the box is part of it, but also zone/man and simply the amount of attention by defenders being paid to run or pass.

The 49ers offense, in particular, thrives on fooling you. Which is ironic, because the heavy run formation is *meant* to run over you even if you do load up against it. A spread the field offense with loads of run/pass options seems like it would be similar. But is it boom-and-bust by design or effect...

ASR is adjusted for down/distance - so it cannot be simply more throws in 3rd and Long. Certainly we know running QBs take more sacks as they try to scramble rather than throw it away. Possibly with precise game charting one could assign all scrambles to passing DVOA instead of QB rushing DVOA - and this would would expect counter balance somwhat. Certainly not very clean when there are a significant number of designed QB runs or options going around.

It is also intriguing that SFs line is ancedotally rated very highly yet is extremely poor in these stats. They were good in ALY last year but not great in pass protection.

It would be interesting to quantify the "obviousness" of a play call in a particular game state.