Tuesday, August 7, 2007

In this post and this post, I've found that the proportion of games won by the home team and those won by the "better" team (according to DVOA) is smaller for interconference games with a larger variance in the former proportion than for intraconference games. Games between divisional opponents have the highest of both proportions and the smallest variance in the former proportion. This led me to hypothesize that coaches are able to better adapt to teams they see more frequently, which makes inuitive sense, but I was also curious whether or not stats such as DVOA (or my VOLA stats) needed to be adjusted based on conference averages rather than league averages. Are NFC teams rewarded unfairly by DVOA for playing three quarters of its season in a conference full of weak teams? Ideally, opponent adjustments will filter out those effects, even if it's by league average. But perhaps 4 games against a stronger conference is not enough to filter out the effect. NFC Team X has played Y% better than league average when adjusted for opponent, but they were more likely to because of an easy schedule. Against AFC opponents, they'd be less likely to play at that level, even when adjusted for opponent strength. This effect might pop up more in games between average to below average opponents rather than when the NFC team is legitimately good. If the effect is real, then we'd see a greater proportion of interconference games being lost by the team with the higher DVOA when that team is in the NFC rather than the AFC.

I went back and looked at games from the entire history of DVOA (1996-2006). The DVOA totals used reflect the entire season's performance, including postseason. We should see the balance of power shift in favor of the AFC in 2000-6, given that only 1 NFC team ('02 Bucs) won the Super Bowl in that time span, and that team had a large and rare tactical advantage.

Year

% Was Better Team

% Was Better Teamand At Home

Better Team Win %Overall

AFC

NFC

AFC

NFC

AFC

NFC

1996

0.56667

0.43333

0.28333

0.21667

0.67647

0.65385

1997

0.56667

0.43333

0.25

0.18333

0.55882

0.53846

1998

0.55

0.45

0.28333

0.23333

0.69697

0.7037

1999

0.65

0.35

0.31667

0.16667

0.74359

0.57143

2000

0.65

0.35

0.31667

0.16667

0.64103

0.7619

2001

0.5

0.5

0.25

0.25

0.8

0.8

2002

0.59375

0.40625

0.29688

0.20313

0.63158

0.57692

2003

0.60938

0.375

0.28125

0.15625

0.66667

0.66667

2004

0.75

0.25

0.40625

0.15625

0.77083

0.5625

2005

0.65625

0.34375

0.32813

0.17188

0.61905

0.63636

2006

0.59375

0.40625

0.29688

0.20313

0.71053

0.5

The first table shows that the AFC might have been the better conference overall since at least 1996. For both conferences, the number of games in which their team was the better team is split evenly between home and away in almost every year, so neither conference's numbers should be unduly affected by home field advantage. For 6 out of 11 years, the AFC had a better win percentage when they were the better team than the NFC did when they were. In 2 of those seasons, the difference amounted to less than one game ('96 and '97). Looking at 2002-6, after the realignment, the AFC has had the better win percentage in 3 out of the 5 seasons. In 2003 and 2005, they were essentially even. In 2004, the NFC would have had to have won 3.33 more games in which they had the better team to match the AFC's win percentage. In 2006, they would have needed 5.4737 games. The predictive power of DVOA (where prediction = team with better total DVOA wins) was only 55.56% in 2006, compared to 66.67% in 2005 and 65% in 2004. Based on the table above, I'm guessing a large amount of the dropoff was because of interconference games, valuing NFC teams too highly given the gap between the conferences. That the year-to-year differences in better team winning percentage favor the AFC more strongly on average also indicates NFC teams being rewarded by a weak conference.

Year

Better Team Win %Home Better

Better Team Win %Away Better

AFC

NFC

AFC

NFC

1996

0.76471

0.76923

0.58824

0.53846

1997

0.6

0.81818

0.52632

0.33333

1998

0.82353

0.78571

0.5625

0.61538

1999

0.78947

0.7

0.7

0.45455

2000

0.84211

0.8

0.45

0.72727

2001

0.8

0.86667

0.8

0.73333

2002

0.78947

0.76923

0.47368

0.38462

2003

0.83333

0.9

0.52381

0.5

2004

0.80769

0.6

0.72727

0.5

2005

0.7619

0.81818

0.47619

0.45455

2006

0.73684

0.46154

0.68421

0.53846

The NFC teams see a bigger gap in better team win % for home and away games. The average difference is 18.517% for the AFC and 22.807% for the NFC. Small sample size has some effect for the games in which the NFC team is better, however. On average, the difference seems to be one or two games that could go either way.

So DVOA does seem to unfairly reward some NFC teams for playing in a weaker conference. Which teams are throwing off the system, though: the good teams being bumped up to very good or the mediocre teams being bumped up to good? In most years, the average gap in DVOA between teams in games in which the AFC was the better team and lost was larger than that for the NFC. And in most years, the average DVOA of the better teams that lost was higher for the AFC than the NFC. I would say, then, that the NFC teams that are mediocre to slightly above average (Rams, Falcons, Panthers) are unfairly rewarded more than good NFC teams (Eagles, Cowboys) for playing in a weaker conference.

Though DVOA is far more sophisticated than my methods, I believe I can apply the lessons learned here to my prediction model, specifically my VOLA stats. For intraconference games, I might start using Value Over Conference Average rather than VOLA. For interconference games, I could use Value Over Other Conference Average. So if Team A average 6.5 yards per play, it could be average in the AFC, but it might above average in the weaker NFC. Perhaps you could do a similar thing for DVOA, but the math of it is rather hazy to me right now.

5 comments:

I guess you read my comment from yesterday. We pretty much said the same thing.

I don't think adjusting for conference is the key, but it would help. It would affect teams differently. Some teams start off the year with 2 or 3 out of 6 inter-conf games. Some teams don't have their first inter-conf game until week 10. Were the first couple inter-conf games against doormats in an otherwise strong conference, or against the 2 strongest teams in the league?

I think the best way would be to have a very good opponent adjustment. Most strength of schedule adjustments are based on opponent records, but W-L records are full of randomness. I'd recommend game-by-game adjustments based on opponents' stats (not including the game against the team in question). But that's a lot of work. I don't know much about DVOA, but I suspect FO doesn't rely on just w-l records for their DVOA.

Now I'm doubting myself. Maybe a sound adjustment for conference is in order, at least toward the end of the season when we know which teams are any good.

I think your thought of creating value over conference is a good one. I'm not sure that it is a matter of DVOA systemically favoring a weaker conference, though. I think there are just more unusual results in AFC-NFC matchups that could skew the results of a rating system, sometimes in favor of overvaluing the better conference overall, and sometimes in favor of the weaker conference overall.

Here are the home records in intraconference matchups, sorted by DVOA difference, the first is AFC, followed by NFC, since 2002:

In some groups, the AFC was better at home, in some, the NFC was better. Look at the NFC group at -30.1 to -40.0, these are teams that are significantly worse than their opponent, and are 8-9 at home. Aaron Schatz of FO has said home field is around +15.0% DVOA on average. However, it appears stronger in intraconference games where the home team is the relatively worse team. Home Teams in intraconference games with a DVOA difference of -10.1 to -40.0 have a winning record 43-42. I guarantee you would not see that kind of winning percentage for worse teams in divisional matchups.

I think uncertainty and unfamiliarity is playing a role here. Thus, I would put less stock in an unusual result in intraconference play than in divisional or conference play.

Also, because there is far less cross over, one result can skew things. Take Oakland's 45-0 win over Tampa in 1999. I dare say that win was not representative of those teams strengths that year, it was a great game for Oakland and a poor one for Tampa. That happens in a conference matchup, and things even themselves out more, because of the number of common opponents. But it happens in an AFC-NFC matchup, and it might cause overvaluing of the AFC West in that case, and undervaluing of the NFC Central.

Special Content

About the Author

My degree is in computer science, and the football research started as an independent study in artificial neural networks. As a lifelong NFL fan, I wanted to explore the relative importance of different factors in winning games. Since the research is still nascent, I wanted to put it out in the public domain and hopefully find others interested in teaming up. Once it becomes profitable, though... I just hope the mafia families running Vegas don't come to hurt me.