Most Recent Extra Points

Final 2010 FEI Ratings

by Brian Fremeau

Same song, different verse. For the fifth straight season, the BCS Championship Game was won by an SEC team that dominated on defense and was in total control of enough of its offensive possessions to win. Auburn's victory over Oregon won't likely be remembered as a classic, though it did match the 2006 Rose Bowl for the smallest scoring margin in a BCS title game. But whereas Texas and USC combined to score 10 touchdowns five years ago, goal-line stands and red-zone defense ruled on Monday night -- along with turnovers and pratfalls.

Until its final game-tying drive, Oregon's offense was forced to work (unsuccessfully) from poor starting field position throughout the second half. Both teams gained plenty of yards, but three drives concluded at the opponent's one-yard line and two others ended at or inside the 11-yard line. Had 35 points been scored instead of nine on those five drives, the game may have been more satisfying for viewers anticipating a shoot out. But the defenses were just as elite according to the opponent-adjusted FEI data -- Auburn's especially. And defensive tackle Nick Fairley produced the night's most prolific highlight reel.

I expected Auburn to win, and I'm satisfied when FEI forecasts do well. It would be bizarre to describe the end of the 2010 season as "inevitable" -- and yet, that word was at the forefront of my mind as the final seconds ticked away in Glendale. Was the game-winning, chip-shot field goal too anticlimactic? Was it the mythology of Cam Newton's destiny? Was it awe of the SEC? Have we already written this college football season postscript?

The mighty SEC won a fifth straight championship. And once again, a non-AQ team ran the table undefeated, won a BCS bowl, and won't be hoisting a crystal football of its own. (I suppose one could be commissioned, if TCU was interested in footing the bill.) Utah in 2008, Boise State in 2009, and TCU in 2010 -- what more could any of the three have done? Nothing really, not within the confines of their respective leagues and schedules and the BCS two-team playoff system. The Utes and Horned Frogs will join automatic qualifying conferences in the next two seasons, but inevitably, others will find themselves in the same predicament down the road. What, if anything, should be done about it?

To make my position clear on this matter, I don't think that any playoff solution will do. I agree with those who believe on any given day, TCU can compete with anyone else in the country. But I believe that Alabama can as well.

And the only reason TCU would "deserve" to be in a playoff and Alabama would not is because one plays in the Mountain West and the other plays in the SEC West. The difference between the two according to FEI cannot be overstated.

An elite team -- roughly equivalent to Alabama or Oregon by the FEI definition, two standard deviations better than average -- would have only a 5.4 percent likelihood of going undefeated against the Crimson Tide's 2010 schedule. The same team would have a 47.6 percent chance of going undefeated against TCU's schedule. Put another way, going undefeated against Alabama's schedule is roughly equivalent to going undefeated against TCU's schedule four times (47.6% x 47.6% x 47.6% x 47.6% = 5.1%) according to FEI; that is, 48-0. Put another way, it is roughly as difficult to navigate a typical SEC West schedule with one loss as it is to go undefeated against TCU's schedule twice. Put yet another way, it is roughly as difficult for an elite team to go 11-2 against Auburn's schedule as it is to go 12-0 against TCU's schedule.

It is critical for us to be honest about the stark differences between conferences and team schedules if we are set on designing a playoff. If automatic bids are part of the equation, there need to be enough at-large bids to accommodate for those differences. Would a playoff that includes TCU but not Alabama truly help us determine a more true and conclusive champion? How large would the at-large field need to be to ensure that the best teams in the country would get to participate?

Judging a team simply by the number in its loss column doesn't cut it absent the context of the strength of its opposition. I don't mean to discredit the accomplishment of TCU's undefeated season -- or Auburn's, for that matter. According to FEI, the Tigers were actually more fortunate to be undefeated than were the Horned Frogs. TCU earned 1.1 more wins than a team with its rating should have expected against its schedule; Auburn earned 1.6 more wins than expected. Both teams beat the odds this year.

Win Likelihoods Based On Final FEI Ratings

Team

0-Loss

1-Loss

2-Loss

Worse

Auburn

15.5%

33.7%

30.3%

20.4%

TCU

26.8%

42.0%

23.4%

7.8%

We're talking about a very small data set for what we're trying to accomplish. Identifying the two best teams in the country to meet for a championship game based on only a few hundred games is guesswork. I'll be the first to admit that a computer-based system like FEI is best used as a guide to assist that kind of guesswork rather than any kind of perfect solution. I try to provide as much context for our ratings to fill in the gaps, but I know what the challenge is. I call it "The Cloud."

Every little gray bubble in this graphic represents a lot of data. The x-axis plot of the bubble is Game Efficiency, the summary of the success rate of the team maximizing the value of its own possessions and minimizing those of its opponents in a game. The y-axis plot is the opponent's FEI rating, a metric produced from a complex algorithm developed over several years. The size of the bubble represents the relevance given that data point in the algorithm. Each of these metrics was ultimately developed for the purpose of identifying and publishing an ordinal ranking for 120 college football teams. I have to remind myself that we're talking about a cloud of data. Pick two teams to compete for a national championship game from that cloud. Pick eight teams, 16 teams, or 24 teams -- more teams doesn't make it easier, but it does push the likelihood towards 100 percent that the best team is in the field.

But I'm not a playoff advocate, I'm a regular season advocate. I harped on this last year, and I'm at it again. We need everyone to table the end-of-year playoff debate for a while until we fix the regular season. There were 90 FBS vs. FCS games played this year, 21 of which were played by Associated Press final Top 25 teams. And yet, in interconference play, only five regular season games were played between AP Top 25 teams. Adding in seven bowl games, a grand total of 12 interconference games were played this year between AP Top 25 teams -- fewer than any of the last 21 years, perhaps fewer than any in the history of the poll. We need these teams to play one another.

I don't bemoan the fact that bowl season has ballooned to 35 games spread over three weeks. I bemoan the fact that only one in five of those games featured a matchup of ranked teams. Twenty years ago, 10 out of 18 bowls featured a matchup of ranked teams. A bloated bowl season doesn't need dilute the level of competition. And the sponsorship money could be spent so much more effectively on more featured regular season games. Conference realignments in 2011 and 2012 will likely disconnect the power teams even more, so it is imperative that something be done soon.

All that said, college football still boasts the greatest regular season in sports. At the end of another terrific year, I thought it would be appropriate to pull together a few tables of the best offensive, defensive, and overall game performances of the year. Thanks for reading FEI throughout the season, and I'll be checking in with a few features (and new data visualizations) over the coming months.

Offensive FEI Single-Game Performances of the Year

OFEI

Date

Team

Opponent

Opp DFEI

Result

2.991

10/23

Navy

vs. Notre Dame

25

W 35-17

2.822

10/30

Houston

at Memphis

114

W 56-17

2.572

12/4

Auburn

vs. South Carolina

28

W 56-17

2.404

10/16

Arkansas

at Auburn

6

L 43-65

2.367

12/23

San Diego State

vs. Navy

75

W 35-14

2.298

11/20

Stanford

at California

66

W 48-14

2.296

1/1

Alabama

vs. Michigan State

32

W 49-7

2.251

10/23

TCU

Air Force

47

W 38-7

2.237

10/9

South Carolina

Alabama

16

W 35-21

2.184

10/16

Oklahoma

Iowa State

89

W 52-0

Defensive FEI Single-Game Performances of the Year

DFEI

Date

Team

Opponent

Opp OFEI

Result

-1.775

11/27

Ohio State

Michigan

2

W 37-7

-1.639

9/9

Mississippi State

Auburn

1

L 14-17

-1.614

1/10

Oregon

vs. Auburn

1

L 19-22

-1.582

9/18

Clemson

at Auburn

1

L 24-27

-1.452

10/2

Air Force

Navy

9

W 14-6

-1.419

1/4

Ohio State

vs. Arkansas

3

W 31-26

-1.414

11/13

Purdue

Michigan

2

L 16-27

-1.380

1/1

Alabama

vs. Michigan State

30

W 49-7

-1.330

9/18

UCLA

Houston

23

W 31-13

-1.308

11/6

Texas A&M

Oklahoma

13

W 33-19

FEI Single-Game Performances of the Year

GFEI

Date

Team

Opponent

Opp FEI

Result

0.928

11/6

Arkansas

South Carolina

16

W 41-20

0.876

1/1

Alabama

vs. Michigan State

39

W 49-7

0.871

10/23

Navy

vs. Notre Dame

23

W 35-17

0.834

12/4

Auburn

vs. South Carolina

16

W 56-17

0.827

9/11

Oklahoma

Florida State

18

W 47-17

0.804

11/20

Stanford

at California

83

W 48-14

0.776

1/3

Stanford

vs. Virginia Tech

5

W 40-12

0.731

11/6

TCU

Utah

47

W 47-7

0.727

9/18

Clemson

at Auburn

1

L 24-27

0.717

10/9

South Carolina

Alabama

3

W 35-21

FEI Final Top 25

The principles of the Fremeau Efficiency Index (FEI) can be found here. FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average. Strength of Schedule (SOS) is calculated as the likelihood that an elite team (two standard deviations above average) would win every game on the given team's schedule to date. SOS listed here does not include future games scheduled.

Mean Wins (FBS MW) represent the average total games a team with the given FEI rating should expect to win against its complete schedule of FBS opponents. Remaining Mean Wins (FBS RW) represent the average expected team wins for games scheduled but not yet played.

Offensive FEI (OFEI) and Defensive FEI (DFEI) are the opponent-adjusted ratings of all non-garbage-time drives from scrimmage. Field Position Advantage (FPA) is the share of the value of total starting field position for the season earned by each team against its opponents. Field Goal Efficiency (FGE) is the point value per field goal attempt earned by the field goal unit.

Only games between FBS teams are considered in the FEI calculations. The FEI ratings published here are a function of the results of games played through January 10.

FEI ratings for all 120 FBS teams are listed in the stats page section of FootballOutsiders.com. Click here for current ratings; the pull-down menu in the stats section directs you to 2007 through 2009 ratings. There are also now separate pages for offensive and defensive FEI ratings for 2010.

The Utes and Horned Frogs will join automatic qualifying conferences in the next two seasons, but inevitably, others will find themselves in the same predicament down the road.

The fact that they're joining an AQ conference is irrelevant: the Mountain West, as it was this year, would've become an AQ conference in 2012 if not for Conference Upheaval 2010 (which left it looking a whole lot like the WAC in terms of difficulty). TCU wouldn't've magically gotten in had the MWC been an AQ conference.

What is relevant is that Utah and TCU are joining much stronger conferences - Utah moreso than TCU, but the Big East is a better conference than "Mountain West 2010 minus TCU."

The whole discussion about college football's championship typically turns into "AQ conferences are the big boys who keep the non-AQ conferences out of the championship game," but that's completely wrong. The non-AQ conferences are what keep the non-AQ conference teams out of the championship game. The media and the coaches actually tend to overrate non-AQ teams - significantly, usually. The problem with college football is the FCS is too big - too many teams with too much variation in ability.

Pretending that Louisiana-Lafayette should even be on the field with Auburn - that's the problem. There's no way, with only 12 games, that you can figure out how to rank all the teams accurately. The only reason you even get close is because the conferences are as disparate as they are (so most games are against... well, closer competition).

Kinda awesome that Oregon's defensive performance was the third best of the entire year - better than Alabama's, even.

From a FEI perspective, should Auburn have won the NCG? I guess I'm asking which team performed better from a FEI perspective in that game, win or lose. From my observations it was very, very close and I'd be surprised if one team had a significant advantage over another.

ETA: I guess this is the answer?
652 1/10 Oregon loses to No. 1 Auburn 19-22

so if I'm reading this correctly, Oregon had one of the top 25 single GFEI performances of the year (compared to Auburn which did not). So from FEI's perspective Oregon did a hell of a lot better than they expected to? or they did well but lost due to weird issues that aren't predictive?

Auburn's performance against Oregon ranked 44th in GFEI. The Tigers had a better OFEI in the game than the Ducks and the Ducks had a better DFEI.

A better GFEI is supposed to mean that Oregon played better, adjusted for the opponent faced, than did Auburn. But I don't think that means they "should" have won. It means Auburn did not need to play as well as Oregon to win.

Fair enough; I was trying to get a similar corrolary to a team that has a higher dvoa than their opponent but loses.

I don't think that Auburn played a worse game than the Ducks or deserved to lose; like I said above, the teams looked very evenly matched, and a game that has lots of ebbs and flows and ends up with the team with the ball last scoring on the final play of the game to win it is usually a pretty close game. But it's interesting that FEI looks at the performances and views Oregon's as slightly better than Auburn's, though I suspect more of that has to do with opponent adjustments and less to do with absolute performance metrics.

Agree with premise that better bowl match-ups are needed, but what do you think is the maximum amount of games that can fall into your criteria of bowl-games between final AP Top 25 teams....
meaning once some of those teams around 19-25 start losing their bowl games (i.e. WV), they'll maybe drop outside of the final Top 25 with previously unranked teams taking their place (i.e. NCSt)

Even though it used to be 10/18, I still think the best we could get is 10/35 and maybe not that much with the amount of Non-AQ schools like Nevada, UCF, etc. that are getting into the top 25 but not pulling ranked teams in the bowls.

Also just a general question regarding FEI... sorry I'm sure you've answered this somewhere before, but in my mind I view FEI as one big attempt at turning the transitive property into rankings by accumulating data.

In this article you "bemoan" the lack of marquee interconference games. How much do you think that affects the accuracy of FEI. I would assume the FEI model would work best when a set of data points are all equally connected or evenly distributed to other data points. Whereas the college football model currently is clusters of data points (intra-conference) with few strands connecting them to other clusters (inter-conference).

I'm not even sure that this could be quantified, but if things keep trending the way they are, we could potentially end up with data clusters that are cut off from other clusters except for bowl games.

I agree with your assertion. If there were more interconnected data points and fewer intra-connected clustered data sets, it would make for better ratings. I think it would be a challenge to determine how much better it would be. Perhaps one way would be to compare closed-set data (a conference-specific FEI, for instance, eliminating all non-conference games) and figuring out a method for measuring its accuracy versus a non-conference FEI. Tough to find a way to measure that comparison, however.

As both a computer rating system developer and college football fan, I would prefer a world of smaller conferences, fewer conference games, and more independents. As recently as 1987, the SEC had only 10 members and played only 6 conference games. That made for some terrific inter-conference football.

I don't think the problem of the isolated data clusters impacts a computer-rating system exclusively. A poll voter, a playoff selection committee, etc would have the same problem making comparisons across conferences as a computer model.

That's true. They just "solve" the issue by ignoring connectivity by doing overly simplistic adjustments, such as opp record as a proxy for schedule strength (which is EXACTLY what the NCAA does for instance in its numbers).

I agree that the problem exists in polls as well, and I honestly prefer these advanced metrics than what is used in the wide world of sports media. However, there are some intangibles that can more easily be connected by pollsters than this computer system... injuries, suspended players, or just plain improvement.

Take FSU for example... OU dominated FSU's defense. But this was the first game with a new HC, new DC, new defensive scheme. Then UF/USCe gets to go against a defense that's had 11/13 games of experience (and USCe is without their best player). I'd imagine that the computer ratings can't adjust for the improved defense that UF and USCe faced verse the lost defense that OU faced, whereas pollsters could theoretically do just that.

Theoretically, humans can adjust for these things, but practically speaking it isn't going to happen. Humans can adjust for the big storylines blasted on ESPN, some additional relevant news that pertains to the team / league they follow, and that's about it.

I'd be surprised if anyone outside the ACC/SEC gave much of any thought to adjusting Oklahoma and South Carolina's performances against Florida St due to coaching experience, much less actually changed a ballot due to it.

To take one example, I'd be stunned if ANY voters attached to AQ teams / leagues (other than maybe Cal or BC) analyzed Nevada's defensive injury/suspension situation, much less adjusted votes for it. Anyone other than Kaepernick pretty much doesn't matter in terms of how people would see Nevada, unless there was a massive injury situation big enough to get serious ESPN attention... and even there I bet most wouldn't bother factoring it in.

Let's say, for the sake of argument, that Auburn is hands down the best team in the country. They play every team in the nation and beat every single one - they are that good. However, if they play the conference schedule of Western Kentucky and a non-conference schedule of Ohio State (2010 versions), the perception of how good they are would be tarnished - it doesn't matter that they went undefeated against that schedule, because it wasn't an SEC schedule (or any AQ conference). It is quite possible that TCU was far better than Auburn, and would have gone undefeated against Auburn's schedule in a more impressive fashion than Auburn did - but we'll never know. And because we'll never know, we don't really know which team is better, or who should be the Champion. Playoff advocates like myself simply expect that every team has an equal opportunity to win a National Championship, and that is simply not the case. The University of Houston may build the greatest team of all time in 2011, but that is irrelevant, they've already been eliminated from Championship contention. That is the fundamental problem with the current system, and why so many oppose it. It's kinda like saying the San Diego Chargers are eliminated from playoff contention in 2011 because the AFC West sucks.
I reject the argument that college football has the best regular season in sports, I believe the NFL does. I believe this for the exact reasons Fremeau cited as problems with the NCAA regular season - too many games that simply tell you nothing about a given team.
Commenter Pat's point that Division 1A (I refuse to use the propaganda title the NCAA insists on) is too big is right on. If non-AQ teams aren't given an equal playing field, then the NCAA should simply break 1A into two divisions, with the Big 6 conferences in Division 1A-1 and the others in Division 1A-2. Games between the two don't count in the standings. That means the Boise State's of the world get shut out of Championship discussion, but that's the only thing they get now - discussion. No real shot. Do this and the college season will improve. I know this will never happen, of course, but if non-AQ schools aren't getting a legit chance to compete anyway it is the right thing to do.

I have a big problem with wanting to give someone a shot at the national championship just because they might be a really great team, we just have no way of knowing based on their schedule. If that policy were ever adopted, what would stop a team like Florida from leaving the SEC, becoming an independent, and scheduling 12 home games a year against the worst teams in division 1? I have no doubt they'd run the table in that hypothetical, which would net them a national championship berth every single season (or a playoff berth, if we ever went to a playoff). Since Florida is Florida, I'm sure they'd win a credible percentage of those postseason games. Some years they might even be the best team in the country. None of that matters, though. In my opinion, you have to EARN your spot in the championship. It's not a birthright. It's not about how good you are, it's how worthy you are of inclusion based on your accomplishments on the field.

I really think that "percentage chance for an elite team to finish with this record or better against this schedule" truly is the best way to determine whether someone deserves a championship berth or not. If, as was the case in 2007, it's more impressive to suffer only two losses against LSU's schedule than it is to suffer no losses against Hawaii's schedule... well then, I have no problem with 2-loss LSU going to the championship while undefeated Hawaii gets sacrificed to Georgia in the Sugar Bowl.

I'm in agreement with you that a team should have to EARN their spot in a championship game and/or playoffs. The problem is half the teams in the country are not allowed to EARN their spot. On the contrary, it is, in the current system, a birthright that the other half of the teams can in any given year get in. Case in point is TCU - this season they had 0 shot, not because they didn't have a team worthy of it but because of the conference they were in. Next season they could be a shell of what they were this season, but by virtue of changing conferences they have as much a shot as any other AQ school. Boise St. has done everything they could to join better conferences, but since they haven't been invited into one of the AQ clubs they are defacto eliminated from the opportunity. I'm a playoff proponet, and believe every conference champion should get an automatic invite, whether it is from the MWC or the SunBelt. Do I think Troy is going to beat Alabama in the 1st round in that scenario? No, not at all (currently, that is - who knows what happens down the road), but they'd have their shot, something the Boise's and TCU's don't have now. There is a lot wrong with a system that gives Vanderbilt a chance but not TCU.
In your '07 example, I agree that LSU's performance was better than Hawaii's, but that's not the point. Hawaii was never, under any circumstance, a contender for the title, regardless of their schedule. Because in the final analysis, it isn't the schedule that matters - it's what conference you are in. Going back to my original example in the previous post, what if Hawaii truly was the best team that year and beat Georgia by 70? It wouldn't have mattered the slightest, just like TCU's win over Wisconsin this year means absolutely nothing. Auburn could have the exact same team they have now, but if they had played TCU's schedule they too would have been eliminating from contention before a game was ever played. The schedule presents a level of difficulty, it does not determine who the best team is.

The best suggestion I've heard for improving college football's postseason is the "midseason playoff". In this scenario, in weeks 6-8ish, teams would play marquee out-of-conference games determined by their win/loss record, conference standings or BCS rankings. These games would not be scheduled until the week before the games begin. One can imagine the excitement these games would create. Postseason play would stay the same - the BCS ranking system would continue to place teams into BCS bowls and the national championship. The difference would be better interconnectivity between the country's top teams.

Another improvement I would make would be to eliminate the AQ-status of conferences and remove conference tie-ins for the BCS bowls, and possibly all bowls. Wouldn't Stanford/Oklahoma and Virginia Tech/Connecticut(or a non-AQ champ like Nevada) have been much, much better?

Also, at this point, you might as well do away with the BCS computer rankings. They add absolutely nothing.

I suppose this is obvious, but you don't get top teams playing against each other because the penalty for a loss is so huge - the end of national title chances. That's one reason I favor an 8 team playoff, with only one representative per conference. Then a non-conference loss doesn't sink your year and schools should be far more willing to schedule good opponents.

I also like that such a playoff would keep the regular season conference games (the vast majority, after all) just as important as they are now.

And the only reason TCU would "deserve" to be in a playoff and Alabama would not is because one plays in the Mountain West and the other plays in the SEC West.

Who said that Alabama would necessarily be excluded from a playoff?

You're not making an argument against a playoff system. You're making an argument against a poorly-designed playoff system.

We need a playoff system to ensure that the best team be recognized. The current BCS system has two major flaws. First, teams from small conferences are almost certain to not get invited to the BCS championship game. Second, regular season games in the true power conferences (i.e. the SEC and whatever other conference might have depth in a given year) are far too important. That's especially true of early-season games. If three of the top five teams in the country are in the SEC, and they've all faced each other by mid-October, there's a fairly good chance that the best of the three will have at least one loss by then.

The current system is just silly for for too many reasons to ennumerate. There is no "champion", there is just a winner of an invitational game where invitations are issued by highly subjective criteria. The reason TCU isn't the "champion" is not because they ever lost to anybody, but because they weren't pretty enough to make the final cut.

Brian posted his FEI bowl confidence picks, which I tracked along a few other sources. FEI did VERY well, a statistical tie with my own system (FEI was just one single point less), both of which were nicely above the 99th percentile according to the ESPN bowl pickem game.

Thanks for doing this and posting the results. Like I said before the bowls, FEI has had mixed results in bowl picking. Great in 2006, 2007; weak in 2008, 2009. Glad the pendulum swung back the other way.