Stat Analysis

Advanced analytics on player and team performance

Historical Draft Efficiency: College Rankings

by Danny Tuccitto

Based on the comments sections the past few weeks, I'm glad to see the Football Outsiders readership enjoying this series. As I stated in the first installment, my aim here has been to try to put numbers on things the best I can, and thereby drum up an entertaining discussion. With franchise rankings and an objective take on the best and worst drafts out of the way, we can now turn to more granular draft splits based on value above expectation (VAE) and return on investment (ROI). Today, I'll stoke the fire that is alumni passion.

As a reminder, like FO's distinction between DYAR and DVOA, VAE is a measure of total value relative to baseline (in this case, the expected career value of a given pick), whereas ROI is a measure of percentage value (or value per pick, given expectations). Interestingly enough, within the same college, there's significant divergence between the two stats, possibly because of the NFL's almost-blind ambition regarding players from certain schools.

Before showing the stats table, let me first discuss a few colleges that didn't make the cut. Easily the most surprising based on its sheer number of picks is Nebraska. From 1970 to 2007, 181 Cornhuskers were selected among the first 222 in a given year. In the aggregate, those picks yielded -52.3 VAE and -13.6% ROI. Among the 70 schools with at least one draft pick per year, Nebraska's VAE ranks dead last, and their ROI ranks 64th. Despite having the fifth-highest expectedCareer AV per Year (384.5), only 38.1 percent of Cornhuskers were positive-VAE picks (average among this group was 40.7 percent). Basically, for every Roger Craig (+4.8 VAE, +181.4% ROI) and Ahman Green (+3.9, +185.3%), there are twice as many Bruce Pickens (-5.0, -84.2%) and Trev Alberts (-4.0, -75.0%).

Though not as much of a disaster as Nebraska, draft picks from USC haven't fared much better in the aggregate. From 1970 to 2007, Trojans were picked the most (200 in the top 222 from 1970 to 2007) and had the highest expected CarAV/Yr (491.6), but were essentially a wash in terms of relative value (+1.2 VAE, +0.2% ROI). Although it's true that USC had an above-average percentage of positive-VAE picks (43.5 percent), for every Anthony Muñoz (+4.3, +72.1%), there was a Matt Leinart (-3.2, -72.3%). For every Marcus Allen (+2.5, +55.9%) and Hoby Brenner (+1.2, +54.7%), there was one Ricky Bell (-3.5, -48.9%).

Nebraska and USC aren't even some kind of big-school anomaly. Among the top 10 colleges in terms of total selections, seven more didn't make the table with respect to top VAEs and ROIs. Here they are, in order of selections: Penn State (180 picks, -6.9 VAE, -1.7% ROI), Ohio State (177, -11.7%, -2.8%), Notre Dame (168, -11.8, -3.2%), Michigan (162, -2.4, -0.7%), Oklahoma (157, -33.2, -9.7%), Tennessee (153, -16.1, -4.7%), and Florida (144, +4.1, +1.2%).

Now you know some of the familiar schools that weren't among the best at producing draft value from 1970 to 2007. Luckily (for ease of presentation, at least), the top 21 colleges in terms of VAE also ranked in the top 21 of ROI. So, below is a table of the top 20-plus-one (min. 38 picks):

College

# Picks

VAE

Rank

ROI

Rank

Louisiana State

112

+39.5

1

+15.7%

7

Pittsburgh

100

+38.2

2

+16.9%

6

Miami (FL)

168

+36.9

3

+8.6%

12

Georgia

109

+32.3

4

+13.9%

9

Virginia

70

+27.3

5

+18.8%

5

UCLA

126

+27.0

6

+9.7%

10

Purdue

91

+26.7

7

+14.7%

8

Louisville

45

+25.7

8

+28.9%

1

Georgia Tech

53

+22.5

9

+22.7%

2

Texas A&M

128

+21.2

10

+7.6%

15

College

# Picks

VAE

Rank

ROI

Rank

Arizona State

114

+20.6

11

+8.2%

13

Jackson State

39

+19.7

12

+22.6

3

Southern Mississippi

40

+14.4

13

+20.5%

4

San Diego State

80

+12.6

14

+8.0%

14

Oklahoma State

51

+11.0

15

+9.5%

11

Maryland

68

+10.7

16

+7.4%

16

Auburn

106

+9.8

17

+4.1%

19

North Carolina

92

+7.7

18

+3.8%

21

Boston College

78

+7.3

19

+4.5%

18

Syracuse

73

+7.2

20

+4.6%

17

Missouri

61

+4.7

21

+3.9%

20

Miami is the only college among the top 10 in total picks to also be among the top 20 in VAE and ROI. It should come as no surprise that the most valuable Hurricanes come from the halcyon days of 1995-2001. Among picks that produced +3.0 VAE are Ed Reed (+5.2 VAE, +150.1% ROI), Frank Gore (+4.7, +204.4%), Reggie Wayne (+4.6, +143.4%), Ray Lewis (+4.6, +136.1%), Clinton Portis (+4.4, +173.9%), Warren Sapp (+4.4, +102.3%), Edgerrin James (+3.9, +88.3%), Vince Wilfork (+3.3, +91.2%), and Jon Beason (+3.0, +88.3%).

Lest we forget, however, those that came before them. Suprisingly, the best pick from the University of Miami in terms of VAE was way back in 1973, when the Minnesota Vikings selected Chuck Foreman 12th overall (+5.6 VAE, +129.9% ROI). In his first five years, Foreman made five Pro Bowls, was voted first-team All-Pro once, and averaged an absurd 13.8 Adjusted CarAV/Yr; he also spearheaded three Vikings Super Bowl appearances.

In contrast to Miami's slew of household names, LSU boasts less-heralded picks that produced at least +3.0 VAE. I mentioned the impeccably named Tommy Casanova last week (+6.1 VAE, +189.3% ROI), but first on the Tigers' list is a running back who had one of the best three-year stretches of the salary cap era. Domanick Williams (nee Davis), selected 101st in 2003 by the Houston Texans, gained at least 1,300 yards from scrimmage in each of his three seasons, good for 8.5 Adjusted CarAV/Yr. Another blast from the past is wide receiver Eric Martin (+6.3, +562.5%), who the New Orleans Saints took at No. 179 in 1985. To this day, and despite the team's pass offense exploits over the past two decades, Martin still holds the franchise record for receptions and receiving yards.

To further feed the narrative of Pennsylvania football, the Pittsburgh Panthers rank in the top six of both VAE and ROI, and have produced the highest percentage of positive-VAE picks (53.0 percent). In the top 222 picks of drafts between 1970 and 2007, they boast six Hall of Famers, with Darrelle Revis poised to make it seven if he successfully returns from his 2012 ACL tear. Among other Panthers that produced at least +3.0 VAE were Giants defensive tackle Keith Hamilton (+4.4 VAE, +242.9% ROI), Oilers cornerback J.C. Wilson (+3.3, +354.3%), and 49ers cornerback Carlton Williamson (+3.1, +133.5%). If you're wondering about Larry Fitzgerald, he's produced +0.5 VAE and +7.9% ROI thus far in his career.

The only other college with an aggregate VAE above +30.0 is Georgia. But unlike LSU, Miami, and Pittsburgh, the Bulldogs don't have a single Hall of Famer to show for their relative draft value -- at least not yet. I'll get into the topic more when I present the best and worst individual picks since the merger, but for now I'll simply point out that Jake Scott produced 10.2 Adjusted CarAV/Yr as the 159th pick in 1970 and Terrell Davis produced 9.9 Adjusted CarAV/Yr as the 196th pick in 1995. Davis has several more bites at the Hall of Fame apple, but Scott's omission is a total mystery to me after having run the stats for this series.

Finally, we come to the "Luke, I am your father!" moment, a point at which we must talk about the thing you least expected. Tiny Jackson State, a member of the Southwestern Athletic Conference, boasts less than one-fifth as many students as my alma mater, the University of Florida. Yet Jackson State shows up in the top 21 while said alma mater does not. With a mere 39 picks, these Tigers have the fifth-highest percentage of positive-VAE picks (48.7 percent), which includes two Hall of Famers: Walter Payton (+4.0 VAE, +72.7% ROI) and Jackie Slater (+2.1, +107.5%).

Interestingly enough, neither Payton nor Slater are among Jackson State's best three draft selections in terms of VAE. Leon Gray (+4.9 VAE, +234.4% ROI), taken 78th overall by the New England Patriots in 1973, was an eight-year starter at left tackle for the Patriots and Oilers, made four Pro Bowls, and was voted first-team All-Pro three times. Rickey Young (+4.3, +354.9%) was San Diego's pick at No. 164 of the 1975 draft, started at either fullback or halfback during his first seven seasons for the Chargers and Vikings, and led the NFL in receptions in 1978.

And then there's Jimmy Smith. Chase Stuart has already made a convincing argument for why Smith is one of the best wide receivers in NFL history, so feel free to click that link. In terms of FO stats, the 36th pick of the 1992 draft had the following DYAR rankings from 1996 to 2001: 4th, 10th, 13th, 1st, 11th , and 7th. After that bit of information, along with his draft value stats (+4.1 VAE, +137.6% ROI), I can't say I disagree with Chase all that much.

I've detailed a mere one-quarter of the VAE/ROI table, so feel free to ask questions and offer opinions in the comments section. Next week, I'll be finishing off this series with a look at the best individual picks from 1970 to 2007. Here's a teaser: Tom Brady is not No. 1.

UPDATE: Comments requested it, so I set up this link, which shows the VAE and ROI stats for all 70 qualifying schools.

While not impossible, any argument that a 6th round QB with Brady's level of historical efficiency (DVOA, DYAR, future HOF, etc.) isn't #1 must be backed up with some strong evidence to the contrary. Danny's been pretty sound in his logic historically, so I think the best approach is to wait and see.

Don't want to get into the issue here, but one general, purposefully vague point I'll make about it is that our memories tend to suffer from recency bias and the availability heuristic. The fact that it took several comments in last week's piece before someone finally guessed Terrell Davis to be among the most valuable picks shows that even the late 1990s are practically out-of-sight, out-of-mind already.

Of course, that's not the only reason (and probably not even the best reason), but it's an underappreciated one nonetheless.

I expected most of the names on the list(except Jackson State). I'm racking my brain about Maryland, though. Randy White and Boomer, OK, sure. But who else? Randy Starks and Eric Barton? Davis and Merriman were drafted pretty high, with Davis only coming on the last few seasons and Merriman out of the league.

Danny, I'll ask again, since I don't believe my question was answered in the comments on the first installment of this series.
When calculating VAE, is the draft pick being studied taken out of the average before calculating his value over said average? In other words, is player X compared to the average of the other 36, or the average of the 37? For players within ~1 standard deviation of the mean, this probably does not change anything (it might move them up or down slightly overall in both measures, but probably does not matter). However, for the huge outliers--Tom Brady and JaMarcus Russell, to name one at each end--this would make a large difference. Brady surely inflates the average value of his pick--thus making his VAE compared to the average of all 37 lower than if his VAE was compared to the average of the other 36. Russell deflates the average of the #1's overall, making him seem "less" of a bust.

Here's an idea that would make you have to redo all the numbers, so I don't expect it done any time soon. Add the number of yrs contributing to Career AV/yr to that CAV/yr. This will give more value to guys who averaged 3 AV for 12 yrs vs those who averaged 3 AV for 5 yrs. (Currently, if I'm understanding your calculations, these two players would be rated approx. the same, depending on their year drafted and pick # overall. I'm basically thinking of an "average" 3rd round pick.)
This would also give kickers, punters, & long-snappers a value to go by, since they don't receive AV like the others. Jason Hanson has contributed much more value than Mike Nugent, for example. This could help reflect that.

Hmmm...sorry for not replying the first time (thought I caught everyone's questions). Yeah, I don't remove the player when calculating VAE and ROI. Know it's not the best excuse in the world, but doing that would take way too much time (as far as I know), and probably wouldn't be an efficient use of that time (i.e., it would be a ton of work, and yet only improve the model at the margins).

Re your second point, Career AV itself takes longevity into account by weighting a player's best years (usually at the end of his career) far less than his worst years. And yet, as I showed in the first piece, using Career AV instead of Career AV/Yr produces a slightly less desirable fit and, more importantly, seems a lot more sample-dependent.

The special teams issue, on the other hand, is something I'll definitely be fixing next time I give this project a fresh look.

Re: my 2nd idea, that's why I think you should use Career AV/Yr PLUS # of years. For an extreme example, let's take Robert Edwards, RB, NE. One good year, one horrific injury at the Pro Bowl, done in the NFL. He then has no fringe years to ever be discounted--his one good year gets counted at 100% in both systems. Someone like Terrell Davis, who maintained great play for 8/9 yrs, also gets helped because he had no "decline" years, plodding along at replacement level vs. previous years of star level. However, even though LdT's years with the Jets aren't weighted much in Career AV nor Career AV/yr, they are still less than his HOF-level years with the Chargers, and thus pull both numbers down slightly. If I understand AV math well, after about 10 yrs, a player doesn't add much to his Career AV unless he's still great, because these years will be counted at 50% or less. But anyone still playing good after 10 years is pretty great to start with, because he should be entering his decline phase (e.g. Ed Reed not re-signed by Ravens).
This brings me to my nitpick with AV (which I voiced in the PFR blog several years ago). The best years should be weighted 100-98-96-94-92-90, then falling off 5%/yr. This makes the best stars burn brighter, IMO. This is where I think adding # of years to Career AV/yr would help.

Re: my first point--really I think it only affects the guys whose VAE is outside of 1 or 2 SDs from the VAE mean. And let's face it--those all-time bests/worsts are the guys we care about anyway. You could probably do it for just the 100 best and worst guys, adjust them up or down as necessary, and the rest of overall big numbers (per team, per college, etc) wouldn't change a bit.

Agreed. I was expecting to see the Boilermakers rate highly, but not necessarily top 10 in both metrics. It passes the eyeball test, though. Their players don't get drafted based on school reputation, so they have to have proven to be good against generally stronger competition (since Michigan, Ohio St, ND can't play themselves) to get drafted. That may be why a decent number of historically weaker and/or smaller schools rate highly here.

Also, as you alluded to, Purdue certainly benefits greatly from Brees (and other QBs, although I believe Griese was drafted before the cutoff of this study), but I also think there aren't too many 1st or 2nd round picks from Purdue that have been busts, either. Add in the number of later round picks that have exceeded expectations as solid to great contributors over their careers, and their ranking makes perfect sense.

It makes me wonder why this is. There are other schools of similar size and historic quality to Purdue (and some of the other top 10 teams) that have not produced as well. Hall of Fame quality players alone can't prop up a team that has had 91 picks, especially since only one current HOFer from Purdue qualifies (Rod Woodson). Brees will be in the HOF, so that only really makes two HOFers drafted since 1970. The school seems to produce a lot of players that meet or exceed expectation without being stars, but why would that be unique to any particular school? I'd be interested in seeing the percentage of drafter players with positive VAE and ROI for each school.

Looking at this list, I suspect there's a mild punishment effect for college teams that were dominant in the 1960s, but fell off some in the 1970s. They get the institutional push that hurts USC, Alabama, U-M, etc, but lose credit for their HOFers who were drafted in the 1960s.

Cincinnati only had 35 picks, so they didn't qualify for the rankings. Their stats, though, are +0.8 VAE and +1.6% ROI. Also, keep in mind that this only goes from 1970 to 2007, so post-2007 Bearcats like Barwin, Nakamura, and Kelce aren't included.

Another way of putting this is this is a list of teams which underacheived in college given the talent they had to work with. Under the theory that playing on a winning team (Nebraska, Alabama) will overinflate your draft status, it would be prefectly expected that schools with great coaching and winning teams and great depth and walk on programs would outperform their players' talent.

Everyone, it's kind of buried in the middle of the comments, so wanted to alert you to the fact that I linked up the full table if you want to see stats for the 49 colleges not included in the table up top.

But it doesn't necessarily have anything to do with how good the team is, or how much talent the team has. It really only has to do with whether or not players from a particular school perform in the NFL to the level they are drafted at.

Or maybe the better way to look at it is that this is simply rating whether or not players from a particular team are drafted earlier or later than they should be, based on their NFL performance.

It's possible that a bad/mediocre team could be that way because of poor coaching, and the players are revealed to be better than expected in the NFL.

It's also possible that a bad college team has a few outstanding players, and is well coached, but doesn't have enough good players to win in college. The outstanding players then could perhaps be drafted too low due to the bad team hiding their true ability.

It's also possible that there's a ton of noise and outliers and one-offs in these stats, and they aren't all that meaningful.

I think the most interesting thing about this table (and sorry if anyone posted something similar already, I haven't read through them) is that during this time period San Diego State had more draft picks than Virginia, Louisville, Georgia Tech, Oklahoma State, Maryland, Boston College, Syracuse, and Missouri.

Tiny Jackson State, a member of the Southwestern Athletic Conference, boasts less than one-fifth as many students as my alma mater, the University of Florida. –ah easy to see why Florida State wasn’t included now.