A look back at the 2010/2011 “SackSEER” projections: Spoiler, it’s ugly

Since we’re in “Combine season,” I thought we’d take a look at a metric that heavily utilizes Combine data. SackSEER, in case you’re unfamiliar, is a model introduced by Football Outsiders that projects 5-year sack totals of highly drafted 4-3 DE’s and 3-4 OLB’s, based on 4 metrics:

Vertical leap from the Combine

Short shuttle time from the Combine

Per-game sack productivity in college

Missed games of NCAA eligibility

I think just about any knowledgeable football fan that is familiar with Football Outsiders is generally very complimentary of the work they do (as they should be), but SackSEER has its fans and detractors. The model was formed from data collected prior to the 2010 season and was first introduced in 2010. Therefore, the model pre-2010 is understandably extremely successful. Its fans point to the pre-2010 success. Its detractors feel that predicting a player’s future in the NFL based on a couple jumps and a few passes through some cones at the Combine is a little ridiculous. It also is a pure data model that ignores things like watching game tape.

I should probably just be upfront and note that I’m not a fan, and often feel that every Tom, Dick, and Harry is trying to come up with “the new formula.” The 26-27-60 QB theory kind of started it, and now people are reachingto find a similar formula. I see SackSEER as one of those reaches. Anyway, here were SackSEER’s projections from last year:

So far, SackSEER seems to have a beat on Justin Houston. If Derrick Morgan, Brandon Graham, and Everson Griffen begin to produce, they might come reasonably close to SackSEER’s projected numbers. But mostly, SackSEER is missing so far since its 2010 introduction… and pretty wildly at that.

SackSEER is the reason I was excited when the Eagles drafted Daniel Teo-Nesheim. But I was always skeptical. Its correlation was fairly tenuous even with all the convincing past data. And it seemed better at predicting busts than predicting future stars. Since then, its biggest bust prediction is the player who became the biggest star. SackSEER should probably go on hiatus for retooling.

I don’t think looking at one year of Data is really helping much. It’s a five year prediction which accounts for the various random factors that affect production across those years. Looking at these stud producers and noting that SackSeer poorly projected them ignores that you’re looking at one year of production and projecting it with no justification, and also ignores that you selected your sample from the beginning to look at outliers in sack numbers for some of them.

I’m not a big fan of the system myself, but looking at a mass prediction agent and judging it using 15 players spanning 22 seasons hardly gives you an impression of its accuracy.

“SackSEER Sleepers – When it comes to sleepers, SackSEER rates two prospects above the rest: South Carolina linebacker Eric Norwood and Washington defensive end Daniel Te’O'Nesheim. Eric Norwood’s Combine numbers are similar to those of Clay Matthews a year ago, and he had better college sack production. Te’O'Nesheim has been billed as a productive, high-energy player who lacks the athleticism to compete at a high level, but his 37-inch vertical leap and a 4.18-second short shuttle at the Combine are enough to convince SackSEER.”

I don’t think Andy Studebaker belongs with DTN and Bryan Smith. DTN and Smith were third round choices that couldn’t make the roster one year after being drafted. Studebaker was a sixth round pick with potential out of a small school who is now starting for the Chiefs. The sixth round is where Reid should look for “fastballs” and take chances on someone coming out of nowhere.