Welcome to Errol Morris Week, a celebration of the legendary director and his work. This is the first of his six shorts for ESPN Films. It’s about electric football, a basement league, and the gang of glorious eccentrics who keep a decades-long tradition alive.

Welcome to the NFL’s SATs

The scouting combine is the most misunderstood event in football. Coverage of the combine has ballooned since 2004, when the NFL Network began offering live wall-to-wall analysis from Indianapolis. The growth of the web has allowed for increased coverage and analysis of assorted workouts and pro days, and an endless number of mock drafts. It has become a public moment on the NFL calendar so quickly, in fact, that fans still don’t really have much of a context to make sense of the combine. We end up talking about the combine through anecdotal examples that don’t make broader sense, as if one player’s path could single-handedly prove anything about the pre-draft process.

So far, we’ve ended up with two dominant competing groups of thought about the combine. Neither of them are particularly accurate.

The draftnik1 side of the conversation tends to take the public-facing aspects of the combine far too seriously. That is to say — and it pains me to say this — the numbers produced by players at the combine really don’t mean very much at all. A player’s 40 time might be a succinct measure of his speed in a vacuum, but it’s a two-attempt sample on one given day being measured both electronically by the league and manually by a bunch of scouts with stopwatches. It’s one thing to suggest there’s a significant difference between a guy with a 4.37 40-yard dash and a 4.43 40-yard dash, but another when the same exact run might produce those two times according to a differing pair of scouts. In addition, the figure in question isn’t being adjusted for a player’s size2 or measured to exhibit any sort of relationship with past or future success at the pro level.

The 40 has caught on because it’s a simple concept to understand and we have a good idea of what a truly fast time looks like. Other draft drills aren’t as popular, even though they might be more meaningful, because they’re obtuse. Do you know what a good score in the three-cone drill would look like? What an excellent broad jump would entail? Probably not. I know I don’t. Maybe that’ll become common knowledge a generation from now, but having paid attention to only a handful of combines, fans and analysts alike are still just beginning to make sense of the information it provides. Much of it is likely to end up as noise.

The rejectors, meanwhile, believe that the combine is a totally useless waste of time, an exaggeration designed to keep the NFL relevant during the dark sporting days of February. Their evidence, almost invariably, consists of anecdotes about how individual players had poor combines and still managed to succeed at the professional level (or vice versa). You can see them in your head shouting, “You don’t play football in a T-shirt and shorts!” self-satisfied, as if they were the first to watch the combine and realize that it doesn’t resemble game conditions.

The problem with those anecdotal examples, of course, is that they fail to tell the whole story. The most famous “workout warrior” is Mike Mamula, whose performance at the 1995 combine and subsequently disappointing pro career has become the stuff of legend. It’s also entirely a myth; Mamula was a star pass-rusher in the Big East who accrued 31.5 sacks across five professional seasons, eventually retiring at the age of 27 because of injuries.

And while Jerry Rice is held up as the classic player who became a Hall of Famer despite running a disappointing 4.7 40-yard dash, there are a number of factors that mitigate his poor 40. Rice’s work ethic was, even among football players, notably legendary; Steve Young once said his former teammate outworked the work-ethic guys. Rice was unquestionably aided by joining a perennially dominant franchise built around its passing attack, developed by a revolutionary coach and executed by two of the greatest quarterbacks in the history of the game. And Rice’s time wasn’t even all that bad; current 49ers president Paraag Marathe is fond of noting that Rice’s “flying 20” time — the final 20 yards of the 40-yard dash — was world-class.

Of course, while there are better examples of players who excelled at the combine without a superlative college track record before failing as pros (Vernon Gholston, Matt Jones), and of players whose work on Saturdays should have meant more than their measurables in shorts (Russell Wilson, Anquan Boldin), there are also plenty of recent examples of players whose combine performance foretold their future. The Chiefs don’t regret falling in love with Dontari Poe, whose incredible performance at the 2012 combine led them to draft Poe with the 11th overall selection; two years later, Poe is a Pro Bowler. Superstars like J.J. Watt and Clay Matthews shook the workout warrior tag on their path to the top, too. And there are plenty of college superstars whose game tape from Saturdays didn’t translate to Sundays after disappointing combine performances or not bothering to work out in Indy at all; Mark Ingram, Matt Leinart, and Sam Bradford are among the recent Heisman winners who have failed to match the lofty heights of their college days at the pro level.

In reality, most of us are thinking about the combine the wrong way. It’s less a test of athleticism and more a test of preparation. A team might move a guy up its draft board if he blows them away athletically or drop him if he loafs through drills, but more so, teams want to see players show up in shape and perform to something resembling expectations. If you show up to the combine and somehow manage to fail a drug test your agent has told you is coming, chances are you’re probably not going to impress at the next level.3 Just about every player who is expected to go in the first few rounds of the draft goes through a training regimen designed to prepare them for the various combine drills, too.

It’s probably better, then, to think of the combine as the football equivalent of the SAT. We all go to different high schools with varying degrees of difficulty, so a 4.0 GPA at one school might not be as impressive as a 3.0 GPA is at a much tougher school. Likewise, throwing for 3,000 yards in the SEC is probably more impressive than making it to 4,000 yards in the Pac-12.

The SAT is standardized. Everybody takes the same test. The scores translate across different school systems because it’s the same scoring system. And it’s hardly an unexpected test; everybody knows it’s coming, and plenty take SAT prep courses to try to improve their scores. A good SAT score likely serves as a useful indicator of a student’s intelligence and ability to perform at the next level of academia, but it’s not exactly foolproof, either; you wouldn’t say one person is smarter than the next because their SAT score was 20 points higher. Just as the SAT helps support a solid high-school transcript, the combine provides some context for a player’s game tape from college.

Instead, the real value of the combine for the NFL’s 32 organizations comes in the stuff that doesn’t occur in front of the cameras. Indy serves as each team’s first chance to interview the prospects of their choice in person, an obviously key part of the evaluation process in a number of ways. That interview can go in many directions. The Giants famously gave prospective draftees a 460-question psychological exam. Some use the interview process to ask questions designed to see how a player handles pressure and/or impertinent behavior, as thankfully deposed Dolphins GM Jeff Ireland’s thoughtless question to Dez Bryant proves.4 More frequently, teams run prospects through questions designed to gain insight into their personality and football acumen. As with the other aspects of the combine, it’s a test of preparation.

The most meaningful test of all in Indianapolis doesn’t involve a coach. It comes courtesy of the team doctor. Each NFL team sends some portion of its medical staff to the combine each year, who poke and prod each player before he steps onto the field to run a single drill. Medical exams are conducted. X-rays are taken. In some cases, those tests can drastically affect how teams view a player, and even they can be wrong.

It was only a year ago that a combine echocardiogram found that Utah defensive tackle Star Lotulelei was suffering from a heart condition so serious that the league sent him home from the combine without allowing him to work out. Once seen as a candidate for the first overall pick in the draft, Lotulelei’s draft stock remained in limbo before the heart ailment was revealed to be a byproduct of a viral infection. Even after he was cleared to return to football and completed a workout at Utah’s pro day, Lotulelei dropped to the 14th pick of the first round, where the Panthers were happy to snatch him up. We obviously can’t see each team’s draft board, but it seems likely that several teams in front of Carolina either took Lotulelei off their draft board or downgraded him because of the ailment. Fortunately, Lotulelei was healthy during a superb rookie season for the Panthers.

There are other benefits to the combine for teams. Having the brain trust of every organization in the NFL in one spot leads to plenty of networking, while meetings with the throngs of agents in town help form the beginnings of free-agent plans. And while teams are more likely to suggest the combine doesn’t mean very much if asked about it publicly, there’s a reason everybody shows up and plants a bunch of scouts with stopwatches in the Lucas Oil seats; this stuff matters, at least a little bit. How much it matters, exactly? One decade into the era of televised combines, the answer remains unclear.

To be fair, there are plenty of draft analysts who place the combine in a more appropriate context, especially among folks who have been part of the scouting and development process for a team. This is more of a fan issue.

As is the case with Speed Score, a metric I built several years ago that contextualized a player’s 40-yard dash by adjusting for his size. That’s a decent measure of athleticism and has some relationship with pro success, but it’s far from even vaguely resembling the whole story.

Then again, one of the more recent candidates to flunk a combine drug test, reportedly, is Justin Houston. Houston promptly fell to the third round of the draft, where Kansas City snapped him up. He’s been a mainstay ever since.

This is the same Jeff Ireland who texted Jonathan Martin this wondrous grammar storm: “Jonathan. Glad your safe. You have a family of brothers that care very much for you. They meant not harm in there prank. We are all here for you. J Ireland”

We know, Grantland publishes a lot of words. Worried you missed something?
Keep up with the best of Grantland — and everything from Bill Simmons & Co. —
with our weekly newsletter.Sign-up is easy. So what are you waiting for?