HASAN: Who Crafted the Best Big Board in 2014?

Too often we hear that “you can’t judge a draft class until three years out,” and though I disagree with the implied conclusion, it does lead us to something interesting—grading the graders.

If we can grade draft classes three years after the fact, we can certainly grade big boards.

I’ve been compiling big boards for three years, which makes now an appropriate time to start rewarding those who did the best job predicting performance.

The first step to that is ranking players from the 2014 draft, something fraught with danger by itself. Zack Martin is one of the top guards in the NFL and a better player than a good portion of players in the draft, including Derek Carr.

On the other hand, as a quarterback, Carr potentially offers more value to his team.

Not only that, it’s difficult to make decisions about which player of two well-regarded athletes is best. Odell Beckham, Zack Martin and Khalil Mack are all at the top of their position groups. Which one is better?

Different people may have different answers, but I chose to employ a method meant to resolve these disputes relatively simply.

Using the same percentile system I used to evaluate wide receivers, I used Pro Football Focus and Bleacher Report Top 1000 grades to create an overall score for a player. If they didn’t play much last year, then I used 2015 grades.

A few players were so disparate between 2015 and 2016 (think Anthony Barr), that if their 2016 grades were too poor, I averaged their 2015 and 2016 grades. Maybe they’ll bounce back.

I also modified those scores for position by adopting the positional premiums implied by the NFL draft over the past 10 years. Based on the average draft capital spent on the top receivers, quarterbacks, guards, etc. we can figure out how the NFL values those positions.

Applying that positional credit to each player and then adding penalties for games missed means we get a rough ranking of players from the draft. You can find that ranking here. Everyone in the world disagrees, so just know that a rough ranking is all that’s necessary for the task at hand.

So, with that out of the way, whose board was best at determining the top players in the draft?

It certainly seems appropriate. They’ve been grading players since 2001 and have done the excellent work of authenticating and gathering mock drafts and top 100 boards from around the internet and have put together Top 100 scores for years.

They did better than the NFL draft itself did, despite the front offices having access to injury information. Beyond that, the NFL front offices are the only ones with the unique information about their own schemes and draft needs. They know which players will play, and how they’ll fit into their own systems.

The consensus boards did extremely well, too. If included, the Evaluator Board, the Forecaster Board and the Consensus Board would have ranked fifth, sixth and seventh, just above Pauline and the NFL.

The full rankings are below:

And which players gave the most credit or forced drafters to incur the highest penalty?

The player who best helped the big boards out in picking correctly was Marcus Smith, who was called a reach by the draft community at the time in Philadelphia, and who later proved the label. Other players who helped included Donte Moncrief (grading him highly helped), Charles Leno (same), Paul Richardson (he was correctly called a reach) and Joel Bitonio.

On the other hand, those who didn’t rank Uani Unga got burned, and that hurt a lot of the big boards. Those who ranked Vikings draft pick David Yankey highly also took serious missteps, and the same is true of former Nebraska cornerback Stanley Jean-Baptiste. Giants center Weston Richburg was ranked too low by a number of big boards, and so was Eagles nose tackle Beau Allen.

We’ll be going over the 2017 consensus big boards throughout the week, including the forecast, evaluator and full consensus boards. Not only that, we’ll reward the quirkiest boards, the ones best fitting the general trends and look at players who the big boards absolutely cannot agree on.

Arif Hasan has written for sites all across the internet and his work has been featured in the Star Tribune, LA Times, Forbes, SB Nation, International Business Times, the Bleacher Report and MSNBC. He's made radio appearances for 1500 ESPN, TSN, KATE 1450 and others. He currently also writes for the Daily Norseman and Optimum Scouting. You can find him on twitter at @ArifHasanNFL.

Arif – have you ever tried looking at how well “risk-adjusted” consensus big boards predict player performance? Evaluators likely do some of this adjusting implicitly, but I’d be curious to see how well a big board that adjusted rankings for opinion variability would look. All else equal, most teams would prefer the sure-bet 50th best player over a player that’s mean is 50th best but with a much wider range of outcomes.

[…] isn’t to pick on any of these analysts: they are all very good at what they do, and in the aggregate they forecast the performance of prospects about as well as NFL teams. In 2014, Kadar and Norris even did a better job of ranking players than actual NFL teams did in […]