On Evaluating Draft Picks

“Since Sidney Crosby scored roughly 10% of his draft year’s goals, that means he outperformed expectations by a whopping 9.2%. It also means that Mark Zagrapan, a center who never played in the NHL, underperformed expectations by -0.78%.”

With no money or cap space at stake and with huge potential for return, draft picks are the most liquid portion of any GM’s portfolio. There’s no such thing as too many, and because there are 217 per year with wide-ranging values, they can be easily used to facilitate trades; packed with players to smooth a deal out or traded to a cap-burdened team to take a player off their hands. Because draft picks are an integral part of the market, it is important to value them correctly.

Drafting is a relative value game – it is not important that the player you pick is better than all the other players in the league, it is important that they are better than all the players in their draft year. You could not criticize Nail Yakupov for being a bad #1 pick because he’s worse than Sidney Crosby, for example, but you could say he’s a bad #1 pick because he’s worse than other pick in his draft year. The metric that matters most for a draft pick is thus their performance relative to their draft class. One way to measure is by taking the percentages of games played, goals, assists, and points that a player contributes to his draft class’s total. For example, 2005’s draft class has collectively played 27,722 games in the NHL and scored 3,380 goals. Sidney Crosby, that year’s #1 pick, has scored 382 career goals or roughly 10% of the total from his draft year. It’s safe to say that when one player scores 10% of the goals by players in his draft year, he was a good pick. Yakupov, on the other hand, has 5.4% of his draft year’s goals. Not a bad number on its own, but not comparable to Crosby’s.

Nail Yakupov – average 1st overall pick?This measure alone is not appropriate for assessing draft picks, however, and must be adjusted. It is not fair to say that a defenseman and wing are equally good picks if they both score 2% of their draft class’s total goals. Defensemen naturally score less than wings do, so a 2% performance from a D is better than a 2% performance from a W. To find an appropriate weighting for each position, we looked at every draft from 2002 to 2012 (stopping at 2012 because this was the last year when all draft picks who would play in the NHL most likely have by now). There were 527 centers, 940 wings, 830 defensemen, and 271 goalies drafted over that period. By now, there have been 260,681 games played, 39,693 goals scored, and 64,976 assists made by players drafted in those classes. Centers made up 20.52% of players picked, wings made up 36.6%, and defensemen made up 32.32%.

Using those numbers, we found the total games played, goals, assists, and points you’d expect each position to have to its credit should those stats be distributed evenly. For example, since centers make up 20.52% of the picks you’d expect them to have scored 20.52% of the goals. With 39,693 total goals, that would mean centers would have 8,145 to their credit. However, centers have instead scored 14,683 goals, or roughly 1.8 times as many as expected. Why is this? It’s not that centers are better players than defensemen, who scored only 48% of the goals you’d expect them to, it’s that the position is naturally higher scoring.

To find the position-weighted expected % of games, goals, assists, and points in draft class for a given player, we first found the expected % should every player score at an even rate by simply dividing 1 by the number of picks. In 2005, there were 230 picks, so each player made up .43% of the class. Then, to modify this number by a player’s position, we multiplied it by their position weighting. For example, a center is expected to score 1.8 times as many goals as expected. Multiplying 1.8 x .43% gives us the % of goals a given center drafted in 2005 should be expected to score of his draft class total. A 2005 center should thus be expected to score 0.78% of his draft year’s goals. Since Sidney Crosby scored roughly 10% of his draft year’s goals, that means he outperformed expectations by a whopping 9.2%. It also means that Mark Zagrapan, a center who never played in the NHL, underperformed expectations by -0.78%.

Marek Zagrapan, none game wonderWe can now identify how good a player truly is relative to his draft class, which will allow for some great statistically-based redrafts. What’s perhaps even more important to consider when evaluating draft picks, however, is their performance relative to their draft position. After all, Crosby was the 1st overall pick and a lauded talent. Drafting Crosby had nothing to do with the skill of the Penguins GM and everything to do with their terrible play the season before that landed them the pick. If an 80th overall pick outperformed by 9.2%, it would signal an even “better pick” because of how far they also outperformed expectations weighted to consider their pick number.

To appropriately weight for this metric, we found the outperformance stats for every player of every draft from 2002-2012. We then averaged those to find the expected outperformance % for every pick. Because 11 picks aren’t a large enough sample size to account for outliers, the variance from pick to pick had to be accounted for. For example, the average 26th pick outperformed in the goals category by 0.89%, the average 27th pick by 0.04%, and the average 28th pick by 0.81%. If this were an accurate representation of trade payoffs, teams would desperately try to trade down from the 27th pick. However, it’s more likely that teams just had bad luck drafting at 27th, not that there was anything wrong with them. To correct for this, we took a 5-pick moving average and assigned that as the expected outperformance of each pick. The 26th pick is now expected to outperform by .6% vs .5% for the 27th and 28th, correcting for the luck of the draw.

Each player’s performance can now be compared to his draft status. Now Crosby and Yakupov can truly be compared as 1st overall picks – with Yak outperforming 0.05% vs Sid’s 4.9%, the distinction becomes even clearer.