One of the more interesting debates that has arisen of late, particularly with the improvement in Calgary's drafting, is how to rate prospects. For example, when I say that Johnny Gaudreau is the best Flames prospect, what does that mean and how do I come to that conclusion?

In the interest of throwing fuel on that particular debate, over the next while we'll be touching on a few different methods of assessing young talent. To kick things off, let's look at NHL Points Equivalence, or NHLE, one of the most relied-upon statistical measures used to evaluate prospects, particularly across different leagues.

To statistics newcomers, NHLE is calculated in a pretty simple manner. Roughly it goes like this: take all the players (over a specified time period) that came from an NHL feeder league (either pro or amateur leagues), add up the points they tallied in both their feeder league and in the NHL. Divide them and that gives you the conversion rate between points in that league and points in the NHL. As a result, at its best, NHLE provides a handy way to compare the offensive production of players – either at the same point in time across leagues, or within the same league, or within the same age group.

When you rely on NHLE to evaluate numbers, though, your analysis can become criticized for being a bit too reliant on counting statistics. For instance, the figure somewhat undervalues players on bad teams or in leagues with lower offensive outputs – although the league effect may be corrected over time if the phenomenon persists.

NHLE is also not great for evaluating players that aren't all that offensively-oriented, such as most defensemen and defensive forwards. For instance, college prospect Matt Deblouw was a key player for Michigan State, playing a good two-way game and taking some key face-offs. But his team didn't score a heck of a lot, nor was he their primary offensive weapon as a freshman, so he posted a disappointing NHLE of 16 For a freshman, he played a lot and did put up some offensive numbers, but because he didn't play a large offensive role his NHLE is comparably low. Thus, while NHLE itself is useful broad brush metric, it doesn't take into account roles or circumstances.

Lastly, one could argue that the biggest difficulty with reliance on NHLE falls in more of a technical realm – sample sizes. As NHL players come from a wider and wider variety of source leagues, the sample-size for creating metrics for the established leagues is becoming smaller. Similarly, some of the lesser-known leagues will be really tough to assess and compare, such as the difficulties analysts face when trying to create metrics to look at, say, Tim Harrison in a Massachusetts high school league, Mark Jankowski in a Quebec prep school and Rushan Rafikov in Russian junior hockey

Similarly, the manner in which NHLE is compiled – essentially averaging approximate production of players who are in the league currently or in the recent past – may be glossing over trends within each source league in terms of how production is changing as pertains to up-and-coming players. Granted, it's unlikely that NHLE for the WHL will change overly for players getting drafted today, but using a conversion factor of 0.30 for the WHL is entirely based upon how players before Hunter Shinkaruk and Morgan Klimchuk performed in both the league, not upon anything those players have done or will do. This means you likely have to "re-calibrate" equivalency rates every so often to ensure the translation factor hasn't drifted for the league relative to the NHL.

That said, despite these flaws, I think NHLE is tremendously useful. The holes it has are, to be honest, fairly easy to work around if you have other evaluation methods at your disposal. We'll go over those in our next installment.

Related

Ryan Pike has covered the Calgary Flames since 2010. He's Senior Contributing Editor at FlamesNation, a Senior Writer covering the Flames and the NHL Draft for The Hockey Writers, and a correspondent for the Fischler Report. You can see his hand or the side of his head on TV sometimes.

The right way to think about NHLE (and any other 'advanced' hockey stat) is that it provides a data point worth considering, but that it must be taken into account in a larger context.

A good example is Sean Monahan. In his first season he posted very exciting offensive totals, and was touted as a potential first overall player. In his second season he posted very similar totals (though slightly better), appearing at first glance using NHLE to have been 'running in place'.

But the context is important. In his 2nd season the team stripped itself of virtually all offensive talent outside of Monahan, and his team was terrible as a result. Monahan though, played a ton, in all situations, lead the team in scoring (by a wide margin), and had gobs of powerplay time.

Just looking at his NHLE wouldn't give you the full picture on his performance, but looking at in the context of the events with his team, it becomes a useful data point in the overall assessment of how he's doing.

Appreciate the effort on explaining this measure. It, along with Corsi and others, are but tools in a very crowded toolbox that needs to include 'gut feel' and "intuition' that are drawn upon to make an overall valuation on any particular player.

Objective evaluations will miss a lot. How would/could you objectively put a valuation on character and work ethic, the intangibles if you will? How would, for example, Gary Suter have measured up on all of the objective evaluations? A ninth round pick out of NCAA (Wisconsin), who would have thought that he would have been a Calder candidate? Bob Johnson, although he did not coach Gary for his full NCAA career, knew. . .

And I guess that this highlights something that we who do remember the history of this Calgary franchise can draw on with some positives. As the Flames developed into the power that they were in the mid 80s and culminating in the '89 championship, they recruited NCAA players who played key roles in the team's success. Joel Otto. Gary Suter. And who can forget the Butcher of Harvard and Gretzky nemesis, Neil Sheehy?

Its for this reason that the Flames management group that includes NCAA alumni (such as Craig Conroy) deserves some credit for having a technical stats system coupled with a solid group of experienced bird dogs hanging out in North American arenas beyond that of the CHL. And for that reason, I look forward to seeing a diamond in the rough evolve from the Kenny Agostinos, the Ben Hanowskis, even the Tim Harrisons that are now part of the system.

Has anyone ever done a real world study to see if NHLE is actually valid? I mean comparisons between what NHLE said they should produce and what they actually produced complete with names and numbers.

Seriously, there are many guys out there who've ripped up the AHL and if you use NHLE you'd expect them to be at least competant NHL'ers (if not all-stars) but they just aren't. Similerly there are guys who produce a high NHLE while in junior but they don't ever end up even matching that number in the AHL. Seems like something of extremely dubious value.

NHLE is like most counting-based stats: it is essentially blind to circumstances and luck. Unfortunately, data for junior and feeder leagues is so spotty it's currently impossible for us to refine it.

That's one of the reasons we started looking at % of team offense and % of offense at ES and the PP - as way to at least partially correct for circumstances that may be inflating or deflating a guys numbers.

I understand the want to compare offensive potential for prospects across various leagues. But this is used out of context so routinely I just don't buy it anymore.

If your going to use it apply context. Compare prospects at similar ages with similar circumstances. Don't write off a 19 year old project because the model says he would only get 18 points if put in the NHL today, for example.

I also question the need for additional confusion. "We don't use counting numbers because those are useless. But when you apply this translation now they are useful!"

Determining which prospects have offensive potential isn't rocket science. Why not just use counting numbers? And the whole NHL prediction model just adds more confusion, especially for prospects not on the fringe of the NHL.

In short I think it has its uses. Comparing potential 1st round picks across various leagues. Predicting possible offensive production from a prospect just joining the NHL. But rarely is it used for those things. It is more commonly used as a big blanket metric used across all prospects without context.

My question on this measure is that when evaluating NHL play (on this site) we are reading about possession stats as the primary determination of a player's value and somewhat dismiss counting stats to situations and luck. However, while evaluating prospects we don't seem to have a way to determine if the player will become a good possession player so we are relying on the very measures that are somewhat dismissed at the higher levels of play. Seems very inconsistent to me?

My question on this measure is that when evaluating NHL play (on this site) we are reading about possession stats as the primary determination of a player's value and somewhat dismiss counting stats to situations and luck. However, while evaluating prospects we don't seem to have a way to determine if the player will become a good possession player so we are relying on the very measures that are somewhat dismissed at the higher levels of play. Seems very inconsistent to me?

I have had very similar thoughts to this myself.

I think it comes down to the fact that there isn't enough data to get the kind of posession info from the lower leauges, so we use NHLE as the best transition tool avaialble based on data available. But I know what you mean; it does seem contradictory.

NHLE is like most counting-based stats: it is essentially blind to circumstances and luck. Unfortunately, data for junior and feeder leagues is so spotty it's currently impossible for us to refine it.

That's one of the reasons we started looking at % of team offense and % of offense at ES and the PP - as way to at least partially correct for circumstances that may be inflating or deflating a guys numbers.

Went to the link... it directs me to Willis's articules in general but not a study with names and numbers, anyhting more direct?

Part of the dubiousness for me lays in the fact that you have to add the caveat "at least for junior-aged players". If someone is selective enough with the data set they can make anything say anything. If NHLE is a valid means of translating projected point levels across leagues why shouldn't older players be a part of the data set?

Question: Given that you have translation factors for mutliple levels/leagues would a players NHLE/0.55 be his AHLE? Curious because the number of CHL/NCAA players who make the intermediate jump to the AHL will far exceed those who jump straight to the NHL and might provide a better test of the assumptions behind the numbers.

Went to the link... it directs me to Willis's articules in general but not a study with names and numbers, anyhting more direct?

Part of the dubiousness for me lays in the fact that you have to add the caveat "at least for junior-aged players". If someone is selective enough with the data set they can make anything say anything. If NHLE is a valid means of translating projected point levels across leagues why shouldn't older players be a part of the data set?

Question: Given that you have translation factors for mutliple levels/leagues would a players NHLE/0.55 be his AHLE? Curious because the number of CHL/NCAA players who make the intermediate jump to the AHL will far exceed those who jump straight to the NHL and might provide a better test of the assumptions behind the numbers.

When analyzing data there are two ways to go about getting a valid result: 1) incorporate as many variables as possible and use a number of increasingly complex calculations to eventually correct for all of them, or 2) control for multiple variables in order to avoid having the data confounded by them.

In the case of NHLE and age, the most straightforward way to use it is to say "this applies most strongly to players between age x and y", as it's generally players between the ages of, say, 16 and 20, who are of most interest, rather than trying to work out a factor by which to adjust for age (although there are guys working on that as well).

Sure in theory. But using translation tool derived from 'averages' to compare 'individuals' from different leagues at different stages of their development without the backing of context numbers like ice time, zone starts, line mates, etc is very limited.

In fact one of the major premises behind advanced stats is that counting numbers aren't useful in evaluating NHL players without context. Yet NHLE is a rough estimation of counting numbers for junior prospects and is being used without context.

I am not saying NHLe should never be used. I think a blog projecting NHL production from the Flames prospects potentially making the NHL next season would be great.

I am just saying that advanced stats already have their cynics. I personally think the community needs to be careful in posting articles making any sort of conclusions using NHLe without the appropriate context. For example ranking the offensive potential of Flames prospects without addressing their stage of development, team strength, ice time, and role would be a poor use of the metric. IMO.

NHLE is like most counting-based stats: it is essentially blind to circumstances and luck. Unfortunately, data for junior and feeder leagues is so spotty it's currently impossible for us to refine it.

That's one of the reasons we started looking at % of team offense and % of offense at ES and the PP - as way to at least partially correct for circumstances that may be inflating or deflating a guys numbers.

I'd suggest that the number one thing no sort of advanced stat or NHLE can measure is how a very specially unique player like Johnny G translates to the NHL.

Its fine to say "players who get X points in NCAA typically get X points in the NHL". This is based on data, averages and math, which is great, and likely the only logical and measurable way to project in a meaningful way. But what about the fact that he is 5'6" and slightly heavier than my 13 year old nephew. I'd suggest its a fact that a smaller player would have an easier time with smaller (less Chara'ish) competition. Has any analysis been done on size VS NHLE? Is it more accurate if the player is average or large size? I'd guess it would....

Of course math can't measure this without huge sample sizes. You could look at similar guys who were tiny and tried to jump up to the NHL, but then you are moving out of math world into abstract tiny sample sizes. But when projecting a guy like Johnny G I think a huge asterix should go beside any NHLE.

**What was Nathan Gerbe's NHLE?** Any other example of extremely small forwards coming up from NCAA?

I realize advanced stats can't factor in EVERYTHING, but I think in a case like this, where the player is SOOO SMALL it needs to be considered in any analysis, and I'd argue they almost make NHLE completely irrelevant. He isn't 2-3 inches undersized.. The guy is 14 inches shorter and 100 lbs lighter than Chara. Wow.

It appears that the complaint about NHLE or advanced stats in general is that it's not a perfect science. It reminds me of what economists and portfolio managers have been trying to do for a century, by developing the perfect market indicators that predict where individual stocks and the markets are going.

There is no single magic indicator that will predict how investment markets will behave, so portfolio managers use a basket of data to make predictions. The real magic is in properly interpreting all the data presented understanding the context of what is being examined.

NHLE is one single indicator, which in my view cannot stand alone and requires context. I like it in general, as it shows a consistent positive correlation with scoring in the NHL, and for that reason alone can't be dismissed.

I like NHLE a lot for scoring forwards and offensive defenseman in juniior leagues. So, for me it has been really exciting to see Baertschi and Gaudreau succeed, and equally disappointing watching Jankowski. At the same time, I try to be mindful and balance the fact that Gaudreau plays for a powerhouse, and Jankowski plays for a very crappy team.

I barely pay attention to NHLE results for guys like Wotherspoon, Seilock and Max Power. I just don't think NHLE is useful for these kinds of players.

Its interesting Emile Poirier was never included in that article everything I read predraft said he would be a first rounder, but anyway his 1.07 points per game would place all three of Calgary's first rounders within the Top. 15 eligible prospects from last years draft for NHLE on that list. Not to mention Monahan & Poirier led their teams in scoring and Klimchuk finished second in scoring to a 20 year-old overager, statistically speaking the Flames had Three Top. 15 Picks that produced with minimal help.

It still surprises me that Max Domi was drafted after teammate Bo Horvat everything I have read puts Domi well ahead of his teammate numberswise and he was always the far better player when I watched London. Phoenix got a steal! The Flames really need to stop trading second round picks, we could also have had a Nic Petan or William Carrier with that Montreal pick and had four Top. 15 NHLE form the draft. Speaking of which surprised William Carrier was not included in that NHLE article as he had 1.23 PPG on the worst team in the CHL. St. Louis got an absolute steal at 57th!

That is a generic comment and doesn't address my specific concerns with the metric. The fact that advanced stats are imperfect but useful doesn't justify dismissing all criticism of one of the metrics.

Also, I am not dismissing the metric. I am just commenting that it (IMO) is commonly used without appropriate context.

I'm not sure about his first year, but he was fourth on his team, so I'm guessing he was on the second line. Last year he played with Tomas Hyka (which I guess explains why Poirier was playing LW last season, since Hyka is a right shooter). I'm pretty sure Hyka had the higher scoring rate, but Poirier had the most points on the team. I have no idea who was playing center (come to think of it, it may have been Hyka after all).

Just something to consider Tomas Hyka & Martin Reway were two good euros on that team but they also missed significant time Poirier played 15+ games without those guys. The only other decent guy he played with was a 20 year-old who has never scored more than 33 points before this season.

Emile Poirier truly did the most with the least. Heard a funny story about Martin Reway this summer apparently he is from a family of gypsies, Hahaha. Most teams didn't even think about drafting him, but Montreal decided to take a flyer on him in the fourth.