Yesterday I listed the top 20 careers by goaltending point shares. This is a system developed by Justin Kubatko at hockey-reference.com. I argue that the goaltender point shares are somewhat flawed because of the way defensive point shares are given out. Each team’s marginal goals against is calculated. This is a theoretical construct that represents the number of goals prevented by the team above that of a replacement level defence. This is shared to each player in different amounts depending upon the player’s position. Goaltenders get the largest share. They get this share regardless of if it is a goaltending effort or a defensive effort preventing goals. As a result, goaltender point shares track very closely with games played for goalies.

The two key examples I gave are Sean Burke who places 13th in goalie point shares and 12th in goaltender games played. I argue this is too high. Burke was a good goalie who played a long time, but he was never a truly top goalie. This system overrates him. Dominik Hasek was probably the best goalie in NHL history. He is 20th in career games played because he didn’t start his career at a young age due to his being stuck in communist Czechoslovakia. He ranks 9th in goalie point shares. I argue this is not high enough.

In the comments Sven22 takes the other side of the argument:

Take your Hasek example. I think he’s the best goalie to come along in at least the last several decades, and maybe the best goalie of all time.

But while I do think ninth place is a little low, I don’t think it’s an unreasonable approximation for this sort of list. Hasek is 20th all time in games played. He is 11th all-time in wins. His ranking on this list suggests that his cumulative total of “points won” based individual play exceeds what one would expect from his raw win total, and is exceptionally high for a goalie with as few games played. Consider that many of the players in Hasek’s neighborhood on this list required 100+ games more than Hasek to “win” a similar amount of points for their respective teams as he did.

The fact that Hasek is ninth on this list, even though most would agree that he’s much better than the ninth-best goalie of all time, does not prove the system failed. Bottom line, just like with wins and shutouts, Hasek didn’t play enough NHL games to acquire more point shares over the course of his career.

I set out to argue that point and realized I would have to look at individual seasons to do it properly. So I thought it was best to list the top individual seasons. Here they are:

Top 20 Seasons By Goaltending Point Shares

Rank

Player

Season

Goalie Pt Shrs

Games Played

1

Roberto Luongo

20.85

2003/04

72

2

Bernie Parent

19.94

1973/74

73

3

Dominik Hasek

18.63

1997/98

72

4

Bernie Parent

18.56

1974/75

68

5

Roberto Luongo

18.51

2005/06

75

6

Tony Esposito

18.22

1973/74

70

7

Jose Theodore

17.40

2001/02

67

8

Martin Brodeur

17.29

2006/07

78

9

Dominik Hasek

17.19

1996/97

67

10

Roberto Luongo

17.08

2006/07

76

11

Terry Sawchuk

17.02

1951/52

70

12

Terry Sawchuk

17.02

1950/51

70

13

Cam Ward

16.83

2010/11

74

14

Dominik Hasek

16.82

1998/99

64

15

Ryan Miller

16.78

2009/10

69

16

Ken Dryden

16.49

1975/76

62

17

Miikka Kiprusoff

16.32

2006/07

74

18

Pete Peeters

16.25

1982/83

62

19

Miikka Kiprusoff

16.20

2005/06

74

20

Curtis Joseph

16.17

1992/93

68

So here is the list of top goalie seasons. There are several things that jump out at me when i look at this list. The one most on topic is how important games played are to rank highly on this list. The lowest games played total is 62 and that is tied between Ken Dryden and Pete Peeters. Both played before individual shots were recorded and they played on very good defensive teams. They faced fewer shots than would have been assumed in the calculation and are thus overrated here. In order to make it to this list with a lower games played total (is 62 really a low total?) you must have the number of shots you faced significantly over-estimated.

Missing from this list is Tim Thomas who had a great season in 2010/11. He set an NHL record for the highest saves percentage since they have started recording these things. He only played 57 games, which would be the lowest total on this list. Thomas places 21st and just barely misses out. However he is not the highest ranked goalie from the 2010/11 season. Cam Ward is 13th. He played 74 games. He wasn’t even nominated for the Vezina Trophy in the NHL. I will write a future post to discuss this situation. It is a clear sign that games played trumps playing well.

Roberto Luongo is listed as having had the top goalie season of all time. He did this in 2003/04. This was a year where he finished a distant third in the Vezina voting. I thought this was a mistake by the voters to rank him so low, but it is quite the claim to say he had the best season ever. I will look at this also in a future post. Luongo ranks three times on this list. The only other goalie to do that is Dominik Hasek and Luongo’s rankings are higher. If we are to believe this list, Luongo may be the best goalie of all time. Does anyone believe that?

Dominik Hasek’s back to back Hart Trophy seasons are 3rd and 9th on this list. I would argue this is low and too much credit has been given to his defence. That is part of the reason for his lower career ranking. I think his career ranking also needs a post in the future.

This list of best seasons by goaltender point shares is not what one would have expected for several reasons. The biggest of these reasons will get their own posts in the future. For the most part playing a lot of games gets a goalie on these lists. That is different from being the best goalie in the league.

Comments

... proving that quantative measures are fun for generating debate, useful in extrapolating probability when understood to not be the only measure, and all but useless when debating hockey greatness. Short careers, clutch performances, personality and swagger ... we’re not measuring cars here. We’re measuring people. And “could have beens” don’t meausre up to “was”. For example, maybe if Hasek got to N.A. earlier in his life he would have washed out in the farm system. His career was what it was excactly because of when he came to N.A. Lengthening his career on the front end would not automatically make him better. It might make him worse too. Enjoy what was, look forward to what will be.

Uh, which is not to say I don’t thoroughly enjoy all this speculation and metrics. It is super interesting.

Posted by
Shaun
on 09/12/11 at 04:13 PM ET

All this shows is that this quantitative measure isn’t perfect. It hasn’t captured the best goaltending performances.

It makes no statement about all other quantitative measures. Many of which are much more successful than this one. The look at this system highlights what it is we need to do better to capture goaltending performance.

I’m a Luongo fan, but I’m objective enough to know he’s not the greatest goaltender of all time. However, I would argue that what we’re seeing here is not “great years” but something more like “workhorse performances”. Which is worth measuring, it’s just not indicative of great goaltending, though there is obviously a relationship between the two.

Posted by
Dan
from hammondking@gmail.com on 09/12/11 at 09:19 PM ET

Hey, as it has been mentioned, it’s a counting stat, and even in a particular season, a counting stat is going to be influenced by how many games you play. What I would like to do is find “replacement level” for goaltending point shares and then add that total to the point share total of the player who didn’t play each game. Now, if you want to be more complex, you could do a per game basis for the stat, and thus find out who had the biggest impact per game ONLY including games in the “prime” of their career. Thus, you don’t penalize a guy for being merely “good” at the end of their career.

Posted by
Justin M
on 09/13/11 at 01:40 AM ET

The harsh thing about this sort of stat is that, traditionally, goalies are judged more by rate stats (save percentage and GAA) than counting stats (wins and games played). But point shares are a counting stat, and that means games played is going to really matter. It’s unavoidable.

The fact that Cam Ward beat out Tim Thomas this year in point shares doesn’t, by this metric, say that Ward is the “better” player. What it literally says is, “4318 minutes played by Cam Ward (versus 4318 minutes played by a marginal replacement goalie) contributes more cumulative points to a team’s total than 3364 minutes by TIm Thomas (versus 3364 minutes of a marginal replacement goalie).”

That’s a debatable claim, but again, I don’t think it’s unreasonable.

A more fair (though certainly still imperfect, for many of the flaws you rightly point out) attempt to find the “best” seasons would be to look at point shares per 60 minutes. That would allow the statistic to adjust to certain practical realities, such as the the fact that competent NHL backup goalies exist and they have a demonstrable effect on how many minutes starters receive. It would also reveal, quite convincingly, that Tim Thomas performed at a much higher level than Cam Ward last year.

That all said,

I want to reiterate that I generally do agree with most of your reservations about the metric—including your claim that Hasek should be higher on both the season and career lists. I didn’t say that I agreed with his ranking—I said it was not unreasonable. Big difference.

As you rightly point out, it’s ridiculous to assume that all goalies in all systems are all responsible for about (or exactly, for years when shots were not tracked) two-sevenths of goal prevention. Goalies that had to stand on their heads to keep their teams afloat (like Hasek) are at a disadvantage versus goalies who played behind much more competent defenses. The shot adjustment probably helps in most cases, but shots against certainly don’t tell the whole story about team defense.

In my mind, that—the 2/7 rule—is the key flaw here. It has a side-effect of making games played even more important in the rankings (since there’s not enough data to differentiate how much of a team’s goal prevention a particular goalie was responsible for), but the fact that games played strongly correlates with point shares is not in and of itself the problem—since again, this is a counting stat we’re dealing with. Games played is always going to matter a great deal with counting stats.

The fact that it is a counting list doesn’t explain away the fact that games played is too important in these defensive and goaltending. The offensive list includes a Howie Morenz season where he played only 45 games. There is no reason that players can’t be on the defensive or goaltending lists under the same circumstances. The fact nothing similar exists while players who were never great but played long careers make the list is a problem. It shows its flaws.

Games played is also extremely important in the offensive portion of point shares.

Howie Morenz’ 27-28 season does not disprove this. In fact, by far the most significant reason that Howie Morenz made the list is a statistical fluke. I’ll explain.

Because point shares use marginal goals for and against, rather than a team’s actual record, to dictate how many “point shares” there are to go around, point shares rarely coincide with points perfectly. Occasionally, there is a substantial disconnect. Montreal’s 1927-28 season is one such instance.

That year, the Canadiens scored 116 goals and allowed 48. Based on the formulas at hockey-reference, I calculated that the Canadiens were “expected” to record 80 points that year—an almost ludicrous total, considering that a perfect record would have yielded 88 points. Likewise, the total number of point shares (offensive, defensive and goalie) awarded to Canadien players that season was 80.7.

The Canadiens actually finished with 59 points on the season, so the almost 22-point gap between their actual points and their total credited point shares is an enormous error. The next greatest (positive) error for that season was less than eight points, achieved by the Ottawa Senators.

Morenz had an incredible season, no doubt. He dominated all other scorers. He set a new points record at a time when scoring was in the pits. As a result, he got a huge chunk of Montreal’s total offensive point shares. Which, as demonstrated, were hugely inflated due to a statistical anomaly.

“Correct” for this anomaly—14.2 * (59 / 80.7)—and Morenz’ offensive point shares decrease from 14.2 to about 10.4, dropping him over 100 places on the list of best seasons.

Morenz’ season ranks 15th all-time (ironically, about where I’d say it deserves to be) not because the system rewards high-scoring players in a short-season era, but because the system broke down in this highly specific case, and the results just happened to coincide with human subjectivity.

Consider that, of the top 250 seasons of all time by offensive point shares, only 2 were recorded in the 1930s and NONE were recorded in the 1940s. There are a number of entries (15) from the 1920s, for reasons I have yet to investigate.

Sean Burke played on some AWFUL teams and maybe the most underrated goalie of all time. Hasek, one of the many overrated goalies of all time played for a team that was ALL defense and defense only. Hasek won the Vezina in 2000/01 with Buffalo and had a .921 and a 2.11 GAA in 67 games. He left the next season for Detroit and Biron his replacement in Buffalo had a .915 and 2.22 in 73 games. Clearly there was a GREAT defense in Buffalo!

By the way, Hasek stats in Detroit that first season was .915 the same as Biron and he had a 2.17 GAA. Clearly Biron is as good as Hasek!

This system is terrible and doesn’t work at all.

Posted by
The Goalie Guru
from Toronto on 09/17/11 at 08:12 PM ET

You’re crazy if you think Hasek is overrated.

Even if you’re right about Buffalo being a great defensive team, it’s unfair to compare Hasek as a non-Sabre against a Sabre in a given year, or cross-years.

The closest thing to an apples-to-apples comparison is to look at Hasek’s performance relative to his backups in a given season.

In every single season during his career in Buffalo, Hasek led all Sabres goalies in both GAA and Sv%. In fact, he usually hammered the next closest backup by wide margins in both categories. The only “technical” exception to this rule is Petr Skudra, who played 1 minute of relief in 2000-01 and didn’t even face a shot.

It is disingenuous to compare Hasek’s first year in a new system, at age 37, after winning six Vezinas in eight years, to what happened to be one of Martin Biron’s two best statistical season by far (the other one coming in 2007-08) and using that as evidence that Hasek is overrated.