Forecasting Future Goalie Performance with Four Year Hockey Marcels:

Evaluating goalies is hard. Goalie performance varies more than anything else in hockey and today’s terrible goalie can randomly turn into an elite goalie next season….and then turn back into a terrible goalie. The best measure we have for evaluating goalies is Save Percentage and so we often tend to use a player’s career SV% as a way of forecasting what to expect from a goalie in the future.

So in my base case, I’m using years 1-4 to try to predict years 5-7. The best predictions came from weighting things like this:

Each shot faced in year 3 counts 60 percent as much as shots in year 4

Each shot faced in year 2 counts 50 percent as much as shots in year 4

Each shot faced in year 1 counts 30 percent as much as shots in year 4

This is particularly similar to the baseball forecasting system invented by Tom Tango, known as the Marcel Forecasting System. Marcel, named after the monkey, is one of the most basic projection systems possible – it simply weights each of the last three years with weights of 5/4/3, adds a very basic regression to the mean, then adds a very basic aging projection. Marcel is very basic on purpose – it’s still pretty damn accurate, and if a more complicated forecasting system can’t beat Marcel in baseball, it’s useless. Surprisingly, most forecasting systems don’t improve upon Marcel by very much.
Eric’s Hockey Weights come out to weights of 5/3/2.5/1.5, which is pretty similar to the 5/4/3 of baseball’s Marcels. So let’s use these weights to create Hockey Marcels.

Of course, as Eric noted, we can’t simply use these weights as is (or well, doing so will work, but won’t be as accurate as you’d like). We still need to regress each player toward the average, especially in the cases of players with smaller than optimal sample sizes – after all, we’re a lot more confident in the weighted average of Lundqvist of a .9228 on 3327 shots than we are in Cory Schneider’s .9295 on 1869.7 shot sample. Tango did this by adding league average at bats until he had a certain # of at bats for each player, and we can do the same thing here. In this case, I added shots saved at the average rate until each player’s sample was 4000 shots strong. This is the weakest part of this method by the way, since my selection of 4000 was kind of arbitrary – 4000 is a general minimum for when we feel somewhat confident in a goalie’s stats, although it’s usually the # used for even strength shots and here we’re using all situations. However, it leads to all goalies facing at least SOME regression adjustment, which is what we would want.

The end result is in the chart down below. However, we shouldn’t forget the last part of baseball Marcels, the aging adjustment. Unfortunately, Hockey aging curves, especially for goaltending, aren’t quite as well founded as for baseball, and I couldn’t find one that I could use to create a very simple formula to adjust the data. So the below data does not include an aging adjustment. However, it should be fairly simple to mentally adjust the data downwards for players on the wrong side of 30, where goalies clearly start to decline.

NOTE: The Following Data is through 1/31/14.

And without any further ado, the data:

Player

Age

Projected 3 Year SV% After Regression

3 Year Weighted Sample

Projected 3 Year SV% w/out Regression

Tuukka Rask

26

0.9223

2295.3

0.9280

Cory Schneider

27

0.9216

1869.7

0.9295

Henrik Lundqvist

31

0.9214

3327.0

0.9228

Ben Bishop

27

0.9198

1711.5

0.9266

Ryan Miller

33

0.9189

3517.2

0.9195

Tim Thomas

39

0.9184

2373.8

0.9210

Pekka Rinne

31

0.9184

2537.6

0.9206

Roberto Luongo

34

0.9182

2718.0

0.9198

Jonathan Bernier

25

0.9179

1837.7

0.9216

Ben Scrivens

27

0.9176

1122.7

0.9251

Jonathan Quick

28

0.9171

2643.2

0.9183

Robin Lehner

22

0.9171

1039.8

0.9238

Carey Price

26

0.9168

3496.9

0.9172

Kari Lehtonen

30

0.9167

3413.4

0.9170

Anton Khudobin

27

0.9166

732.1

0.9254

Jimmy Howard

29

0.9166

2857.4

0.9174

Braden Holtby

24

0.9165

1885.9

0.9186

Semyon Varlamov

25

0.9164

2909.9

0.9171

Antti Niemi

30

0.9164

3390.8

0.9167

Marc-Andre Fleury

29

0.9157

3092.2

0.9160

Sergei Bobrovsky

25

0.9156

2404.0

0.9162

Mike Smith

31

0.9153

3142.6

0.9154

Jaroslav Halak

28

0.9152

2100.5

0.9156

Brian Elliott

28

0.9150

1750.9

0.9154

Craig Anderson

32

0.9149

2951.5

0.9150

James Reimer

25

0.9144

2145.2

0.9142

Jonas Hiller

31

0.9141

2852.4

0.9139

Jhonas Enroth

25

0.9138

1181.3

0.9118

Jean-Sebastien Giguere

36

0.9136

1295.5

0.9115

Al Montoya

28

0.9134

1130.3

0.9102

Corey Crawford

29

0.9132

2670.4

0.9125

Cam Ward

29

0.9131

2605.2

0.9123

Peter Budaj

31

0.9130

1093.0

0.9085

Justin Peters

27

0.9130

1149.0

0.9087

Michal Neuvirth

25

0.9129

1409.1

0.9097

Niklas Backstrom

35

0.9125

2299.5

0.9109

Evgeni Nabokov

38

0.9123

1989.4

0.9099

Ray Emery

31

0.9118

1260.6

0.9055

Dan Ellis

33

0.9112

1069.7

0.9018

Scott Clemmensen

36

0.9108

1218.0

0.9019

Ilya Bryzgalov

33

0.9106

2578.1

0.9084

Jonas Gustavsson

29

0.9103

1429.5

0.9023

Devan Dubnyk

27

0.9101

2645.1

0.9078

Steve Mason

25

0.9099

2620.4

0.9074

Anders Lindback

25

0.9099

1164.0

0.8982

Kevin Poulin

23

0.9096

1052.8

0.8952

Martin Brodeur

41

0.9081

2214.3

0.9029

Ondrej Pavelec

26

0.9062

3507.1

0.9050

You’ll note only three players show expected SV%s, after regression toward the mean, above .920, the standard for the very elite, over the next three years (Rask, Schneider, Lundqvist). Of these three, we’re clearly the most confident in Lundqvist due to his 3300 shots, which is why his projection barely changes after regression, whereas Rask and Schneider’s projections drop a ton.

Similarly, only 5 goalies show up under .910, which is kind of terrible for starting goalies facing this # of shots. Admittedly, for two of these goalies we have very tiny samples (Poulin and Lindback both give us samples barely over 1000 shots), but those samples have been so poor, that even the huge regression hasn’t put them above .910. By contrast, Ondrej Pavelec somehow gives us our 2nd biggest sample of the above-goalies – so the regression barely helps him. Of course, when we consider aging, Brodeur probably should project to have a worse performance in the future than Pavelec, but that’s not saying much.

One final note: These #s act as if the league itself isn’t going to change, but of course, SV% has been rising for years. If it does, one would expect most of these guys to perform “better” than the results above, although their rankings should still be the same. The point isn’t really to project absolute SV% for the next year as much as it is to project the quality of goalies over the next three years. I think this seems like a pretty solid way of doing so.

First time visiting your site, like it. I’ve read some interesting stuff on how coaches can influence save percentage. Can that be considered? Albeit in an admittedly small sample Pavelec looks way better with Maurice coaching. From watching the games it makes sense, the Jets were often guilty of incredible defensive breakdowns that left the goalie out to dry, which can’t help but kill save %. Thoughts?