Thursday, January 15, 2009

Shots and Save Percentage Revisited

I wanted to do some more study of the relationship between shots and save percentage. Earlierlooks at this topic have shown little to no relationship, but my recent look at third period play shows a large effect of teams playing to the score. This is likely to be at least partially blocking the underlying shots/save percentage relationship, if there is any, so I thought to use my sample of playoff shots by period to revisit this issue.

I have compiled shot numbers from all playoff games played by Belfour, Brodeur, Hasek, Joseph and Roy from 1994-2008 (not including 1997). I was especially interested in the results from the first two periods, which we would expect to be less affected by scoreboard effects. All the shot rates in this post are expressed in terms of shots against per 60 minutes of play, both because that is a familiar scale for shot numbers and also to allow us to compare to the third period results. For the first two periods, the breakdown was as follows:

Those numbers certainly do suggest that more shots against result in a higher save percentage, although we have to be a bit careful with conclusions from this sample since the numbers reflect the performances of just 5 goalies playing on 10 different franchises. Also, even though the correlation seems pretty clear, there are some other variables that we need to take into account.

One such variable is special teams play. Power play shots are more likely to go in than even strength shots, and if power play shots are a higher percentage of the total shots taken in a more defensive contest we would expect this to lower the average save percentage. If this is true, we would expect that save percentage would be correlated with the total shots in the game from both teams, so I ran the numbers on that one as well (again, first two periods only) and they support the theory:

less than 45 total shots per 60 mins: .912from 45 to 60 total shots per 60 mins: .919more than 60 total shots per 60 mins: .923

Running this type of breakdown on a larger sample consisting of only even strength shots during the first two periods would remove both the special team and playing to the score variables, and would therefore be a good way to test if there is any direct relationship between number of shots and save percentage.

What about the third period numbers? Does the relationship continue when teams start playing to the score? Here are the third period numbers, again expressed in terms of shots per 60 minutes (e.g. 10 third period shots would be equivalent to 30 shots per 60).

The relationship is not as clear, but there still seems to be some evidence that more shots means a higher save percentage. The sample size is smaller so a bit more randomness should be expected, but we also know there is a strong game score effect.

The logical step is to break the third period numbers down by game situation. The numbers are still expressed as shots per 60 minutes, but the sample sizes are smaller so I'm going to compress the groups.

Third Period Shots When Leading After 2 Periods:0-20 shots/60: .92721-30 shots/60: .93431-40 shots/60: .92341+ shots/60: .939

It looks like a similar relationship when tied. However, there is not a clear relationship for either the shots while leading or the shots while trailing. The shots when trailing look, if anything, to be negatively correlated with save percentage, but I would expect that this could partly be explained by strength of opposition, as only dominant teams would be badly outplaying and outshooting the opposition while holding a one goal lead late in the game.

The problem with the tie-game numbers are that they are from third periods that start out tied, but don't usually stay that way. When one of the teams scores a goal to take the lead, both teams would then adjust their style of play according. Based on what we know about score effects the numbers may make sense, if the teams that faced few shots are teams that got scored on early and then had to attack for the last period, and the teams that faced a lot of shots scored early and then sat back to defend the lead. Note that special team factors are not very likely to be at play here, as referees do not generally make many penalty calls in the third periods of close playoff games.

The "playing to the score" effect may be similarly impacting the first and second period numbers as well. I assumed that teams would start playing to the score in the third period, but in a playoff game that they were trying their hardest to win they would quite possibly start playing to the score earlier. That seemed to be the case in many games that I saw, especially when one team went up by a few goals early on and then got outshot for the next two periods. It would be pretty unlikely for two teams that were tied 0-0 in a playoff game through 2 periods to have allowed 25 shots each to that point, as generally teams play cautiously until they are forced to open it up.

Some have argued that goalies are less focused and more likely to let in goals when shots are less frequent. This one is very difficult to test without play-by-play data. As a goalie myself, I can believe that this would have some small effect, but I really doubt it is anything significant enough to even worry about. One possible way to test this would be to look at the performance of backup goalies who come into the game off the bench and see how they do in their first couple of shots against. I'm not sure what kind of situational effects they would be facing, since that would usually be a blowout scenario, but that would be an interesting number to see.

A final consideration is whether chances against the run of play are more dangerous than shots generated in the offensive zone, since the defence is less organized and there are less defenders to beat. There is some evidence in favour of this one from the scoring and save percentages I collected for teams when leading and trailing in the third period. However, team strength is also a factor - if the stronger team is more likely to be leading in the third, and the stronger team is more likely to score on any given shot, then we would expect that teams leading in the third period would have better shooting percentages.

I'd like to see this theory tested by some more advanced metrics like measures of shot quality or average shot distance, with game score taken into account. Note that the third period leading and trailing numbers suggest that this effect, if real, would only apply if one team was substantially outplaying the other team. It seems that in the third period, the leading team usually plays it safe while the trailing team is just putting more pucks on the net, and in that situation we don't see any relationship between the number of shots against and save percentage. Again, however, we may expect that the team leading is more likely to be the better team, which would be counterbalancing any effect.

I think that special teams, playing to the score, and strength of opposition effects are mainly responsible for the stats that show a relationship between shots against and save percentage. More advanced studies could be done using play-by-play data to test each of these variables individually and see the effect on the results, which would allow us to come up with a more definitive answer to this question.

One thing that seems clear from this is that shots taken are at least partially discretionary. The way we see both shots against and save percentage increase for the goalie on the leading team in the third period is evidence of that. This finding has implications for the ability to compare goalies based on shots against totals. A goalie who faces more shots per game than his teammates might be every bit as good in terms of puckhandling and rebound control and so on, and the difference might stem entirely from the team being in the lead much more often with him in net. Or maybe opposing coaches told their shooters to fire from everywhere on that particular goalie (see this game for an example of how that can happen to anyone). There may be certain goalies that have inflated reputations which cause opposing shooters to pass the puck rather than shoot from all angles, or, on the flip side of the coin, there may be goalies who are falsely considered to be weak or have poor rebound control that end up facing a barrage of rubber by opposing teams who are eager to challenge them. In both of those cases, it seems to me a bit unfair to give the goalie the credit or blame for that extra shot differential.

7 comments:

CG: More excellent work. I agree with your caution about how to interpret the various bits of information, all your caveats are certainly worth considering and speak to the complexity of the game and its analysis.

For now just one question, about your very last comment:

it seems to me a bit unfair to give the goalie the credit or blame for that extra shot differential.

Are you referring to the "credit" that goes with a higher Sv% even though it might be influenced by circumstances, or the "credit" that is assigned by folks like me to the goalie's personal impact on the flow of play. I could read this either way. And I daresay the truth, as usual, lies somewhere in the middle.

One of these days one of the code guys is going to figure out how to strip shots/goals by game score right out of the play-by-play data, which should provide an exponentially higher degree of resolution on the effect of game state on shots. But this brute force study you've undertaken is an excellent step.

Yes, I'm definitely looking forward to seeing the results when somebody does a detailed play-by-play breakdown. Especially if they do some of the things I suggested to try to analyze the root causes of shots/save percentage relationship.

Are you referring to the "credit" that goes with a higher Sv% even though it might be influenced by circumstances, or the "credit" that is assigned by folks like me to the goalie's personal impact on the flow of play. I could read this either way. And I daresay the truth, as usual, lies somewhere in the middle.

I had in mind the the issue of assessing a goalie's impact on flow of play through shot totals, but you are correct that save percentages could also be affected by either scenario.

I know you have done a little bit of work determining the relationship between each of the individual goal statistics and their influence upon each other, so maybe you could do a follow up, or at least give you 2 cents on this. I believe you mentioned, and I will agree, that GAA is largely team influenced, and save percentage is not. However, I have noticed that very often, a goalies league rank in one category, is very close, if not often exactly the same in the other. Thus, whatever the reason may be, there would definitely be some sort of relation between the two. If I had to guess, I would say that save percentage is more team influenced than we like to admit. Now, this is not always the case, but way more often than not it is, and I find it interested to note that if the league ranks for one, often indicate where a goalie will measure up in the other, than there is without a doubt relation.

Anonymous: There is certainly a relationship between GAA and save percentage. That is because save percentage is a component of GAA.

GAA = (1 - Save Percentage) x Shots Against Per Game

Therefore, GAA is just another way of expressing save percentage, but there is an additional factor of the number of shots against, which most people would agree is mostly determined by the rest of the team.

That is why I don't like GAA - we have save percentage, why should we muck it up by adding an extra variable that the goalie has little control over (i.e. shots against?)

Now a goalie with an average shots against total will usually end up ranked in a similar spot in both GAA and save percentage for that reason. There will always be a few goalies, however, who face unusually few or many shots and therefore will be ranked very differently in GAA and save percentage.

I agree with you that save percentage is team-influenced, but that is because teams allow different types of shots to get through, not because there is often similarities in rankings between goalies in terms of both GAA and save percentage.

@Contrarian: I recently consulted with my physiology prof (and lab supervisor) on the subject of goaltending, and put together this article. The bit that's relevant to this post is about muscle fibre recruitment, and how it could explain some of the trend towards high save percentages with high shot totals. Obviously, when you're already down in the third, psychologically it may not be there regardless, especially if you continue to get no help, but being physiologically, as well as psychologically, ready may at least help explain the effect.

more shots/60 has a better Sv%--------------------------------These are stats for the two most recent seasons, as of this post. One says that facing more shots will give you a better chance at a good Save%, one says that facing less shots will. I was just thinking that a goalie may face 25 shots/60 like Brodeur but he also may face 20 in one game and 30 in another. Game by game, and different game scenarios, a Save% may fluctuate, but, at the end of the season, it all equals out. I don't think there's much of a correlation between shots and Save% over an entire season. I think if I did this with more seasons, I would get a fairly even split. The best goalies are still the goalies with higher Save%s, whether or not the face 25 or 35 per game.

About Me

I call myself a contrarian because I rely on stats-based analysis over "watching them play", think large sample sizes are more important than "big saves", prefer to rank goalies on save percentage rather than team success, and rate Dominik Hasek as the GOAT. In other words, pretty much the mainstream views of the hockey stats community, but until the far-off day when hockey broadcasters everywhere start talking about ES SV% instead of GAA and wins, I'll have to keep the moniker and my soapbox about the importance of separating a goalie and his team. I am also on Twitter (@tcghockey).