Investigation into the Sustainability of Team Percentages, Part I: PDO

In the evolving world of advanced hockey metrics, one of the simplest and most useful observations has been that shooting percentages are highly variable in the short term, allowing players like Jordan Eberle to shoot 18.9% last year. Certain players are able to sustain higher than normal personal and team on-ice shooting percentages, but what about teams?

We use a concept known as PDO to describe the encapsulated luck of a team or player. Even strength PDO (which I’ll refer to just as PDO) is a combination of a team’s even strength shooting percentage and their even strength save percentage. For instance, if they are saving 0.920 of shots made by the other team, but scoring on 0.080 of their own shots, this adds to 1.000, meaning they’re shooting as well as they save. If one of these components gets off kilter, say in this year’s Calgary Flames, things start to look wonky: the Flames have a normal 8.3 shooting percentage, but an historically bad 0.885 save percentage, meaning they have a PDO as a team this year of 968, the lowest in the league.

Casual theory of PDO suggests that it regresses towards 1000 over the long run. Various teams that have had PDO’s well above 1000 while putting up great records are assumed to be punching above their weights on the strength of luck, and are expected to fall in the standings. It follows that truly good teams tend to out-shoot other teams, meaning that they can score more goals than they allow through the simple observation that you tend to score more goals when you shoot more often (making a normal shooting percentage perfectly fine). If you’re scoring a lot because of a lucky shooting percentage, you tend to fall back down to Earth.

In this series of posts, I want to explore this concept to test whether teams truly do tend towards PDO of 1000. Today I’ll start with PDO itself.

For this post I’ll use team shooting data obtained from behindthenet.ca. I gathered each team’s shooting and save percentage at even strength for every season since 2007-2008, or 6 seasons of data (including this partial one). I then found the mean and standard deviation among the 30 teams for each year’s data for each of shooting percentage (SH%), save percentage (SV%), and PDO. Using these, I transformed each team’s data for each season for each measure using Excel’s NORMDIST function, which translates each number into a normal cumulative distribution score between 0 and 1. If a team’s score was 0.50 for a certain metric, that means it was right on the average. Anything above 0.50 means it is above average, anything below 0.50 means it is below average. Here’s a listing of each team’s PDO since 07-08:

Here you can see each year’s score between 0 and 1, along with an average of the 6 years of data and the rank of this overall average. Right away, we can see that some teams have consistently below average scores, and some have above average. Here’s a list of the top and bottom 5 teams in terms of overall average scores:

So, teams like Vancouver and Boston are able to sustain very high average PDO scores of 6 years of data, while a team like the NY Islanders can be incredibly below average over this timeframe. You would expect teams to tend towards 50% over such a long time frame if PDO really regressed towards average under any circumstance. How are the Canucks able to sustain such a high degree of “luck” over 6 years of data?

I plotted each team’s average PDO on a scatterplot to visualize how far or close each team is to this 50% long run average:

You can see that while the data does seem to tend towards 0.50, there are many dots outside of my arbitrary yellow band that are within +/- 10% of the average line. I could potentially buy a few outliers here and there, but this graphic shows that there are many teams who sustain long-run PDO’s well above or below league averages.

Here’s another way to think about this: we have an assumed probability that for each season, a team has a 50% chance to be above league average, and a 50% probability of being below league average. It’s essentially a coin-flip, in the assumptions of many (including my own). Some years you get lucky, some years you don’t, or so the thinking goes.

To test this, I needed to create a massive probability tree with 6 branch stages with two branches of 50% probability each. You can conceptualize what I’m getting at here by thinking about what the probability is of a team posting 6 straight seasons above league average PDO: 50% * 50% * 50% * 50% * 50% * 50% = 1.56%. The probability tree fleshes out all these options to come up with a full picture of what our expected probabilities are.

After the tree is constructed, I can add up all the probabilities to create a table of expected outcomes:

So, from this table, you can see that our expected probability of having 3 above average seasons and 3 below average seasons out of 6 seasons is 31.3%. Out of 30 teams, this means I’d expect about 9 teams to have such a track record. So what does the data say?

This can be graphically represented as follows:

To me, this chart is fascinating. We see that only 20% of teams had 3 above and below average seasons — when compared to our expected mark of 31.3%, we begin to make our case that PDO does not, in fact, randomly swing up and down. If we’re missing teams with even performances, where are they? Almost entirely in the far left and right of this graphic, or teams that had either 6 or 5 seasons of either above or below average PDO.

This suggests that PDO is a concept that does not even out over time in some cases — certain teams can attain an established level of “luck”, either good or bad, and remain consistent in that level. In the velocity of the NHL, this is not surprising. Some teams have consistent rosters, goalies, top lines, coaches, etc that will all tend to reproduce similar results year after year. Look at the top and bottom 5 PDO teams again. The top teams tend to be good teams, and the bottom teams tend to be bad teams. Now, it’s been well-established that teams who outshoot other teams consistently are also pretty damn good teams, like the LA Kings or Detroit Red Wings. But we can start to get the sense from this data that good teams can also be ‘good’ because of extraordinarily high PDO’s; teams like the Canucks, Penguins, and Bruins. In short, outshooting is incredibly useful and important, but it’s not the only means to become a consistently good team.

In the following posts of this series I’ll treat the components of SH% and SV% similarly to how I’ve broken down PDO here today in order to get a better sense of which component drives this stability in PDO more than the other.

But I want to close with one final statistical test. If PDO really was random, a team’s PDO in one year would have no statistical power to predict their next season (ie, no autocorrelation would be seen in the data). I set up an experiment where I regressed k-1 seasons as an explanatory variable against k seasons as the dependent variable for each team for each year. I found a formula of:

Predicted k season PDO = 0.36 + (k-1 season PDO) * 0.26

Now, this only had an r-square correlation of 0.067, but what’s important is that the previous season variable had a P-value of 0.0014, meaning that I can reject a null hypothesis that a previous season’s PDO has no explanatory power for a subsequent season’s PDO. It may be a small influence, but it is apparent and statistically significant.

This means that teams with high PDO seasons tend to have high PDO seasons afterwards, and vice versa.

4 Comments

It does make sense that some teams consistently have PDO’s above or below what you’d expect if PDO’s were just randomly distributed. This is because some goalies are better than others. Teams that can keep good goalies for several years should therefore have higher PDO’s.

I knew the algebra police might drop by . Thanks for the link, that’ll come in handy. I chose the tree format here because it’s a bit more intuitive for people to visualize the concept, and also because I miss using that Excel Add-In.

4 Trackbacks

[...] Tuesday I wrote a post Part 1 of this series looking into the sustainability of team shooting and save perce…, with a concentration on PDO (which combines both shooting and save percentage into an index that [...]

[...] team PDO and team save percentages — today I will concentrate on team shooting percentages. We’ve seen evidence suggesting that PDO is not just a random walk through peaks and valleys, it can be sustained in a fashion more [...]

[...] post continues a series of examinations into the concept of team ‘luck’. I initially wrote a three part series on team PDO, testing both it and its constituent parts to find out if they were truly random over time. I [...]