Thursday, September 11, 2008

David Sparks is the Arbitrarian. His stats column runs weekly here at HP. This week he discusses depth and its impact.

The survey responses to last week's post were so interesting, I decided to do an immediate follow-up (if you haven't read it, you may want to do so before continuing here). Last week, we focused on team rotation size, as measured by minutes played. Today, we will look at a very similar, but somewhat more interesting concept: team depth.

Depth and rotation are not necessarily the same. Since there must be five players on the court per team at all times, the theoretical minimum for rotation size is five, which you would see if a team played only five players, all game, every game. However, depth concerns not playing time, but production, and it is easy to imagine one of those five players contributing more than 20% of the team's total production, while one or more of the others produces less than their share. (There is a metric, called the Valuable Contributions Ratio, which I use to measure players' productive contributions relative to their floor time.)

If each player produced in proportion to their allocation of minutes, it would make no difference which players were on the floor, but obviously this is not the case. Rather, better players produce a greater proportion of their team's production than their proportion of a team's minutes played. This implies, of course, that a team's rotation size will likely not be the same as its productive depth, and further, that depth will likely be smaller than rotation.

In fact, depth can be calculated in exactly the same way as rotation (see last week's column), except instead of using minutes as the variable of interest, we use Model-Estimated Value (MEV), a productivity metric.

So many theories

Last week, I invited readers to speculate about the relationship between rotation size and team success. You submitted countless interesting ideas in response to this question, and made many other interesting suggestions about ways to assess rotation consistency, variations in rotation size by coach, and differences between regular-season and playoff play, among others. I hope, in time, to investigate some of these great ideas.

For now, let us turn to the relationship between rotation size and success. In response to my question, the plurality of respondents said that wins and rotation size would positively correlate, many noting that deeper rotations would probably enhance a team's chances in the playoffs.

Others suggested that the relationship would be negative, due to the fact that poorer teams needed to give more playing time to younger, weaker players, to aid in their development.

A large minority of answers indicated that there should be no consistent relationship. Several of these claimed that rotation size was too idiosyncratic: a function of the coach, playing style, and available personnel, and successful teams could make any sort of rotation a winner.

Several others predicted a parabolic relationship, in which the smallest rotations would find success on the back of a few stars, the largest rotations succeed through roster flexibility, and those in the middle, by failing to follow either strategy, will not do well.

I must admit that I was intrigued by all of these arguments, especially the parabolic prediction. My personal hypothesis was that increased rotation size would lead to greater success, due to the positive effects of diversification, as in the stock market. With more diverse contributions, I thought, would come greater insurance that even if one player failed to show up, one or more of his teammates would pick up the slack and ensure victory.

There were a number of other interesting hypotheses: one was that since defense requires a greater exertion of energy and offense requires time to find a rhythm, defense would correlate positively, and offense negatively, with rotation size. Other noted that faster-paced teams may require longer rotations, due to greater energy expended per minute. Several others suggested that the age of the team would vary positively with rotation size, as younger players can typically play a greater number of minutes without hurting productivity.

The empirical evidence

Who was most correct? Well, first I should mention that part of the problem with my question last week was that rotation size was often conflated with depth, which I define as separate concepts. That said, after reviewing the graphical relationships, I must sadly rule out the parabolic hypothesis. The rest of the relationships (between all suggested variables), are depicted in the correlation matrix below:

Rotation and depth are measured as described previously. Game age is the playing-time-weighted age of the team. Possessions are a measure of pace. Offensive efficiency is a measure of a team's scoring per possession, while defensive efficiency measures the same thing for their opponents (so better defensive teams have a lower defensive efficiency as constructed here). Efficiency difference is a measure of absolute quality, subtracting defensive from offensive efficiency.

Many of these results (the ones close to zero) indicate no relationship: Rotation size seems to be unrelated to anything but depth. However, depth appears to be positively correlated with offensive efficiency, and thereby, also positively correlated with efficiency differential--apparently teams with greater depth (at the per-game level) see improved efficiency differentials. One problem is that we cannot tell which direction causality moves in. Do deeper teams play better, or do teams who are winning by a lot give bench players increased minutes and thus increased time to produce?

To some extent, the likelihood of the second option can be tempered by the fact that rotation size has no real relationship with efficiency differential, but this question is still not definitively settled.

Expanding our scope

How have rotation sizes and depth changed over time? Which teams, historically, are the deepest? Due to data limitations, to investigate these questions, I must change the way I measure rotations and depth. Instead of assessing these at the per-game level, to make historical comparisons, I will measure at the season level, meaning that from this point on, rotation is best understood as the inverse of the concentration of minutes played over the course of the season, and depth is best understood as the inverse of the concentration of production over the course of the season. In general, these figures will be higher than each team's mean per-game figures, due to changes in the roster and substitution patterns over the course of a season. However, error ought to be normally distributed, and so I will press forward using these slightly modified metrics, which are interesting enough in their own right.

As you can see in the plot above, both rotations and depth have increased over time. Rotation is denoted in red, and depth in cyan, and both are greater now than they were in the early years of the NBA. There could be any number of reasons for this--expansion, and the dilution of the talent pool, could be responsible; or merely a realization that heavy minutes' loads may shorten player's careers. Incidentally, I have scaled the size of each team-year marker to their winning percentage, but the relationship between depth, rotation, and winning is unclear in this depiction.

Below, I plot team winning percentage (jittered) against team depth. The color scale indicates rotation size, going from small (red) to large (blue), so that if you see a blue team amongst several red ones, you know that that team has a relatively large rotation given its depth. I've also scaled markers by year, so that more recent teams stand out more.

The first thing I notice is the outliers. The most concentrated teams appear to be several Chamberlain squads, in which he was an absolutely dominant producer, and carried his team more than any other player ever has on a consistent basis.

The least concentrated teams are several more recent, and fairly bad teams, topped by the 2002 Chicago Bulls, who were very deep with potential that had yet to develop into actuality.

As noted above, depth has increased over time, and so it is interesting to note the most concentrated teams in a more modern era (which I mark with the inception of the three-pointer, 1979-present). There are two very shallow Utah teams, lead by Malone and Stockton, and supported by almost no one else. The pre-Pippen Bulls show up here, as do the Kobe-only Lakers--teams with one star who did a substantial amount of the producing. We also see the '87 Celtics, '04 Timberwolves, and '08 Hornets, each of which had a couple of extremely good players dominating the contributions to winning, and then filled the rest of the roster out with players who couldn't hope to match the same level of productivity.

Among the very best teams, there is a decent variety of concentration, although it is interesting to see the '08 Celtics at the high end of depth among this elite. Their big three may have gotten the headlines, but it the entire roster made important contributions. Further down and to the right, we see the '08 Rockets, which put on the least likely 22-game winning streak in history, on the back of role players, a different one of which stepped up every night. This team was very successful, given its depth, and it will be interesting to see how this translates to future success.

What does it mean?

The overall trend is a slight but definite negative relationship between team depth and success, but it is unclear what conclusions can be drawn from this. Is this proof that a superstar (or a Big Two, or a Big Three) is key? Does it reflect the fact that it's easier to field a team of equally poor players than a team of equally excellent players?

Since this graphic is based on season-level data, it may just mean that teams with less volatility in their rotation and minimal personnel turnover are more successful. However, I must admit to being unsure of what to make of these preliminary findings. Should teams dump their midlevel players (in salary and productivity terms), in pursuit of a bimodal roster of two stars and ten inexpensive warm bodies? Obviously, constructing a roster requires more than just collecting players at varying levels of talent--the interaction of their abilities is a key consideration--a team is more than the sum of its parts. I would love to hear your insight, explanations, and questions in the comments. Also, I would appreciate your taking the time to fill out the short survey below.