Category Archive

Note: This was originally a series of obnoxiously long comments on this post at BoxscoreGeeks.com. If somehow you’ve arrived here without having previously visited that site, you should go there, it’s really good!

There’s no real back and forth, so I’m preserving the posts as originally written without extra editing with the exception of converting the dynamic posting time indicators to something that makes since on a static blog. The times are approximate, and refer to the date this was published 2015 June 4.

At Noon Shawn_Furyan wrote
OK, so you got me. I’ve spent all morning looking at this research. I eventually ended up reading all of Miller and Sanjurjo 2014 after skimming the study cited in your quote: Miller and Sanjurjo 2015.

I did a heavy skim of the later study, and realized that its conclusions were essentially completely dependent on the statistical methodology developed in the earlier study. After reading the earlier study and rereading the abstract and introduction to the later study, my preliminary conclusion is that this research really only applies to repeated shots from a particular location. The first study was a controlled study that looked at shots from a specific point designed on an individual shooter level to be made at a rate of 50%. They did multiple 300 shot sessions with several players on a single Euro team.

They conjectured that the results applied to game situations based on significantly less rigorous evidence. Unless the 2015 study controls for the potential stationary shooter effect in a way not foreshadowed by its introduction, then really all of the new evidence will lack controls for the stationary shooter effect, and so really can’t be inferred for game situations.

In the 2014 study, they do a good job discussing this issue in a kind of buried section, but IMO, their conclusion and abstract don’t discount the claims appropriately (meaning they didn’t find evidence that Kobe can just have a stupid hot streak in a game, when I think that it’s pretty clear that this is the takeaway many people, particularly reporters, are likely to pull from reading those portions of the study).

One situation where the effect could potentially apply is that thing where Curry will try to do a quick pullup from the spot he just hit from on the last possession. Based on the evidence I’ve seen here, I’d recommend trying to keep him from doing that if you’re defending him. This is a conjecture, but it’s closer to the ‘demonstrated’ ‘hot hand effect'[1] than the conjecture by the authors that the effect is general for particular shooters.

That’s the other thing. A primary aspect of the proposed effect is that the authors assume that it’s not really a wide-spread phenomenon. Rather they think that they have identified that particular players are able to get significantly hot, but that most players don’t or that the effect is very subdued in most players.

[1] I would note that the 2015 study will be simultaneously discredited if the 2014 study gets discredited, and that the 2014 study has not been replicated as far as I can tell.

.At Noon Shawn_Furyan wrote
I’ll note that I’m not saying anything about the statistical methodology. They don’t look particularly convoluted, and shouldn’t take a huge amount of statistical expertise to verify most of the methods, but that would have taken a lot more time and I’ve already devoted a lot of time just examining the claims.
Shawn_Furyan wrote 5 hours ago
That is to say that my first comment should be taken as an interpretation of what we will have evidence for should the statistical methods of the 2014 study stand up to significant scrutiny. I think that this already diverges significantly from what people are likely taking away from the reporting on the studies in question.

At 3PM Shawn_Furyan wrote
I’m in the middle of the second study now. I REALLY don’t like one of their citations to the previous study. I would say that they are mischaracterizing the evidence, which I actually read earlier today:

“Miller and Sanjurjo (2014) [edit: to be clear, this is a self reference, so it’s not plausible that they just misunderstand the findings they are citing] find, in a detailed questionnaire administered to the expert players who participated in their controlled shooting experiment (and did not observe each others’ shooting sessions), that not only do all of the eight players believe that many of their teammates generally shoot better when on a streak of hits, but only one player believed that all of them do. Further, both their ratings (-3 to +3), and rankings, of how well their teammates shoot when on a hit streak are highly correlated with the actual shooting performance of the teammates in Miller and Sanjurjo’s experiment. When this evidence is taken together with the fact that expert coaches and players have much more information to go by than simply whether a teammate has made or missed the last several shots (e.g. how cleanly shots are entering or missing, the player’s shooting mechanics, body language, etc.), what is suggested is that not only can experts identify which shooters have a greater tendency to get the hot hand, but that they may also sometimes be able to tell the difference between a lucky streak of hits and one that is instead due to an elevated state of performance.”

OK, so 8 dudes collectively, but with significant disagreement, identified the ONE SINGLE GUY on their team who’s probably the main shooting threat as most likely to get a hot hand. One of the 8 players gave everyone the same ranking. In the original article, this is presented as a corroborating sort of sanity check (though they don’t discuss alternative hypotheses like the one I just presented, that the agreement is mainly driven by identifying who the shooter on the team is), and kind of spin that into a conjecture that it’s plausible that this means that the study authors have identified an effect that exists in in-game scenarios. I didn’t take much exception to that, but citing it in another paper as if it’s some sort of significant finding, and stripping out the context is laundering the result of it’s lack of rigor. They even take pains to couch the citation in authoritative language: ‘a detailed questionnaire administered to the expert players’, ‘controlled shooting experiment’, ‘highly correlated with’ (remember, n = 8 with one of those possibly not being taken seriously), as well as language that downplays the lack of substance: ‘only one player believed that all of them [benefit from the hot hand][or alternatively, one of them just picked ‘3’ across the board because questionnaires are annoying], spelling out ‘eight’ (as opposed to ‘n=8’) which is kind of easy to miss when reading because of the density, but which is also completely unscannable, and then finally DOUBLING the length of the section by spouting a bunch of completely unsubstantiated theorizations in order to bolster the perceived effect!
I take huge exception to that.

At 4PM Shawn_Furyan wrote
I guess I was closer to the end than I thought, and the appendices weren’t significant. Yeah… I don’t really see the 2015 study as really adding anything of note.

They seemed to recognize in the 2014 paper that having to shoot against a defense and from different locations is a confounding variable, but then they kind of hand wave that away. I would have expected a followup to confront that issue directly, but instead, the produced essentially more of the same, and then doubled down on and ballasted the hand waving.

Also, they haven’t really dealt with a warm up effect directly, they just observed that in some of their trials that it took about 2 shots for everyone to hit their average shooting efficiency. But in games, shots are much scarcer, and there’s a lot more chaos, so I don’t think that the 2 shot warmup period is directly transferrable to game situations.

They also acknowledge, but don’t really take on, the issue that players shoot less efficiently in the beginning of the season.

These are two major effects that could explain any findings of clustered of makes beyond what one would expect if a players shooting efficiency was constant over time. In light of those unanswered issues their attempts to transfer the effect to game situations are very weak, and honestly the papers would have been better if they’d put a lot less emphasis on trying to stretch their research to cover that case. The research just doesn’t address the situation in any meaningful way.

At 5PM Shawn_Furyan wrote
Another thing that I haven’t looked into at all, but which gives me pause, is that in both papers, the authors essentially slam everybody else who’s studied the effect, including explicitly calling out Amos Tversky (who’s obviously no whippersnapper), and tout their pretty simple metrics as the solution to the errors in all of the previous research. Their (in english) claim is essentially that past methods weren’t sensitive enough to clustering, and that the methods developed in their 2014 study are tuned just right to catch the signal [presumably while filtering out the noise adequately]. The number observations in the two studies look on their face to be pretty high, but overall they include only 41 players, and in neither study can the case be made that the player samples are particularly representative of general populations of basketball players within particular leagues, et cetera. The first study has an alarmingly small population of players (n=8), and second has selection criteria so stringent that it only finds 33 candidates that have met the criteria since the mid 80s (and of course we’re talking about the 3 point competition, so we’re already selecting for the most effective shooters). I don’t recall any indication of how many 3 point shooters were selected against, and I’m not sure that it even makes sense to do a second selection if you’re already selecting specifically for the best shooters. Also, in the second study, the median number of observations was like 143, but the biggest outlier, craig hodges was among the hot handiest, but at 454 observations, puts him 4 standard deviations above the median. The researchers report only the average in the study, and use the average to come up with their conclusions, but I do wonder if that one player is significantly positively skewing the distribution.

I also wonder if the study wouldn’t be significantly improved by not filtering out shooters that are already supposedly very effective, but rather looking at all of the shots so that the potential ‘hot hand’ shots are presented in terms of all shots rather than isolating them out on some pretty darn synthetic criteria. I’m not 100% positive that this would be an improvement, but the authors don’t attempt to justify their decision to filter out players from an already highly specific non-representative sample.

One last thing… the authors take pains in both studies to insist that their findings are the ones that are consistent with conventional wisdom (which I guess is assumed to be consistent as well… they don’t really address that…). But the findings of the second study really don’t strike me as consistent with conventional wisdom once you get past the headline. So… the ordering of their hot handedness metric doesn’t seem to be in any way predictable ahead of time. Reggie Miller is apparently inverse hot handed, Peja is the most inverse hot handed… OK Glibert Arenas seems to make sense as being among the inverse hot hand group, but Dirk Nowitzki has less hot handiness than Glen Rice? I’m cherry picking a little, and I’m sure you could make a case for yourself after seeing it, but look on page 13[2]. I mean, is your head nodding vigorously?

It finally happened. The event that embodies the culmination of the hopes and dreams of each of the NBA’s less fortunate fans, whether they hail from New York, Northern California, New Jersey or Minnesota. The NBA Draft is a time when such fans look to the future, confident about their favorite teams prospects for improvement. It is a time of infinite possibilities, a time when a single draft pick can change the face of a franchise. Fans hope to nab the next MJ, Lebron, Shaq, or Duncan. Will John Wall be the next Chris Paul or the next Earl Watson? Will undrafted Brian Zoubek be the next Ben Wallace or the next John Doe? In this Draft review, we will try to answer those questions, and more.

Let’s kick things off with a summary of the order in which players were drafted on Draft Day (See figures 1.1 and 1.2)

Fig. 1.1 - 2010 NBA Draft - Round 1

Fig. 1.2 - 2010 Draft - Round 2

Nobody was surprised when the Wizards used the #1 draft pick to take John Wall. Every major Mock Draft available had Wall filling the top slot. Similarly, Ekpe Udoh, and Greg Monroe were considered consensus top-10 talent, and Eric Bledsoe, Avery Bradley, Daniel Orton and Craig Brackens were all considered solid first round picks. When we look at overall production via the PAWS40 metric however, we see a different story.

This metric, which is adjusted on a per 40 minute basis (see the PAWS40, and PAWS40 Rank columns in Figure 1), shows us that John Wall was a slightly below average producer this last year in college with a PAWS40 of 10.0 (average PAWS40 from 1994-2005 for college players was about 10.2), and there were 41 players in the draft that were more productive on a minute adjusted basis. Udoh, and Monroe fared very slightly better, each producing PAWS40 of 10.1. As for Eric Bledsoe, Avery Bradley, Daniel Orton and Craig Brackens, despite being first round draft picks, each were among the 10 least productive of the 79 draft prospects we looked at. A full 27 draft prospects that were more productive than these first round picks went undrafted.

Draft Winners

So which teams came out ahead in this years draft?

Oklahoma City Thunder

The Thunder were this year’s Blazers. They were involved in 4 first round and 4 second round picks. They made the most notable move in the draft by trading away their 21st and 27th picks, Craig Brackens (horrendous PAWS40 of 6.3, especially for a 22 year old) and Quincy Pondexter (who was actually quite a bargain at #26 with a PAWS40 of 12.5), in return for the 3rd most productive player in the draft, Cole Aldrich (PAWS40 of 15.2). In the second round they acquired big man Tibor Pleiss of Germany from Atlanta (who had acquired him from the Nets previously), and Latavious Williams who spent last season in the NBA Development League playing for the Tulsa 66ers where he achieved a PAWS40 of 11.8. Essentially, the Thunder went into the Draft with 3 first round draft picks, the highest of which was 18th, and came out of the draft with the 3rd most productive player, one player that was productive in the D-League, as well as a slew of longshot prospects which includes a 7 foot tall German who scores fairly efficiently and rebounds at a high rate (any perceived parallels to Dirk Nowitzki are strictly unintentional).

Sacramento Kings

The Kings went into the draft with the 5th and the 33rd pick. At 5, they drafted DeMarcus Cousins, the 19 year old center from Kentucky who was the 2nd most productive player in the draft. Cousins is probably the best overall prospect from this years draft given the high level of production he was able to achieve at such a young age. NBA draft commentators knocked him for his lack of athleticism, but college production has been shown to be correlated with NBA production, whereas, say, vertical leap has not. At number 33, the Kings drafted another center, Hassan Whiteside, who was the 10th most productive of the 79 players analyzed. It should also be noted that the Kings recently acquired the very productive Samuel Dalembert (2010 wp48 of .243), in exchange for the unproductive Spencer Hawes (2010 WP48 of -0.007) and Andres Nocioni, an old and formerly, but no longer productive small forward (2010 WP48 of -0.015). This leaves the Kings with two fewer unproductive players and a productive veteran center, and two rookie centers that have been productive at the collegiate level.

New Jersey Nets

The Nets took power forward Derrick Favors with the 3rd Pick in the draft, and while Favors was only ranked 23rd in terms of PAWS40, he was the youngest player in the draft, and still managed to produce at an above average clip. They also acquired the Hawks first round selection, the 23rd pick Damion James, who turns out to have been the most productive player in the draft (and I might add, has also spent the last 4 seasons playing here in Austin, TX, for UT), in return for their 27th and 31st picks, Jordan Crawford (PAWS40 9.9) and Tibor Pleiss of Germany.

Milwaukee Bucks

The Bucks took 4 players in the draft. They ended up with an above average power forward (Larry Sanders, PAWS40 12.5) with the 15th pick, an above average small forward (Darington Hobson, PAWS40 11.6) with the 37th pick, and an above average center (Jerome Jordan, PAWS40 11.1) with the 44th pick. They also acquired 19 year old power forward Tiny Gallon with the 47th pick, who was a bit below average, but at 19 years old, could potentially improve relative to his current PAWS40 rank of 53.

Honorable Mentions

The 76ers came up with the most productive guard in the draft, Evan Turner, with the second pick. The Raptors got Ed Davis, who is only 20 and was also the 8th most productive player with the draft, but if they lose Chris Bosh to free agency in two weeks, then they’ll likely have lost ground overall as well. The Blazers had the 22nd pick and the 34th pick. Though many above average picks were available at both positions, both of these picks, Elliot Williams and Armon Johnson respectively, were well below average. However the Blazers were somehow able to foist unproductive veteran forward Martell Webster on the Timberwolves in exchange for their 16th pick, Luke Babbitt, who was one of the most productive small forwards in the draft.

Draft Losers

If there are winners, then there must be losers. Let’s take a look at the teams that fared worst on Thursday.

Wizards

The Wizards ended up with 5 draft selections, including the number 1 overall pick, and didn’t manage to come up with a single player that produced at an above average level this last year. As for the #1 pick, it’s difficult to be too hard on the Wizards for doing what everyone expected them to do. In 5 years if John Wall is a merely average player, nobody will blame the Wizards front office for drafting him. If they had gone with another draft pick though, and John Wall ended up being more productive than his college production would indicate, then the Wizards front office would be widely seen as inept. On the up side, Wall is only 19 years old, and still managed to produce at a level that was close to average, so there is some room to hope that he ends up living up to the hype, though the odds do not favor this outcome.

Golden State Warriors

With the 6th overall pick, the Warriors chose Baylor junior Ekpe Udoh, a 22 year old power forward. There really are no redeeming qualities to this pick. Udoh was below average, and is too old to be expected to improve his position relative to the rest of this draft class. Further, he will be guaranteed a number 6 pick’s salary while most likely being unproductive in the NBA.

New Orleans Hornets

If financials are not considered, the Hornets are one of the the biggest losers of the draft. They traded the third most productive player in the draft, Cole Aldrich (PAWS40 of 15.2), for Quincy Pondexter (PAWS40 of 12.5) and the 6th least productive draft prospect, Craig Brackins (PAWS40 of 6.3) along with Morris Peterson who produced in the negative range for the Hornets last year. That being said, this move was clearly motivated by the desire to dump Peterson’s $6.6M salary while avoiding having the guaranteed contract of an 11th pick (rookie salaries of first rounders scale according to draft position such that the 1st pick makes the most, and the 30th pick makes the least). The Hornets have saved a lot of money, and many teams came out with worse picks than Quincy Pondexter, but it’s always hard to see bad basketball moves made for financial reasons.

Detroit Pistons (Sorry DJ, maybe next year)

Greg Monroe tied Udoh for second least productive top-10 draft pick with a PAWS40 of 10.1. Unlike Udoh, he is only 19, so he has time to improve. Still, there were many productive big men in this draft. There was no reason to gamble the 7th pick on power forward who hasn’t shown that he can be productive.

Boston Celtics

The Celtics threw away the 19th pick of the draft on Avery Bradley (who also, though I’m somewhat less excited to say it, played here in Austin, TX this past season). Avery was the third least productive player in the draft, and even though he is only 19 years old, at half the production of an average college player (his PAWS40 was an abysmal 5.4), can not be justified as the 19th overall pick.

Undrafted Gems

There were several productive players in the draft that sat through 60 picks without hearing their name called. Many of the players from the figure 2 below achieved good numbers in smaller conferences. Most are also 21 years old or older and thus don’t have the allure of limitless upside. Still many of these players are certainly worth a 14th or 15th roster slot.

Fig. 2 - Undrafted Prospects

Brian Zoubek

Brian Zoubek filled up every category in the stat sheet last season for Duke (excluding 3 pointers, but including turn overs and especially fouls). His stats don’t look great at a glance, but he only played 18.7 minutes a game. If we look at his stats per 40 minutes, he looks much better. Of course, he fouls at a very high rate, so his minutes will be limited for the foreseeable future, but he is likely to be very productive for the minutes that he is able to play. Really, what team couldn’t use a productive center to soak up 10-20 minutes per game? In order to put things into perspective, Figure 3 compares the per 40 minute production of the undrafted Brian Zoubek, and that of the 13th pick, Ed Davis since the two played in the ACC, played comparable minutes, had the same PAWS40, and are both big men.

Fig. 3 - Brian Zoubek vs. Ed Davis

In per 40 minute terms, the two are evenly matched. Both are efficient scorers, but Davis shoots more, so he gets a bit of an advantage. Zoubek is much better than Davis in getting extra possessions for his team and has a huge advantage in that category. Davis though has a large advantage in blocks and personal fouls. On the whole though, the per 40 minute production of the two players this last season was equal. Being that Brian Zoubek compares well with Ed Davis who was a (legitimate) lottery pick, it is clear he would be a very nice pickup for any team that could use a center that doesn’t have to start immediately.

Zoubek is the only player at the top of the undrafted players list to come from a major conference. Still any teams that were to pick up Jeremy Lin, Artsiom Parakhouski, or Omar Samhan are likely to find that they have a more than serviceable player on their hands.

Other Thoughts

Many a Kentucky player seems to have profited from playing on the same team as DeMarcus Cousins. Cousins had a stellar PAWS40 of 15.6. The four other players from Kentucky drafted (all in the first round, and of course Wall was even drafted before Cousins) had an average PAWS40 of 8.3. Perhaps 4 thank you notes are in order. Avery Bradley also seems to have profited from the success that Damion James and Dexter Pittman brought to the Longhorns. He was drafted before both though he did not achieve even half of the production of either.

-Shawn Ryan

P.S. This article was written well before the the main free agent moves that have happened at this point. I will try to add a follow-up post or two in the near future to look at some of the issues raised given the current, and quite different landscape of the NBA. Also, it looks like Chris Bosh did leave Toronto, so acquiring Ed Davis, while certainly not a bad move, is still not likely help a whole lot.

Update: Big thanks to Sportsfanatic613 for corrections.

If anyone else has any comments or criticism, let me know. I’m new at this, so I’m very interested in knowing what you all think, and how the I can improve. I’ll respond to every comment that I can.