Stopping momentum, part II

In the last installment, I investigated one case of what sports commentators refer to as “momentum” (where a team that makes a successful play will continue making to be successful): outcomes in overtime games. Looking through the CFBStats data from 2005-2011, I found that not only did teams that came from behind to force overtime fail to come out on top at an unusual rate, their outcomes were not affected by other factors such as being the home team or coming back from large deficits. However, I was not entirely exhaustive in my analysis, and two commenters, SpyinColumbus and cgnost, pointed out that it might be interesting to see what, if any role, rankings might play in determining outcomes in overtime.

As it turned out, integrating Sagarin rankings into the CFBStats data was fairly straightforward, and I created a table that matches the CFBStats ID codes (which are the same as used in the NCAA data that CFBStats is built from) with the names that Sagarin uses in his published data. So if you are working with these two data sets and want to put them together, here is the file to integrate these data sources, which is covered by an ODC PDDL (public domain).

With this in mind, the first order of business was to address the issue of what, if any, differences emerge in terms of Sagarin rankings in determining overtime outcomes compared to whether or not teams come from behind. In essence, do teams that come from behind beat their Sagarin predictions? If so, this might suggest that teams coming from behind are bringing some momentum into overtime.

Again, I am considering the set of 230 overtime games from 2005-2011 (dropping the 2005 Arkansas State-Florida Atlantic 0-0 EOR game). I will focus on Sagarin’s “PREDICTOR” model as he regards this as the most useful predictor of game outcomes, though I will also present some analysis using “RATING” and “ELO_CHESS.” PREDICTOR accounts for margin of victory, while ELO_CHESS only considers game outcomes (Sagarin describes it as more “politically correct”). RATING is a synthesis of the two. I also used the year-by-year home advantage values to adjust these ratings, including the 2011 addition of separate values for home advantage for each of the ranking systems. Neutral site games are not adjusted.

One important limitation of this analysis is that, because historical week-by-week Sagarin rankings are not available to my knowledge, all of this analysis is based on his year-end rankings. Because end-of-year rankings are determined by performance in-season, this brings up considerable endogeneity issues that cannot be easily dismissed. The best way to address this would be with the week-by-week rankings, and so if anyone knows of historical data, please let me know and I will see if this changes the results in any meaningful way.

To characterize the results, the first analysis I considered with regard to the ranking was general prediction of overtime outcomes. Sagarin’s rankings use scales with higher values indicating a higher ranked team, and, at least with PREDICTOR, the expected margin of victory. To predict outcomes based on Sagarin’s rankings, I subtracted the PREDICTOR, RATING and ELO_CHESS values of the losing team from the winning team. Thus, positive values indicate that the higher ranked team won (a “normal” outcome) and negative values indicate an “upset.” Based on this, we see the following results for overtime games:

PREDICTOR

RATING

ELO_CHESS

Normal

123 (53.5%)

129 (56.1%)

132 (57.4%)

Upsets

107

101

98

Total

230

230

230

Sagarin’s hit rate for overtime games is about 57% at best and 54% at worst, depending on which of his models is being used. It is worth noting that among non-overtime games, his hit rate is much better (between 78.4% and 80.2% in games between 2005-2011), but this is not surprising because overtime games represent a small sample of games between more closely matched teams (average difference between teams for the ranking systems in non-overtime games is between 10.0 and 10.2 while for overtime games it is between .3 and 1.1). How do Sagarin’s rankings look when considering the way in which overtime is forced?

To do this analysis, I modified my measures somewhat to make the results more interpretable. Since I was focused on teams coming from behind, I subtracted the PREDICTOR rating of the leading team from that of the team that came from behind. This difference therefore represents Sagarin’s predicted outcome for the team coming from behind – if it is less than zero, then the team coming from behind would be predicted to lose the game, while if it is greater than zero, they would be predicted to win.

The overall average for the from behind PREDICTOR difference score is -1.44, which is significantly different from zero (t(229) = -2.19, p < .05), indicating that, on average, teams coming from behind were predicted to lose. A logistic regression with the from-behind PREDICTOR difference score as the independent variable and the game’s outcome as the dependent variable revealed that these differences in PREDICTOR scores did not predict the games’ outcome (Exp(β) = 1.002, p > .85). To further clarify this relationship, I split the data into games where the team coming from behind was predicted to lose (that is, had a PREDICTOR score less than zero) and where these teams were predicted to win (PREDICTOR>0), and compared this to the games’ overall outcome:

From behind loss

From behind win

Total

From behind predicted loss

76

58

134

From behind predicted win

50

46

96

126

104

230

(χ2(1) = .49, p >.48)

What this tells us is that rankings and game outcomes are independent of one another. More directly, while teams coming from behind to tie the game up are more likely to have been predicted to lose, these predictions did not affect how they performed in overtime.

In the context of momentum, this provides further evidence that coming from behind has no effect on game outcomes. Overall, Sagarin rankings are a barely weighted coin flip in overtime games, and how the teams became deadlocked in regulation does not affect this coin in any way.

Thanks, again, for reading, and to cgnost for prompting this analysis. In the next installment, I’ll continue the search for evidence of momentum in traditional defensive stops (those not ending in fumbles, interceptions or safeties), with a special focus on my favorite play in all of football: the goal line stand. Go blue.

I'd be interested to combine overtime rules with that Game Theory Predictor that Brian uses all the time. That is, if you're on offense first in OT, should you go for the 4th and 1 from the 19? It would require figuring out both your odds of success, and the probability that the other team will score a TD vs a FG.

Of course, even armed with the knowledge, coaches will still pull a Mark Richt and manage to lose the game for their teams in OT.

We know game results are highly dependent on turnovers, which might suggest that some teams "play better" than their opponent but are forced into overtime because of turnovers. I would guess that overtime performance is negatively correlated with turnover margin during the regulation portion of the game.

Is there a site where the scoring sequence in games is available, preferably in a digestible spreadsheet format?

I'm looking for something like the "scoring summary" here, but for an entire season's worth of games. And if it included every team in cfb, that would be even better. I'm not afraid to do a little data mining if I must.

eg Tversky tested for such effects in sequential free throws by Larry Bird and found that such effects were nil after correcting for the base rate (eg an overall 90% FT pct by LB). Similar results were found from ordinary shots from a distance on the court.

Your work is very interesting and supports the above conclusions nicely.

The results also make me think about presumed momentum effects in stock prices. Some research has shown these effects to occur--and there are plausible reasons for why they could persist in the market. (eg a tendency of some investors to try to recoup their losses. Thus, they hold on to losing stocks too long, resulting in prolonged periods of selling by different investors at different delays due to differences in the magnitudes of their own losses).

Recently, however, some people who analyze the results of high frequency trading assert (without much empirical evidence) that these momentum effects have largely disappeared. The high frequency (HF) traders presumably capitalize on evidence of price momentum in an instant, leaving little room for other investors to capitalize on further momentum trading. I am not sure if that is true, but if so, it would suggest that an illusion of momentum now also exists in the market.

The big question, however, is whether momentum effects could be nil in very short time frames (because HF traders absorb all the trends) and whether they could still exist in longer time frames, such as weeks or months. As you suggested, a similar question could exist about the time frame of possible momentum effects in FB games.