Scapegoating the Shift for the Decline in Offense

As defensive shifts have become more pervasive in baseball, so has the mystique surrounding their supposed effect.

Early this year, Jonah Keri identified defensive shifts as one reason for the recent decline in baseball offense. By early July, David Lennon placed basically all of the blame for baseball’s offensive challenges on the shift. Like Keri, he pointed to declining batting averages as proof that the shift was working too well. Things reached a fever pitch a few weeks ago when Tom Verducci ran with an idea that Lennon had proposed: banning the shift in some form, lest baseball offense never recover.

This false association is worrisome. No one feels sorry for declining millionaire pull hitters, but many people are sincerely concerned about declining offense in baseball. Successfully associating the rise of the shift to the decline in baseball offense could convince people to support drastic changes in the game. In 1968, the Year of the Pitcher famously resulted in the mound being lowered and changes to the strike zone.

Let me be clear about one thing: shifts are not affecting overall baseball offense. Shifts are not producing less value on balls put into fair play. We are facing a troubling decline in baseball offense, but if we want to treat the illness, the first thing order of business is to diagnose it correctly.

The primary reason people blame the shift for declining offense is because they are citing the wrong statistics. It won’t surprise you to know that batting average, the primary diagnostic used by the columns cited above, is as misleading in this context as it is in most others. But statistics that are usually more reliable, such as On Base Plus Slugging (OPS) and weighted On Base Average (wOBA), also provide the wrong answer in their typical form.

This is because understanding the effect of defensive shifts requires you not only to use the correct statistics, but to use the right denominator under those statistics—specifically, limiting their focus to balls put into fair play. By reweighting the best of these statistics—wOBA—with the correct denominator, we can both correctly ascertain the current value of baseball contact and understand why the decline in baseball offense has nothing to do with the shift.

Let’s start with something indisputable: over the last 15 years or so, baseball offense is down, and down by a lot. The currency of the game is runs, and the trend is dramatic and indisputable:

At 4.11 runs per game through the end of July—admittedly with plenty of warm weather yet to come—the 2014 season seems unlikely to post any improvement.

There are certainly reasons for this decline that go beyond defensive strategy. At the end of the 1990s, baseball was coming out of an expansion era that had diluted the quality of league pitching. The use of performance enhancing drugs (amphetamines, steroids, and the like) also was increasingly scrutinized and heading toward decline.

Overall, you can see three general time periods on the chart: the end of the high-flying 90s; a time period from about 2001 through 2009 when run scoring more or less stabilized around 4.7 team runs scored per game (“runs per game” or “R/G”), and from 2010 through 2013 when the bottom fell out again, down to under 4.5 runs per game.

Because that last, four-year period somewhat correlates (albeit imperfectly) with the rising popularity of the defensive shift, you can see why someone might hypothesize the shift is at fault. Moreover, if you test that hypothesis with commonly-cited offensive metrics, it seems like decreased success on balls in play—which is the entire point of shifting your infield defense—could be responsible for some of the offensive decline. For example:

Major League Batter Production by Year

Season

AVG

OPS

wOBA

1999

0.275

0.792

0.346

2000

0.275

0.795

0.346

2001

0.268

0.772

0.332

2002

0.265

0.76

0.331

2003

0.268

0.767

0.333

2004

0.27

0.776

0.335

2005

0.268

0.762

0.331

2006

0.274

0.782

0.337

2007

0.272

0.77

0.336

2008

0.268

0.761

0.333

2009

0.266

0.763

0.334

2010

0.261

0.74

0.326

2011

0.259

0.731

0.321

2012

0.259

0.737

0.32

2013

0.257

0.725

0.318

2014

0.256

0.717

0.317

The chart presents three composite offensive statistics of varying quality: (1) Batting Average; (2) On-Base Plus Slugging (OPS); and, best of all, (3) weighted on-base average (wOBA). We all grew up with batting average, but it is imprecise and often misleading. OPS is an improvement, though it is still somewhat imprecise but usually not misleading. At the top of the ladder is wOBA, which accurately averages the reasonable values of each offensive event.

All three statistics suggest the same conclusion as the runs chart: very high batter production during the late 1990s, a reduced period of production during the early to mid-2000s, and finally the last four to five years when production has declined steadily even further. Offense is down, and the on-base production statistics are plunging at the same time the shift is growing in popularity. Therefore, the shift could fairly be blamed as a contributing factor, right?

Well, no. The first rule of using a statistic correctly is to know what the statistic actually is measuring. Batting average, OPS, and wOBA are not limited to balls in play and sometimes can mask on-field results in larger trends. Dave Cameron pointed this out recently when he introduced the new Instagraphs feature at Fangraphs. Citing a BABIP (batting average on balls in play) chart, he noted that the league BABIP has actually held steady over the past few years. Here’s what a subset of that chart looks like in graphical format over the time period we’ve been considering:

As Dave notes, this chart obviously is inconsistent with the idea that balls in play are being stifled by the shift. BABIP is higher than it was 10 years ago, and BABIP has been increasing, rather than decreasing, over the last few years as more shifts have been implemented.

But Dave’s point, although accurate, is more of a first step than the final answer. Old-fashioned BABIP pretends that all hits are of equal value, which obviously they are not. Extra-base hits are much more valuable than singles. And the goal of defensive shifts is not merely to turn singles into outs, but to turn doubles into singles and perhaps home runs into bloop hits as hitters supposedly try to salvage their plate appearance by hitting the ball around the shift, compromising their power as they do so.

More importantly, in appreciating the value of hits (and lost hits), we need to remember that the offensive values of each type of hit are dynamic in nature: if tracked over the course of a season, they fluctuate along with the overall run environment. In particular, the fewer the hits we have, the greater the value of the hits that still remain. The increasing value of the remaining hits can—and in this case do—compensate for hits that might otherwise be getting stopped or limited by defensive shifts.

In short, analyzing the shift requires us to appreciate: (1) the need to look specifically at the productivity of balls in fair play, rather than all plate appearances; (2) the need to correctly weight the value of each hit that does occur in fair play; and (3) to recognize that the value of each type of hit is connected to the run environment in which it occurs.

None of the standard statistics cited above allows us to satisfy all three requirements. Batting average and BABIP flunk criteria (2) and (3), because they treat all hits as equally valuable; OPS and wOBA fail criteria (1), because they have the wrong denominator, using all plate appearances (which would include walks and strikeouts) instead of just fairly-hit balls. We need something different.

BABIP already has the right denominator, so one step might be to take BABIP and apply dynamic linear weights to it (a “wBABIP,” if you will). But remember, BABIP excludes home runs, and home runs are the ultimate equalizer: even if a double is reduced to a single, or a would-be single ends up being a mere “productive out,” the home run still exists as a means to clear the base paths and score most, if not all, of the desired runners.

So we can’t just use “balls in play,” as that concept is generally used. We need to include all fairly-hit balls. In sum, to understand the possible effect of the shift on baseball offense, we need to include home runs, we need to include the correct weights for each season, and we need to have the correct denominator.

The formula I’ve chosen is this:

The numerator consists of each type of hit, multipled by its offensive value (the “w” variable), with the results summed. The denominator is all at-bats, minus strikeouts. This allows us to capture all balls in that are put into fair play.

I’m not the originator of this formula: it previously has been described as “wOBA on contact” or “wOBAcon.” wOBAcon pops up from time to time but really has not stuck, in part because no one seems to have a consistent use for it. wOBAcon is perfect for our purposes, though, because it allows us to measure whether offense on balls in fair play has truly declined over the last 15 years or not. If it has, then and only then is it fair to question whether the shift is reducing baseball offense. If the value of baseball contact has not gone down, though, the shift is irrelevant. It is a convenient scapegoat and nothing more.

Now that we finally have the correct metric, let’s chart and then plot our decline in runs against the league’s non-pitcher wOBAcon and answer the question once and for all: Is offense down on balls being put into fair play?

Here is a table comparing League wOBAcon over the same Runs/Game time period we looked at originally. I’ve italicized the time period of particular interest:

League wOBAcon vs. Team R/G

Season

League wOBAcon

Team R/G

1999

0.374

5.08

2000

0.374

5.14

2001

0.367

4.78

2002

0.362

4.62

2003

0.363

4.73

2004

0.367

4.81

2005

0.362

4.59

2006

0.370

4.86

2007

0.370

4.80

2008

0.368

4.65

2009

0.370

4.61

2010

0.364

4.38

2011

0.361

4.28

2012

0.366

4.32

2013

0.364

4.17

2014

0.365

4.11

And here is another version of our Runs per Game line graph, this time with the league-wide wOBAcon superimposed over the same time period:

Although there is a connection between wOBAcon and Team Runs per game (more on that in a moment), the values we care about are from 2010 onward. As you can see, there is no discernable connection between the rise of the shift and the plunge in baseball offense during that time period. In fact, production on balls in fair play (as measured by wOBAcon) has been trending upward at the same time scoring overall has been trending downward.

This finding would seem to exonerate the shift. If the value of balls in fair play is not actually declining, then the shift, by definition, cannot be responsible for a coincidental decline in baseball offense. The claim that the shift is responsible for decreased baseball offense is wrong, and changes to the game designed to limit the shift would make no difference, to say nothing of their unintended consequences. So please, stop blaming the shift. Declining offense is a problem, but it is not being caused by the results of balls in play.

Two final points. First, in his Instagraphs piece, Dave Cameron blamed the decline of offense on the increase in strikeouts. I think the data suggest that the decline in walks (since 1955, r=.6, p<.00001), is more responsible than the increase in strikeouts (since 1955, r=.3, p<.02), but the two obviously are related. Either way, this means that people’s misguided focus on the shift would be better directed toward the size of the strike zone.

Second, I think it’s interesting to look at the relationship between seasonal wOBAcon and wOBA. It turns out that the seasonal differential between the two (wOBAcon minus wOBA) is moderately correlated with team runs scored. (Since 1955, r=.39, p<.003). In other words, the greater the distance between major-league wOBAcon and wOBA, the more runs we usually have getting scored. What is interesting about the last few years, though, is that this relationship has started to break down:

As you can see from the chart, the differential between wOBAcon and wOBA is greater than it has ever been, which means that run-scoring should be going up, not down. And yet the opposite is happening, and dramatically so. In other words, something in the system appears to be broken—something like the strike zone, for example.

Moreover, the deviation from the predicted trend is growing stronger. In 2011, the standardized residual was -1.07. In 2012 it was -1.19. In 2013, it was -1.66. So far, 2014 is on pace to deviate even further in this unexpected direction.

We have seen negative deviations of this magnitude before, during the so-called Second Deadball Era of the 1960s. We first saw them from 1963 through 1966, when these z-scores ranged from -1.08 to -1.27. The trend continued as we approached the Year of the Pitcher, reaching -1.89 in 1967 and an astonishing -2.93 in 1968. After changes were made to the mound and strike zone, the trend then rebounded briefly before plunging again to -1.17 in 1971 and -1.86 in 1972.

I mention this because in 1973, the American League adopted the designated hitter. It took 39 years after the addition of the DH for the wOBA differential to veer back into the territory of the 1960s. This finding is interesting because one idea being floated for increasing baseball offense would be adding the designated hitter to the National League. Would adding the DH to the NL help address this most recent decline in baseball offense? Maybe next time, we’ll give it a look.

References, Resources & Notes

According to Baseball Information Solutions, teams shifted 2,400 times in 2010 and 2011, 4500 times in 2012, 8,100 times in 2013, and are on pace for 13,000 shifts in 2014.

All data in this article combine American League and National League teams. The data have not been park-adjusted.

Fangraphs has a “Guts” page that displays the seasonal value for each batting event.

Comments

Since the main/only difference between wOBAcon and wOBA is the effect of HR’s, wouldn’t it make sense that the decline in overall offense is a function of how teams are adjusting to the specific flaws of HR/power batters/pitchers in the ‘post-PEDs’ (hahaha – well in theory) era. Seems to me that a reasonable stereotype of the power hitter is that they a)have more swing-and-miss (should lead to more K’s) and b)have lower-than average ability/incentive to spray the hits around the field (even if they have the ability, their road to a big contract and/or bigger arbitration likely goes through pull power and HR – not through LD or ‘small ball’ stuff) and c)they tend to suck defensively. A reasonable stereotype of the power pitcher is that a)they give up harder hit balls and b)they get injured more often

The shift itself may only be partially directed at the batter. It may partially be intended to cover for the defensive limitations of the power position players still on teams rosters (typically the corners – where a shift will cover for their limited range). And it takes advantage of teams’ continued willingness to spend much of their payroll on power starters recovering from TJ surgery and on gobs of roster slots to relievers who don’t pitch much.

To the degree that teams drafting strategies may have changed post-PEDs (contact/defense more important, raw/flawed power less so) that could account for the more recent trend reversal. Those younger players still have the flaws of youth (strike zone discipline – leading to fewer walks) but don’t yet have the playing time or batting order position to really affect overall runs on either offense/defense. Likewise, it is possible that teams may start looking at the financial risk of both high-K power pitchers and one-out/inning reliever types – and revert back to more finesse/command pitchers and multi-inning relievers which will take time to work its way up to the MLB level.

What may be happening underneath the data is that the replacement level itself is changing significantly for different subcomponents of both offense and defense. A similar thing happened in the immediate aftermath of WW2. Kids went from HS to war – but didn’t return to the minors – so MLB expanded its player pool a bit by integrating and, basically, played a slightly different version of baseball until 1955 when the post-WW2 refilled the pipeline. In this case, the combination of PEDs, post-1994 strike owner emphasis on both power pitching and power batting, and the ‘Latin expansion’.

If I’m right, the shift is probably a temporary phenomenon and runs scored will begin to head back up – but they will depend less on HR’s/power.

Nice writeup. It is intriguing to see that players batted balls have been increasing productivity according to your wOBAcon formula.

It would be interested to bring an analysis of BB and rates and how the shift is affecting them. Are more players swinging for the fences leading to higher K rates and a higher HR/FB rate? Are pitchers going after hitters more so the BB rates will decrease?

It is tough to straight up quantify the shift in isolation since it will affect other aspects of the game. Much like a team picking up a closer in trade. The closer only may pitch 1 inning every other game, but his presence picks up the confidence of every other pitcher on the team. So the value added may be more indirect.

This is the key flaw in the argument that contact performance rates define the impact of shifts. It seems intuitive that defensive shifts would influence hitter approaches, and if that’s true, all outcomes could be impacted, included Ks and BBs. If, for example, a hitter, who would normally pull an inside fastball, tries to wait a little longer to beat the shift by going the other way, he may be more prone to swinging and missing (i.e., more likely to strike out). We know K rates are up exponentially, but aren’t sure exactly why. Increased velocity, strike zones, etc. may all be reasons, but perhaps hitters compensating for shifts is also a culprit? If so, we can’t simply look at BABIP and wOBAcon and say shifts have no impact on offense.

No, it is making no prescriptive conclusion. The article is stating that by the measure employed there is no apparent correlation between the increased use of the defensive shift and the continuation of the trend of depressed offense in baseball. That’s it.

The purpose of the shift is to decrease the probability that a given hitter will have a productive plate appearance, and/or to the reduce the productivity of said PA. Given that the majority of plays and players do not see a shift, we would not expect that successful employment of the shift would necessarily noticeably suppress offense across the board.

Further, we do not know how effective the shift is. We can measure actual results against theoretical projections, but those projections did not happen, and performances change season-to-season. Maybe a hitter would have had a “great” season but is merely “good” because of the shift. Maybe he would have fallen apart anyway, and is having a lousy year not impacted by the shift. How often are defenses shifting? Are they doing it properly? How extreme is the movement? Is pitch selection maximizing the utility of the shift, and is the pitcher executing? Etc.

What we can say is that the Runs per Game value has been decreasing for several years, but the productivity of balls in play, while fluctuating over the greater span examined, has increased over the last several seasons (by the terms of wOBAcon). We may then suggest that some other factor has greater responsibility than the popularity of the shift for the relative lack of runs scored. Is it strikeouts? Walks? Overall defense? One of the thousands of other variables we could examine?

I just had a thought and did some quick math to check it; There has been significantly more strikes thrown since 2010. If you take the run value of a strike to be 0.13 the amount the extra strikes lowered R/G is almost the same as the amount the R/G went down over that time. Actually it’s a little bit more which jives with your wOBAcon findings, I think. I’m not 100% sure that’s a valid thing to do but there ya go. Maybe it just means the run value of a strike is changing over time. If so, looking at my data I would guess that run value was calculated based on the 2007 season, since it’s almost a 0 differential (-0.011). Anyway, if anyone has done any work looking at this I’d be interested to see it.

Another thought I had while figuring this out – what if the extra strikes are entirely due to increased usage of or talent in pitch framing? The 0.13 number came from the first article to study pitch framing, which I think, did come out in 2008.

I do feel a bit like you started from a conclusion (that the shift isn’t affecting anything at all) and worked backwards from it, and found one statistic from the mountain of them that backed up those conclusions.

I don’t want to dismiss what you did out of hand, but there’s a reason that all those team analysts and managers have been using the shift more and more. There’s a trend which is continuing this year, and if it keeps going one way more and more fans are going to drift away from baseball. If the shift is making games less fun to watch, then get rid of it. It can’t support a multi-billion dollar industry on a few handfuls of statheads.

Nope. The data analysis was done independently to look for trends on batted ball contact, and this one jumped out. Since the finding was inconsistent with the claim that shifts were decreasing baseball offense, and it became clear that proponents of that claim were not citing the correct statistics, that’s where the article came from.

AC of DC said it correctly. We can have an effective system of shifts that neutralizes certain batters without causing some overall offensive decline. And more importantly, the use of shifts is not causing some larger damage to the game, as far as I can tell.

I, for one, do not find the decline in scoring in baseball “troubling.” One team tries to score, the other tries to prevent it. It’s not basketball. Nothing wrong with a well-pitched, well-defended 2-1 game.

Some of the claims in this piece seem a bit dicey. In particular, over 2010-2014: “wOBAcon has been trending upward”? I mean… if I told you prices for gas over the last few years were: $3.64, $3.61, $3.66, 3.64, and $3.65… would you call that an upward trend? Between the first year and the last year, the differences were -3, +2, 0, +1. That falls squarely in the range of “random noise” rather than “trend,” given that the mean of the next 4 years is exactly equal to the one you started with…

The shift can’t possibly account for most of the decrease in runs, because (as noted), there just simply aren’t enough batters who need to see shifts. However, the “trends” and correlations here are using such small, noisy samples I wouldn’t hang my smallest hat on it. A better approach might be to just count up how many shifts actually occur in a season and what percentage of batters are affected, then work backwards to see how detrimental a shift would need to be in order to account for the drops in overall run production league-wide. I would bet the shift would have to almost completely floor the values of the affected hitters to even make close to the dent we’re seeing.

I’m betting that a lot of the difference just has to do with much better utilization of pitchers than before. Pitchers used to “develop” more and show up later, and pitch later into their careers. Now, instead, there is a revolving door of young fireballers pounding the zone for K’s and fewer walks. Plus, modern analytics and a bigger bullpen lets you do better “rock-paper-scissors” matchups against a batters’ specific weaknesses.

I should also note that ‘develop’ is in air-quotes because the research found that pitchers’ quality pretty much peaked in the early 20’s, which is where we’re seeing a lot more pitchers hitting the majors (rather than in their later 20’s).

The former is an easy fix that is transparent to most fans and will have a dramatic effect. We’ve seen it before. But instead of a ham fisted 1987 style juicing it needs to be done incrementally over five years. Make the ball a bit livelier each year.

More home runs and extra base hits lead to more offense, leads to more entertaining games.

Right now baseball has the combined disaster of four hour long games with the offense of the late 60s. It’s nearly unwatchable for many casual fans without deep rooting interests.

The latter is much harder to accomplish but something has to be done. Strikeouts are longer at bats that ultimately end with no action or movement occurring.

Movement and dynamism are the life blood of entertainment. Strikeouts reduce the game to two players one of whom accomplishes nothing after 8 minutes.

I have no suggestion for accomplishing this goal but I honestly believe we are approaching a crisis of offense and a flood of 2-1 games where %80 of the at bats and on strikeouts or ground balls.