Bullpens: A Sign of Royalty?

Every year, one team wins the World Series. They get a parade in their town, get to visit the White House, get a cool ring, and have a lot of articles written about them on the internet on sites like this one. However, the 2015 World Series champion, the Kansas City Royals, have continued to receive more attention for their win than almost any other recent winner, even two full years later. There is a simple reason why: Royals did not fit any of the usual descriptions of a World Series champion.

Most baseball experts will tell you a team needs quality starting pitching to win. The Royals did not have that. Their rotation ranked in the bottom quarter of MLB teams in Wins Above Replacement (WAR), a statistic designed to be an all-encompassing measure of how many wins a player added to his team over what a typical AAA player would have. The Royals also did not have an ace to carry them: not a single Royals starting pitcher ranked in the top 30 in the league in ERA.

The same experts would likely say a team needs a lineup with some powerful hitters. The Royals did not have this. Mike Moustakas and Kendrys Morales tied for the team lead in homeruns with 22. That total was enough for 42nd best in the league. The Royals had one batter (Lorenzo Cain) bat above .300. Not a single Royal had a slugging percentage above .500.

So how did the Royals manage to win, especially when it counted? The 2015 Royals were defined by speed, defense, and, perhaps most importantly and definitely most relevantly for this article, dominant relief pitching. Led by closer Greg Holland and setup men Wade Davis, Kelvin Herrera, and Ryan Madson, the Royals bullpen that postseason basically shortened games by three innings. If the Royals starter could cobble together six innings and somehow pass off a lead, the game was essentially over. Teams could not touch that bullpen.

The World Series served as a national platform for the value that a bullpen like that could offer a team, especially in the playoffs. Since more off-days are built into the schedule during the playoffs, teams can use their top relievers far more frequently than they can during the regular season, further enhancing their value in the playoffs.

Because of this scheduling advantage, many successful teams have begun to place higher values on relievers than ever before. This phenomenon can be observed in the trade market, where ace relievers Aroldis Chapman and Andrew Miller each netted the Yankees prospects ranked in top 20 league-wide as well as several other top 100 prospects. Notably, the teams acquiring these relievers, the Cubs and the Indians, met each other in the World Series.

This inflated valuation of relievers carried over into the free agent market this winter, where three relievers (Chapman, Kenley Jansen, and Mark Melancon) each signed contracts with guarantees vastly exceeding Jonathan Papelbon’s previous record deal with the Phillies.

With teams so focused on acquiring shutdown relievers, it begs the question of whether teams with better bullpens truly are better equipped to win in the postseason. In order to answer this question, I took every playoff team since 2000 and looked at their record in the postseason, their record in the regular season, and the strength of their bullpen. To attempt to measure bullpen strength, I averaged out the ERA- and FIP- of each bullpen. FIP is a statistic that attempts to strip all luck and defense out of a pitcher’s results by looking only at things completely under his control (namely walks, strikeouts, and homeruns). Basically, whereas ERA assumes that a pitcher has complete control over batted balls against him, FIP assumes he has none. Since obviously neither of these assumptions are completely accurate, I like to combine the two statistics. ERA- and FIP- compare a pitcher’s ERA and FIP to league average, controlling for park and league factors, with each point below or above 100 representing one percentage point better or worse than league average, respectively. Essentially, the lower, the better. I called the average of the two MIX-, but this stat doesn’t really have a name. You can call it whatever you want. I’m going to call it MIX-.

The average bullpen MIX- for this sample was 90, or 10% better than league average. Playoff teams’ bullpens tend to be above average for two reasons. For one, relievers generally have better statistics than starters because they only have to pitch one inning at a time, so league-wide bullpen MIX- will always be above average. Secondly, playoff teams tend to have above average stats because they have more good players. That is why they are playoff teams. I divided the teams into two groups, those with a bullpen MIX- above 90 and those below 90.

Here are how those two groups fared in the playoffs:

Group

Playoffs wins

Playoffs losses

Playoff Win %

Regular Season wins

Regular Season losses

Regular Season Win %

Bullpen ERA-

Bullpen FIP-

MIX-

WS Winners

WS Win%

Bottom Half BP

302

302

0.500

6784

5039

0.574

95

97

96

10

13.7%

Top Half BP

268

268

0.500

6911

4915

0.584

79

88

83

7

9.6%

Having an above average bullpen has not led to any advantage in playoff winning percentage. In fact, teams with above average bullpens had a higher regular season winning percentage, so this data suggests teams with better bullpens perform the same in the playoffs as worse teams with lesser bullpens. Additionally, the teams with lesser bullpens have actually advanced further into the playoffs on average, as shown by their higher playoff win total and their advantage in World Series victories.

It might be tempting conclude that the reason teams with worse bullpens have advanced farther into the playoffs is that these teams have more dominant starting pitchers who can go deep into games, rendering a strong bullpen less meaningful. If that was the case, the question might become whether it was more cost-effective to build a great bullpen or a great pitching staff, since even elite relievers still cost far less than good starters.

I would caution against this conclusion for several reasons. For one, if teams with worse bullpens actually had superior starting rotations, one would expect this to show up in the regular season results. It did not – better bullpen teams had a higher regular season winning percentage. For another, this study looked at the quality of bullpens, not at the amount of innings pitched by that bullpen. There is no reason to think that if a team has a good bullpen, they therefore must have a weaker rotation. If anything, the correlation should go the other direction: good rotations pitch a lot of innings, limiting the total amount of innings needed from a bullpen, so the better relievers in that bullpen can pitch a higher percentage of the innings, inflating the whole bullpen’s stats. I can’t conclusively rule out the role of rotations, though.

Here is a scatterplot of playoff winning percentage against bullpen MIX-:

In this graph, a slight trend towards teams with better bullpens performing better in the playoffs seems to appear. However, the R2 of this trendline is 0.00855, which implies that Bullpen MIX- explains just 0.855% of the variation in playoff winning percentage. Obviously, a metric that explains less than 1% of why some teams do better than others in the playoffs isn’t telling you very much. Perhaps bullpen strength just does not impact playoff success.

However, this conclusion should be taken with some caveats. By looking at team bullpen as a whole, I am likely underrating teams with top-heavy bullpens. Since teams’ top relievers can pitch more often in the playoffs, it is possible that the top 3 relievers in a team’s bullpen matter more than the entire bullpen’s stats. The relievers for whom teams have been paying exorbitant costs to acquire are the truly elite relievers, as opposed to the merely very good. It is still possible that elite relievers take on greater significance in the playoffs than this analysis has captured. Additionally, I did not account for players who were acquired midseason and thus played a larger role in the playoffs than regular season team bullpen stats would suggest.

Without more granular data than I have, it is impossible to account for the difference between bullpens with a few elite relievers and those with depth of very good relievers. However, by simply splitting my sample into top half and bottom half bullpens, I likely merged the truly elite with the very good on the team level. On the team level, this can be accounted for.

I created two more groups, teams with a MIX- larger than 100 (below league average) and those with MIX- below 80, which gave me two groups of 15-20 teams. Here’s how those two groups fared:

Group

Playoffs wins

Playoffs losses

Playoff W%

Regular season wins

Regular season losses

Regular season W%

Bullpen ERA-

Bullpen FIP-

MIX-

WS Winners

WS win %

MIX- >100

40

53

0.430

1290

978

0.569

104

103

104

2

14.3%

MIX- <80

82

74

0.526

1851

1227

0.601

70.9

84.0

77.5

3

16.7%

Here, a difference emerges. The teams with truly elite bullpens tend to outperform the general average, while the teams with very poor bullpens, relative to other playoff teams, tend to very significantly underperform the average. Although the teams with elite bullpens tend to be better regular season teams, the difference in winning percentage between the two groups in the playoffs (96 points) dwarves the regular season difference (42 points). Finally, the elite bullpens tend to advance much further as shown by the much larger playoff win total (the elite bullpen group has slightly more teams, but teams with elite bullpens also average almost a win and a half more per postseason run).

However, the most striking data is how poorly teams with especially poor bullpens fared. Teams with poor bullpens had a win percentage 70 points worse than average, while the elite bullpen teams had a win percentage just 26 points better than average. Perhaps the real conclusion from this data is that building an elite bullpen helps in the playoff, but the most important issue for GMs is to avoid a true dumpster fire of a bullpen.

Out of the teams in this year’s playoffs, three (the Red Sox, Yankees, and Indians) posted MIX- scores better than 80, including a new record of 69 from the Indians, while the Nationals posted a MIX- of exactly 100. The Nationals serve to illuminate a shortcoming of this study, as Washington drastically upgraded their bullpen at the trade deadline. Thus, full season statistics do not capture their bullpen’s current quality. However, two of the three elite bullpen teams were eliminated in the first round, although notably the Indians were knocked out by the Yankees, who had the second-best bullpen in the league by MIX-. So far this postseason, bullpen strength has not been very valuable.

In short, at the end of this study, I still cannot say much about an individual team’s playoff chances based on their bullpen. So much of the data is inconclusive, and all interpretations can be debated. This study is not a death knell for teams with poor bullpens, as the two world series winners from the bad bullpen group show (2003 Marlins and 2012 Cardinals, both with a 102 MIX-). Nor should one consider it a guarantee of success for teams with amazing bullpens. For an example, look no further than the 2006 Twins who, led by potential Hall of Famer closer Joe Nathan, posted a MIX- of 70, the best since 2000 until this year’s Indians. They were swept by the Athletics in the first round of the playoffs. Maybe what this study shows is that there is no right way to build a World Series champion.

However, the 2015 Royals and their 79 MIX- also fit into the elite group. Maybe, just maybe, we should have been less shocked at their success. Maybe they were a typical championship team after all.