The History of Relief Pitching Part Six: Comparing Closers Across Eras

The History of Relief Pitching Part Six: Comparing Closers Across Eras

Comparing baseball players from different eras is always an endeavor that requires we acknowledge that, in the history of the sport, some periods of time favored hitters, others favored pitchers. We also know that ballparks are different. You can put the best pitcher in baseball on the Colorado Rockies and he’s going to struggle to keep his ERA under 3.50.

With relief pitchers, we have the added challenge of measuring not just performance but evolving patterns of usage. The firemen of the the 1970’s and ’80’s often came into the game with runners on base. In the last 30 years, most elite closers have had the luxury of coming into a clean 9th inning, with a one-to-three run lead and the bases empty.

Here is the list of the top 10 pitchers all-time in saves, plus those in the Hall of Fame.

Courtesy Baseball Reference

Rank

Most Saves All-Time

SV

SV%

IP

ERA

1

Mariano Rivera

652

89.1%

1283.2

2.21

2

Trevor Hoffman

601

88.8%

1089.1

2.87

3

Lee Smith

478

82.3%

1289.1

3.03

4

Francisco Rodriguez

437

85.2%

976.0

2.86

5

John Franco

424

80.8%

1245.2

2.89

6

Billy Wagner

422

85.9%

903.0

2.31

7

*Dennis Eckersley

390

84.6%

789.2

2.96

8

Joe Nathan

377

89.1%

923.1

2.87

9

Jonathan Papelbon

368

88.2%

725.2

2.44

10

Jeff Reardon

367

77.6%

1132.1

3.16

13

Rollie Fingers

341

75.8%

1701.1

2.90

23

Rich Gossage

310

73.5%

1809.1

3.01

T-26

Bruce Sutter

300

74.8%

1042.0

2.83

39

Hoyt Wilhelm

228

75.7%

2254.1

2.52

*IP and ERA as relief pitcher (1987-1998)

Well, it’s clear that you can’t just evaluate save totals and conversion percentages when comparing pitchers across eras, otherwise you would have to conclude, based on their save conversion percentages and ERAs, that Joe Nathan and Jonathan Papelbon should be in the Hall of Fame ahead of Dennis Eckersley, Rollie Fingers, Goose Gossage and Bruce Sutter.

It’s not uncommon for professional athletes to look with disdain upon successive generations. One man who has never been reluctant to share an opinion is Richard “Goose” Gossage. This spring, in an interview with N.J. Advanced Media, the Goose went off on today’s one-inning closers and decided to compare himself to Mariano Rivera, who is universally regarded as the best relief pitcher in the game’s history:

“I’m not taking anything away from what Mo did, but don’t compare me to him. It’s insulting. It really is.”

“These guys that pitch one inning with the three-run lead (and get a) save. It shouldn’t even be a save for one inning and a three-run lead. This is not a knock against Mo. I’m just trying to make a point that I’d like to know how many of Mo’s saves are of one inning with a three-run lead. If everybody in that (expletive) bullpen can’t save a three-run lead for one inning, they shouldn’t even be in the big leagues.”

— Goose Gossage (to N.J. Advanced Media, Feb. 18, 2017).

Well, Goose, let’s answer your question. A year ago, when researching the Hall of Fame case for Trevor Hoffman, I did a deep dive into the game logs for all of the top relief pitchers in history. Using the power of sorting and data splitting through Microsoft Excel, I was able to break down the top relief aces and what their save percentages were in multiple scenarios.

In his career, Rivera saved 652 games (compared to Gossage’s 310). Of those 652, a whopping 483 of them occurred in a “clean” 9th inning (no runners on). Here is how Rivera and Gossage stack up in terms of the situations they pitched in to accumulate their saves.

Career Statistics

Mariano Rivera

Goose Gossage

Situation

Saves

BSV

SV %

Saves

BSV

SV %

Start 9th or extra inn., none on, 2 or 3 run lead

316

10

97%

30

3

91%

Start 9th or extra inn., none on, 1 run lead

163

34

83%

21

5

81%

Start 8th inning, no runners on

24

1

96%

59

21

74%

Enter with runners on base 9th inning

50

3

94%

57

27

68%

Enter w/ runners on base 9th (4 or 5 run lead)

46

3

94%

16

2

89%

Enter with runners on base in 8th inning

94

29

76%

82

18

82%

Enter with runners on base before 8th inning

1

1

50%

43

29

60%

You’ll notice that Rivera, in the 9th inning games in which he entered with runners on base (the “fireman” situations) didn’t have to put out much more than a small campfire. It’s a popular managerial strategy to use a set-up man in the 9th inning when the team has a 4-to-5 run lead but, if that pitcher allows two or three runners to reach base, then they reach for the closer. Remember that one of the criteria for a save is when the tying run is on deck.

You can also see that, in situations in the 8th inning in which runners were on base, Gossage’s numbers are superior. And, of course, as a modern closer, Rivera had virtually no save situations before the 8th inning in his entire career.

In his autobiography, this is how Gossage described the role of a closer:

“It’s imperative that closers realize that, no matter how overpowering their stuff, they are always one pitch away from disaster. Being a closer on a baseball team is like working a high wire without a net. There’s no margin for error.”

— Goose Gossage (in “The Goose is Loose” (2000))

The key line here is “no margin for error.” The truth is that, in the modern game, the closers actually do have a margin for error more often than not. You’ll notice that, in 395 of his career saves, Rivera had the safety net of being able to give up a home run without blowing the game (this is the bottom row on the next chart).

Career Statistics

Mariano Rivera

Goose Gossage

Situation

Saves

BSV

SV %

Saves

BSV

SV %

Enter w/ tying run on base

46

23

67%

83

57

59%

Enter w/ tying run at bat (but not on base)

211

43

83%

122

42

74%

Other saves (tying run NOT on base or at bat)

395

14

97%

105

13

89%

In his career, Rivera pitched in 113 more games than Gossage did in his career and, no matter what situations the two men faced, Rivera’s success rates were a little bit better. That’s not a slight on the Goose. If you stack up any relief pitcher against the Super Mariano, Mo is going to look better because he was the best at his craft in the game’s history.

Still, when it comes to raw save totals, 602 for Mo and 310 for Goose doesn’t tell remotely close to the whole story.

In 193 of his 310 career saves, Gossage worked more than one inning. Rivera only recorded four or more outs in 119 of his 652 career saves. By the way, that’s still quite a few compared to his contemporaries of the 1990’s and 2000’s but a small amount compared to the Goose.

Gossage pitched two innings or more in 125 of his saves; Rivera only did this 11 times. These are traits also found in Goose’s contemporaries.

The chart below shows the number of innings pitched in all games that ended with a save. We’re showing the same 14 pitchers we showed before plus Dan Quisenberry.

Ranked by number of total innings pitched in all saves

Rank

Career Statistics

Saves

IP

IP per SV

1

Mariano Rivera

652

688.0

1.06

2

Trevor Hoffman

601

602.0

1.00

3

Lee Smith

478

593.0

1.24

4

Rollie Fingers

341

547.1

1.61

5

Goose Gossage

310

487.1

1.57

6

Bruce Sutter

300

473.1

1.58

7

John Franco

424

471.0

1.11

8

Jeff Reardon

367

470.1

1.28

9

Dennis Eckersley

390

432.1

1.11

10

Billy Wagner

422

431.2

1.02

11

Francisco Rodriguez

437

430.0

0.98

12

Dan Quisenberry

244

422.0

1.73

13

Hoyt Wilhelm

228

420.2

1.85

17

Jonathan Papelbon

368

375.0

1.02

19

Joe Nathan

377

370.2

0.98

Courtesy Baseball Reference

I included Quisenberry for a reason. The side-arming closer for the 1980’s Kansas City Royals only finished his career with 244 saves but pitched nearly as many innings in those saves as Billy Wagner and Francisco Rodriguez, members of the 400-save club. The Quiz and his Hall of Fame contemporaries (Goose, Sutter and Fingers) often had to save the game twice.

The next chart shows the sheer volume of multi-inning saves that the firemen of the 1970’s and ’80’s had compared to the one-inning closers of the modern game.

Courtesy Baseball Reference

Number of outs in saves

3 or less

4 or 5

6 or more

Mariano Rivera

533

108

11

Trevor Hoffman

546

48

7

Lee Smith

309

75

94

Francisco Rodriguez

405

30

2

John Franco

334

53

37

Billy Wagner

386

32

4

Dennis Eckersley

284

78

28

Joe Nathan

366

10

1

Jonathan Papelbon

331

36

1

Jeff Reardon

215

74

78

Rollie Fingers

140

66

135

Goose Gossage

117

68

125

Bruce Sutter

112

58

130

Dan Quisenberry

84

40

120

Hoyt Wilhelm

80

30

118

I think any serious fan would not be surprised by the numbers here but it’s still striking when you look at them. The four Hall of Fame pitchers not named Eckersley all logged at least two innings in more than a third of their career saves (for Wilhelm, it was more than half). So, the fact that all four have career save percentages in the 75 percent range should absolutely not lead to a conclusion that they were inferior to their successors. Think about the difference between a 3-out save and a 6-out save. Again, if you’re getting 6 outs, you in essence have to save the game twice.

So, what does this all mean? How do we properly quantify the added value of the multi-inning saves authored by the stoppers of the ’70’s and ’80’s compared to the high number of one-inning saves put up by the closers of today? And how do we quantify the differing degree of difficulty of a “clean 9th inning” save as opposed to a “on fire” runners on base save?

The answer is imperfect but there is one statistic that actually gets us close. It gets us close, not all the way there.

That statistic is called WPA (Win Probability Added). We introduced the concept of WPA in Part Five of this series but, for those who may have skipped it (shame on you), a recap:

WPA takes each individual play and assesses to what degree it increases or decreases the player’s team’s chances for a win. Therefore, an out to end a one-run game has a dramatically higher WPA than an out in the seventh inning of a 10-0 contest. If you’re the type to lose patience with some of the advanced statistics, hang in with me for a bit on this one. It’s easy to understand how WPA works and it’s important for the evaluation of relief pitching.

WPA is a “counting” stat that can count backwards too. If you’re a closer and you enter a game with a 3-run lead in the 9th, you should close out that game over 96% of the time so, depending on the ballpark and the season, you would get approximately 0.04 WPA “points” for bringing your team from a 96% win probability to 100% (an actual win). If, on the other hand, you give up four runs and lose that game, you’ll get -0.96 WPA “points” if it’s a road game. (If you’re pitching at home, your team still has a chance to come back to win so you can’t go from 96% to 0%).

Now, if you enter the game in the 9th with just a one-run lead and convert the save, that’s worth anywhere between 0.16 and 0.24 WPA. Why such a big difference? It’s because of the ballpark and the season. Using Trevor Hoffman as an example, a 9th inning save with a one-run lead at PetCo Park was worth about 0.16 WPA. A 9th inning save with a one-run lead at Coors Field was worth about 0.24 WPA because it’s much easier to score runs at Coors Field.

To help you understand how it works (and impacts relief pitching), here are some famous examples from October baseball with the relief pitcher involved and the positive or negative impact they had on each game.

Courtesy Baseball Reference

Pitcher

Team

Year

Round

Game

When they entered, what happened

WPA

Madison Bumgarner

SF

2014

WS

7

Bot 5th, 3-2 lead, throws 5 scoreless IP for save & Series win

0.60

Sparky Lyle

NYY

1977

LCS

4

Bot 4th, 2 runners on, 5-4 lead, 5.1 scoreless IP, gets the win

0.57

Rollie Fingers

OAK

1973

WS

7

Bot 8th, 3-1 lead, 2 RISP, 0 out, sac fly & 6 outs to clinch Series

0.46

Donnie Moore

CAL

1986

LCS

5

One out from WS, 2-run HR to Dave Henderson, Sac Fly in 11th

-0.91

Dennis Eckersley

OAK

1988

WS

1

Bot 9th, 4-3 lead, 2-out, 2-run HR to Kirk Gibson

-0.83

Mitch Williams

PHL

1993

WS

6

Bot 9th, 6-5 lead, 3-run HR to Joe Carter to end the Series

-0.79

Don’t you love it when the value of a mysterious, previously unknown metric is understandable and actually displays some good old fashioned common sense?

So, here is a list of the career Top 10 for relief pitchers for Win Probability Added plus Lee Smith and all of the existing Hall of Fame relievers. These are regular season numbers and they ONLY include games in which the listed pitchers appeared out of the bullpen.

Courtesy Baseball Reference

Rank

Career WPA (games in relief only)

WPA

G

IP

1

Mariano Rivera

57.3

1105

1233.2

2

Trevor Hoffman

34.2

1035

1089.1

3

Rich Gossage

33.9

965

1556.2

4

Joe Nathan

30.8

758

761

5

Billy Wagner

29.1

854

903

6

Jonathan Papelbon

28.0

686

709.2

7

Hoyt Wilhelm

26.5

1018

1872.1

8

Tug McGraw

25.6

785

1301.1

9

Francisco Rodriguez

24.4

948

976

10

Troy Percival

23.6

702

707.2

11

Lee Smith

22.0

1017

1252.1

25

Rollie Fingers

18.7

907

1505.2

26

Bruce Sutter

18.2

661

1042

29

Dennis Eckersley

17.1

710

807.1

Hall of Famers in BOLD

Well, this is interesting, isn’t it? A moment ago I discounted the high save percentage numbers of Nathan, Rodriguez and Papelbon because they were one-inning specialists. And yet the three of them (plus Billy Wagner and Troy Percival) are all in the top ten of this game-context-dependent metric. In the meantime, Hall of Famers Fingers, Sutter and Eckersley are not even in the Top 20.

What potential conclusions can we draw from this?

Fingers, Sutter and Eckersley may have been overrated as relief pitchers.

WPA favors the pitcher who enters with a clean inning as opposed to the pitcher who comes in with the game “on fire,” runners in scoring position. Think about it. If you’re entering a clean inning and you give up a single, doom has not occurred. If you enter a game with two runners in scoring position and you give up a single, you may have instantly lost the lead for your team. A two-run swing will punish your WPA much more than a single with the bases empty.

The modern usage of top closers for just the 9th inning, while perhaps now unimaginative, yields predictable results. Rivera, Hoffman, Nathan, Wagner, Papelbon, Rodriguez, and Percival all closed at remarkably high efficiency and that brought high value to their teams.

We’ll address #1 in Part Seven of this series.

Point #2 is important: there is definitely a bias that favors the one-inning closers when compared to the multi-inning stoppers of yesteryear. If you take a list of all of the relief pitchers of the LCS era with over 100 saves and calculate their WPA per inning pitched (not overall WPA), the top 20 are ALL one-inning closers and include nine hurlers who are still active (Craig Kimbrel, Greg Holland, Kenley Jansen, Aroldis Chapman, Joakim Soria, Mark Melancon, Huston Street, David Robertson and Zach Britton). Rivera, of course, tops the list, with Nathan (9), Wagner (10) and Hoffman (11) also in the top 12 positions.

To find a closer from the ’70’s or ’80’s, you have to go down to 35th place (Dan Quisenberry). Gossage is in 39th, Sutter 44th.

That bias is more than just a statistical flaw. Rather, it’s a one of many statistical manifestations of a human factor. Simply put, the job of a one-inning closer of today is easier than the job of a multi-inning closer of yesterday. Today’s bullpens contain multiple pitchers who throw fastballs in excess of 95 miles per hour. There are a variety of reasons for this:

Here, point #3 is the key. The majority of pitchers of today are conditioned to throw with maximum effort all of the time. This is especially true with closers. Could Aroldis Chapman throw nine innings’ worth of 100 MPH fastballs? Probably not. But for one inning? Relief pitching today is the equivalent of a 100 meter sprint whereas relief pitching of the era of Gossage was like a 400 meter race.

If you watch MLB Now on the MLB Network (Brian Kenny’s show), he’s constantly waxing poetic about the way teams used relief pitchers in the era of Gossage, Fingers, and Sutter. He often talks about how teams aren’t properly using their best relief weapons when they save them for just the 9th inning. He’s correct of course when it comes the idea that managers actually manage according to whether their “closer” can get a save or not (see Part Four of this series).

Kenny envisions a game in which all teams utilize their best guns the way Indians’ manager Terry Francona utilizes Andrew Miller. When the tall Indians’ lefty is sitting in the bullpen, he doesn’t know when he’s going to pitch (if at all) or for how many batters. The vast majority of high leverage relievers today know that they’re almost always going just one inning with the occasional 4-out save request. Simply put, the 6’8″ Miller is an anomaly in today’s game. Whether he turns out to be a prototype for a new type of relief pitcher (or a “retro” type, if you will) or whether he remains a spectacular oddity remains to be seen. However, the bias of WPA metric towards the one-inning pitchers who “know their role” and come in for a clean inning indicates that maybe, to some extent, the games’ managers know what they’re doing. What a concept?

So, have I led you down a rabbit hole with WPA? The answer is “no” but with conditions. What WPA measures does matter. If the majority of innings that you pitch involve the risk that you could turn a win into a loss, that’s significant. However, since we’ve seen that there’s an inherent bias favoring the relievers of the “one-inning closer” variety, it’s most relevant when measuring relief pitchers against others who pitched at the same time in history.

Despite its flaws, no less an authority than statistical wizard Nate Silver of (fivethirtyeight.com) calls WPA “arguably the best way to value relief pitchers.”

But Silver, being the numbers guy that he is, decided to try to find a better way to evaluate relievers. Just as Jerome Holtzman coined the “save” statistic, Silver has coined the term “goose egg” (in honor of Gossage, or to “troll him,” he writes). The basic version is that a relief pitcher gets credit for a “goose egg” in each inning pitched as follows:

They pitch a scoreless inning without giving up a run in situations in which the pitcher’s team has a lead of two runs or less in the seventh inning or later or when the score is tied.

A pitcher can get credit for a goose egg with leads of more than two runs provided the tying run is on base or at bat.

If the pitcher records less than three outs, he can still get a goose egg if the total number of outs recorded and inherited runners on base totals three or more.

A pitcher can get more than one goose egg in a game (which helps credit the stoppers of the 1970’s and 1980’s when “saving” a game twice or more).

In order to get the goose egg, no runs can cross the plate after the pitcher enters the game in that inning.

Anyway, from Silver’s research (you can access the full piece by clicking here), here are the top 20 pitchers with most “goose eggs” all-time:

Most "goose egg" innings all time (see criteria above)

Rank

Career (1930-2016)

Goose Eggs

*Saves

*SV rank

1

Goose Gossage

677

310

23

2

Rollie Fingers

663

341

13

3

Hoyt Wilhelm

641

228

39

4

Mariano Rivera

614

652

1

5

Lee Smith

589

478

3

6

John Franco

589

424

5

7

Trevor Hoffman

580

601

2

8

Bruce Sutter

557

300

T-26

9

Tug McGraw

521

180

63

10

Jeff Reardon

520

367

10

11

Sparky Lyle

520

238

37

12

Kent Tekulve

517

184

T-59

13

Lindy McDaniel

507

174

67

14

Mike Marshall

489

188

55

15

Gene Garber

468

218

41

16

Ron Perranoski

444

178

64

17

Francisco Rodriguez

430

437

4

18

Todd Jones

425

319

19

19

Billy Wagner

421

422

6

20

Jesse Orosco

416

144

T-87

Hall of Famers in BOLD

Courtesy: www.fivethirtyeight.com

*Career Saves and Saves Rank through 2016

What you have here is another counting statistic that still requires some context but it is the best that I’ve seen that properly credits the extra work of the multi-inning stoppers. At the very least, it’s a significant improvement over the “save” statistic. By the way, if you’re wondering where Eckersley is here, Silver credits Eck with 352 goose eggs (compared to 390 saves), which is not even close to the top 20.

Of course, there’s the flip side of the “goose egg,” which Silver calls the “broken egg,” when a pitcher gives up at least one earned run and does not finish the game with his team on the winning side. And yes, at the top of this category are the same three leaders in “goose eggs,” Fingers, Gossage and Wilhelm. Silver then proceeds to go really deep into the weeds, creating a “goose egg” version of WAR, which he calls GWAR. I won’t go into detail here because I have one issue with the methodology (it’s either that I don’t fully understand it or that it favors the modern closer, probably that I don’t fully understand it). Suffice it to say that Mariano Rivera still tops it, by a lot.

There’s one final way that I can recommend that one can compare pitchers across eras and that’s by comparing, on a year-by-year or multi-year basis, how they ranked against their contemporaries. If you want to look at which relief pitchers had lowest ERA on a year-by-year basis that’s one way to do it. Or you could check out the leaders each year in WHIP, or strikeout-to-walk ratio, or the advanced metrics of WAR, WPA and ERA+. I like using ERA+ more than the traditional ERA because it adjusts for ballpark effects. An ERA of 3.00 in the Oakland Coliseum is not the same as an ERA of 3.00 at Wrigley Field.

I’m going to share the full results of the research I did about all of this in the Lee Smith comment in Part Seven of this series, but I’ll tease you with this: Mariano Rivera is still #1. In ten different seasons in his career, Mariano was in the Top 5 of the major leagues in ERA+. By comparison, Goose Gossage did this five times. Nobody else has even managed it more than four times.

Ultimately, even with the various methods shared in this piece, comparing relief pitchers across eras remains a complex endeavor and one that is easily influenced by personal bias. But this, of course, is what the Hall of Fame debate is all about. The exercise is to get 75% of the baseball writers with the voting privilege to concur. More often than not but not always, the writers have gotten it right.

In Part Seven, we’ll take a look at whether there are already too many relief pitchers in the Hall of Fame (or whether there will be too many after the inevitable inductions of Hoffman and Rivera). And, of course, we’ll give a thumbs up, thumbs down, or TBA for the best relief pitchers in the history of the game.