Thursday, February 07, 2008

WPA and Tigers Relievers in 2007

It's difficult to measure the performance of relievers for a couple of reasons: (1) They pitch so few innings that their statistics can be influenced heavily by a couple of really bad outings. (2) Their actual value depends on game situations more than any other player. Using ERA to evaluate relievers is problematic because relievers often come in with runners on base and give up other pitcher's runs. So a pitcher could have a low ERA without actually being that effective. FIP ERA which is based on walks, strikeouts and home runs allowed rather than runs allowed is better but it still does not consider the game environments in which a reliever pitched.

In one instance, Fernando Rodney comes into the game with a one run lead and two runners on base in the 8th inning. In another game, Bobby Seay comes into the game with a 6 run lead and nobody on base. Now suppose each pitches a perfect inning. Using ERA or FIP, they would both get the same credit for that inning but Rodney's performance had more impact on the outcome of the game.

The Win Probability Added (WPA) statistic gives players (hitters, starters, relievers) credit based on the effect each play has on a team's probability of winning. These probabilities vary depending on the game score, the runners on base and the number of outs before and after each play. They are based on the results of thousands of games worth of data looking at every possible situation over and over.

More concretely, WPA works as follows. Suppose Bobby Seay comes into the game in the top of the 8th with a 2 run lead, 0 outs and a runner on first. There is a .787 (78.7%) expectancy that a team will win the game given that situation. Suppose Seay strikes out the first batter. There is now one out and the probability of winning has gone up to .848. Thus, the strikeout was worth .848-.787=.061

Now suppose the next batter after that doubles home a run. The Tigers now have a one run lead with a runner on second and one out. The probability of winning goes down to .693. So Seay loses points on that batter: .693-.848=-.155.

If you add up all the gains and losses for all the batters Seay faces you get his WPA. WPA doesn't solve the problem of small sample sizes for relievers but it's a reasonable stat because it gives more weight to plate appearances which have a strong impact on winning and losing games. Table 1 below lists The WPAs for Tigers relievers with 40 or more innings in 2007. Table 2 lists the American League leaders. These data were abstracted from the Fan Graphs web site.

Bobby Seay was the leading Tigers reliever according to WPA (1.66) in 2007. It may surprise some that Todd Jones finished with a pretty solid 1.57 WPA. Todd Jones did not have a low ERA (4.26) but he (like most closers) came into a lot of critical situations and got positive results a lot more often than negative results. The Tigers lowest WPA belonged to Fernando Rodney (-0.37). No Tiger finished among the top ten AL relievers, although Seay and Jones did finish in the top twenty.

Table 1: WPAs for Tigers relievers in 2007

Name

G

WPA

Bobby Seay

58

1.66

Todd Jones

63

1.57

Tim Byrdak

39

0.29

Jason Grilli

57

-0.10

Zach Miner

33

-0.27

Fernando Rodney

48

-0.37

Table 2: AL reliever WPA leaders in 2007

Name

Team

G

WPA

J.J. Putz

Mariners

68

6.17

Rafael Betancourt

Indians

68

5.38

Joakim Soria

Royals

62

3.85

Jonathan Papelbon

Red Sox

59

3.72

Joe Nathan

Twins

68

3.63

Francisco Rodriguez

Angels

64

2.95

Hideki Okajima

Red Sox

66

2.93

Pat Neshek

Twins

74

2.83

Joaquin Benoit

Rangers

70

2.48

Rafael Perez

Indians

44

2.41

Now, which pitchers typically worked in the most pressing situations? To answer this question, we can use Leverage Index (LI) which measures how critical a given plate appearance is to determining the final result of a game. An LI of one is average. An LI of more than one indicates a high leverage plate appearance which has a potentially high impact on the outcome of the game. An LI of less than one is a low leverage plate appearance. pLI is Leverage Index per Plate Appearance.

Table 3 below lists The pLIs for Tigers relievers with 40 or more innings in 2007. Table 4 lists the American League leaders. Not surprisingly closers dominated the leader boards. In fact, Jones finished second in the American League. So, the criticism that he gets a lot of easy saves might not be warranted. Of course, it's also true that some of the high leverage situations in which he pitched were caused by his own base runners. If we instead look at the leverage index at the beginning of his appearances (gmLI=1.76) Jones is still among the leaders but not quite as high (8th in the league).

Many complained last year that Jim Leyland used Jason Grilli too often in tight games. In fact, Grilli's pLI of 0.73 suggests just the opposite. He pitched in lower leverage situations than any of the six most often used Tigers relievers. Again, all of these data were pulled from Fan Graphs.