Shopping Cart

My Account

How to Measure Team Efficiency

Measuring team efficiency

By Tom Tippett December 5, 2002

It goes without saying that wins and losses are the most important things to consider when judging a team's performance. They are, after all, what the game is all about and what determines who gets to keep playing until there's only one winner left.

The next most important things are runs scored and runs allowed. You win games by outscoring your opponents, so the connection between runs and wins is very strong. It's not perfect, though, and every season produces a few teams that win more or less than you'd expect given their run differential.

If runs are one step removed from wins, then the baseball events that produce runs are two steps removed from wins. You score runs by putting together singles and walks and doubles and steals and homers, and you prevent runs by holding the other team to a minimum of those things.

In most cases, there's a very direct relationship between wins and runs and the underlying events that produce runs. But that's not always the case, and in this review of the 2002 season, we'll identify teams where those relationships didn't hold up. If the past is any guide, this will give us some very strong hints about what is likely to happen with those teams in the future.

To explore the relationship between runs and wins, we'll use the pythagorean method that was developed by Bill James. To explore the relationship between offensive events and runs, I'll introduce a new statistic that I'll call the run efficiency average. This number will tell us which teams were unusually good at turning offensive events into runs and unusually good at keeping the other team from doing the same.

We'll end up with three measures for each team -- one for offensive efficiency, one for defensive efficiency, and one for pythagorean efficiency -- that will tell us which teams squeezed more wins out of the hits and walks and homers and other events that occurred during their games. And which teams squandered their output to the greatest degree.

And we'll take a look at some history. We'll see that teams that are unusually efficient (or ineffecient) have exhibited a very strong tendency to revert back to the norm the next year. In other words, if your team was especially inefficient in 2002, there is every reason to believe things will be better next year. And the opposite is true, too. If your team was very efficient this year, don't count on a repeat performance next year.

That's good news for the Cubs, Brewers, Devil Rays and Tigers. And bad news for the Angels, Braves, and Twins. It's way too early to start predicting what's going to happen in 2003, and all thirty teams are quite capable of improving or regressing based on their off-season moves and the development of their younger players and prospects. But we can say that these seven teams (and a few others to a lesser degree) go into the offseason in better or worse condition than it might seem based solely on their 2002 win-loss records.

Converting runs into wins

Others, notably Rob Neyer and the Baseball Prospectus crew, have written extensively on ESPN.com about the Bill James pythagorean method, a well-established formula that says that a team's winning percentage is tightly coupled with runs scored and runs allowed. The expanded standings on ESPN.com include run margins and expected win-loss records that are derived using this formula, and Rob's home page showed pythagorean standings every day. So I'm not going to go over that ground again.

I will, however, try to put the 2002 results into historical context. For instance, the Red Sox and Cubs won 8 fewer games than their run margin would normally produce, while three teams (Minnesota +7, Oakland +6, and Detroit +6) won at least six more than expected. How unusual is this? And what tends to happen to teams that stray from their expected win totals?

I started by computing the expected and actual win totals for every team since 1962, the first year the 162-game schedule was used in both leagues. The more games you play, the larger the differences between expected and actual wins, so I didn't want to mix seasons with different schedules. For that reason, I left out the strike-shortened 1972, 1981, 1994, and 1995 campaigns, leaving a total of 72 league-seasons.

In those 72 league-seasons, the average team "missed" its pythagorean projection by only 3.2 games, indicating that there is indeed a very strong relationship between runs and wins. How does 2002 compare? This year, AL teams were off by an average of 4.1 games while their NL counterparts missed by an average of 3.0 games. Overall, the 30 teams had an average difference of 3.5 games, slightly higher than the historical average but well within the normal year-to-year fluctuations.

The wackiest season, in the pythagorean sense, was the 1978 National League, whose teams missed their projections by an average of 5.3 games. In the NL West that year, the Dodgers led the league in both scoring and fewest runs allowed, outscoring their opponents by 154 runs, while the Reds were only +22 on run differential. But Cincinnati won nine more games than expected and the Dodgers five fewer, turning what could have been a runaway win by the Dodgers into a close battle that saw LA win by 2-1/2 games. In the NL East, the story was much the same, as the Phillies (-6) edged the Pirates (+2) by two games in a race that was much closer than it could have been. That same year, the Expos matched the Reds with a run margin of +22, but Cincinnati (+9) won 92 games and Montreal (-8) only 76. (The Expos run margin foreshadowed their improvement; they went on to win 95 games in 1979.)

In contrast, the 1991 American League was closest to pythagorean form, with an average difference of only 1.8 wins.

What tends to happen to teams with large pythagorean differences? Here's a list of the 22 teams that have exceeded their projected win total by at least 8 games, along with their differences in the next year:

As you can see, only a few teams came close to matching their pythagorean differences in the next season. In fact, these 22 teams were collectively 18 wins above their projection the year after, an average of less than one win per team.

(Just to be clear, these next-year numbers don't represent the change in actual win-loss record from the year before, so they don't measure the team's tendency to get better or worse. They represent the difference between actual and pythagorean wins the next season. In other words, they measure the tendency to consistently win more or fewer games than the run margin suggests, not the tendency to produce a better or worse run margin in the first place.)

On the flip side, here are the teams with the biggest negative differences since 1962:

These 31 teams were collectively 4 wins above their projection the year after, about as close to zero wins per team as you can get.

These extreme teams do leave us with a few unanswered questions. Why are there more NL teams than AL teams on these lists? Why do the Astros show up as often as they do? How miserable must the mid-1980s Pirates fans have been when their team posted a three-year pythagorean difference of -32 wins from 1984 to 1986? These answers, if they exist, will have to wait for another day.

I'm not going to suggest that I have proven this beyond a reasonable doubt, but I believe luck plays a large part. If you wanted to argue that the over-achievers had big pluses because their manager was especially astute or their roster was full of clutch players, it would be a tough case to make based on the next-year records of these teams. And looking at the under-achievers, it would be even tougher to argue that their manager and players are fundamentally flawed based on their next-year results.

But teams change from year to year, and the under-achievers are much more likely to fire their managers and turn over half their rosters. Perhaps those changes were responsible for bringing them back to pythagorean normalcy. Even though I don't believe this argument would hold up under closer examination, it muddies the water a little.

Still, if managerial skill and clutch performance were the biggest piece of this puzzle, why wouldn't the over-achievers, the teams that would not be making many changes from year to year, be able to maintain their performance to a much greater extent?

Converting offensive events into runs

In the previous section, we took one step back from wins and losses to examine runs. In this section, we'll take another step back and look at the offensive events -- the hits and walks that lead to the runs that generate the wins -- that were produced and allowed by each team.

Just as there is a strong relationship between runs and wins, it's almost always true that the more hits and walks you produce, the more runs you'll score. Sometimes a productive team comes up short on the scoreboard because they didn't hit in the clutch or just because they happened to hit line drives right at people in key situations. Or the opposite could be true. But this relationship holds up most of the time.

To shed some light on this relationship, we need a way to take batting stats and turn them into a measure of overall offensive production. There are several good options here, including Runs Created (Bill James), Batting Runs (Pete Palmer), Equivalent Average (Clay Davenport), and OPS (on-base average plus slugging average). But many of them require a computer, and although we do computer analysis all the time, we also like to use simpler measures that anyone can use whenever they have a page of stats in front of them. The best of these simple methods give up very little accuracy in return for a big gain in usability.

For this exercise, I'll use the sum of total bases and walks, or TBW for short. TBW is not a perfect measure, but it does have a few things going for it. It captures the most important things a team does to produce runs -- singles, extra-base hits, and walks. It's easy to figure without a computer. In the past, I've used both TBW and OPS for this type of analysis, and the results are almost exactly the same, so the accuracy is more than acceptable.

And sometimes it just seems to tell a story more clearly. For instance, the 2002 Yankees had a team OPS of .809 compared to the .769 mark of the Mariners. Even though I've been working with OPS figures for a number of years, I still need to stop and think about what a 40-point advantage means. But if you tell me that the Yankees produced 224 more total bases and walks than the Mariners, that's something I can grasp right away.

The following table shows the offensive and defensive TBW figures for the American League, along with the difference between these two figures and each team's league rank based on those differences. It also shows runs for and against, the run differential, and the rankings based on run differential. Finally, because we're trying to trace a path from TBW to runs to wins, it also lists the team's win total for the year.

As you can see, the team rankings using TBW and those using run differentials are very similar. In fact, they're identical except for Anaheim's move from fourth in TBW to first in run margin. The Angels were very efficient on both sides of the ball, finishing 4th in scoring (and only 8 runs out of second) despite trailing six other teams in TBW, and leading the league in fewest runs allowed even though three other teams gave up fewer TBW. (That efficiency didn't carry over to the relationship of runs to wins, however, as they led the league in run margin but were only third in wins.)

In terms of raw production, the Red Sox nearly matched the Yankees, but still managed to come up ten short in the win column. (The same thing happened in 2001.) This comes as no surprise to the long-suffering Boston fans or the incredibly smug New Yorkers who just knew the Sox would find a way to lose despite all their talent.

It's interesting to note that the White Sox were a match for the Twins in production even though Minnesota ran away with the division. For all the talk about the Twins superior pitching and defense and the problems the White Sox had in those areas, Chicago gave up only 50 more TBW, roughly one base every three games.

And we see yet another example of how strong the AL West was this year, with three teams in the league's top five in TBW and run differentials and the Rangers only a little below the league average. Oakland was a clear winner in TBW but trailed the amazing Angels in run margin, and it took an excellent 32-14 record in one-run games to keep the A's in first place.

Let's take a quick look at the National League before pausing to put these TBW numbers in historical context.

While it's clear that Atlanta was the division's top team, their TBW differential wasn't much better than that of the Phillies, who somehow managed to turn a big edge in raw production into a negative run differential and a losing season. Most of the problem was on offense, where the Phils were 3rd in TBW but only 8th in runs scored. (Before the season, our computer simulations had the Phillies finishing a close second behind the Braves. In the real season, they were a very close second statistically, but that didn't translate into the things that really matter, runs and wins.)

The biggest surprise in the Central division was the Cubs. In fact, most of what I just wrote about Philly applies here, too. Our preseason simulations put Chicago third with a .500 record, and the real Cubs put up TBW numbers that were entirely consistent with being a .500 team. But they ranked a few places lower in runs than in TBW on both sides of the ball and they couldn't win the close games (18-36 in contests decided by one run). By the way, the 2001 Cubs were the division's best team statistically (+175 TBW) but failed to win the pennant; with two straight seasons like this, it's no surprise that a managerial change was made, regardless of whether the manager was to blame.

In the West, San Francisco outproduced Arizona but came up a little short in the standings. Both teams were very strong across the board, however, and the Giants showed during the second season that they really were the best team in the league. Statistically speaking, Los Angeles was much closer to a .500 team than their 92-70 record suggests. In fact, the Dodgers were the anti-Phillies, turning a 12th-place ranking in offensive TBW into a 7th-place finish in scoring. (Warning to LA fans: the Padres had the most efficient offense in baseball in 2001 -- 13th in OPS, 6th in runs -- and look what happened to them in 2002.)

A little TBW history

I've been putting these tables together for a few years now, and I can tell you that TBW differentials are usually in the plus or minus 300 range. With eight teams more than 400 from the midpoint this year, I wondered how these figures stacked up against other teams from the past. Thanks to Retrosheet's database of play-by-play accounts, I ran the numbers for all seasons back to 1974. (It would be nice to go back further, but the official stats don't include doubles and triples allowed by pitchers.)

Notes: Only one of these strong teams, the 2002 Red Sox, failed to make the postseason ... only five won the World Series, a reminder that surviving the expanded postseason format is very tough ... could have been a great 1998 World Series if the Braves hadn't lost to the Padres ... the 1978 Brewers also had the best run margin in the AL that year, so this could have been one of the great three-way pennant races in history.

Notes: Eleven of these twenty teams were expansion franchises in the first seven years of their existence ... four 2002 teams made this list ... maybe two rounds of expansion since 1993 is the reason so many recent teams made both lists ... Billy Martin took over as manager of the A's after their disastrous 1979 season, led them to a winning record in both 1980 and 1981, then made this list again in 1982 before being fired ... the Twins have come a long way since 1999 ... in 1982, Bill James wrote that the Blue Jays might be the worst expansion team in history, but they got better in a hurry after that, so maybe there's reason for Devil Rays fans to have some hope as their young prospects move up.

Run Efficiency Average

Earlier in this article, when discussing the relationship between runs and wins, we saw that teams sometimes win quite a few more or less games than their run margin would normally produce. And that those differences don't tend to repeat the next year. It's very rare for a team to over-achieve (or fall short) two years in a row, and there's a very strong tendency to revert to a normal runs-to-wins relationship. Is this also true of TBW and runs?

To identify teams with particularly efficient or inefficient offenses, ones that produce more or less than the expected number of runs given the TBW they produced, I divided runs by TBW to get something I'll call the run efficiency average (REA). As you can see in the following chart, which plots TBW versus runs scored for every full team season since 1974, there's a very strong straight-line relationship between TBW and runs. In other words, we can predict runs scored from TBW with a high degree of accuracy.

It turns out that run efficiency averages look an awful lot like team batting averages. From 1974 to 2002, team batting averages ranged from a low of .229 to a high of .294 with a midpoint of .261. Baseball fans know from experience that a team batting average of .280 or higher is very good, and one below the .245 mark is woeful.

In this time period, run efficiency averages have ranged from .225 to .305 with a midpoint of .264. The midpoint and the spread are slightly higher than for team batting averages, but the benchmarks are basically the same. Anything over .280 indicates a very efficient offense, while anything under .245 indicates a team that squandered a lot of its chances.

Like team batting averages, run efficiency averages tend to be higher in the American League (because pitchers don't bat) and rise and fall by a few points from season to season. (They also appear to be higher in good hitters parks, but I'm leaving park effects out of the equation for the time being. I'll be looking at both offense and defense, so the park effects should cancel out when we subtract one from the other.)

So the best way to evaluate teams is to compare their run efficiency averages to the norm for their league that season and to rank them based on those differences. Here are the offenses that were most efficient in the 1974 to 2002 period, relative to their leagues, and what they did the following year:

A majority of these teams were above average again the next year, but all but one made a move back toward the middle of the pack. On average, they lost 19 points relative to the league. On a base of 2800 TBW, that's a loss of 53 runs, enough to cost a team five to six wins.

Again, all but one team moved up the next year, with an average improvement of 22 points. It's abundantly clear that extreme REA values don't repeat themselves; no matter what the environment, and no matter how good or bad the team, the REA tends to make a big move toward the norm the next year.

This is good news for teams that were the most inefficient this year. Detroit, Baltimore, Philadelphia, Milwaukee, and Tampa Bay can expect to improve their efficiency in 2003. Of course, it's bad news for this year's over-achievers, namely Anaheim, Arizona, Colorado, St. Louis, and the White Sox.

I won't take the space to show top-20 lists for pitching efficiency, but I can tell you that the same pattern held on the other side of the ball. The twenty most efficient pitching staffs moved an average of 21 points toward the norm the next year, while the least efficient improved by 22 points. Not a single team on either list moved further away.

The five most efficient pitching staffs this year, and the five most likely to struggle to match that performance, were Atlanta (which had the lowest run efficiency average in this 29-year period), Anaheim, Oakland, Minnesota, and Los Angeles. On the other hand, improvement is bound to be in store for Colorado, Detroit, Cleveland, Tampa Bay, and the Cubs.

By the way, Detroit was very inefficient on both offense and defense this year, and while their park might have something to do with that, I don't think it's a major factor. If the Tigers move 20 points toward the norm on both sides of the ball, they're looking at a favorable swing of 108 runs, or about 11 wins, even if nothing else changes. (Of course, moving from 54 wins to 65 wins isn't anything to write home about. They need to do even better than that.)

In contrast, Anaheim was on both top-five lists for 2002, and they stand to move back toward the pack offensively and defensively in 2003. That could take a 99-win team and bring them back to the high 80s.

Converting efficiency into wins

Let's try to wrap all of this up into one neat package. We started by showing that runs scored and runs allowed are an accurate predictor of wins and losses. Teams that deviate from this prediction usually revert to form the next year.

Then we showed that offensive production (as measured by total bases plus walks) is an accurate predictor of runs scored. Likewise for defensive production and runs allowed. For both offense and defense, teams that deviate from the predicted number of runs tend to move significantly toward the norm the next year.

In other words, these three forms of efficiency -- which I'll call pythagorean efficiency (turning runs into wins), offensive efficiency (turning TBW into runs scored), and defensive efficiency (limiting runs allowed per TBW allowed) -- can have a major impact on the standings in any one season. But that effect isn't likely to carry over to the next year.

Pythagorean efficiency is already expressed in wins and losses. I'll translate offensive and defensive efficiency into wins by taking the surplus or deficit in runs and dividing by nine. Why nine? According to the pythagorean method, that's the number of runs it takes to add one win in a league where the average team scores about 750 runs. By converting all three types of efficiency to wins, we can add them up to see which teams gained or lost the most due to efficiency in 2002. Here are the figures for all thirty teams:

Let's work through a few examples to make sure it's clear what we're trying to say with this table:

Oakland won 103 games ... six more games than the pythagorean method says is normal for a team that scored 800 runs and allowed 654, mainly by posting an extraordinary 32-14 record on one-run games ... offensive inefficiency (REA of .266 versus a league average of .270) cost them two wins ... efficient pitching and defense (REA of .252) added five wins ... overall efficiency adds up to nine wins ... a team with Oakland's offensive and defensive stats with average efficiency would therefore be expected to win only 94 games.

Anaheim won 99 games ... with the best run differential in the AL, they should have won more games than anyone, but they fell four games short of their pythagorean projection ... but they had the most efficient offense in the majors, picking up seven extra wins (63 runs) because their offensive REA was .292, twenty-two points above the norm for the league ... they also had the majors second most efficient pitching/defense (REA of .243, saving 71 runs), good for another eight wins ... overall, their efficiency on offense and defense overtook their pythagorean inefficiency for a net gain of 11 wins ... an 88-win season would have been more in line with their offensive and defensive stats.

Philadelphia won 80 games ... one more than normal for a team that was outscored by 14 runs ... the offense was the third least efficient in the majors (REA of .239 versus a league average of .256), costing them 51 runs and 6 wins ... defensive efficiency was also a problem (REA of .261), robbing them of two more wins ... total impact was a loss of seven wins for a team that was the league's fifth-best statistically ... with normal efficiency, they should have won 87 games.

the Cubs won 67 games ... eight fewer than normal for a team that was outscored by only 53 runs ... offensive inefficiency (REA of .247 versus a league average of .256) cost them another three games ... defensive efficiency (REA of .270) lowered their win total by another four wins ... this triple whammy of inefficiency cost them 15 wins ... with average efficiency in all three areas, they win 82 games and finish a respectable third in the division.

Before leaving this topic, I want to emphasize that I'm not trying to diminish what the Angels accomplished this year by pointing out that their offensive and defensive stats are more consistent with those of an 88-win team. They did win 99 games in a very tough division by doing all the little things that count: putting the ball in play so even their outs were able to move runners over, hitting in the clutch, playing great defense, getting key outs when they needed them, and so on. They did all that again in the post season, when time after time they got themselves into a hole against very good teams and found a way to get the job done when it mattered most. It was a great run by a team that was awfully fun to watch.

The Angels remind me a lot of the New England Patriots. Both were expected to do very little before the start of the season. Both got off to slow starts and reached the playoffs by putting together winning streaks late in the year. Both were more impressive on the scoreboard than in the statistical leaderboards. Both were intelligent, fundamentally sound teams that had to scrap for everything they got and came up with big play after big play when things looked bleak. And because of all that, both teams were a lot of fun to watch and served as great examples of why championships are decided on the field, not on paper.

Summing up

I could have used a more sophisticated statistic like Runs Created to measure the efficiency of each team's offense and defense, thereby factoring in things like stolen bases, hit batsmen, and a few other stats that contribute to success. But I'm partial to simpler measures like TBW that are easy to figure, easy to interpret, and tell essentially the same story as the more complicated stats. I especially like the fact that runs divided by TBW, what I'm calling the run efficiency average, produces a figure that looks a lot like a batting average, a happy coincidence that makes it easier to get a feel for what's good, what's normal, and what's bad.

It was also very interesting to discover the strong tendency of teams that are highly efficient or inefficient in these three areas to move significantly toward the norm the following season. It's very rare for teams to excel (or fall short) in this way two years in a row. That's a good thing for team executives to know as they plan for next season.

I recall being very impressed with the Houston Astros, who refused to panic after a disappointing 2000 season that saw them fall 8 games short of their pythagorean projection. Many teams would have fired the manager and turned over half the roster in a futile attempt to blame someone for their poor showing. Instead, they chalked it up to one of those years when things just didn't go right and were rewarded with a tie for the division title in 2001. (Of course, after the 2001 season, they fired the manager for failing to win in the postseason, but that's a topic for another day.)

A number of this year's most inefficient teams have changed managers in recent weeks, and some of those managers are going to look like geniuses when their clubs make big gains in the win column next year. I wouldn't mind being Dusty Baker right now, assuming the front office doesn't destroy the team with ill-advised personnel moves this winter. The Cubs are the team most likely to get a large efficiency-related bounce, and with one of baseball's best-regarded farm systems, they are poised for a strong run in the NL Central.