Aim For The Head

PAP^3 FAQ

PAP^3 is the name for the new system for measuring pitcher abuse via pitch counts introduced in Baseball Prospectus 2001.
Though it shares a similar name and goal with a system previously introduced by Rany Jazayerli, it was developed independently,
and replaces the older system.

Since the publication of the book, I've received many thoughtful questions and comments about the research and the PAP^3 system.
Probably the single most complete critique is Sean Forman's writeup at Baseball Primer. The questions in this FAQ aren't
necessarily taken verbatim from reader comments, as I've tried to amalgamate various questions raised.

A: This was a goof on my part. The report is sorted by average PAP per start, but I neglected to actually print that column.
We'll have it fixed for this year's reports.

Q: Hey, there's a problem with the graphs published in the book!

A: Yes, the graphs do have a mistake, though not, as it turns out, one that unduly affects the conclusions of the article.

The problem is essentially that the data points are taken from the leftmost point of the data range, rather than the midpoint.
E.g., for 90-99 pitches, the data point is plotted at 90 instead of 95 (or 94.5 to be more accurate). I've corrected the graphs
from the ones that appear in the book such that multiple of ten appears in the midpoint of the range, and they are the ones
found in the web version of the original articles, and that I've reproduced below. (Thanks to Sean Forman, who was the first to
bring this to my attention.)

Q: The star pitcher on my favorite team just threw 130 pitches! Shouldn't the manager be fired?

A: Not necessarily. Though PAP is measured on a per-start basis, a single long outing is probably not enough to cause
significant injury risk. You might expect your pitcher to be a bit less than his usual dominating self for the next few starts,
however.

For example, if the pitcher in question started 30 games, one in which he threw 130 pitches, and the rest with exactly 100, his
Stress level for the season would be 8.9, which is not worth worrying about. It is a pattern of high-stress workloads over the
course of a career that is of most concern.

What's worth more scrutiny is the circumstances under which the pitcher was allowed to throw so many pitches. In a close game
against a tough opponent, or versus a division rival during a pennant chase, or even if the pitcher is nearing a personal
accomplishment such a no-hitter, shutout, or career high in strikeouts, there may be justification for leaving him in. Letting
him burn himself out in a 15-2 blowout is more suspect (unless, perhaps, the game is at Coors Field or Enron Memorial Stadium).

Q: What about age effects? The old PAP system had an adjustment for young pitchers.

A: The new PAP research did not study the role of age in injury susceptibility beyond grouping pitchers of similar age and
career workload in studying injury risk. Thus, the new PAP system itself is silent on the question of age--nothing in the study
confirms or refutes the notion that young pitchers are at more risk, though some other research (notably Craig Wright's)
suggests that it may be the case.

Similar, PAP may not be valid for high school, college, and low-minors pitchers who are significantly younger and in earlier
stages of physical development than most of the pitchers in the major-league study. It's probably better than nothing, but the
threshold (100 pitches in PAP), in particular, could be quite a bit lower for less developed arms.

Q: Does PAP address whether having additional days of rest can offset the damage of a high-pitch start? For example, Bud
Smith was skipped once in the Cardinal rotation following his 134-pitch no-hitter in 2001.

A: No, PAP in its current form captures "typical" usage patterns, which essentially means the five-man rotation that
is standard today. No attempt was made to look at how pitching on short or long rest affects performance. One difficulty with
studying the effect of the Bud Smith example is that it's very hard to tell from historical records whether a pitcher skipped a
turn in the rotation as a preventative measure, or because he was already hurting.

Q: Does PAP address pitching on three days of rest versus four days?

A: No. PAP, because it's based on averages across pitchers from the late '80s and 1990s, is very much a product of that
particular baseball environment. The five-man rotation has been a fixture throughout that period, and thus PAP can generally be
said to be based on the four days of rest common in modern pitching rotations. There was no attempt to distinguish starts based
on the amount of rest the pitcher got.

Q: Why does the PAP model start with 100 pitches instead of 130 where the curve really takes off?

A: As mentioned in the article, I based the system on the lowest pitch count in which we see a consistent decline in one of the
four components of the performance index. This index that declined first happened to be strikeout rate. Furthermore, referring
to the distribution-of-personal-thresholds model described elsewhere in this FAQ, if there are pitchers with low thresholds,
there would be a real chance of pushing one of them past their personal threshold with a relatively low number of pitches. A
good system would capture this small, but real, risk even at lower pitch-count levels.

Q: There are examples of pitchers throwing over 200 pitches a game. With the amount of abuse PAP would assign to that start,
their arms would have fallen off! Does this mean that PAP is fundamentally flawed?

A: PAP, like any tool, should only be used within the ranges where it's been validated. There were very few starts above 140
pitches in the study, and therefore we can't say with any certainty how much worse the results would be as pitch counts rise
above that level. Using PAP in its current form for starts much above 140 is not advised. To be conservative, capping PAP at the
140- or 150-pitch level makes some sense. Otherwise, the PAP from a 200 pitch start would probably suggest that the pitcher's
arm would spontaneously combust.

As a purely hypothetical example, suppose that once a pitcher starts tiring, his decline continues until he's "fully
fatigued," or to use baseball slang, "out of gas." If a typical pitcher's threshold for full fatigue is around
160 pitches, then it's possible that the loss of effectiveness as measured by PAP would flatten out with the next 10-20 pitches
above the studied range. In such a case, 200 pitches would not have more impact on a pitcher's short-term performance than 180
pitches, and PAP would need to be modified to reflect that relationship once it's discovered.

Q: Curveballs are supposedly more damaging to a pitcher's arm than fastballs. Does that have an effect on PAP?

A: The type of pitches thrown is not currently a factor in PAP. In fact, the historical records do not include breakdowns for
the number of fastballs, curves, change-ups, sliders, split fingers, and the like thrown in a given start, so that kind of
detailed analysis will probably have to wait for better recordkeeping. It may be possible to do some study involving pitchers
who were "known" to be curveball pitchers (Bert Blyleven) or fastball pitchers (Nolan Ryan), but barring
watching videos for every start in their careers, the estimation of pitches by type is extremely dicey.

Q: Injuries come from pitchers being tired, not an arbitrary number of pitches.

A: Pitch counts, and systems like PAP, are ways to quantify or derive a measurable index that is related to injury or
ineffectiveness, which are in turn caused by fatigue. However, fatigue cannot be reliably and objectively assessed based on
information available to the manager at the time that a decision needs to be made. Self-reporting of fatigue by competitive
athletes is notoriously inaccurate. Expert assessment, in the form of coaches' observations of a pitcher's mechanics, may help
determine when a given pitcher gets tired, although the quality of those assessments is difficult to evaluate. PAP can be
another tool in a manager's arsenal, and a useful one, but it is not a one-stop shop for making these kinds of decisions.

Q: Does PAP predict the type of arm injury that is most likely to occur? Do older pitchers have a higher change of reinjury?
Are certain types of injuries more likely to recur?

A: No, unknown, and unknown. The PAP research to date has not included age, previous injury, or breakdowns of different types of
arm injuries, and thus can't be used as a basis for concluding anything about the role of age or type of injury.

Q: Is there any connection between PAP and pitchers trying to come back from injuries?

A: It's possible that there's an effect on recovery. A reasonable person might suspect that high stress has greater impact on an
already damaged arm, but this hasn't been explicitly studied. It's a great area for further research, perhaps even more
important that focusing on age.

Q: How can throwing one more pitch change a "Moderate" risk to a "Significant" risk just because it
happens to be the 122nd pitch of the night? It seems like an arbitrary distinction.

A: Any classification system that organizes a continuum of possible effects into a finite number of discrete groups is going to
have break points or discontinuities where the local transition is overstated. In other words, whenever you simplify a complex
phenomenon into categories, the borders of those categories will be more similar than you'd expect from looking at the average
member of each category. Quality Starts is another baseball example of this. You can argue about whether "six innings,
three earned runs" ought to be a Quality Start or not, but wherever you draw the line, the average QS will be significantly
better than the average non-QS.

This isn't limited to baseball. A person who is 18 years and 0 days old is not necessarily more mature or capable than one who
is 17 years, 364 days old. Yet the legal system has a clear boundary in which the former has rights and responsibilities that
the latter doesn't based only on that one-day difference.

Q: Can you use the equation presented in the article [ Prob(Injury) = .06 * LN(Stress) ], plug in a pitcher's Stress for
2000, and forecast his likelihood for injury in 2001? For example: Pedro Martinez had a Stress level of 60.13 in 2000.
Could this mean that he has a 24.58 % chance of injury in 2001?

A: Answering this question now is rather ironic, given what we know happened to Martinez in 2001. The short answer is "no,
that's not an appropriate way to use the Stress-to-Injury formula in the book, based on the research to date."

What was presented was, at best, an estimate of the lifetime risk of at least one major arm injury (missing 30+ days) given a
sustained career Stress level. Even this estimate is dubious, as the study included several pitchers whose careers are not yet
complete, and thus we can't say with certainty whether a pitcher who has been healthy to date will suffer a qualifying injury,
and thus affect the results of the research. It's a work in progress, and it's far too early to say with any confidence that a
Stress load of X means an injury probability of Y even for a career, let alone one season to the next. I'll emphasize that
again--PAP^3 is a rough tool for assessing career injury risk, and any specific probability that comes out of it will have very
wide error bars pending much more detailed research.

Q: I am a graduate student in statistics and one comment in the article struck me as a little off. The chance of injury for
one group was near 30% and the chance for the other group was 10, so you state that the risk was about three times as great. In
categorical data analysis, this is called the relative risk.

The problem is that this is not an appropriate measure of association in this case. Relative risk is good when you have
independent binomial sampling as in clinical trials where you set the members of the group beforehand. Your study is more of a
case control where the injuries are fixed and then you notice which pitchers are in after the fact. It is more of an
observational study. Future studies could be more clearly defined, but this is observational in my opinion.

When you have case control studies, the relative risk is a function of the total sample size, not a desirable property since
sample sizes vary. This means that as more pitchers are added, the rates could differ. A better measure of association would be
the odds ratio. A probability of .3 equates to odds of 0.4286 (p/1-p) and .1 is odds of .1111. That means the odds ratio is
3.858 which means the odds of injury are 3.858 times greater for the high PAP group. I believe this to be more telling statistic
and it is more correct. I know this is a minor detail but you guys are detail-oriented and I have recently lost points on a
take-home exam for the same mistake.

A: I defer to your professor, and appreciate the correction. It's important to get the methodology correct, and I welcome input
from those more statistically sophisticated than I to keep me honest. Good luck on your next test.

Q: The effects measured in PAP seem to be rather small, on the order of a few percent. It is at all significant in practical
terms?

A: A single start by itself produces only a small effect. A 5% decline in performance for three weeks amounts to an expected
increase in scoring of about six-tenths of a run. But it is a pattern of overuse, say one overlong start every 3-4 weeks, that
can potentially keep a starter from his optimal performance level for the entire season. If a manager rides his top 2 starters
hard, that can amount to about 10 runs over the course of a season, or about one win. Over the course of a season, the cost of
not keeping starting pitchers fresh could be significant.

It's also worth noting that in comparing the pre-start and post-start performance levels, no attempt was made to see whether the
pre-start time period included a high-pitch-count start or was affected by an earlier high pitch count. If the effects of high
pitch counts are accumulating on top of each other, the magnitude of the effect itself could be understated.

Another secondary impact is that starts subsequent to a high-pitch-count start tend to be shorter (in innings pitched) than
those immediately preceding. Throwing fewer innings may have a ripple effect on the bullpen, although this is counterbalanced by
the fact that most high-pitch-count outings have a pitcher throwing deep into the game, reducing the pressure on the bullpen.

Q: Pitchers are all different! PAP treats them all the same without considering that Randy Johnson is more durable
than Steve Woodard.

A: That's a good point, and we can use it to demonstrate how PAP can be interpreted in different ways.

Take a start of 118 pitches. The PAP^3 study finds that a start of that length has a moderate risk of a short-term decline in
performance (118 pitches is a high Category III start) when measured across all starters who've thrown that many pitches. In
other words, assuming you know nothing else about a pitcher, you'd expect a mild decline. However, there are other ways to
interpret what PAP tells us. It is not necessarily true that all pitchers are expected to decline modestly after such an outing,
even if the group as a whole does.

One of the apparent assumptions in PAP is that all pitchers have the same physical reaction to a given number of pitches. Thus,
any pitcher with a 118-pitch start is assigned the same level of risk, or expected decline. The reality is far more complex, as
pitchers differ significantly in their ability to handle a workload, and how their bodies react to being pushed past a
threshold. Let's introduce the idea of a personal threshold for a pitcher--the level at which a specific pitcher's performance
can be expected to decline slightly as a result of marginal overuse. This is comparable to the 100-pitch level in PAP, where
abuse points start accumulating.

To illustrate with a hypothetical example: suppose there are 100 pitchers, each of whom have a personal threshold of 101-200
pitches (one pitcher per pitch level uniformly spread out across the interval). Beyond that point, each of them suffers an
immediate 25% decline in performance (and no more, regardless of how far past their personal threshold). In this simplified
model, each pitcher is either definitely overworked or definitely not, with no shades of gray. If we let each pitcher throw 120
pitches in a game, 80% of them will suffer no ill effects, while the other 20% will decline by 25%. Measured across the entire
population of pitchers, the effect of a 120-pitch outing will appear to be just a 5% decline. But you could also say that there
is a 1 in 5 chance of performance dropping by 25%. There's a difference between surely losing a small amount and a small chance
of losing a lot (indeed, that's the basis of the insurance business).

This is a roundabout way of saying that at 118 pitches, rather than PAP suggesting a mild decline in pitcher performance, PAP
could also be interpreted as that there is a fraction of pitchers who would decline a lot (as in the example above). This is not
necessarily the case, it is just one possibility. That's part of the reason why I described the categories in terms of the
amount of risk, rather than the amount of decline. In reality, pitchers probably do have different thresholds of work they can
handle, but the decline for each pitcher isn't constant above that threshold--the further above the threshold you push him, the
bigger decline you can expect, up to a point. Finding a pitcher's personal threshold is a much bigger challenge, and one where
the professional discretion of the manager and coaching staff can be given some leeway. I don't think anyone has tackled the
problem sabermetrically yet, which is why we have generic tools like PAP that describe the expectation across populations
instead of individuals.

The shape of the PAP curve that would result from combining the different profiles of each pitcher depends on how personal
thresholds are distributed. In the example above, thresholds were uniformly distributed between 100 and 200 pitches. It's more
likely that pitchers fall on some sort of bell curve (though perhaps skewed towards higher-endurance pitchers becoming
starters). I've done some informal mathematical modeling of whether certain probability distributions could yield a PAP^3 kind
of curve, and there's indication that a lognormally distributed threshold (with a mean around 125 pitches) could produce the
results observed. However, keep in mind that PAP was developed by focusing on the higher-endurance pitchers to begin with.

Given a distribution of thresholds, the perceived decline in the overall population of starting pitchers would be smaller than
the observed decline in a pitcher actually pushed beyond his limit. In a hypothetical world where this was the only effect of
high pitch counts, PAP^3 would be a measure of the likelihood that a random pitcher suffers a large decline from throwing that
many pitches, rather than the amount of decline a typical pitcher would suffer.

This does not mean, though, that other sources of knowledge can't inform our assessment of the wisdom in letting someone throw
that many pitches. Craig Wright's research suggests that young pitchers are at higher risk of injury from overuse, and
biomechanics probably tells us that muscles get torn more easily at cold temperatures. If you take a young pitcher, like
Kerry Wood, who has a history of arm problems, there is good reason to think he might be a pitcher whose risks from high
pitch counts are higher than the "moderate" risk level assigned by PAP^3 to a generic pitcher. Perhaps even more so if
he's starting on a cold night in April. The PAP framework is a baseline, and should be modified based on knowledge of the
individual pitcher and game circumstances.

Q: Why did you use total career pitch counts instead of pitches per start in the injury study?

A: Average pitch counts alone are not sufficient, because they do not capture the total amount of work thrown at that pitch
load. To illustrate with an extreme case, a rookie pitcher with five starts of 120 pitches each is considered to be as much
"at risk" as a staff ace who has thrown 120 pitches/start for 30 starts in each of the past three years. Since the arm
injuries under consideration are repetitive stress injuries, the quantity of work performed has to be considered in conjunction
with how that workload was structured.

In terms of the total amount of work extracted from an arm, total pitch counts is one way to measure for the total energy
expended in the act of pitching, or the number of repetitions of the pitching motion required over the course of a career. Thus,
grouping pitchers by career pitch counts seemed to be a reasonable basis for the study.

Q: Why do both the career pitch total, and the length of individual starts matter?

Let's look at an example. Consider two pitchers, A has 10 starts of 110 pitches each, B had 11 starts of 100 pitches each. They
have each thrown the same total number of pitches. We can chart them as follows:

The last chart is the most significant. The area under the curve (or in this case, the area of the rectangles for each pitcher),
represent the total amount of work done (pitches thrown). Placing one workload over the other, we see that A and B had a large
subset of work in common--namely, the first 100 pitches of 10 starts. Each pitcher has 100 more pitches not in common with the
other. Pitcher A threw 10 more pitches in each of his 10 starts, while Pitcher B had one more start of 100 pitches.

If we determine that Pitcher A is at more risk for injury than Pitcher B, then what this means is that the extra 100 pitches
thrown late in the game collectively cause more damage (or create more risk) than the 100 pitches throw while starting fresh in
an extra game. Thus, throwing a larger fraction of pitches late in the game would leave a pitcher more susceptible to injury.
If, on the other hand, you believe that every pitch contributes an equal amount of risk (rather than an increasing amount on the
margin), then total pitch counts would be proportional to risk, and the results of the long-term injury risk study showed that
they are not.

Looking at average pitch counts actually is actually conceptually similar to PAP's Stress metric, assuming that you've already
grouped pitchers by career workload. To use the examples of Shawn Estes and Steve Woodard (suggested by Sean
Forman), we have two pitchers with similar career workloads and vastly different average pitch counts. If you believe that they
are not comparable in injury risk, then one possible reason would be that the 16-pitch-per-game difference results in a higher
injury risk for Estes. In other words, concentrating the same number of pitches in high-pitch-count outings is worse than having
more low-pitch-count outings for injury risk. That's an idea very similar to Stress.

Now, this average pitch count differs from PAP^3 in two significant ways--it's not a cubic function, and it doesn't have a lower
threshold cutoff. But fundamentally, it's employing a similar concept of increasing injury risk with increasing pitch count.

Q: Can you chart the injury risk versus average pitch count the same way you did with Stress in the article?

Somewhat ironically, the best and simplest polynomial curve to fit the average pitch count results is a cubic function similar
to the one found in PAP^3, though this is coincidental.

A better way to compare average pitch count to Stress is to look at the how well different models taken from each fit the
observed injury data. Here, we'll look at R^2, the percentage of variance in injury attributable to changes in the underlying
metric (Stress or average pitch count, transformed by the indicated model).

The formula presented in the book--Prob{Injury} = .6*LN(Stress)--has an R^2 of 0.621 (in the range where it returns a
non-negative number), and while it's not as good a fit as the Stress-squared model, it has two features that make it more
attractive:

it has the appropriate behavior even as we reach the top of the range (the polynomial version decreases at high Stress
levels, implying that higher stress leads to lower injury rates)

it is a simpler and more elegant form to remember and implement. The loss of accuracy compared to the best curve-fitting we
can find for a tool that is still quite rough to begin with should not be a major concern, as the real world variance dwarfs any
difference between the models.

That being said, it would not surprise me if eventually a better injury metric than PAP^3's Stress is discovered (and I would
endorse it when/if it comes), but at the moment it does appear that PAP^3 has some usefulness in predicting injuries.

Q: What about the regimen of pre-game pitches to get loose, the warmup pitches to start each inning, throwing between turns
in the rotation, spring training, post-season starts, the All-Star Game, and the like? You're not counting every pitch.

A: That's true, and ideally we would count them all. It may be particularly critical that we aren't counting post-season starts
and, to a lesser degree, spring training games. Especially with the longer post-season, a starting pitcher could end up making
20% more starts beyond his season total.

However, as long as the other factors either (a) are more or less constant among pitchers, or (b) are roughly proportional to
the number of pitches thrown, then it doesn't have a big impact on PAP's evaluations. Many of those examples are pitches not
thrown at full force, and presumably are not as stressful on the arm. The pitched thrown in pre-game warmups (especially since
the arm is fully rested at that point) and between turns are probably comparable for every major-league starter. Pitchers
throwing lots of pitches tend to throw lots of innings, and adding a fixed number of pitches per inning started would add
similar amounts to every pitcher with similar PAP totals.

It's even possible that those warmup pitches are part of the reason we see the decline in performance that we do. Assuming eight
warmup pitches per inning, a pitcher who totalled 80 pitches in five innings has "thrown" 40 extra (120 pitches),
while a pitcher whose made 120 official pitches in eight innings has "thrown" 64 extra (184 pitches). This hidden
workload, some of which occurs while the pitcher is getting fatigued and which is larger the deeper into the game you go, could
explain part of the tendency for high official pitch counts to yield non-linear declines in performance. This is purely
speculative, though.

Q: People who count pitches are akin to someone who notes death and funerals almost always go together and then concludes
that funerals cause death.

A: That isn't really an accurate analogy in a couple of ways. At the very least, the temporal ordering of death and funerals are
reversed from pitching resulting in injuries. A better analogy would be to say that doing 120 mph on a highway is associated
with higher death rates, and objecting because the thing that actually causes death is blood loss from the flying piece of metal
that used to be the other guy's bumper piercing your jugular vein. The speeding doesn't cause the death, but it increases the
risk of circumstances that cause death. In other words, B may cause C, but A leads to a higher risk of B happening, so A
similarly leads to a higher risk of C.

Keith Woolner is an author of Baseball Prospectus. You can contact him by
clicking here.

Keith Woolner is an author of Baseball Prospectus. Click here to see Keith's other articles.
You can contact Keith by clicking here