I leave the baseball analysis to my brothers at baseball-reference.com, but I know enough to be dangerous. There’s a stat called BABIP, which stands for Batting Average on Balls In Play. A “ball in play” is simply any at bat that doesn’t end in a home run or a strikeout. The thinking goes that luck and randomness is mostly responsible for the variance in BABIP allowed by pitchers to opposing batters. Pitchers can control the number of strikeouts they throw and control whether they allow home runs or not, but they can’t really control their BABIP.

Therefore, if a pitcher has a high BABIP, sort of like an NFL team with a lot of turnovers, he’s probably been unlucky. And good things may be coming around the corner. A high BABIP means a pitcher probably has an ERA higher than he “should” and that his ERA will go down in the future. In fact, you can easily recalculate a pitcher’s ERA by replacing the actual BABIP he has allowed with the league average BABIP. And that ERA will be a better predictor of future ERA than the actual ERA. At least, I think. Forgive me if my baseball analysis is not perfect.

Are you still awake? It’s Monday, and I’ve brought not only baseball into the equation, but obscure baseball statistics. Let’s get to the point of the post by starting with a hypothesis:

Assume that it is within a quarterback’s control as to whether he throws a completed pass on any given pass attempt. However, if he throws an incomplete pass, then he has no control over whether or not that pass is intercepted.
Essentially, we’re saying that all incomplete passes are “passes in play.” Therefore, a quarterback’s average of “Picks On Passes In Play” — the number of interceptions per incomplete pass he throws — is out of his control. This is just a hypotheses; how would we go about proving or disproving this theory?

Manning and Roethlisberger obviously discussing the intricacies of POPIP.

Let’s look at an example. In 2010, Eli Manning led the NFL with 25 interceptions, throwing an interception on 4.6% of his pass attempts, and on 12.5% of his incomplete passes. That same year, his draft classmate Ben Roethlisberger had just five interceptions on 389 attempts. Roethlisberger averaged only 1.3 interceptions per 100 passes, and threw an interception on just 3.4% of his incomplete passes.

Eli actually had a higher completion percentage than Roethlisberger. Maybe that’s a sign that completion percentage and interception rate aren’t strongly related, or maybe it’s a sign that interception rates on incomplete passes are random. In 2010, 7.5% of all incomplete passes were intercepted. If 7.5% of all of Eli Manning’s incomplete passes were intercepted, he would have thrown 15 interceptions in 2010 instead of 25, and therefore would have an estimated interception rate of 2.8% based on a league average POPIP ratio. For Big Ben, if 7.5% of his incomplete passes had been picked off, he would have thrown 11.2 interceptions instead of just five; that would give him an estimated INT rate of 2.9%.

Okay, so what does any of this mean? Manning had an actual INT rate of 4.6% but an estimated rate of 2.8%, while Roethlisberger was at 1.3% and 2.9%, respectively. Obviously this doesn’t impact what’s already happened. But just like how BABIP can help predict future INT, POPIP could help predict future INT rate — well, at least that’s the theory.

As it turns out, in 2011, Manning and Roethlisberger both had actual INT rates of 2.7%. That gives us two pieces of evidence that estimated INT rate based on POPIP is a better predictor of future INT rate than actual INT rate. But those are just two pieces of evidence.

Since 1970, there have been 813 quarterbacks to play for the same team in consecutive years, and throw at least 224 passes in Year N and at least 100 passes in Year N+1. Why those cutoffs? 224 passes in the current minimum number of attempts needed to qualify for passing crowns in rate statistics; meanwhile, I don’t want to lose out on including quarterbacks who were benched early the next year because they were bad. But we can play around with several cutoffs.

I went through and gave a win to the actual INT rate if the player’s interception rate in Year N+1 was closer to his actual INT rate from Year N than his estimated INT rate. On the other hand, if the player’s estimated INT rate in Year N was closer to the actual INT rate in Year N+1 than the actual INT rate, I gave a win to the estimated INT metric.

The results? 335 times, the actual interception rate proved to be the better predictor, while 478 times the estimated interception rate closer to the future rate. So 59% of the time, using estimated INT rates based on POPIP proved to be helpful.

If we make the quarterbacks throw 224+ times in both years, we’re left with 693 pairs of quarterback seasons. The estimated INT rate was a better predictor in 403, or 58%, of those pairs.

If we go back to only 1978 instead of 1970, the estimated interception rate was better 58% of the time out of 595 pairs. If we change the cutoff to 1990, the estimated interception rate was better 58% of the time on 385 pairs.

If we bump the attempts threshold to 350 in both Year N and Year N+1, since 1990, the estimated interception rate was the better predictor on 157 of 253 pairs, or 62% of the time. If we limit it to just the past ten years, we have 122 pairs of quarterbacks — and the estimated INT rate was better on 63% of those pairs. Change the cutoffs to 400 attempts both years and look at only quarterbacks over the last five years, and 60% of the time on 57 pairs the estimated interception rate was better.

I think you get the point. Estimated interception rate isn’t perfect — there are so many fluky interceptions that no one could come very close to predicting future interception rate — but I feel pretty confident in telling you that estimated interception rate is better at predicting future INT rate than actual INT rate. In that vein, it is similar to how Pythagorean winning percentage is a better predictor of future winning percentage than actual winning percentage.

We can also use POPIP to find quarterback outliers. We can calculate how many interceptions a quarterback was estimated to throw for his career along with how many he actually threw. The difference there might be pretty informative. The table below lists all quarterbacks since 1970 with at least 50 interceptions; as always, quarterback who played prior to 1970 are included but only their stats since 1970 are reflected in the table below. The table is currently sorted based on the quarterbacks who came in under their estimated number of interceptions the most. Let me use Donovan McNabb as an example.

The first row reads: McNabb for his career has 5,374 attempts and 2,204 incomplete passes. His career INT rate is 2.2%, and has thrown an interception on 5.3% of his incompletions. Based on league average POPIP, we would have expected him to throw an interception on 3.2% of his passes. For his career, he has 117 interceptions and we would estimate that he would have thrown 171.1 interceptions. As a result, McNabb has thrown 54.1 fewer interceptions than we would have estimated (this is what the table is sorted by). He has thrown one fewer interception per 100 passes than we would have projected.

It’s tempting to attribute much of the spread between actual and estimated interceptions to luck. No doubt, luck plays a big part, and is perhaps the biggest single factor. I don’t know if there is a lot of talent involved in keeping your POPIP low; both Peyton Manning and Drew Brees have almost exactly hit their estimated interception numbers. Greats like Brett Favre and Kurt Warner came in with a bit more interceptions than expected, while Donovan McNabb, Bernie Kosar and Ken O’Brien were great at having a low POPIP average. And, of course, we should need to guard against circular logic such as ‘Keeping your POPIP low is a skill because some of the best quarterbacks kept it low, and those quarterbacks are some of the best quarterbacks because they didn’t throw many interceptions.’

But just like we can look at the relationship between sack rate and interception rate to see a quarterback’s style of play rather than just consider stats as good and bad, we can do the same here. Commenter Red astutely pointed out that McNabb had a low interception rate even though he was inaccurate, and he did seem to throw a lot of bad passes towards his receivers’ feet and not by defenders’ hands. I don’t really think of Warner and Favre as similar quarterbacks, but both did seem to throw a fair share of interceptions.

Ken Stabler is the one that really throws me. His INT/INC rate is off the charts, even for his era. Let me know your thoughts in the comments. I’ll close with a cool chart, showing the INT/ATT and INT/INC rates for every year since the merger.

Isn’t your predicted interception numbers basically just regression? If a guy has a lot of interceptions in year N, isn’t he usually going to have fewer in N+1 due to regression?

As for the general theory, I think there are lots of different types of interceptions. If you could filter those out it might be more informative.

First of all, if all hail mary-type interceptions could be entirely removed from the analysis, that would be helpful. Those aren’t the same, because an INT and an incompletion are equivalent to the QB at that point.

Also, if there was a way to remove interceptions where a defender clearly sneaks in front of a receiver to grab the pass, would be helpful. I’m not sure if that is really an uncontrollable situation for the QB. That pass probably wasn’t going to be an incomplete anyway. But the QB probably could have avoided it if he had been watching the defenders better.

Tipped passes seem the most obvious candidates for this analysis. Once a pass gets tipped, it seems much more likely that what happens next (completion, incompletion or interception) is random.

Anyway, those are just some initial thoughts.

Chase Stuart

Agreed with all of those thoughts. Ideally, you would want to analyze each interception thrown by a quarterback (and really, each pass thrown by a quarterback) and grade the throw, not the result.

Re: your regression comment, that’s true. If I had time, my plan was to compare these results to a few other models (comparing EST_INT_RT to just leave average rate to predict future INT rate for each QB; comparing EST_INT_RT to a regression-adjusted model for each QB’s INT rate (maybe give him 50% of his own INT rate, 50% of league average), etc.). But Sunday’s games took too much out of me. I think this is just step one of analysis, not the final step.

Yeah, no, I wasn’t linking to it saying we’ve done your thing before. POPIP is definitely unique, and I like its straightforwardness. I was responding to Richie’s recommendations.

Danish

Very interesting, and a nice idea. Those to graphs are extremely similar, to the point where I suspect you have made a mistake in excel. I guess the lack of spike in INT/ATT in ’82 gets you off the hook 🙂

I used a slightly different set of data (just QBs, not all passers) but I imagine the results will be just about identical. Unless I made a mistake, of course. 🙂

Red

Could it be that accuracy and decision making are two independent components of QB play? Brett Favre is probably the best example of this. He had several seasons in which he completed over 65% of his passes, yet still had awful INT % in those years (2003 and 2008 come to mind). Favre was always known as a great thrower, but a poor decision maker, which is backed up by the discrepancy between his completion % and INT/INC %.

Oh, and I’ve never had anyone link to one of my comments before 🙂 Thanks, Chase!

Chase Stuart

Yes, it’s possible that accuracy and decision making are pretty independent, which would go along with your theory. I’m not sure what that says about McNabb, though — that he wasn’t accurate but made good decisions?

I also think we’re going to be dealing with really small sample sizes. Outside of the few outliers like McNabb and Stabler, I’m not sure if we can make too many definitive statements.

Red

I agree that POPIP is mostly sample size noise, and for 90% of QB’s it’s probably random. Outliers like McNabb are interesting, because frankly he just doesn’t make any sense. He wasn’t very accurate or very smart, and he often played in pass-heavy offenses that required him to throw in sub-optimal situations. I really have no idea why his POPIP is so consistently low throughout his career. I’m too young to have seen Stabler play, so I couldn’t even begin to formulate an explanation for his high POPIP.

Just out of curisosity, how many pass attempts does a QB need for his INT % to become statistically significant? I would guess it has to be at least 2000+ attempts before INT numbers mean anything.

It would be very interesting to see an alternate version of your GQBOAT rankings, using expected INT’s instead of actual INT’s. I’m thinking that would give us a more accurate picture of who was good and who wasn’t, with the randomness of interceptions pretty much eliminated from the equation.

DC

Stats can only go so far in this case. You’d have to look at the game film for each interception to assess blame. Did the quarterback miss his target? Did the receiver forget his route? Did defensive line deflect the ball? Did the ball bounce off the target’s hands?

Derek

I know this is a late post, but I just came across your article and I like the idea behind it. I’m a fan of advanced baseball statistics and would like to see similar work in football. Here are my thoughts on INT/INC…

I like the concept of determining the repeatability of avoiding interceptions. I don’t think the research should start by comparing individual seasons though. The sample sizes are too small to produce meaningful data.

You might glean more useful data when examining careers. If POPIP is primarily luck, the career rates should be fairly close because we could expect each QB’s POPIP to regress toward the average over the course of his career.

I took a cursory glance at the numbers. Excluding the five highest and lowest values, the range of INT/INC is 6% to 12.8%. The average is 8.82%. Out of 143 samples, 58 are within 1% of the average, and 114 are within 2% of the average. That’s far from a complete picture, but my gut tells me there is too much variance between each QB for INT/INC to be a non-repeatable skill.

The big caveat though is that the pool of qualifying QBs is only 143 (after five excluding highest and lowest values). I’d like to see more research on the subject with a bigger sample of quarterbacks, though I don’t think you’d want to push the minimum career length any lower. I wonder if a look at college numbers might provide more information. Perhaps you could establish a baseline for team quality and minimum career passes attempted, and compare the INT/INT career rates of those QBs.