This time around, I’m not going to tweak the formula much (that’s for GQBOAT VI), but I do have one big change that I suspect will be well-received. Let’s review the methodology.

Methodology

We start with plain old yards per attempt. I then incorporate sack data by removing sack yards from the numerator and adding sacks to the denominator.1 To include touchdowns and interceptions, I gave a quarterback 20 yards for each passing touchdown and subtracted 45 yards for each interception. This calculation — (Pass Yards + 20 * PTD – 45 * INT – Sack Yards Lost) / (Sacks + Pass Attempts) forms the basis for Adjusted Net Yards per Attempt, one of the key metrics I use to evaluate quarterbacks. For purposes of this study, I did some further tweaking. I’m including rushing touchdowns, because our goal is to measure quarterbacks as players. There’s no reason to separate rushing and passing touchdowns from a value standpoint, so all passing and rushing touchdowns are worth 20 yards and are calculated in the numerator of Adjusted Net Yards per Attempt. To be consistent, I also include rushing touchdowns in the denominator of the equation. This won’t change anything for most quarterbacks, but feels right to me. A touchdown is a touchdown.

Now, here comes the twist. In past year, I’ve compared each quarterback’s “ANY/A” — I put that term in quotes because what we’re really using is ANY/A with a rushing touchdowns modifier — and then calculated a value over average statistic after comparing that rate to the league average. For example, if a QB has an “ANY/A” of 7.0 and the NFL average “ANY/A” is 5.0, and the quarterback has 500 “dropbacks” — i.e., pass attempts plus sacks plus rushing touchdowns — then the quarterback gets credit for 1,000 yards above average.

This time around, I’m going to incorporate strength of schedule by comparing each quarterback in each game not to league average, but to the “ANY/A” allowed by the opposing team in each of their other games. Add up the Value relative to the defense in each game of each season, and you get season grades. This step combines a SOS and era adjustment all in one, and frankly, I’m not sure why it took me so long to get around to this.2

There are three final steps. After creating season grades, we need to include two more statistics and then make one last adjustment. First, we subtract for “net fumbles” lost by the quarterback, which represents the number of fumbles by a quarterback minus the number of fumbles he recovered. Based on prior research, a net fumble is worth about -30 yards, so -30 is subtracted for each net fumble. I calculated the average fumbles lost per “Dropback” by quarterbacks in each year (which was just north of 1.0% for most years) and then gave each quarterback credit (or a penalty) for their net fumbles above/below average.3

We also must add a rushing component. I have never figured out a good way to handle quarterback rushing, but what I’ve done in the past is to add all quarterback rushing yards over 4.0 yards per carry. So 500 yards on 100 carries goes down as +100. That’s not very scientific, but the results have been noncontroversial, so I’ll continue to use it until someone suggests something better.

Finally, I made adjustments for non-16 game NFL seasons. For seasons with fewer than 16 games, all stats have been pro-rated to 16 games. And for the first five years of the AFL, I reduced the value of quarterback statistics.4

Last time around, Tom Brady’s 2007 season ranked not just third, but a somewhat distant third to Dan Marino 1984 and Peyton Manning 2004. As it turns out, the SOS adjustment changes that. The table below shows the top 100 quarterback seasons since 1960. Let’s walk through Brady’s ’07 to guide you through the table. In 2007, Brady threw for 4,806 yards and 50 touchdowns, while recording just 8 interceptions, 21 sacks, and 128 sack yards lost. He had a “Relative ANY/A” of +3.63 (this includes the rushing TD bonus). He had a harder than average schedule, although that’s already been factored into his “RANY/A” (I’m including the SOS column for reference only). Brady had 601 dropbacks, and since he had fewer fumbles than average, gets an 88 yard fumble value bonus (but no rushing bonus). Add it up, and Brady produced 2,270 yards of VALUE over average, the most of any quarterback in any season since 1960.

San Francisco leads the way with 17 top-200 seasons: seven by Young, five by Joe Montana, four by Brodie, and one by Jeff Garcia. The Chargers are second with 12 such seasons, with Brees and John Hadl (!) each contributing one year, and Fouts and Philip Rivers delivering the rest.

And now, the huge caveat: It’s important to remember that this is just a measure of each team’s passing game, assigned to the quarterback on the field for those plays. Obviously the quality of the offensive line, the ability of the receivers, the versatility of the tight ends and running backs, the philosophy of the coaches, the strength of the schedule, and good old randomness have a significant impact on the above numbers. The reason for these posts is to accurately measure quarterback statistics, and nothing else. Once we have strong measures of QB performance, we can then judge QBs based on how much of their success (or lack thereof) we want to assign to the QB and how much to other people/factors.

And, of course, even if supporting casts are the same, numbers don’t tell the full story. The point here is simply to get the most out of the numbers we have. Anyway, I’m sure this gives you more than enough to chew on. Leave your thoughts on the methodology and the single-season leaders here; tomorrow, we get to the career list.

I have individual game sack data for every quarterback back to 2008. For seasons between 1969 and 2007, I have season sack data and team game sack data, so I was able to derive best-fit estimates for each quarterback in each game. For seasons between 1960 and 1969, I gave each quarterback an approximate number of sacks, giving him the pro-rated portion of sacks allowed by the percentage of pass attempts he threw for the team. [↩]

Ideally, I would iterate the strength of schedule adjustment. That’s for GQBOAT VI, I suppose. [↩]

If I had net fumbles lost on a per-game basis, I would simply include them in the prior step. But I don’t think much changes by simply including them in the aggregate on the season-level. [↩]

In 1960, the final grade gets multiplied by 0.5; in 1961, by 0.6, in 1962 by 0.7, and so on. [↩]

Will this include a small modifier for each season for the AV of his offensive teammates? Presumably, getting a better “ANY/A” and therefore more yard value with “weaker” teammates is more worthy of praise.

Obviously it would have to be small because of the problem of interrelated effects and the reliance on postseason honors for offensive linemen.

Arif

Just realized that the “QB help” work you did may contribute to this.

Chase Stuart

Nah, not yet. That is something that will take a lot of work to do; realistically, it’s probably in GQBOAT VII or later. I think for now, we’ll have to be content with our subjective modifiers based on supporting cast.

Brian Griese’s 2000 season is in the top 200 even with the SOS adjustment dragging him down. I still wonder how differently his career would have turned out without that weird shoulder injury and the accompanying bizarreness.

If you sort by the SOS adjustment, the two highest positive adjustments are Richard Todd and Tommy Kramer in 1982. They did play nearly the same schedule, so it makes sense for them to be close. If you sort for the highest negative adjustment, you get Ken Anderson in 1982. 1982 was of course a strike-shortened season. It could just be by chance that those guys ended up on the extremes, but I’m wondering if the vagaries of that season, since it’s obviously an oddity, are to blame. Did the smaller number of games lead to wider variation in teams’ defensive ANY/A? Was there something else?

Chase Stuart

My guess is that it’s just the small number of games combined with the unbalanced schedule. For example, the ’82 Dolphins had a great pass defense, and Todd played them in 2 of his 9 games. The Oilers pass defense was a joke — 18 TDs, 3 INTS!!! — and Anderson played them twice.

Red

The SOS adjustments are illuminating. Tom Brady has apparently played his entire career against tough schedules, which changes my view of his legacy. I’ve long believed Brady to be somewhat overrated, but now I must reconsider that notion. On the other hand, Kurt Warner faced a comically easy schedule during his GSoT years (and played in a dome) so his best seasons weren’t as great as they seemed at the time. In 2013, I couldn’t shake the feeling that Nick Foles’ terrific season was smoke and mirrors, and his super easy schedule lends some credence to that. I really hope you apply SOS to playoff stats as well…

I am strongly against fully prorated numbers for 1982. Dan Fouts essentially played half a great year, and you’re saying it’s the fourth best season of all time? Sorry, that’s just plain wrong.

Are you going to include pre-1960 seasons in the career rankings? If so, I’m hoping you include adjustments for the 1940’s seasons that had weakened rosters because of WWII.

I don’t understand why your sack data only goes back to 2008 for individual players. Game level QB sack data is readily available on many other websites going to back to the merger, including nfl.com and footballdb.com.

Most underappreciated season ever: John Brodie 1970. Sack rate of 2% in the dead ball era seems borderline impossible.

More proof that RINGZ are overrated: Eli’s best season ranks 120th, Big Ben’s is 193rd, while Plunkett and Flacco don’t even crack the top 200.

Dan Marino only had one truly awesome year, which is surprising given his career totals. I checked out his p-f-r page and was shocked to see his career YPA was only 7.2. Perhaps Marino’s “stat stuffer” rep needs to be re-examined?

Interesting that Peyton’s 2013 is the only top 20 season with negative fumble value, especially since he’s usually great at avoiding fumbles.

Are you going to publish a list of the worst QB seasons of all time? Kim McQuilken will be offended if you don’t.

Malene, CPH

Apropos RINGZ, the 5 Brady seasons in top 14 obviously all came long after last ring. We might laugh about the Jets circus, but apparently facing 2xJets D, Ravens D and Steelers D every year is fairly tough on average.

I don’t think most places have individual QB sack data on the game level going back before 1999. If they do, hey, please send it to me 🙂

Yes, I will include pre-1960 seasons in the career rankings. But nothing has changed for those grades since I ran this two years ago 🙂 And yes, there was an adjustment for the WWII years.

Agreed that Brodie’s 1970 season was outstanding.

I have also noticed that Marino’s ’84 really was an outlier. It is kind of surprising.

I probably am not going to do a worst QB season of all time list this time around. I don’t think too much will have changed in the past couple of years.

Richie

Yep, as a Marino fan, most of these analyses are disappointing to me. Clearly his 84 season was an all-time great one. In 86 he was really good. 85 he was pretty good, and it was really the last time the Dolphins were a really good team. (In 92 they made the AFCG, but it was kind of fluky and as the final score shows, Buffalo was a much better team.)

So he had a 3-year run where he was historically good, and after that he was still very good, but never really close to the dominance from 84-86.

That just got me thinking. If you removed the best 2-year (or maybe 3-year) run from every player’s career, it might be interesting to see how they rate out. There might be a handful of guys who built a large part of their reputations during a small piece of their careers.

Robin

Just thought I would chime in and say that I find this to be a really interesting question!

Jeff

Should Dropbacks be adjusted for era as well if we are looking for the greatest QB of all time, rather than the most valuable. I find it hard to believe that 13 of the 25 greatest statistical seasons since 1960 happened in the past 10 years when truly adjusted for era. If Unitas played now would he really pass less than 50% of the time, and conversely. would Drew Brees have nearly 700 dropbacks in any other era?

Chase Stuart

This is the million dollar question. In theory, I think yes, but in practice, it leads to some strange results. Nobody wants to see a 250-attempt season that was kind of fluky grade out as better than some of Drew Brees’ best seasons. I’ve run some era adjustments for dropbacks before, and it never really passes the eyeball test. But I agree with your main point. It’s something to think about.

Great read, and the “huge caveat” at the end is perfect. Although the aim for these types of lists is to get us closer to the truth, they’re main benefit is that they provide context and are a baseline for further discussion. As you state there are all manner of other factors that go into a QB having a great season, but this list covers the most important ones. Looking forward to the next installment!

Ever thought of splitting home/away SOS adjustments? I used to do that for some scoring categories . Just quickly looking at the numbers Id imagine that Marino and Bradys top 2 would rank even better while Peytons would take a hit.

Richie

Drew Brees has thrown for 5,000+ yards four times! I guess I never really thought about that before.

It kind of reminds me of Sammy Sosa hitting 60+ HR three times, yet not really being considered the best HR hitter of his time.

YW

I’d like to request some background reading on the basis for the coefficients in ANY/A. 20 yards for a TD and -45 yards for an INT seems arbitrary, and even if there is a basis for their use in the raw ANY/A I don’t know if those numbers are approrpiate for a more holistic analysis such as this. Even if those coefficients are not arbitrary they seem to be fairly imperfect as they punish QBs on teams with good short-yardage run games (fewer passing TDs) and QBs on teams that often play from behind necessitating late-game high-variance throws (such as hail marries) that will more often result in INTs

I am aware from Chase’s previous work that ANY/A correlates as well as any basic QB stat with team wins but I wonder if it isn’t possible to devise a formula with different coefficients for this expanded exercise that correlates even better with wins.

Richie

YW, I believe that the book “Hidden Game of Football” calculated the value of an interception to be -45 yards, and a TD to be +10 yards. Later, Chase ran some calculations that adjusted a TD to be worth 20 points. (Actually, he said a TD is worth 19.3 points.) http://www.pro-football-reference.com/blog/?p=633

Very much appreciate all the effort that has obviously gone into these interesting studies. Two questions come to mind (and I apologize if I missed the answers that might have already been provided elsewhere).

One, it is not clear to me why QBs should be penalized for getting sacked? This seems to be primarily dependent on the quality of the offensive line, as indicated by the fact that sacks go down when the line is good, and go up when they are poor. Differently put, QBs are not generally sacked at consistent rates over their careers. Or think of the unfortunate Archie Manning, who had all the talent in the world to have a low sack rate — strong arm, quick release, great running ability — and yet he regularly got hammered most seasons because he played on weak teams with poor offensive lines. Just consider his 79 and 80 seasons. Saints were 8-8 in 79, and Archie was sacked only 17 times. Next year the Saints crashed to 1 and 15, and he was sacked 41 times. And oddly enough, Archie actually had a better statistical passing year in 1980! If you put Archie behind Bradshaw’s Steeler line, or behind Bart Starr’s Packer line, don’t you think he would be in the HOF?

Second question concerns awarding 20 yards for every TD pass. Doesn’t this risk skewing the results rather dramatically in favor of contemporary QBs who pile up their stats with a very high and seemingly growing % of TD passes under 5 yards? Peyton Manning and Brees come easily to mind (~40% under 10 yards for both over their careers). Prior to the rule changes that opened the passing floodgates, runners were generally called upon to punch in from short yardage, and passing TDs more often came from mid range to longer distance. Perhaps this would be too complicated, but why not just add the actual distance for each TD pass, or pro-rate by short/medium/long. Just a thought.

Chase Stuart

Joseph,

I, and several others, have done countless studies showing that sack rate is heavily tied to the quarterback. That doesn’t mean it’s not tied to the OL, but sacks are no more on the OL than say, passing touchdowns are on the receivers.

Re: point two, the thing you are forgetting about is the unsuccessful pass. Y/A and cmp% are significantly depressed near the goal line. The average ANY/A gained on pass plays inside the 5 last year was only about 7.4. And that number is a bit skewed because the best QBs are more likely to be the ones passing inside the 5. Manning, for example, averaged 12.1 ANY/A inside the 5. And since he was responsible for nearly 6% of all such plays, the league average without Manning was only 7.1. That’s not far enough from the 5.9 league average to concern me.

Joseph Bryant

Stuart,

Could you briefly explain why the sack is primarily on the QB? I appreciate that qualities like speed of reading defenses, quickness of set-up and throwing release, and basic mobility are crucial here, but when great QBs discuss the challenges of the position, they almost always come back to “time to throw.” And that’s why, historically, when coaches and GMs talk about improving their passing game, they usually say things like “we have to get Archie/Jim Plunkett/x… better protection”; or we have to “give Sonny a running game to keep defenses honest”, etc.

I mentioned Archie Manning as a case in point — i.e., where talent is there and consistent, yet sacks fluctuate significantly — and looking at a couple of others just now, I hope you can understand why some might think that the variations in sacks do not appear to track QB ability so much as quality of team, the protection granted, and whether a poor or great ground game placed QBs in better or worse passing situations.

It’s hard for me to see how Aikman or Anderson either improved or declined in their set-up speed, release, or defensive reads over this time — their talents had “settled”, I would say. Of course injuries play a key role, but that would be devilish to quantify.

One might also reflect on game to game variability. How many times is it said that a QB had a bad passing game because he was harried and rushed hard all day, yet follows up with great passing next week when his offensive line stops the defenders from crashing in? Even at the college and high school levels, QB passing effectiveness seems to depend crucially on whether the QB has enough time to set and throw.

So, yes, I do agree that QB skills make a difference, but few QBs will pass equally well against a great defensive rush as against a weak one, and that QBs with a great line and strong ground games have a passing advantage over those without.

On the passing TD question, I was thinking more about the contrast between the so-called “Dead Ball era”, when “pounding it in” with the running backs seemed more common near the goal line, whereas today I think the 1 to 5 yard TD pass is much more in play. The average length of TD pass has dropped significantly over this time, hasn’t it, or am I mistaken?

Ty

5 of the top 25 seasons had Randy Moss on the team (I didn’t count 2010 Tom Brady).

Brett Favre’s best season being 2009 looks like it has a lot of merit (it is also somewhat surprising that his highest rank is at 61). Brett Favre was a great player, though quite overrated. He was arguably the best QB in the mid-90s, but there were circumstances that made that happen during that time (Steve Young kept getting hurt, Montana retired, Marino, Kelly and Elway were declining, then Elway revitalized his career, Moon was old). Even with all that, Favre still put up some great seasons in Green Bay, before Holmgren left.

When Daunte Culpepper was on his game, he was unbelievable. I’m not too surprised that he is on the top 25. He never really learned how to play the QB position, and he got hurt. Randy Moss definitely helped Culpepper a lot, but I think Culpepper had a lot of ability in him as well.

Manning and Brady are amazing, Young was, too; it is a shame that Young kept getting hurt in the mid-90s… and how his career ended. He would definitely rank higher (career-wise), otherwise.

BringBackTheFlex

“I’m including rushing touchdowns, because our goal is to measure quarterbacks as players. ”

This makes no sense to me. You aren’t evaluating them as players. Are you taking into account any tackles they might have made? Fumble recoveries? Receptions? Those are all things offensive PLAYERS do as well.

A running QB is a QB who doesn’t know how to read defenses. It should be a penalty in your calculations for rushing yards – a QB isn’t doing his job if he’s running all over the field.

Ty

QBs aren’t in a position to tackle, and if they are, then it is because they made a mistake (or their teammates did, so they shouldn’t be blamed for that), QBs obviously don’t catch passes, and fumble recovery rates are pretty fluky (I would understand if you had said actual fumbles).

QBs are able to do 2 things when in a passing situation, pass and run. As long as you are efficient (with passing weighted more heavily), then what is the problem? Obviously you would want your QB to be great at passing, and anything else is a bonus. I would assume that this metric weights passing QBs much more heavily than running QBs, which is why statues like Manning, Brady, Marino, and Fouts are at the top. If a running QB can’t throw, his stats will reflect that, and he won’t usually be playing for long. If you want a QB that is good at both, there is Steve Young and Aaron Rodgers.

BringBackTheFlex

The fact that Michael Vick, Sonny Jurgensen, Ron Jaworski, Roman Gabriel, Nick Foles, and Randall Cunningham are on this list proves how invalid it actually is. Whatever you are using to calculate this it woefully inadequate.

You can’t rank a quarterback’s season just based on his regular season statistics, alone. You have to include playoffs, and most importantly, if they won the Super Bowl. That is why Brady’s 2007 season is practically null & void, along with Rodgers’ 2011 season and Manning’s 2004 disappointment.

In short, the greatest season by a quarterback is Joe Montana’s 1989 season.

That year, Joe set the all-time passing record with a rating of 112.4, then played even better than that in the playoffs. He threw 11 touchdowns and no interceptions in those 3 playoff games. His cumulative quarterback rating in the post season was over 130.

Who can possibly match that?

Steve Young’s 1994 season comes the closest. He edged Joe’s regular season mark of 112.4 by finishing with a 112.8 rating. In the playoffs Steve threw 9 TDs and no interceptions and his cumulative rating was just a little over 100. Still phenomenally good, but not as good as Joe.

Joe and Steve have one feat that no one else in history: setting the passing record and winning the Super Bowl that very same year.

When Manning broke Steve’s mark in 2004, Peyton fell well-short in the playoffs with a pathetic loss to the Patriots. Manning took three 1000+ yard receivers and a 1200+ yard running back and amassed a total of 3 points. Sad.

Rodgers in 2011 broke Manning’s mark, put together a 15-1 regular season record, then lost his first playoff game to a mediocre 9-7 Giants.

Greatness is about winning the big game. Sure, maybe there can be some luck involved, but a great quarterback steps up at the most important time and prevails. That is why Joe Montana is number one and someone like Peyton Manning or Steve Young fall outside the top-10.

As many touchdowns and regular season wins that Manning has amassed, anybody knows he would happily trade his accomplishments for 2 Super Bowl rings. Instead, Manning is 11-12 in the playoffs, threw a game-deciding interception against the Saints in the 2009 Super Bowl, and lost 43-8 just this last year.

Ever notice how Manning never smiles? It’s because he is extremely unsatisfied. He knows he has failed and underachieved.

Lastly, if you rank quarterbacks based on true “greatness,” it would look like this:

I don’t know you, or anyone, can compensate for the following.
Bob Greise, he of like 8 attempts in Super Bowl VIII, called his own plays.* When it was 2nd and goal from the two he could either throw a TD or take the ball out of his hands and give it to Czonka.

* According to Terry Bradshaw, it does not mean a QB was like a coach. There were like six options; long pass left, medium pass left, long pass right, medium pass right, run left, and run right. Probably slightly more complicated than that with 6 holes to run at and two RBs, but still.