If you haven't done so yet, you'd better read Approximate Value I. (I know you're thinking, "yeah, yeah, whatever." I do the same thing. But I mean it. Go read the old post.)

I left off last time with this bunch of questions:

What metric do I use to determine offensive points at the team level?

What fraction of points should go to the line?

What is the pass/run split?

On the passing side, what is the throw/catch split?

We need to figure a way to give some of those offensive line points to fullbacks and tight ends, many of whose jobs include a lot of blocking.

Hopefully, I'll be able to answer all these today, and run through an example or two. Before I do, I'd like to make a few comments about the method and about my style of doing these sorts of things:

I'll just state upfront that this is a case where I'm not necessarily opposed to tweaking the metric until it gives us results we're happy with, instead of picking a theoretical basis and forcing ourselves to stick with it. As I quoted Bill James in the last post: "These approximations are not intended to tell you anything at all about the player that you do not already know." They're not supposed to teach us new things; they're merely supposed to codify the things we already know, so it's OK to cook the books a little bit until they do tell us what we already know. The problem here is that none of us really knows how to compare Tarik Glenn's 2006 to Gary Clark's 1991. And to the extent that we do "know," we all "know" different things. The point is: while I do think we need some sort of theory to get us started in certain areas, I won't be too apologetic about making some arbitrary changes if a strict application of the theory leads us to "wrong" answers.

The reason I write a blog instead of writing books --- well, one of them --- is because I'm not the kind of guy who thinks everything through completely before running the numbers and writing it up. I'm the kind of guy who comes up with a vague idea, gets excited about it, and tries to get to (and share) some preliminary results as soon as possible, occasionally making a few admittedly half-hearted choices along the way. There are some calculations below that I know are wrong. But I think they're close enough that they won't do any real damage to the conclusions. I'll go back and re-examine them later.

A reminder that this method is for the purpose of establishing approximate value. From yesterday's post, this is James: "The approximations are intended only to distinguish as quickly and reliably as possible between large contributions, very large contributions, gigantic contributions, medium-sized contributions, small, smaller, and negligible contributions." I bolded "as possible" to remind us that there's only so much we're going to be able to do. We're not going to be able to give Hines Ward the credit he allegedly deserves for blocking. We're not going to be able to distinguish between a tight end who posted 20/110/1 because he stunk and another who posted 20/110/1 because he was basically a third tackle. If a team has a very good right guard and a very bad left guard, we're not going to be able to distinguish them unless the good one makes the pro bowl. Is this unfortunate? Yes. Does it make this whole exercise useless? That's for you to decide. But if you think it does, there's not much point in reading on. So if you're still here, remember: I will liberally and sometimes arbitrarily play the "we're keeping it simple" card. Daniel Graham and Steve Tasker will simply have to accept my apologies.

At least for now, I'm only going to attempt to rate players from 1978--2006 (I'm still filling in a few stats from '07). First, lots of things changed with the rule changes that preceded the 1978 season. They have continued to change since then, of course, but I'm fairly comfortable assuming that 78--06 constitutes an era. Also, games started will be a key part of the metric, and GS data gets a little sketchy in my database right around 1978.

OK, here goes.

The metric for measuring team offense

At least for now, I'm going to go with offensive points per drive, which is a stat I should have thought to compute a long time ago, but didn't. Unless I'm missing something, every drive ends with either a rushing touchdown, a passing touchdown, a field goal attempt, a punt, a turnover, a failed fourth down conversion attempt, or the end of a half. On the team level, the new database has touchdowns, field goals, punts, and turnovers. The other two I don't have data for. A half-ending possession that doesn't result in one of the four main outcomes probably wasn't much of a possession anyway, so I don't mind not counting those. I wish I had turnover-on-downs data, but given the relative rarity of these kind of possessions, I'm not too uncomfortable excluding them. So we'll just call it offensive points per estimated drive:

Now divide the team's OPPED by the league OPPED, multiply by 100, and that's how many points a team's offense has to distribute. So an average offensive team will necessarily have 100 points. The 1982 Chargers, 2000 Rams, and 1984 Dolphins all have about 170. The 1992 Seahawks have about 44 and the 2006 Raiders have 47.

Dividing up the points

Here's where it's going to get a little controversial.

In the comments to yesterday's post, Neil outlined an idea that is probably better than mine:

I tried to look at salaries to see how GMs value each position. For instance, QBs made 17% of the salary cap # devoted to offensive players from 2000-07, so I allocated 17% of “Offensive Wins” to the QB position; RBs were paid 12% so they got 12% of wins, etc

If I had that data, and if I'd thought of it, that's probably what I'd use. But I don't and I didn't. I did, however, realize that last April I ran a study that can provide a similar theoretical basis for allocating the points. I calculated the percentage of draft value chart "points" that NFL teams have historically used on each position. Here is the chart:

Assuming teams actually believe the pick value chart, and assuming they know what they're doing, then it might make sense to make it a goal to set up our system so that, in the long run, the number of total points awarded to players at each position matches the above distribution. We're going to start with that idea and work from there.

I wish I had a RB/FB breakdown, but I don't. I'm just going to make one up: 80/20. So we have this:

qb 7.4
rb 10.0
fb 2.5
wr 11.7
te 3.9
ol 15.5

Looking at just QB, RB, WR, TE, and OL, it adds to 51. Our task is essentially to divide that 51 into four buckets: (1) blocking, (2) running, (3) pass throwing, (4) pass catching. Let's make the following assumptions, which I do know are not exactly right:

an OL's job is 100% blocking

an average QB's job is 95% pass-throwing and 5% running

an average RB's job is 70% running and 30% pass-catching

a WR's job is 100% pass-catching (sorry Hines)

an average TE's job is 70% pass-catching and 30% blocking (yes, I know this varies a lot from TE to TE, more on that later)

an average fullback's job is 10% running, 20% catching, and 70% blocking.

So, for blocking we have 100% of 15.5, 30% of 3.9, and 70% of 2.5.

For running we have 5% of 7.4, 70% of 10.0, and 10% of 2.5.

For pass-throwing we have 95% of 7.4.

For pass-catching we have 100% of 11.7, 30% of 10.0, 20% of 2.5, and 70% of 3.9.

It all adds up to:

Blocking: 18.42
Running: 7.62
Throwing: 7.03
Catching: 17.93

So the proportions are:

Blocking: 36.1%
Running: 14.9%
Throwing: 13.8%
Catching: 35.2%

I really, really like the above method. The problem is that it just doesn't seem to work. Offensive linemen turn out to be undervalued by just about anyone's standards. In a preliminary version of this method using these percentages, Jon Ogden came in right between Brian Sipe and Ronnie Harmon in terms of total career approximate value. So let's tweak up the blocking percentage just a bit and keep the remaining relationships fixed. Here's one that I think works pretty well.

Every OL, FB, and TE gets points proportional to his percentage of the team's total pre-points.

Now we move on to the skill guys. The percentages above dictate that an average team should have 23.5% of its remaining points devoted to running, and 76.5% devoted to throwing and catching. [Here come a couple of calculations that I think are basically in the ballpark, but could use some improvement...] For each team, we take its ratio of rushing yards to total yards and divide it by the league average ratio. This gives a number like 1.15 for a run-heavy team or .93 for a team with a slight tendency toward the pass. Then we multiply that number by 23.5 to see what percentage of that team's non-blocking points will go to runners. Now we know how many points to give the runners, and how many points to give the passers/receivers.

Remember from last time, I declared that the passer/receiver split should stay constant from team to team. So passers get 11.8/(11.8+29.9) = 28.3% of the passing game points and receivers get the other 71.7%. Remember, there are a lot more people that have to split the receiving points.

The individual runners get points proportional to their share of the team's rushing yards. We're keeping it simple here. Likewise with the individual passers (passing yards) and receivers (receiving yards).

That's it. Now let's look at some results and see if it's believable. Here are the top two and bottom two offensive teams of the era, along with a couple of average-ish teams:

You may agree with these lists in spots and strongly disagree in others. That's OK. But it's worth pausing for a minute to remind ourselves just how impossible the task we've undertaken is. All we're using is passing yards, rushing yards, receiving yards, games, games started, position played, and pro bowl status. And we're only using the pro bowls when we really have to (for linemen). Given that that's all we have to work with, I happen to think the method does a pretty good job of getting close to an assessment of approximate value that most people would generally agree with.

Kinds of players who are over-valued:

On the career list, quarterbacks who played for a long time. That's unavoidable, given that QB is the most important position on the field, quarterbacks tend to have long careers, and this is a career total metric. But it does raise some questions. Namely, if a team has a terrible offense and/or a terribly inefficient passing game, why are we giving the quarterback any points at all? Stan Gelbaugh got three points for compiling dreadful stats on the worst offensive team of the last 30 years. Why aren't we giving him negative points? I'm not sure I have a good answer for that, except that I don't see how to do it in a way that would keep the method for QBs consistent with the rest of the positions and I'm not quite ready to have totally different sets of rules for different positions. And I definitely don't want to go the route of making this a metric where an average player is zero and a below average player has negative points.

Post-James Wilder era running backs, compared to pre-James Wilder era backs. Wilder was the first true workhorse back. Before him, no coach ever attempted to let a single RB get 80--90% (or more) of the RB work on a team. I'm not completely comfortable with Shaun Alexander having three seasons as good as Earl Campbell's best one, simply because Tim Wilson was getting some touches.

I was hoping to avoid this, but I may have to re-work the running game / passing game percentages, creating a slightly different set for pre-1985ish versus post-1985ish.

Bad offensive tackles. Of course, they won't generally be on good teams and/or won't stick around long anyway. So I don't see this as a major issue.

Offensive linemen who make the pro bowl based on reputation long after their best years are behind them.

Offensive linemen (especially tackles) who were the only pro bowl lineman on a very good offensive team. Orlando Pace and Tarik Glenn have probably been overcredited a little.

Guys whose nominal position is fullback, but who don't do as much blocking as a real fullback. Yes, Mike Alstott, I'm talking about you. Probably Larry Centers too. They get as big a share of the blocking credit as Dan Kreider and Lorenzo Neal do, but they don't do near as much actual blocking.

Kinds of players who are under-valued:

Good blocking tight ends and wide receivers.

Linemen (especially guards and tackles) who were good, but not good enough to make the pro bowl.

Next, we move to the defensive side of the ball. That may happen this week, or it may be next week.

This entry was posted on Wednesday, January 16th, 2008 at 4:47 pm and is filed under Approximate Value, General, Statgeekery.
You can follow any responses to this entry through the RSS 2.0 feed.
Both comments and pings are currently closed.

One thing that catches my eye is Vinny Testaverde being nestled snugly between Peyton Manning and Marshall Faulk. That suggests to me that the formula overvalues compilers. Testaverde obviously isn't even half the player that Manning or Faulk is, but he's way up there because he hung around forever.

Obviously, this is really cool and I'm not complaining about it, but I think it's pretty clear that there needs to be some sort of efficiency stat involved, because Manning and Testaverde are getting roughly equal credit for their total yardage, even though Manning's yardage was obviously produced more efficiently.

Wow. Great work, Doug. Like you said, it has known flaws and probably could use some tweaking over time, but this is really a remarkable effort -- and something I don't think has ever been attempted before. Fantastic stuff!

Oh, and I meant to add further that it's not just true for quarterbacks, as you said, but also for every other position. Bruce Matthews, Emmitt Smith, and other players who lasted a really long time have fairly charitable ratings relative to guys with shorter but more stellar careers, like Orlando Pace and Barry Sanders.

The best way to modify it would probably be to further widen the point spread between great offenses and bad ones, or to further widen the point spread between great players and poor ones for any individual season.

Yaguar, there's no question that compilers rank too high on the career list. I think I'd prefer to deal with that not by changing the metric itself, but simply by being aware of that the leaders in total career AV aren't necessarily the best players.

Isaac Bruce is currently 3rd in career receiving yards and no one anywhere believes he's the third best receiver in history. But that doesn't mean there is anything necessarily wrong with receiving yards as a measure of receiving effectiveness. You just have remember to be a little careful, depending on what you want to use it for.

One way to generate an all-time list that feels a little better is to rank players by "Total AV of their N best seasons," where N is whatever you like. Here's an N=10 list:

Perhaps one problem with the draft values is that the length of a career is vastly different depending on position, and injuries are much more common for some positions. Running backs take a lot of punishment and tail off while relatively young but QB's may be productive until they're 40. So I'd expect RBs to be overvalued in the draft chart simply because they get replaced so often. QB's last longer, so even though they're highly valued, many teams may already HAVE their star QB.

Then again, for a career metric, perhaps it makes sense to factor in the average length of a career for a given position. A RB is probably going to have a lot less time to compile points than a QB.

The best single season list has exactly one WR on it, and that's from the strike shortened season. Neither of Rice's record setting seasons (the 1848 yard season or the 22 TD season) are on that list.

- It looks like you set the non-OL run/pass split by taking the proportion of total yards that are rushing, compared to other teams that same year. I think it would be better, and fairer to non-workhorse RBs in rushing-heavy years, if you just took the proportion and did not compare to other teams in the same year. In other words, the share of non-OL points that goes to rushers would just be the team's rushing yards divided by total yards times some constant, with the constant set so that this averages .235 over the whole period.

- Could you set up the blocking points to separate pass blocking and run blocking? For pass blocking, tackles are very important while fullbacks are not so important. For running, the importance is more evenly balanced among the OL positions, TEs, and FBs.

I'm not sure if this is even an issue - but there is exactly 1 WR on the all-time best seasons list. As I think about the value of a real game-breaking WR, I wonder if there is some skew against them having huge seasons here.

I haven't spent enough time looking at the formulas to really be certain, but I thought it might be worth examining.

If you could draft any player in history for your team, as a rookie, and you knew exactly what his career would be like, is it better to pick the player who will last longer or the one with a higher peak?

Testaverde or Manning?
Terrell Davis or Earnest Byner?

It sure would be nice to be able to take Testaverde and know you'd have an average-to-above-average QB for 15 years. (Of course, Manning may end up being a great QB for 15 years.)

...which I guess comes back to the question of what is better, to be a team that wins 1 Super Bowl, sandwiched by a bunch of bad-mediocre seasons ---- or a team that never wins a Super Bowl, but is a consistent contender?

Have you considered using your outputs (player value) to double check and adjust your assumptions (draft value chart and it's derivatives, the positional split, etc)?

Eventually you should reach a fixed point and have produced an accurate draft value chart for your system (ie GMs with the #1 pick get X value per this system). Or else your system of valuation has no stable solution.

The other problem with using draft value is that some positions are easier to draft. 1st round running backs, in particular, are generally quite successful, usually as rookies, so a lot of running backs get drafted early. Quarterbacks - (other than overall #1s, and there have been a lot of those) much more guessing & waiting. (cf Starr, Unitas & your Quinn vs Anderson posts)

You have the right idea - the N=10 list looks remarkably credible - but that's not an ideal means to accomplish the intended goal. Someone's 11th best season should obviously still produce some value. (Especially silly if you're talking about, say, Jerry Rice's 11th best season, which was still awesome. A system where Andrew Walter's worst season counts for 2 points and a good Jerry Rice season counts for 0 is obviously silly.) The modification I would make, if I had your awesome database and spreadsheet skills, would be to increase the value of great seasons relative to poor seasons. In other words, Marshall Faulk's 1999 could count for 30 points instead of 23, while you keep Aaron Brooks's awful 2006 at 3 points.

That would produce a better "greatest of all time" list because it would tone down the advantages for compilers. But as you said, producing a "greatest of all time" list might not be your goal. You're calculating "approximate value."

Nonetheless, I think everyone would say that 15 years of Joe Montana are at least twice as "valuable" as 20 years of Testaverde. And yet, Montana is only barely ahead in career approximate value.

It's all about what Richie was talking about in posts 8 and 9 - would you rather have a greatest of all time-level guy for 15 years, or a slightly-above-average guy for 20? I think we all know the answer, and it's not even close.

As much as you hate to do it, Doug, I think you should give more consideration to the idea of a replacement-level or average season being worth 0 points. That way if a player is merely compiling, he will only be compiling zeroes. And if he's bad enough to be scoring negative, shouldn't that be reflected as well?

The list is, at least intuitively, a good start. I don't understand what the problem is with sorting by all-time accumulated value (emphasis on long-term achievement) and separately by peak performance (over a shorter period).

I also think your approach for quarterbacks, linemen, and running backs is somehow fundamentally skewed. For instance, how the hell is Barry Sanders' 2000 yard season not on the list?

I understand that you are taking on a huge, "impossible" task, but to do this well, you have to incorporate some chart statistics, and valuable information isn't available on the back of a sports card.

For instance, for QBs, I'd love to see how the following would change the results:

If one wants to make a larger gap between stellar seasons and mediocre ones, one could just raise each season number to some power. ie. if you square all values, then a few stellar seasons would greatly outweigh a long mediocre career. Probably too much. But one could see if a certain power comes up with more "right" answers.

Overall the idea looks great, and makes a lot of sense. The problem with the James's approx value was that he didn't tie it into team performance.

Why do you reward passing so much more than running? Isn't a yard a yard, no matter whether you run for it or throw for it? If a team gains 2,000 rushing yards and 2,000 passing yards, isn't its offense 50% rushing? Maybe I haven't looked at your numbers enough.

But here's a question: how can Steve Young have greater value than Joe Montana? Young could not take the starting job away from Montana until Joe was 36 (1992) and coming off a completely missed season (1991). Young was obviously a great player but subjectively Montana was better. Have you ever done a Montana vs. Young comparison? If head to head Montana is better, shouldn't his AV reflect that (accepting it's approximate and all)?

First of all, I noticed the same thing as Jacob: this system does not think much of wide receivers. Maybe that's right -- these are guys who touch the ball five times a game -- but it's certainly counter-intuitive to see Art Monk and Cris Carter ranked the same as Phil Simms.

And not to create extra work, but I think you need some kind of efficiency rating. That would help get Barry Sanders and his terrific rushing average to a level that seems more appropriate, and it would punish workhorses who are the football equivalent of inning-eaters. I'd also want to see a David Garrard 2007-type ahead of a Jon Kitna-type who gets a ton of yards AND a ton of interceptions.

Richie - your points may be true in general. But in '89-90, Montana was a Pro Bowl and All-pro selection. The team went 28-4, won one SB, and lost to the eventual winner the other year. '89 might have been Montana's best season; he slipped a bit in'90. Young was a great player but Montana was better and Young only beat Joe out when age and injuries caught up to him.

I once did a study where I calculated the normalized passer rating of every qb-season: his rating in a given year/divided by the league rating for that year. A normalized rating of 1.0 = an average qb that year; over 1 is above average, below 1 is below. Montana's average normalized rating for his career was 1.24 (#4 all time), Young was 1.23 (#5) so it's very close between them for career value, more so than perhaps I thought when I wrote my previous post. #1 was Graham, then Van Brocklin and Jurgensen. This method punishes qbs who played well into their decline phase.