The Bills on Fourth Down: Bad Strategy and Yet Not Out of the Ordinary

Week in and week out, when the Bot tweetsaway its disagreements with coaches’ illogical fourth-down decisions, it’s Buffalo fans who chirp back most reliably, complaining about their coach, Doug Marrone.

But recently the Bot, which exists only on the Internet, had some effect in the real world of the N.F.L. Last Friday, two Buffalo-based reporters grilled Marrone about his fourth-down strategy while referring to the Bot. (Marrone’s response, transcribed in this tweet, is an instant classic in coach-speak.) On Monday, the Bot was even quoted by The Buffalo News.

On Tuesday, a reporter for Buffalo’s WGR 550 radio station posted an interview with Bills General Manager Doug Whaley. When asked about the use of analytics in strategizing on fourth down, Whaley said it was a “layer of information,” but he also said he would have to do more research and added that “a lot of it goes with gut feeling.”

What analytically minded fans want to know is simple: Just how bad are the Bills on fourth down?

Today we’re making good on a promise the Bot made last week: a post dedicated to the fourth downs of the 2014 Buffalo Bills. The short version: Marrone has done his team few favors on fourth down, but he may not be as bad as fans think.

Let’s start with the red meat: The Bot has disagreed with only one team more than it has with the Bills: the 0-10 Oakland Raiders. Marrone’s poor calls on fourth down probably played a role in at least two Bills losses.

The Bills’ worst call of the season came in Week 10 against Kansas City. Down by 4 points in the fourth quarter with seven minutes remaining, the Bills punted on a fourth-and-one on their 41. We estimate that decision dropped their chances of winning to about one in five, from one in four. They lost the game, 17-13.

Image

Bills quarterback Kyle Orton being sacked last Thursday in a loss that led to questions about Buffalo's fourth-down strategy.CreditAlan Diaz/Associated Press

After that, a handful of calls stand out as second worst, including another call in the Week 10 loss against the Chiefs and another in last week’s loss against the Dolphins, which prompted the local press’s questions.

There’s no denying the conservatism that has frustrated Bills fans so much. Nine times this year, the Bills have faced fourth-and-one, and we almost always recommend going for it in that situation regardless of field position. TheBillskickedorpuntedonallofthem. Leaguewide, the rate for going for it on fourth-and-one is about 43 percent.

And when the Bills have gone for it, it has been a matter of necessity rather than strategy. All eight attempts were when the Bills were losing in the fourth quarter, and six happened with the Bills trailing by 10 points or more. Put another way, if the Bills are going for it, it’s because they’re probably going to lose.

But don’t get your pitchforks out just yet.

One reason the Bills have had so many failures on fourth down is because they have so many failures on third down. They have one of the worst offenses in the league, ranked 25th, which has translated into more fourth downs than any other team. Simply put, the Bot gets more opportunities to disagree with Marrone than any other coach.

A more meaningful measure, the share of fourth-down decisions the Bot disagrees with, is about 16 percent for the Bills — fifth highest in the league. But that ranking is noisy, largely reflecting the particular fourth-down situations each team has had. Most coaches make similar decisions on fourth down. Cade Massey, a professor at the University of Pennsylvania who has written frequently about analytics and the N.F.L., has argued that fourth-down strategy has not changed appreciably in 15 years.

Perhaps the best comparison is slightly more technical: Every time a team makes a decision that the Bot disagrees with, we believe the team is forfeiting a small fraction of wins. If we divide the total forfeited win percentages by the number of fourth downs, we’ll get a measure that we’ll call win percentage lost per fourth down. This measure puts the Bills in the middle of the pack.

These more nuanced measures don’t exonerate Marrone. But they do provide him some cover: He’s just as bad as everybody else.

The Upshot provides news, analysis and graphics about politics, policy and everyday life. Follow us on Facebook and Twitter. Sign up for our weekly newsletter here.

NYT 4th Down Bot is a news application that Brian Burke, Shan Carter, Jennifer Daniel, Tom Giratikanon and Kevin Quealy created in 2013. Jeremy Bowers, David Leonhardt and Ken Schwencke contributed development and editing in 2014.