Wednesday, February 04, 2009

My 2008 accuracy analysis is up on the Fifth Down! Check it out, Ranking the Rankings in Fantasy Football, 2008 and let me know what you think. Does my system make sense? Will you be thinking about this at all when you decide which rankings to look at next summer? What else would you like me to do with this?

If you're wondering which sites I looked at, here's a full list of all 23 sites in alphabetical order:

ADP (via FF Calculator)

Ask the Commish

CBS Sports

ESPN

Fantasy Hulks

Football Outsiders

FantasyFootball.com

Fantasy Football Nerd

Fantasy Football Sherpa

Fantasy Football Starters

Fantasy Football Trader

Fantasy Football Xtreme

FFToday

FFToolbox

Fantazzle Fantasy Sports Games

FootballDocs

FOX Sports

KFFL

Robocoach

Rotoworld

Sports Illustrated

SportsBuff

Yahoo! Fantasy Sports

My plans for the next few months include re-running the analysis with even more sites - I've already heard from a few sites that would like to see how they stack up - and I'd also like to take a look at this from a more historical perspective as well. Consider this my solicitation to fantasy football experts for their historical position rankings. The Wayback Machine can only be of so much help so if you happen to be have your rankings by position on hand for the past several years, send them my way; I'd love to run this same analysis on them.

In case you were wondering about the difference between how all of these sites actually did, you can see the battle was pretty tight based on the numbers for the Top 10 sites below. If you take the square root of the average number listed below, you would get the average amount of ranking spots "off" that each site was per player across all positions. I'm not showing that number here since I think knowing how "off" a site was per player is really most valuable when looking at specific positions, so you'll see that number in the details for the position rankings below. As you can read in my Fifth Down post, I ranked sites based on the sum of squared differences between rankings as averaged by the number of rankings for that position; the square root number is just being shown since I think it's easier for most of us to wrap our minds around than an obscure average number.

Top 10 sites across all positions:

SITE

AVERAGE

1

Fantazzle

55.025

2

Yahoo

56.793

3

Fantasy Trader

57.033

4

ADP

58.293

5

FFToday

60.843

6

FantasyFootball.com

61.475

7

FOX Sports

63.205

8

ESPN

63.253

9

Ask the Commish

63.802

10

Rotoworld

64.815

Top 10 sites for QBs:

SITE

SQ ROOT

1

FFStarters

9.249

2

CBS Sportsline

9.309

3

Football Outsiders

9.463

4

ESPN

9.558

5

Robocoach

9.674

6

FFXtreme

9.742

7

FFToolbox

9.750

8

Fantasy Hulks

9.805

9

FOX Sports

9.887

10

Yahoo

9.907

Top 10 sites for RBs:

SITE

SQ ROOT

1

KFFL

5.638

2

Fantazzle

6.477

3

FantasyFootball.com

6.876

4

FFHulks

6.930

5

Yahoo

7.063

6

SI

7.123

7

FOX Sports

7.208

8

Fantasy Trader

7.293

9

FFDocs

7.334

10

FFToday

7.353

Top 10 sites for WRs:

SITE

SQ ROOT

1

Fantasy Trader

6.309

2

Yahoo

6.327

3

FFNerd

6.374

4

Fantazzle

6.414

5

FantasyFootball.com

6.421

6

FOX Sports

6.445

6

Rotoworld

6.445

8

FFToday

6.446

9

FFXtreme

6.470

10

Ask the Commish

6.501

Top 10 sites for TEs:

SITE

SQ ROOT

1

FantasyTrader

5.007

2

CBS Sportsline

5.018

3

ADP

5.249

4

FFStarters

5.555

5

Fantazzle

5.852

6

Yahoo

6.264

7

FFToday

6.655

8

Ask the Commish

7.519

9

FantasyFootball.com

7.595

10

Football Outsiders

7.736

A few things to note - you can see that on average the experts were far more "off" on their QB rankings than other positions. I attribute much of this to the Tom Brady effect. I left him in the rankings (and all other injured players) so to some extent it skews how off a ranking was on QBs, but it should have penalized all sites in roughly the same way.

I also think it's fascinating how close the WR predictions were - sites #4 through #9 are separated by less than 1 ranking spot. Very, very close.

Anyway, I could go on at great length here about all of the interesting things I found in looking at the details (like Laurence Maroney's difference in rankings single-handedly wrecked one site's RB rankings) but I think I'll pause and maybe give you all a breather from my rankings analysis talk.

10 comments:

Great analysis! I look forward to even more in the near future. I'm amazed at the consistency of FF Trader; I had not heard of them before this post, but they made the top ten lists for everyone but QBs, placed #1 in two lists, and finished #3 overall. That's a pretty strong record.

Thanks Sara! We, at Fantazzle Fantasy Sports Games, appreciate all the hard work that went into this. I think this will be incredibly useful for the years to come and we hope to prove we weren't a fluke.

Thanks bigboid. We're very happy with 3rd place among this competition. Apparently, don't ask us about QB's though b/c we don't know squat (23rd)! I'm anxious to see librarian's follow up to everyone's ranking in terms of longevity in the coming years. That'll begin to tell a clearer story in my opinion.

About Me

So why Fantasy Football Librarian? I have been playing fantasy football now for 12 years. I'm also a librarian at a museum in New York City (though my family and I now live in Knoxville, Tennessee) and I love to research stuff. What could be better than combining my love for football and research? So in September 2006, I launched FFLibrarian to try and create one place for fantasy football fans to find the info they need to succeed in their leagues.
From 2007-2013, I conducted an accuracy analysis of the experts' rankings with support from the Fantasy Sports Trade Association, and have written columns for The New York Times' Fifth Down blog and the Bruno Boys.
Please feel free to contact me if you have any questions or suggestions. Here's to the 2016 season!