Fantasy Movie League observations from an obsessed nerd

October 26, 2015October 26, 2015

Data Deep Dive: Fall Season ShowBuzzDaily Accuracy

Traditionally I’ve relied on ProBoxOffice.com forecasts in my analysis but several readers and regular forum posters have pointed out the value of ShowBuzzDaily as well. Two weeks ago I looked at ProBoxOffice.com accuracy over the first six weeks of the Fall season so for an apples to apples comparison, I’ll give ShowBuzzDaily the same treatment here.

Why should you care?

Professional forecasts are helpful when compiling lineup possibilities each week of the Fantasy Movie League season. Understanding the limitations of those forecasts, as evidenced by “Jem and the Holograms” this past week, is important.

Where’d I get my data?

ShowBuzzDaily forecasts for the first six weeks of the Fall FML season and weekend actuals from BoxOfficeMojo (who will get a similar analysis of their forecasts next week).

What were my findings?

The easiest way to look at this is to simply compare the two forecasts vs actuals graphs:

The first thing that jumps out is that ShowBuzzDaily has a much tighter bounds on new films in this sample, from -1.26% to -9.73%. Their hit ratio on new films is slightly less than that of ProBoxOffice.com, but when they hit the bounds are much smaller.

For returning films, the patterns between the two sites are pretty similar, with ShowBuzzDaily not doing quite as well with films in its 3rd week of release with only 58.3% of films in a decent range, but again we see that their ranges are slightly smaller.

As I was playing around with this data, though, I made a nice discovery: ShowBuzzDaily and ProBoxOffice.com were often oppositely incorrect about individual films. In other words, ShowBuzzDaily was commonly high on its forecast for a film that ProBoxOffice.com was low on and vice versa. If you average the two, you get the following:

While the new releases don’t get much better with this approach, the other groupings do, making the bounds closer to +/- 10%. This is why, starting last week for forecasted films, I began to update the FML Comparator with ShowBuzzDaily numbers on new releases but an average between ShowBuzzDaily and ProBoxOffice.com for returning films and changed my variance simulator to run +/- 10%. These numbers reinforce the idea that the professional forecasters are quite good and fairly predictable when it comes to returning films. The challenge, really, is those truly outrageous new films.

Next week I’ll run a similar analysis on BoxOfficeMojo forecasts. That typically comes on on Thursdays, which doesn’t help my publishing schedule since I write my weekly picks articles on Wednesday evening so they can get published on the main FML site on Thursday morning, but may reveal something else useful for all of us.

Bonus Graph

This past week has the highest percentage of players ever reach Perfect Combo (5.86%) although not the most by volume. Most weeks are still well below 2%, meaning that Perfect Combos are typically rare.