Combining forecasts

The PollyVote is based on the principle of combining forecasts. That is, PollyVote combines forecasts from different forecasting methods, the so-called component methods, each of which relies on different data. The PollyVote forecast is calculated by following a two-step approach:

The rationale behind choosing this procedure is to equalize the impact of each component method, regardless of whether a component includes many forecasts or only a few. For example, while there is only one prediction market that predicts the national popular vote, there are forecasts from numerous econometric models. In such a situation, a simple average of all available forecasts would overrepresent models and underrepresent prediction markets, which we expect would harm the accuracy of the combined forecast. Another advantage of this approach is that it allows for comparing forecasts from different component methods.

The principle of combining forecasts

Combining forecasts is well established in the forecasting literature as a powerful method to reduce forecast error. The research interest in combining forecasts has increased since the publication of the frequently-cited paper by Bates and Granger almost half a century ago. Since then, hundreds of studies have demonstrated the benefits of combining forecasts in different fields.

Why combining forecasts works

One intuitive explanation as to why combining improves accuracy is that it enables forecasters to use more information, and to do so in an objective manner. Moreover, bias exists both in the selection of data and in the forecasting methods that are used. Often the bias is unique to the data and the method, so that when various methods using different data are combined to make a forecast, the biases tend to cancel out in the aggregate.

How to best combine forecasts

A widespread concern when combining forecasts is the question of how best to weight the components, and many scholars have proposed different methods for doing so. However, an early review of more than two hundred studies from different fields concluded that the question of how to combine forecasts does not seem to be critical to forecast accuracy. In fact, it was found that the simple average (i.e., assigning equal weights to components) often provides more accurate forecasts than complex approaches to estimating ‘‘optimal’’ combining procedures. Empirical research since then repeatedly confirmed these findings.

One reason for the strong performance of equal weights is the fact that the accuracy of the component forecasts varies over time and depends strongly on external effects. For example, when analyzing the predictive performance of six econometric models across the ten election from 1976 to 2012, one study found a negative correlation between a model’s past and future performance. In other words, models that were among the most accurate in a given election tended to be among the least accurate in the succeeding election. The reason is that every election is held in a different context and has its idiosyncrasies and, therefore methods that worked well in the past might not work well in the future. In such a situation, weighting the forecasts based on past performance is of course of limited value.

Another more technical reason is estimation error in the differential weights. In general, the simple average will be more accurate than estimated ‘‘optimal’’ weights if two conditions are met:

The combination is based on a large number of individual forecasts and