PollyVote just published its first forecast for this year’s German federal election, which will be held on September 24.

This is the second time (after 2013) that the PollyVote provides forecast of German elections. For this, the PollyVote applies the same basic method of combining forecasts, which has been used for US elections since 2004. That is, the PollyVote combines forecasts within and across four component methods, namely polls, prediction markets, expert judgment, and quantitative models.

The first forecast predicts Angela Merkel’s CDU/CSU to remain the strongest party, with a vote share of 33.1%, followed by the Social Democrats (SPD), which are predicted to gain 30.2% of the vote. The right-wing AfD (Alternative for Germany) is predicted to come in third with 9.9% of the vote, followed by the Left Party (7.9%) and the Greens (7.7%). The Free Democratic Party (FDP) is currently predicted to gain 6.7%, which puts them over the necessary 5% threshold to enter parliament.

Since its first launch last January, the combined PollyVote forecast has consistently – and correctly – predicted Hillary Clinton to win the popular vote. And so she did, albeit barely.

In this preliminary post-mortem of how the PollyVote performed in predicting the 2016 popular vote, we compare how each component did relative to the other and to its record on the last six elections. The analysis is based on current projections according to which Clinton will end up winning 50.9% of the popular two-party vote.

Polls

Across the last 100 days, which is the time frame we usually look at in our publications, the MAE of combined vote-intention polls was 1.8 percentage points. That is, polls were in fact considerably more accurate than in previous elections.

The problem is that there were large polling errors in certain key states, such as Michigan, Wisconsin, and Pennsylvania, that ended up deciding the election in the Electoral College and which no forecaster got right. We will do a separate analysis on whether the PollyVote reduced the error in the EC forecast in due time.

Prediction markets

In prediction markets traders bet on the outcome of an election, and the betting quotes provide a forecast of what is going to happen. Depending on the accuracy of their individual predictions, participants can either win or lose money, and thus have an incentive to be right. They should only participate if they think they have information that improves the current market forecast.

As in previous years, the PollyVote relied on the Iowa Electronic Markets’ (IEM) for predicting the popular vote. The IEM is operated by the Business School of the University of Iowa for teaching and research purposes. The market is relatively low volume and participants are not allowed to invest more than $500. The IEM is the only prediction market for forecasting the vote share, so one cannot combine across markets. We did, however, combine across time by calculating one-week averages to protect against short-term spikes.

Across the last six elections from 1992 to 2012, the IEM vote-share market was the second most accurate among the PollyVote’s components, after citizen forecasts. In 2016, however, the IEM performed poorly and provided the worst forecasts of all components throughout the campaign by wildly overestimated Clinton’s vote share.

Prior research shows that participants in these markets tend to be well educated and belong to middle and upper income groups. It may have been the case that this certain demographic group allowed their wishes to get the better of their judgment.

Expert judgment

Asking experts to predict what is going to happen is one of the oldest forecasting methods available. When it comes to forecasting elections, for example, experts have knowledge and expertise about how to read and interpret polls. In particular, experts know that polls have errors. Thus, they could be expected to be able to figure out in which direction the error is, and thus revise the forecast in the right direction.

In 2016, the combined expert forecast overestimated Clinton’s popular vote share relative to polls in five of the six surveys conducted since the parties’ conventions. On average, the expert forecast was 0.6 percentage points higher than the combined polls. In other words, the expert consensus was that the polls underestimestated Clinton’s support.

Citizen forecasts

As in previous elections, a forecast derived from survey respondents’ answer to a simple question “Who will win?” was among the most accurate methods for predicting the 2016 popular vote. Across the final 100 days before the election, citizen forecasts missed on average by only 1.4 percentage points, an error that is only slightly higher than the average across the six previous elections.

Models

The PollyVote used combined forecasts from two separate groups of models: index and econometric models.

Index models are based on the idea of prospective voting. That is, they rely on the idea that voters evaluate the candidates and how they stand on the issues when deciding upon for whom to vote. On average, the five available index models overestimated Clinton’s support, particularly due to two models that were especially far off (the bio-index and the issue-index). Three of these models (e.g., the big-issue model, the Issues and Leaders model, and the Keys to the White House) were close the final election outcome.

In contrast, most political-economic models are based on the idea of retrospective voting. That is, voters are expected to look back at how well the incumbent government has done its job, particularly in handling the economy. In addition, many models include some measure about how long the incumbent party has been in office to account for Americans’ desire for change, and some also include a measure of the president’s popularity. These models predicted a very close race, at least on average, and thus were the most accurate component method this election, which is the first time that has happened since 1992. That said, the final forecasts from the 18 individual models differed by as much as 10-points, ranging from 44.0% to 53.9% of the two-party vote.

Combining forecasts

We know from prior research that the relative accuracy of the different methods varies from one election to the next, not just for election forecasting but also in other fields. We can see this again in 2016. Prediction markets, which were among the most accurate methods historically, were dramatically off, while econometric models, historically high in error, turned out to be more accurate this time. That is one of the reasons why combining forecasts usually works well. It’s extremely difficult to predict ex ante which method will end up being most accurate.

In 2016, five of the six component methods erred in the same direction, over-predicting Clinton’s vote share by an average of more than 2 percentage points. As already mentioned, the IEM, in particular, was way of the mark. The only component method that slightly underestimated Clinton’s support were econometric models. As a result, there was little “bracketing” of the true value, which limits the benefits of combining forecasts.

Given the little bracketing, the PollyVote performed only slightly better than the typical forecast. It performed worse than econometric models, citizen forecasts, and polls but outperformed expert judgment, index models, and prediction markets.

The principle of combining forecasts does not claim that the combined forecast will always be more accurate than any of its components. While that can happen – and did happen, for instance, in 2004 and 2012 – it is unlikely to happen in any single election.

The claim is that over time, as the relative accuracy of other methods varies, the combined forecast will outperform its components. This can be seen by looking at the mean absolute forecast errors across all seven elections, including this year’s. On average, the PollyVote error was lower than any of its components.

Interestingly, citizen forecasts performed nearly as well as the PollyVote. So why not just use this one method in the future, you might ask? One major advantage of combining forecasts is that it’s often among the most accurate methods and, most importantly, it avoids large errors. There is not guarantee that citizen forecasts will perform as well in future elections.

How to improve

At the PollyVote, we are currently reviewing what we can learn from this election. This process includes reviewing which forecasts to include, how to better combine them, and how to better communicate their surrounding uncertainty.

The PollyVote aims at communicating scientific principles that can be used by any organization.

We do know from prior research that the combined forecast will always be at least as accurate than the typical component forecasts, in any single election. So, it prevents you from making large errors.

We also know that, over time and across many elections, the combined forecast will be among the most accurate forecasts available, because the performance of individual forecasts varies wildly.

If you accept that it is extremely difficult to predict which forecast will turn out to be most accurate in a particular election, there is no better way to forecasting than combining forecasts.

The 2016 presidential election result may well be one of the biggest upsets in the history of election forecasting. Usually, after an election, people look for the best forecasters. This time, it’s hard to find anyone who got it right, us included.

Regarding the Electoral College, PollyVote combined state-level forecasts from 20 different sources, none of which predicted Trump to win a majority of electoral votes.

Yet, these models predicted Trump to win the popular vote, which, according to the latest projections, he most likely won’t. Norpoth’s model, for example, predicted Trump to gain 47.5% of the two-party vote and thus might miss by about three points, which is a larger error than most of the other econometric models will end up with.

So are there any winners? Well, you could look at the individual models’ vote share predictions to judge their accuracy. For example, Jim Campbell’s well-known trial-heat model predicted Clinton to gain 50.7% of the vote and might thus be very close. But then, these models don’t provide us with the most important information of who will become president. So, no winners in sight.

Based on the latest results, Donald Trump won the election with 305 electoral votes (vs. 233 for Hillary Clinton). Most likely, the final outcome will show a split between the Electoral College and the national popular vote. The New York Times currently projects that Clinton will win the popular vote by 1.3 percentage points.

These results are a major upset for the PollyVote and other election forecasters, as virtually no one saw this coming.

Electoral College

The Electoral College outcome came at a big surprise. The PollyVote’s final Electoral College forecast predicted Clinton to win 323 electoral votes, compared to 215 for Trump, which was very much in line with other forecasters.

For these state-level forecasts, the PollyVote combined predictions from eight poll aggregators, five models, four sources of expert judgment, two prediction markets, and one source of citizen forecasts.

Not a single one of these twenty different sources predicted that Trump would win a majority of the electoral votes! Accordingly, the combined forecast failed as well.

Popular Vote

The final PollyVote forecast predicted Clinton to win the popular vote by 5 percentage points, 52.5% vs. 47.5%. Based on the latest projections, Clinton is expected to win the popular vote by about 1.2 points, which would translate to about 50.6% of the two-party vote. If these numbers are correct, the PollyVote will have missed by 1.9 percentage points, which is more than three times the average error from previous elections.

However, the accuracy of forecasts shouldn’t just be judged based on final predictions. Rather, one should look at which forecasting method has performed best over the course of the campaign.

The PollyGraph below shows the mean absolute error for the PollyVote and each of its component methods, across the remaining days to election. That is, at each day, the PollyGraph shows the error that you would have received by relying on that method’s forecast until Election Day. For example, starting March 15th, which is the day from which forecasts from all six component methods were available, the PollyVote’s average error until Election Day was 2.3 percentage points. This is makes it rank third after citizen forecasts (MAE: 1.1 percentage points) and and econometric models (MAE: 1.8 percentage points). Prediction markets were least accurate and missed on average by 5.9 points. (Note that these numbers will still change as the final vote tally changes. The chart below will always show the latest figures.)

NOTE: THE CHART IS CURRENTLY BROKEN

The relative accuracy of the different methods changes as we get closer to the election. For example, the average of econometric models became more accurate, whereas the accuracy of citizen forecasts decreased somewhat.

Initial thoughts

It’s been a long night and it’s still too early to draw conclusions. But Polly the forecasting parrot feels miserable about her miss. While she did of course perform as well as the typical forecast – which is the minimum what you would expect from a combined forecast – she did not outperform the best individual method as in previous elections. We will work hard to find out what went wrong and what information we missed in order to provide her with even better information next time.

But a look at the method’s relative accuracy in predicting the vote shares provides some interesting first insights. Prediction markets, which were among the most accurate methods historically, experienced a huge error. In contrast, econometric models, which were among the least accurate methods over the last six elections, outperformed all other methods in 2016.

These results conform to what we found in prior research on combining forecasts:

The accuracy of different methods changes over time and across elections. There is simply no one best method to predict elections. The methods’ relative accuracy for forecasting an election strongly depends on the context and the idiosyncrasies of that particular election.

It’s very difficult to determine a priori which method will provide the best forecasts. This is also why it makes little sense to weight forecasts based on their historical accuracy, particularly in applications such as election forecasting, where we have limited historical data to derive valid estimates about the method’s relative performance.

This is why Polly’s method of combining different forecasts is still valuable, and perhaps even more important in the future. First, it’s generally not a good idea to trust any single forecast. Second, in aggregating all the different forecasts that are out there, the PollyVote enables us to learn about the accuracy of the various methods – and the biases they may entail – over time.

If we can better understand the conditions under which different methods work best, we might be able to use this information to improve the accuracy of our combined forecast and to better estimate its surrounding uncertainty. In fact, estimating uncertainty is one of the most interesting areas for future research when it comes to combining forecasts.

We will provide additional analyses and thoughts over the next days and weeks. So stay tuned.

All numbers are in and, according to the PollyVote, Hillary Clinton will become the first female president of the United States.

Popular and Electoral Vote Prediction

Clinton will win the popular vote by 5.0 percentage points in the two-party vote (52.5% vs. 47.5%). Based on the PollyVote’s historical error, Clinton’s chance to win the popular vote is above 99%. In terms of the Electoral College, Polly predicts Clinton to receive 323 electoral votes compared to 215 for Trump.

Forecast over the course of the campaign

Forecasting is useful only when it can aid decision-making. Thus, Election Eve forecasts are only for entertainment and betting— and writing victory and concession speeches.

The PollyVote has predicted Clinton to win beginning with its first release in early January 2016. Since the first Super Tuesday on March 1st, when it became clear that Trump would become the Republican nominee, Clinton’s lead in the PollyVote forecast averaged 5.6 points. Throughout the course of the campaign, it ranged from a minimum of 3.8 points to a maximum of 8.2 points. In other words, the PollyVote never had Trump close to winning.

Since its launch in 2004, the PollyVote has always correctly predicted the winner on any given day, months in advance, and with little forecast error. Tonight, we will find out whether Polly the Parrot will keep her clean slate. Check back in the days to come as we will provide analyses of the different forecasting methods’ accuracy.

The PollyVote team has conducted the fourth and final round of its state-level expert survey. According to the forecasts of 638 political scientists, Hillary Clinton will win 334 Electoral Votes, compared to 204 for Donald Trump.

This Electoral College prediction thus differs from the results of the previous survey round, when the experts’ aggregate forecast was that Clinton would gain 358 Electoral Votes, compared to 180 for Trump. The difference is due to Iowa and Ohio, which the experts now see leaning towards the Republicans. For Arizona, the experts predict essentially a pure tossup in both median winning probabilities and vote shares. However, since the average forecast for Clinton’s vote share is slightly above 50%, Arizona is called for the Democrats.

The following map visualizes the experts’ median estimates regarding Clinton’s chance of winning each state.

Nebraska and Maine deserve special attention, since these two states allocate two Electoral Votes to the popular vote winner plus one each to the popular vote winner in each Congressional district. The experts predict that all districts will go with the state-wide popular with two exceptions.

In Nebraska’s 2nd district, the Democratic candidate is expected to win 51.0% of the two-party vote.

In Maine’s 2nd district, the Republican candidate is expected to win the popular vote with a two-party vote share of 50.6%.

Method

We reached out to political scientists across the country and asked them two short questions:

What share of the vote do you expect the nominees to receive in your home state?

What do you think is Hillary Clinton’s chance of winning the election in your home state?

For experts who regard Nebraska or Maine as their home state, we additionally asked them to predict the outcome in each congressional district.

The survey was conducted from November 6 to 7. A total of 638 experts made estimates as requested. The number of experts by state ranged from 2 to 44. The table below shows the number of respondents per state as well as the median answer for each question.

The PollyVote team has completed its 13th and final survey of elections experts to forecast the 2016 presidential election. In this survey, conducted between November 6 and 7, 12 academics from a variety of colleges and universities responded.

As in previous rounds, all respondents expect a Clinton win. However, her lead has narrowed again compared to the results of the previous survey conducted a week ago.

Whereas, in late-October, the experts expected that Clinton will win the popular vote by more than 5.5 points, the new average forecast is half a percentage point lower, at 52.2% of the two-party vote (or about a 4.4-point margin). The individual forecasts ranged from 51.0% to 53.3%, with a standard deviation of only 0.8 points.

Results of a new national poll administered by Economist were released. The poll asked respondents for whom they will vote: Republican Donald Trump or Democrat Hillary Clinton.

Economist poll results

49

Clinton

45

Trump

Of those who responded, 49.0% said that they plan to vote for former Secretary of State Hillary Clinton, whereas 45.0% said that they would give their vote to real estate developer Donald Trump.

The poll was conducted between November 4 and November 7. The sample size was 3669 participants. There is a sampling error of +/-1.7 percentage points. Considering this error margin, the gap between both candidates is statistically significant.

Putting the results in context

Individual polls should be treated with caution, since they can include substantial errors. Instead of relying on results from single polls, research in forecasting recommends to use combined polls or, even better, the combined PollyVote forecast that relies on different methods and data.

In order to make the results comparable to benchmark forecasts, one can translate them into shares of the two-party vote. This procedure results in values of 52.1% for Clinton and 47.9% for Trump. On November 1 Clinton obtained only 51.6% in the Economist poll and Trump obtained 48.4%.

Comparison to other polls

An average of recent polls sees Clinton at 52.0% of the two-party vote. This value is 0.1 percentage points lower than her respective numbers in the Economist poll. This deviation is within the poll's error margin, which suggests that the poll is not an outlier.

Comparison to the combined PollyVote

The latest PollyVote expects Clinton to gain 52.6% of the two-party vote. This means that Polly's forecast is 0.5 points above her polling numbers. Again, a look at the poll's margin of error indicates that this deviation is negligible.

This article was automatically generated by the PollyBot, which uses algorithms developed by AX Semantics to generate text from data stored in our API. The exact dataset underlying this particular article can be found here.

Please let us know if you find any typos, missing words, or grammatical errors. Your feedback helps us to further improve the quality of the texts.

Siena published the results of a new poll. In this poll, respondents from New York were asked for whom they will vote: Donald Trump or Hillary Clinton.

Siena poll results

51

Clinton

30

Trump

The results show that 51.0% of participants will give their vote to former New York Senator Hillary Clinton, while 30.0% plan to vote for billionaire Donald Trump.

The poll was conducted from September 11 to September 15. A total of 600 likely voters responded. There is a sampling error of +/-5.0 percentage points. Considering this error margin, the gap between both candidates is statistically significant.

Putting the results in context

Single polls can incorporate large errors, and should be treated with caution. Rather, one should check how a poll's results compare to benchmark forecasts.

For the following analysis, we translate Clinton's and Trump's raw poll numbers into shares of the two-party vote. This yields figures of 63.0% for Clinton and 37.0% for Trump. The most recent PollyVote foresees Clinton to gain 63.0% of the two-party vote in New York.

This article was automatically generated by the PollyBot, which uses algorithms developed by AX Semantics to generate text from data stored in our API. The exact dataset underlying this particular article can be found here.

Please let us know if you find any typos, missing words, or grammatical errors. Your feedback helps us to further improve the quality of the texts.

KSTP/SurveyUSA released the results of a new poll, in which respondents from Minnesota were asked for whom they will vote: Donald Trump or Hillary Clinton.

KSTP/SurveyUSA poll results

42

Clinton

45

Trump

The results show that 42.0% of participants said that they would cast a ballot for former First Lady Hillary Clinton, while 45.0% intend to vote for businessman Donald Trump.

The poll was conducted from October 29 to November 2. A total of 516 registered voters responded. The margin of error is +/-4.4 points. This means that the poll results for both parties' candidates do not differ significantly.

Putting the results in context

Individual polls often incorporate large errors, which is why they should be interpreted with caution. Rather than trusting the results from single polls, one should rely on combined polls or, even better, the combined PollyVote forecast that draws upon forecasts from different methods, each of which draws upon different data.

For the following comparison, we convert the candidates' raw poll numbers into two-party vote shares. This yields figures of 48.3% for Clinton and 51.7% for Trump. For comparison: 55.8% was obtained by Clinton in the KSTP/SurveyUSA poll on October 25, for Trump this number was only 44.2%.

Results in comparison to other polls

Looking at an average of Minnesota polls, Trump's two-party vote share is currently at 46.4%. Compared to his numbers in the KSTP/SurveyUSA poll Trump's poll average is 5.3 percentage points worse. This margin is outside the poll's margin of error, which suggests that the poll is an outlier.

Comparison to the combined PollyVote

The current PollyVote forecasts Trump to gain 44.5% of the two-party vote in Minnesota. That is, the PollyVote is 7.2 points below his polling numbers. Again, a look at the poll's sampling error reveals that this difference is significant.

This article was automatically generated by the PollyBot, which uses algorithms developed by AX Semantics to generate text from data stored in our API. The exact dataset underlying this particular article can be found here.

Please let us know if you find any typos, missing words, or grammatical errors. Your feedback helps us to further improve the quality of the texts.