Thank you!

September 11th, 2012, 3:56pm by Sam Wang

If you look at the left…the $25,000 target on the thermometer has been exceeded, two months early. T. Roberts, a reader of this site since 2004, put us over the top. Thank you all.

Soon I may have to reset the thermometer. As indicated by the rumblings, Team Romney is now trying to avoid discussing the possibility of panic. But the sense is unmistakable on the right that the Presidency is slipping away. Indeed, Paul Ryan is starting to run television ads to keep his current House seat. And the degree to which the Senate is in play is starting to be generally appreciated.

17 Comments so far ↓

This election is going to set the long term direction of the United States for a long time to come. Over the next four years, the US economy will get a lot better. Robert Gordon’s pessimistic assessment notwithstanding, productivity growth will resurge. Sure, there are challenges like the national debt. But, if you look around among the OECD countries, it is far better than the most. It is times like this, where small contributions can make a big difference. This is one of the rare moments I wish I still had my Green Card so that I could donate money in this election. Sam Wang is doing a stellar job. I really really hope that anybody reading this can and does help a worthy cause. At this juncture, it is far beyond a political donation. It will determine the trajectory of health, education and other critical matters for you and your children and grandchildren if you are a resident/citizen of the US. (And I join Sam in thanking T. Roberts).

Does anyone have information on the pedigree of Gravis Marketing and who might be funding them for the state polls. Superficially, they seem to be skewed Republican and are certainly affecting the averages on RCP???

Precisely, Tapen. If Romney wins, >50% of the country will be fooled into thinking the Obama policies that laid the groundwork for the recovery were actually because of Romney. He would be easily reelected and probably would propel a GOPer to win 2020. On the flip side, if Obama wins, the GOP will double its primaries focus on getting a “true conservative” on the ballot for 2016, which will make them more vulnerable than otherwise to lose to the Dem candidate

Sam, I have a question about the error bands on the Median EV estimator plot. I presume these are 95% confidence level bands. I really like that you calculate these.

I am wondering about how the recent uptick in Obama’s polling numbers should affect these error bands. I would expect that the upper edge should go higher now. It’s been stuck at approx. 345 for a while now.

If I look at the polls.medians.txt file, some states have not been polled for ages. And why should they be? It’s obvious that Texas or Montanna are not going blue, no matter how well Obama polls.

But maybe Indiana, Missouri, etc get more interesting. Maybe they go from 10% to 33%. Maybe that does not move the Median EV, but it could move the mean, and the upper error band.

In other words, before the conventions 350 EV was just outside the 95% CL. After the conventions it might be just inside.

Having said all this, I realize I don’t know how to go about estimating how to move these un-polled states. I suppose you could extrapolate based on neighboring polled states with similar demographics. And maybe blow up the SEM a bit to compensate, before doing the integral (can we call it the “Wang Integral” or the “Wang Sum”)?

Anyway, that’s my long winded way of saying there are a lot of red states which have not been polled since mid-August and could have become a little more blue.

Amitabh: One thing Silver does is use the national trend to inform the older polls. He adjusts these based on his elasticity I believe. I think Sam’s mentioned it before as potentially useful.

I’m not as concerned as you with figuring out if the right tail has become longer, but even battleground polling has been sparse… I would want to be sure that unnecessary error is not introduced however.

It should be easy to backtest, but even then you have to take care not to cheat on parameterization. Idk.

Peter: It does not surprise me that Nate corrects for stale polls. He has house effect corrections, individual poll weights, economic data, phase of the moon…

The next improvement in his analysis might well be a neural net.

Don’t get me wrong, I like what Nate does. Some things are a bit confusing, like having two trend lines (Nov 6 and Nowcast) which by definition have to converge by election day, so what does the difference between them signify now? But overall he creates a model, integrates it numerically and publishes the answer (though not the model itself).

Of course, there is no uncertainty band in Nate’s number. How would you estimate the systematic uncertainty in say, pollster house effects?

I think Sam’s approach is conservative (I mean that in the academic sense, not political). He has a closed form solution and can propagate errors through.

If he decided to extend the stale polls, he might have to blow up errors on those extrapolated states, but it wouldn’t be a showstopper.

By the way, the reason I want the confidence level bands (other than every measurement should have an uncertainty quoted) is that if Obama is consistently above 95% CL I will shift all my paltry donations completely over to Senate races.

Silver does post a lot of figures that break down uncertainty in various ways (the EV histogram, “odds of winning”), so he probably could provide a CI in that sense – it’s a little surprising to me as a reader that he doesn’t. I have a feeling those numbers just literally tally the 10k simulations, though, and there’s some uncertainty from that sort of Monte Carlo thing. If you want to go one more ‘meta’ level up to account for the simulation sampling (such as to get a CI on the “odds of winning” number), maybe he could bootstrap confidence intervals on everything.

I have to point out a couple of things too…

(1) Silver seems to take the house effect ‘zero point’ (for lack of a better term) as just another thing that the pollsters themselves are providing estimates for. He takes a mean of house effects, weighted by ‘pollster rating’, which contains a component for how off-the-mark each pollster’s house effect has been in the past. So that seems relatively easy to put a CI on (just pick out the 5%- and 95%-iles, post-weighting), at least in the context of that model.

(2) The difference between “now-cast” and “Nov 6” is, if I understand correctly, that “now-cast” goes along the lines of what Sam does (just polls), while “Nov 6” does various things that Silver thinks will predict where polls will go in the future (economic data, convention bounces, regression to the mean in accordance to a historical/socio/etc. model, etc…) and expands the uncertainty bars by time (if he were to draw the bars in!).

AySz88 – no, I think the Now-cast is also a non-snapshot because it’s got all the econometric whatnot. In some sense it’s a calculation of today’s conventional wisdom. I really think you are taking it more seriously than it deserves.

As far as calculating a CI from his probabilities, that is not a good idea because states covary over time. This is why I calculate the Meta-margin, which basically dials all states up/down to find what point gives an EV tie. Also, as you point out (if I understand you), it’s not clear if those probabilities are outputs of the simulation.

If you want to do something bootstrappy, then take today’s polls, subtract their known house effects, re-add a randomly chosen house effect to each poll, and recalculate the EV estimator. Repeat this 1000 times and see how the EV estimator is distributed. All the moving parts are in front of you and you don’t have to guess what all those “probabilities” are.

Figuring out what might happen in very red and blue states is a difficulty. I was just toying with a post on the subject of Obama’s ceiling, which I think is about 360 EV because the EV vs. amount-of-future-movement graph starts going flattish around 340 EV before it takes off again. To put it another way, everybody is focused on about 10 “swing states,” but there is a larger set of “swingable” states that don’t depend on 2012 conditions. And *then* we finally get to true red/blue states, i.e. Utah, New York, Vermont…

To model the change, the following thoughts come to mind, in ascending complexity.

(1) Take current polls, stale or new, and add/subtract a constant bias to all. Then recalculate using EV_estimator.m.

(2) To test if shifts in opinion are not evenly distributed across states, study how those changes are correlated over time. Use that.

(3) Do a complicated version of (2) for sex, ethnic group, age, and so on.

>>>

Regarding putting all your contributions into Senate races (which I wish you would do): To estimate when P(re-elect) might get above 0.95, imagine the ideal case that the snapshot’s CI is <1 EV. Then the question becomes: when do polls predict enough about November 6th to seal the deal?

This will happen if the t-statistic (3 d.f.) gets to 2.3, for example if the Meta-margin were 3.1% (its average to date) and the SD of drift from today to election were 1.35%. If we believe that SD of the Meta-margin over a general campaign is 2.2%, and polls get progressively more predictive at <40 days, t=2.3 by around 25 days before the election, i.e. October 12th. So you may have to wait until then at the earliest.

As to calculating a CI – put more simply, I was actually wondering why Silver wouldn’t just mark the 5% and 95%-ile lines on his EV histogram and call that a(n estimate of the) CI. Does the fuzziness of such a CI make it not kosher? (That’s why I initially mentioned bootstrapping – to estimate the uncertainty in that sort of fuzzy CI. But I love that method of estimating the impact on house effects!)

Er, no offense intended in comparing your work with Silver. I thought it didn’t use economic stuff, because Silver did say that his “now-cast” doesn’t include economic variables… Except maybe he forgot that his model still does use them in that “state fundamentals” number? It looks like that’s used as a sort of Bayesian prior. *shrug*

I would agree that he seems to be re-expressing conventional wisdom (and a whole kitchen-sink-ful of it), just in a quantitative way. But I’d say that’s an improvement! At least various pieces of his model can be verified (or thrown out), unlike the conventional pundit that always seems to manage to worm their way out of accountability. I look at it as a step on the long journey toward getting rid of the junk and superstition.

As an aside: I’m not entirely sure by what you meant by “probabilities are outputs of the simulation”. But judging from the way he has used his simulations to explore different scenarios, I do believe that, for example, an 80.0% “odds of winning” for Obama actually meant that ~8000 of his daily simulations were Obama wins.

AySz88, I think we agree on most points, including the possibility of future hope for detailed models. Yes, the state probabilities reported on his site are probably outputs of many simulations, not usable as inputs to someone else’s calculation.

Adjusting a single-state win probability downward using “fundamentals” includes hidden assumptions in the sense that the effect is to move away from a direct measurement of opinion. I could imagine doing this to estimate the range of future movement, but not to estimate conditions “now.”

Wheeler’s Cat, it is doubtful that the house effects correction can account for the difference, since this is a step that should result in more certainty and therefore a narrow CI. One possible contributor to a win probability closer to 50% is “state fundamentals.” Another possibility is that the Now-cast is heavily filtered in time. A third possibility is that the individual simulations assume that states drift independently from one another. All of these factors may blur the picture substantially.

I find the topic of the Now-cast to be awkward because the methods are a bit hard to sift through. I think his honesty and intuitions are strong, but what I can see of the implementation is not. If someone can offer specific methodological documentation, great. But please, no more speculations…

Amitabh, Dr Wang, it is my understanding that there is no econometric whatnot in the Nowcast and its purely based on poll data. At least that is what Nate said it was.
So I postulated that the significant difference in the PEC probability to win and the Nowcast has to be the treatment of house effect, since Nate uses parametric statistics and PEC uses non-parametric statistics.
Of course, CLT would imply PEC and the Nowcast converge over time, and that seems to be what is happening.
Nowcast is now 88.2.

It makes sense that the 95% CI is resistant to going above 347. Once O takes North Carolina there is a big gap to his next state in terms of Meta-Margin and odds. Given correlation of movement of states, the odds of going over 347 should be no greater than his odds of taking Indiana (what are those odds?).