Thoughts about math, modeling, music, midlife, Montclair, and occasional things not beginning with the letter M.

Main menu

Monthly Archives: November 2016

At the start of October, I went to see Elvis Costello at Town Hall, a small theater near Times Square with a history of hosting political meetings. You can see framed programs from some of those meetings, hazy, fading memories of the past, as you walk into the building:

The show had been loosely billed as a review of Elvis’s life and career, aligned with the memoir he released last summer, and I was looking forward to a few hours spent listening to him resculpting his songs in new and deeply satisfying ways. (I still felt warm from the time I saw him carry this off several years ago, at Carnegie Hall of all places, when he opened the show by playing the entire first side of My Aim is True, sang songs from all over the map, and told the kind of stories you might hear from the cool uncle you wish you had.)

For the first half hour, the show felt much like that night at Carnegie Hall. The stage was dominated by a giant TV, which showed 80’s Elvis videos as we waited for him to come on, then, once he did, ran a slide show of old family photos (hazy snapshots from the past again) and pictures from his early tours behind him as he played (solo, with acoustic guitar). He gave us both old and recent standards, culminating in a lovely rearrangement of Everyday I Write the Book. But as I burrowed deeper into my plush chair, getting comfortable for a couple more hours with Elvis’s fabulous songbook, he walked stage right, sat down at the piano, and smashed the time machine to bits.

In a halting, almost shattered voice, accompanied by the most austere, jagged piano figures you could imagine, he sang Shipbuilding, perhaps his greatest song. Written at the time of the Falklands war, Shipbuilding is a biography of out-of-work men in the English shipyards, waiting for the war to restart the shipyards and bring them jobs,

A new winter coat and shoes for the wifeAnd a bicycle on the boy’s birthday

and also for the warships that emerge from the shipyards to take those boys away, and bring them to their graves. “Is it worth it,” Elvis asks, and then asks again in a later verse that I hadn’t known before,

A small bunch of flowers is all that you get And a box to bury the baby

I had heard Elvis sing the song before, but not like this. I wish I knew how to describe the cocktail of regret and loss and compromise and inevitability I heard in his voice, but all I can do is point you to another song: if you listen to Springsteen’s Highway Patrolman, you’ll catch the same emotions when you hear Bruce’s voice trail off as he sings When it’s your brother, sometimes you look the other way at the end of the second verse. Hearing Elvis now, you could see shipyards, but also car factories on the outskirts of town, and mom and pop stores on Main Street, all of them filled with people’s dread of decisions they might know they’ll come to regret but still can’t avoid making. By the time the song’s resolution — diving for dear life, when we should be diving for pearls — exploded from the stage, you felt the full weight of the present day, of this election. Somehow, in just four minutes, Elvis had managed to make you feel the dilemmas of Trump voters deep down in your bones, more convincingly than any analysis I’ve read before or since the election. But he was only getting started.

He sang Deep Dark Truthful Mirror, an answer record in some ways, about facing up to the consequences of whatever choices you’ve made. Then, still at the piano, he started to tell us about a musical he was working on with the playwright Sarah Ruhl. It is an adaptation of the 1957 film A Face in the Crowd, which tells the story of the media-driven rise and fall of an American demagogue. If hearing Elvis sing Shipbuilding wasn’t enough to convince you he was addressing today’s political climate, the new songs he had written for the musical made it even plainer. He sang as the anti-hero, promising the angry and ignored to be their champion. He sang silly jingles that scoot back and forth across your screen and sell you silly things. And he sang for and with the faces in the crowd, all of us looking ourselves in the mirror and trying to find some common ground. The songs were theatrical, raw, maybe still works in progress — but they felt necessary and Elvis got them across. I am no fan of musicals, but I can’t wait to see what these songs will be like, and what they will mean, by the time they hit the stage, in the full blossoming of Trump’s America.

And then the finale. Given the range of Elvis’s songbook, it’s ironic that the song he is most associated with in the public imagination, What’s So Funny ‘Bout Peace, Love, and Understanding, is one he didn’t even write. But it is, and you won’t feel like you got your money’s worth unless you hear him play it, will you? Still, I’m not sure the song ever felt like it had more at stake than when Elvis sang it at Town Hall to close this show. And, as he sang it, the giant TV behind him projected a late 70’s, Armed Forces-era poster with the headline of the evening, Elvis’s answer to the America of Donald J. Trump, the next president of the United States: DON’T JOIN!

Soon it was over, and it was hard to believe we actually had to leave the building. But we did, and as I walked downstairs, the faded programs on the walls, from Town Meetings of long ago, were suddenly as sharp and timely as anything you might see on TV or on your Facebook feed. The past is not dead, not even past, as Faulkner said, and now all of us have many more town meetings to go to.

Modeling this election was never about win probabilities (i.e., saying that Clinton is 98% likely to win, or 71% likely to win, or whatever). It was about finding a way to convey meaningful information about uncertainty and about what could happen. And, despite the not-so-great headline, this article by Nate Silver does a pretty impressive job.

First, let’s have a look at what not to do. This article by Sam Wang (Princeton Election Consortium) explains how you end up with a win probability of 98-99% for Clinton. First, he aggregates the state polls, and figures that if they’re right on average, then Clinton wins easily (with over 300 electoral votes I believe). Then he looks for a way to model the uncertainty. He asks, reasonably: what happens if the polls are all off by a given amount? And he answers the question, again reasonably: if Trump overperforms his polls by 2.6%, the election becomes a toss-up. If he overperforms by more, he’s likely to win.

But then you have to ask: how much could the polls be off by? And this is where Wang goes horribly wrong.

The uncertainty here is virtually impossible to model statistically. US presidential elections don’t happen that often, so there’s not much direct history, plus the challenges of polling are changing dramatically as fewer and fewer people are reachable via listed phone numbers. Wang does say that in the last three elections, the polls have been off by 1.3% (Bush 2004), 1.2% (Obama 2008), and 2.3% (Obama 2012). So polls being off by 2.6% doesn’t seem crazy at all.

For some inexplicable reason, however, Wang ignores what is right in front of his nose, picks a tiny standard error parameter out of the air, plugs it into his model, and basically says: well, the polls are very unlikely to be off by very much, so Clinton is 98-99% likely to win.

Always be wary of models, especially models of human behavior, that give probabilities of 98-99%. Always ask yourself: am I anywhere near 98-99% sure that my model is complete and accurate? If not, STOP, cross out your probabilities because they are meaningless, and start again.

How do you come up with a meaningful forecast, though? Once you accept that there’s genuine uncertainty in the most important parameter in your model, and that trying to assign a probability is likely to range from meaningless to flat-out wrong, how do you proceed?

Well, let’s look at what Silver does in this article. Instead of trying to estimate the volatility as Wang does (and as Silver also does on the front page of his web site, people just can’t help themselves), he gives a careful analysis of some possible specific scenarios. What are some good scenarios to pick? Well, maybe we should look at recent cases of when nationwide polls have been off. OK, can you think of any good examples? Hmm, I don’t know, maybe…

Aiiieeee!!!!

Look at the numbers in that Sun cover. Brexit (Leave) won by 4%, while the polls before the election were essentially tied, with Remain perhaps enjoying a slight lead. That’s a polling error of at least 4%. And the US poll numbers are very clear: if Trump overperforms his polls by 4%, he wins easily.

In financial modeling, where you often don’t have enough relevant history to build a good probabilistic model, this technique — pick some scenarios that seem important, play them through your model, and look at the outcomes — is called stress testing. Silver’s article does a really, really good job of it. He doesn’t pretend to know what’s going to happen (we can’t all be Michael Moore, you know), but he plays out the possibilities, makes the risks transparent, and puts you in a position to evaluate them. That is how you’re supposed to analyze situations with inherent uncertainty. And with the inherent uncertainty in our world increasing, to say the least, it’s a way of thinking that we all better start becoming really familiar with.

The models were plain as day. What the numbers were telling us was that if the polls were right, Clinton would win easily, but if they were underestimating Trump’s support by anywhere near a Brexit-like margin, Trump would win easily. Shouldn’t that have been the headline? Wouldn’t you have liked to have known that? Isn’t it way more informative than saying that Clinton is 98% or 71% likely to win based on some parameter someone plucked out of thin air?

I went to bed around 11 when the outcome seemed obvious, woke up around 2 and couldn’t resist making sure it was really true, then lay awake in bed for the next few hours trying to process it. I can’t remember another time when I found it so hard to even catch hold of a coherent thought. Here are my three best attempts.

A lot of us are saying this isn’t the country we thought we knew. Look, here’s what you thought you knew a day ago: if you picked 100 people across the country at random, about 49 would vote for Hillary, about 46 for Trump, the remaining 5 or so for someone else, and things would trundle along as usual. Now it turns out that one or two of those people that you thought would vote for Hillary actually voted for Trump. That’s a big deal, and it has huge consequences, but it’s not like you were wrong about all, or even most, of those people, your fellow Americans. This is still the same country that voted for Obama twice. You were just wrong about one or two people out of 100, and anyway you don’t know what they were thinking.

One thing that *is* different is that the world looks a lot more uncertain now than many of us are used to. So let’s stop saying, like we know for sure, that Trump won because of racism, or misogyny, or stupidity, or whatever you’re absolutely sure is the reason. You were absolutely sure yesterday that Hillary would win, and look what happened. If we’re entering a very different world, let’s not reduce it to easy judgments, or leave the responsibility to figure out that world to other people.

Let’s not forget too that the level of anxiety that many of us are feeling now is closer to the level of anxiety that many other people feel, and cope with gracefully, every day. Yes it sucks to lose your Certainty Privilege, but we’ve probably been unusually lucky to have had it up until now. Right now I am coping with my own uncertainty by hanging on to the things that I am genuinely certain of, like how my sons, age 9 and age 7, deserve better than Trump. Going to try to make this a better world for them. Onward.