Thoughts about math, modeling, music, midlife, Montclair, and occasional things not beginning with the letter M.

Main menu

Monthly Archives: September 2016

I have an 8:00 meeting this morning, so I’m going to take the early (6:36) bus in. I almost never go in this early, so to be safe, I am already downstairs putting on my shoes at 6:29. Sometimes I joke that I live so close to the bus stop that if I hear the bus from the second floor window of my house, I still catch it as it goes by. That’s an exaggeration, but not by much. Now I hear a sound outside that could be the bus, but I know it’s not, it’s never this early. Still, I peek out the door.

It’s the bus, already on the corner. Crap.

I dash out. The bus stop is diagonally across from my house, and fortunately, the bus is waiting at a red light. This gives me a chance: I run down to the corner and across the street with the light. Now the bus and I are facing each other, I wave, cross the other way when the light turns as the bus waits an extra couple seconds, and get on. I am feeling good about the world and my place in it.

Still, 6:29, that’s super-early. Unless maybe this is the previous bus, running super-late? I think I should say something to the driver. “Excuse me, which bus on the schedule is this?” It’s the morning, everything is rushed, who knows if I actually said “Excuse me.”

The bus driver, not one I’m familiar with, is not pleased. He barks at me: “It’s whatever bus it is when I get here!” I look behind him, see the bus is mostly empty. I know from experience that if it were the previous bus, running late, it would be overcrowded, so this must really be the 6:36. I give the driver my ticket and try to explain: you’re running way too early, people who count on this bus will miss it, then the next buses are overcrowded and run late, etc. I ask him to wait a few minutes to get back on schedule. He remains very hot, says (yells) that I better stop talking to him, threatens to call the police if I don’t get away from him. The good thing about him yelling at me is that the bus is still not moving. A couple people, breathless from running down the street, get on the bus while we’re going back and forth. I count this as a small victory.

A large man walks up the aisle from the back of the bus. I am hoping for a little support from a fellow commuter. “You better fucking stop talking to him so he can go!” he yells at me. “You’re holding the bus up! He’s going to call the cops!” This guy looks like he’d like nothing better than to slug me. “He’s calling the cops!” The bus driver is not calling the cops. My fellow passenger, though he is louder than the driver, does not actually take a swing at me. I sit down in one of the many empty seats.

The bus doesn’t move.

I take my book out of my bag and try to disappear into it. It is eerily silent. In moments of suspense, time feels suspended. The bus still doesn’t move.

Finally, after what must have been only a short wait but didn’t feel that way, the bus huffs and trundles forward. I check the time. It is exactly 6:36. We have been waiting for just a few minutes.

I mull things over as the bus heads into the city. I feel good that I managed to stay calm through the whole episode. Maybe this will be a good story to tell the children — the importance of keeping your cool. It occurs to me that the bus driver ended up doing exactly what I asked him to — waiting to leave until he was scheduled to, to the minute. Or was that just a coincidence? Something makes me decide to talk to him again, though I can’t tell exactly how or why I make that decision.

The bus pulls in to Port Authority. I am going to wait for everyone else to get off and then try to approach the driver. As the people ahead of me are getting off, I hear the usual end-of-trip courtesies: passengers say thank you, driver says you’re welcome or have a good day. That’s encouraging: this isn’t one of those guys who’s so nasty that people give up on saying thanks at the end of the ride.

Once is everyone else is off, I get off too. The driver is standing to the side. “Thanks for waiting a few minutes at the bus stop,” I say to him. “I wasn’t trying to pick a fight, I just…”

The driver is looking me in the eye. “Yeah, I know,” he says. “I get what you were saying.” I talk to him about the schedule, he says he understands, sometimes they get off schedule and it’s hard to keep track. We are talking to each other like people now. He says he’s sorry, and he seems totally sincere. I tell him my name, because that seems like a talking-like-people thing to do, and he tells me his. I was hoping to make peace, but this is more than I expected; in under a minute, our conversation has turned comfortably fraternal. I am feeling good about the world again as I head inside the bus terminal.

Now another man slides over to me. He is fiftyish, tall, thin, with slightly graying hair, wearing jeans and a blazer. “You know,” he says,”I saw the bus driver yelling at you, I got a video on my phone. I’m going to call the bus company.”

I don’t actually want him to call the bus company. I tell him that I just had a good talk with the driver, and that he seemed very direct and sincere. My companion isn’t impressed. “He’s just afraid for his job,” he says. He tells me that the driver has done this (I think he means run ahead of schedule) once before, that people were running for the bus and he didn’t care. He looks at me a little conspiratorially. “I think this guy just hates white people.”

Well. The driver, like the majority of bus drivers I encounter, is black. I am white. The guy I’m talking to now is white. The passenger who was yelling at me earlier is white. Most people on the bus are white, with a few exceptions. What is the logic here, how do you decide that a black man driving a bus full of mostly white people hates them because they’re white? Does the white man who came up to yell at me before hate white people too? Why do we look differently at angry black people than at angry white people?

The man I’m talking to is going his way and I’m going mine, and there isn’t time to ask these questions. But I as I walk up 8th Avenue on a sunny end-of-summer morning, I realize that I’ve lost the tidy story I was going to tell my kids about the virtues of keeping calm. And perhaps a bit of my confidence in those virtues as well.

So here you are on Amazon’s web page, reading about Cathy O’Neil’s new book, Weapons of Math Destruction. Amazon hopes you buy the book (and so do I, it’s great!). But Amazon also hopes it can sell you some other books while you’re here. That’s why, in a prominent place on the page, you see a section entitled:

Customers Who Bought This Item Also Bought

This section is Amazon’s way of using what it knows — which book you’re looking at, and sales data collected across all its customers — to recommend other books that you might be interested in. It’s a very simple, and successful, example of a predictive model: data goes in, some computation happens, a prediction comes out. What makes this a good model? Here are a few things:

It uses relevant input data.The goal is to get people to buy books, and the input to the model is what books people buy. You can’t expect to get much more relevant than that.

It’s transparent. You know exactly why the site is showing you these particular books, and if the system recommends a book you didn’t expect, you have a pretty good idea why. That means you can make an informed decision about whether or not to trust the recommendation.

There’s a clear measure of success and an embedded feedback mechanism. Amazon wants to sell books. The model succeeds if people click on the books they’re shown, and, ultimately, if they buy more books, both of which are easy to measure. If clicks on or sales of related items go down, Amazon will know, and can investigate and adjust the model accordingly.

Weapons of Math Destruction reviews, in an accessible, non-technical way, what makes models effective — or not. The emphasis, as you might guess from the title, is on models with problems. The book highlights many important ideas; here are just a few:

Models are more than just math. Take a look at Amazon’s model above: while there are calculations (simple ones) embedded, it’s people who decide what data to use, how to use it, and how to measure success. Math is not a final arbiter, but a tool to express, in a scalable (i.e., computable) way, the values that people explicitly decide to emphasize. Cathy says that “models are opinions expressed in mathematics” (or computer code). She highlights that when we evaluate teachers based on students’ test scores, or assess someone’s insurability as a driver based on their credit record, we are expressing opinions: that a successful teacher should boost test scores, or that responsible bill-payers are more likely to be responsible drivers.

Replacing what you really care about with what you can easily get your hands on can get you in trouble. In Amazon’s recommendation model, we want to predict book sales, and we can use book sales as inputs; that’s a good thing. But what if you can’t directly measure what you’re interested in? In the early 1980’s, the magazine US News wanted to report on college quality. Unable to measure quality directly, the magazine built a model based on proxies, primarily outward markers of success, like selectivity and alumni giving. Predictably, college administrators, eager to boost their ratings, focused on these markers rather than on education quality itself. For example, to boost selectivity, they encouraged more students, even unqualified ones, to apply. This is an example of gaming the model.

Historical data is stuck in the past. Typically, predictive models use past history to predict future behavior. This can be problematic when part of the intention of the model is to break with the past. To take a very simple example, imagine that Cathy is about to publish a sequel to Weapons of Math Destruction. If Amazon uses only purchase data, the Customers Who Bought This Also Bought list would completely miss the connection between the original and the sequel. This means that if we don’t want the future to look just like the past, our models need to use more than just history as inputs. A chapter about predictive models in hiring is largely devoted to this idea. A company may think that its past, subjective hiring system overlooks qualified candidates, but if it replaces the HR department with a model that sifts through resumes based only on the records of past hires, it may just be codifying (pun intended) past practice. A related idea is that, in this case, rather than adding objectivity, the model becomes a shield that hides discrimination. This takes us back to Models are more than just math and also leads to the next point:

Transparency matters! If a book you didn’t expect shows up on The Customers Who Bought This Also Bought list, it’s pretty easy for Amazon to check if it really belongs there. The model is pretty easy to understand and audit, which builds confidence and also decreases the likelihood that it gets used to obfuscate. An example of a very different story is the value added model for teachers, which evaluates teachers through their students’ standardized test scores. Among its other drawbacks, this model is especially opaque in practice, both because of its complexity and because many implementations are built by outsiders. Models need to be openly assessed for effectiveness, and when teachers receive bad scores without knowing why, or when a single teacher’s score fluctuates dramatically from year to year without explanation, it’s hard to have any faith in the process.

Models don’t just measure reality, but sometimes amplify it, or create their own. Put another way, models of human behavior create feedback loops, often becoming self-fulfilling prophecies. There are many examples of this in the book, especially focusing on how models can amplify economic inequality. To take one example, a company in the center of town might notice that workers with longer commutes tend to turn over more frequently, and adjust its hiring model to focus on job candidates who can afford to live in town. This makes it easier for wealthier candidates to find jobs than poorer ones, and perpetuates a cycle of inequality. There are many other examples: predictive policing, prison sentences based on recidivism, e-scores for credit. Cathy talks about a trade-off between efficiency and fairness, and, as you can again guess from the title, argues for fairness as an explicit value in modeling.

Weapons of Math Destruction is not a math book, and it is not investigative journalism. It is short — you can read it in an afternoon — and it doesn’t have time or space for either detailed data analysis (there are no formulas or graphs) or complete histories of the models she considers. Instead, Cathy sketches out the models quickly, perhaps with an individual anecdote or two thrown in, so she can get to the main point — getting people, especially non-technical people, used to questioning models. As more and more aspects of our lives fall under the purview of automated data analysis, that’s a hugely important undertaking.