Chelsea Manning says software developers need a code of ethics

Chelsea Manning, the former Army intelligence analyst and whistleblower who gave diplomatic and military documents to WikiLeaks, took the stage at SXSW this morning to talk about what life has been like for her post-prison, the current political landscape, and the misuse of technology and data.

In Iraq, Manning said she had gone in with the mindset that algorithms could make things better. But during her service, Manning said she noticed people would pick and choose the data they would and wouldn’t believe.

“The algorithms themselves are not unbiased,” she said. “We put our biases in there when we write it.”

It’s also possible that engineers feed it biased data, she said, and pointed to predictive policing. With predictive policing, if a police department is already over-policing a neighborhood due to racism and biases, for example, the data they put into it will already be tainted.

“It’s a feedback loop,” Manning said.

In systems that affect millions of people, technologists and developers have a responsibility to be aware of the potential consequences of the tools they develop. Manning, herself, said she’s been in the “hot seat” to ship code quickly and has gotten caught up in the development process without thinking too much about the bigger picture.

“There’s also dangers in writing software for one thing and expecting it to only be used for one thing,” she said.

Manning pointed to how before she served in Iraq, she worked with marketing data to write predictive algorithms based on purchase history to try to determine where future customers may be, and how to maintain customer loyalty. In Iraq, Manning said she reused that knowledge to write algorithms for war.

“Because of the modularity and the ability to reuse code, we have to put in caveats on our code and take into account the fact that when we write software for one purpose we may be writing it for a very nefarious purpose down the road,” she said.

The same goes for collecting data for one purpose but then using it for another later on, she said. Manning went on to propose software developers, like doctors, have a code of ethics.

“We should have a sense of what our responsibilities are as software developers that transcend our bosses,” Manning said.

Additionally, developers should feel a responsibility to think about how their software can be misused. Ahem, Facebook getting misused to influence the American presidential election.

“That’s going to be the norm unless we become more self-aware and more cognizant of the systems we build,” Manning said in response to a question about Facebook. “Because our systems are going to be exploited. Every piece of software can be misused.”

Since getting out of prison, Manning said it has been hard adjusting to life. She also said living alone has been hard, and sometimes brings back bad memories of being in solitary confinement. But on the positive side, Manning said it’s nice to be able to live her life as she is.

“It’s really nice to not be self-conscious about who I am,” she said. “Beforehand, there was always this internal questioning inside my brain. What am I? Am I being masculine enough? Am I being man enough? Am I meeting people’s expectations? There’s none of that anymore.”

Had Obama not commuted her sentence, Manning would have remained in prison until 2045. While President Obama was responsible for her early release from prison, a different president was in office by the time she reentered society.

While some people have expressed shock and distaste over Trump’s policies and rhetoric, Manning says this didn’t come out of left field. Instead, this has been decades in the making, she said.

“The political rhetoric and style of governance that we’ve been seeing is not an aberration,” she said. “It’s a conclusion of the systems we’ve built.”