How algorithms rule the world

The NSA revelations highlight the role sophisticated
algorithms play in sifting through masses of data, but more surprising is their widespread use in our everyday lives
— should we be more wary of their power?

By Leo Hickman / The Guardian, LONDON

Illustration: Tania Chou

On Aug. 4, 2005, the police department of Memphis, Tennessee, made so many arrests over a three-hour period that it ran out of vehicles to transport the detainees to jail. Three days later, 1,200 people had been arrested across the city — a new police department record. Operation Blue Crush was hailed a huge success.

Larry Godwin, the city’s new police director at the time, quickly rolled out the scheme and by 2011 crime across the city had fallen by 24 percent. When it was revealed that Blue Crush faced budget cuts earlier this year, there was a public outcry.

“CRUSH” policing is now perceived to be so successful that it has reportedly been mimicked across the globe, including in countries such as Poland and Israel. In 2010, it was reported that two police forces in Britain were using it, but their identities were not revealed.

CRUSH stands for “Criminal Reduction Utilizing Statistical History.”

Translated, it means predictive policing, or, more accurately, police officers guided by algorithms. A team of criminologists and data scientists at the University of Memphis first developed the technique using IBM predictive analytics software. Put simply, they compiled crime statistics from across the city over time and overlaid it with other datasets — such as social housing maps and outside temperatures — then instructed algorithms to search for correlations in the data to identify crime “hot spots.” The police then flooded those areas with highly targeted patrols.

“It’s putting the right people in the right places on the right day at the right time,” said Richard Janikowski, an associate professor in the Department of Criminology and Criminal Justice at the University of Memphis, when the scheme was launched.

However, not everyone is comfortable with the idea. Some critics have dubbed it Minority Report policing, in reference to the science-fiction film in which psychics are used to guide a “precrime” police unit.

The use of algorithms in policing is one example of their increasing influence on our lives and, as their ubiquity spreads, so too does the debate on whether we should allow ourselves to become so reliant on them — and who, if anyone, is policing their use.

Such concerns were sharpened further by the continuing revelations about how the US National Security Agency (NSA) has been using algorithms to help it interpret the colossal amounts of data it has collected from its covert dragnet of international telecommunications.

“For datasets the size of those the NSA collect, using algorithms is the only way to operate for certain tasks,” said James Ball, the Guardian’s data editor and part of the paper’s NSA files reporting team.

“The problem is how the rules are set: It’s impossible to do this perfectly. If you’re, say, looking for terrorists, you’re looking for something very rare. Set your rules too tight and you’ll miss lots of, probably most, potential terror suspects, but set them more broadly and you’ll drag lots of entirely blameless people into your dragnet, who will then face further intrusion or even formal investigation. We don’t know exactly how the NSA or GCHQ [Britain’s Government Communications Headquarters] use algorithms — or how extensively they’re applied, but we do know they use them, including on the huge data trawls revealed in the Guardian,” Ball said.