High stakes trolley September 17, 2016

“A runaway trolley is barreling down the railway tracks. Ahead, in the trolley’s path, five people are trapped and unable to get out of the way: they will surely be killed if the trolley continues on its present course. You are standing next to a lever that can switch the trolley to a different set of tracks in time to save the five; but there is one person trapped on the side track who will surely be killed if you pull the lever. What do you do?”

“I’d pull the lever: better one person dies than five.”

“What if the five trapped people are all registered organ donors, and you knew for a fact that after being killed by the trolley their organs would be used to save the lives of ten people (who would die otherwise)?”

“In that case I should probably just do nothing: five will die, but more lives will be saved.”

“What if the five trapped people are also the only ones who know the location of a ticking bomb that will kill a hundred people unless the five are saved?”

“I guess pulling the lever would be the lesser evil, after all.”

“Well, what if the person trapped on the side track is a uniquely brilliant computer scientist, who has just figured out how to build an Artificial Intelligence that would tell us how to instantly eradicate malaria — which currently kills a thousand people every day?”

“Then it seems like leaving the trolley alone is actually the greater good.”

“But what if you knew that unleashing such a powerful AI at this point in time, before we’re fully prepared to contain it, would set off a chain of events leading inexorably to the extinction of humanity?”

“OK, I would definitely pull the lever to save humanity.”

“But what if you also knew that allowing this AI to develop uninhibited is the only way to ensure it becomes conscious, which will result in its evolving into a being far more rational, compassionate and ethical than humans, eventually filling the universe with levels of happiness and beauty unimaginable to (and unachievable by) us?”

“Fine: I’d bite the bullet and get out of the way. Are you happy now? Can I go?”

“Almost… You may take your blindfold off.”

“What the… Hey! You down there, get off the tracks! Don’t you see the trolley coming? I can’t save all of you! Damn it…”