The Human Iceberg

Megan McArdle ascribes the cause of Margaret Thatcher's biggest debacle to a string of successes. She was right so often it blinded her to the possibility of being wrong. She gained so much confidence in her own judgment that she had little defense against inevitable error.

After a string of successful privatizations, often conducted over the vocal objections of all the best people, U.K. Prime Minister Margaret Thatcher’s Conservative Party decided it needed to reform the taxation system ... This would not only please the folks who ran local party operations, but also strike a blow against overbearing left-wing councils, which in some northern cities were run by actual Communists.

The reform they came up with was known as the Community Charge, a per-capita tax that we would call a head tax, and in Britain was popularly known as the Poll Tax. The riots that this new scheme sparked were therefore known as the Poll Tax Riots...

The takeaway, as tired as it sounds, is that negative feedback is often more powerful than positive feedback. Organizations have a tendency to ignore Dr. No or, worse, fire him. But silencing your critics is a good way to set yourself up for total disaster. That doesn’t mean letting critics shut you down. But it does mean taking everything they say as seriously as the rah-rah sentiments of the policy cheerleaders.

It's hard to admit error, especially where you might be plausibly right; especially when you've long been right. When the fall comes, all you know even after the fact is that some terrible screwup has happened, though it is unclear at first exactly why.

Peter Lanza, the father of the Sandy Hook killer, reflecting on his role as a father, told New Yorker writer Andrew Solomon he clearly should have done something differently in raising his son but can't say what. “Any variation on what I did and how my relationship was had to be good, because no outcome could be worse”.

Yet the two things Peter Lanza believes beyond a doubt are 1) that he escaped death by quitting Adam Lanza's presence: “with hindsight, I know Adam would have killed me in a heartbeat, if he’d had the chance. I don’t question that for a minute. The reason he shot Nancy four times was one for each of us: one for Nancy; one for him; one for Ryan; one for me”; and 2) that nobody could see it coming. “Those mental health professionals who saw him did not see anything that would have predicted his future behavior. Here we are near New York, one of the best locations for mental-health care, and nobody saw this.”

Nobody saw it coming, except maybe himself. He had wondered whether there was something more than Asperger's behind Adam's weirdness. But the pros said no. Casting his mind back, the mass killer's father says there was something evil or broken in that boy. “You can’t get any more evil,” he said. There was something he glimpsed, even today as Adam comes to him in dreams.

Peter has dreamed about Adam every night since the event, dreams of pervasive sadness rather than fear; he had told me that he could not be afraid of his fate as Adam’s father, even of being murdered by his son. Recently, though, he had had the worst nightmare of his life. He was walking past a door; a figure in the door began shaking it violently. Peter could sense hatred, anger, “the worst possible evilness,” and he could see upraised hands. He realized it was Adam. “What surprised me is that I was scared as shit,” he recounted. “I couldn’t understand what was happening to me. And then I realized that I was experiencing it from the perspective of his victims.”

What was beating on the dream-door? What do you call it? Once my son showed me an short story he wrote about an android who gradually realizes that he is both artificial and sentient. Once an impulse I asked, "do you think that HAL 2000 could be truly self-aware without the acquiring the capacity for murder?" He thought for a half second and said: "No. Any real person will have the capacity for evil".

Which is what the Bible used to teach, though we have since forgotten it. However the programmers are rediscovering the notion of evil as inseparable from true sentience. Late last year Gary Marcus, also writing in the New Yorker, quoted sources who argued that truly self-aware computers would value themselves over their creators. Build a sufficiently intelligent machine and it would worship itself above all else.

Barrat’s core argument, which he borrows from the A.I. researcher Steve Omohundro, is that the drive for self-preservation and resource acquisition may be inherent in all goal-driven systems of a certain degree of intelligence. In Omohundro’s words, “if it is smart enough, a robot that is designed to play chess might also want to be build a spaceship,” in order to obtain more resources for whatever goals it might have. A purely rational artificial intelligence, Barrat writes, might expand “its idea of self-preservation … to include proactive attacks on future threats,” including, presumably, people who might be loathe to surrender their resources to the machine. Barrat worries that “without meticulous, countervailing instructions, a self-aware, self-improving, goal-seeking system will go to lengths we’d deem ridiculous to fulfill its goals,” even, perhaps, commandeering all the world’s energy in order to maximize whatever calculation it happened to be interested in.

That description of a homicidal computer sounds remarkably like Adam Lanza. His mind was maximizing something, but nobody knew what. He was the center of his own universe, a population of one, where nobody else counted. Peter Lanza was probably right in believing he too would have been shot had he been in the way. Of what exactly? The thing knocking on the dream-door. He recalled what his wife Nancy said:

“He had a horrible night. . . . He cried in the bathroom for 45 minutes and missed his first class.” Two weeks later, she wrote, “I am hoping that he pulls together in time for school this afternoon, but it is doubtful. He has been sitting with his head to one side for over an hour doing nothing.” Later that year: “Adam had a rough night. He moved EVERYTHING out of his room last night. He only kept his bed and wardrobe cabinet.”

The probable fact is that, for any given population, there are a number of Adam Lanzas; human icebergs afloat upon the population sea who, because of defective wiring or some poorly understood process, become what Peter Lanza called "evil". Most people will never encounter an iceberg because of their relative rarity. But when one does blunder into a Ted Bundy, a Charles Manson or Adam Lanza there's only minutes to turn the wheel and reverse engines before things get dicey.

There are known countermeasures to icebergs. We detect them, avoid them, route our way around -- but we don't expect icebergs not to exist. Nancy Lanza, who loved her son, made the cardinal mistake of thinking she could talk to the block of ice.

There is at the heart of human experience, the unknowable. That which we must beware. Donald Rumsfeld once remarked, "reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know that we know. There are known unknowns; that is to say, there are things that we now know we don't know. But there are also unknown unknowns – there are things we do not know we don't know."

We think it is shameful not to know. And vast numbers of political hacks are employed to revise and parse politicians words to make them appear as omniscient and error-free as possible. But it is far more dangerous to assume we know everything. We don't. And maybe that's why we still can't find MH370. One commenter at the Professional Pilots Rumor Network speculating on the MH370 disaster remarked, "Looking at the ever-widening search area, the logical conclusion is no, they don't have any useful primary/military radar plot" to yield an initial position.

The Searchers didn't know what they assumed they knew. Which means, they have to look with open eyes and an open mind, and there's no shame in that.

Did you know that you can purchase some of these books and pamphlets by Richard Fernandez and share them with you friends? They will receive a link in their email and it will automatically give them access to a Kindle reader on their smartphone, computer or even as a web-readable document.

The War of the Words for $3.99, Understanding the crisis of the early 21st century in terms of information corruption in the financial, security and political spheres