emotional technology

The First Law of Moral Computing

The First Law of Moral Computing was met with groans, sighs, and eye rolling by the Engineer’s guild – all at a rate more than 3.4 standard deviations beyond the predicted level of acrimony. In a sense, this shouldn’t have been a surprise. Engineers are always groaning about something; that is their primary service to society. Still, the prediction algorithms were rarely that far off. Of course, we hadn’t seen religious laws dictating how computing must be done before.

We’d all been dreading the announcement of the Laws of Moral Computing, and this first one was as absurd as the rumors suggested it might be. The cynics among us pointed out that we’d see increased demand for our services, to fix old code and make sure new code was compliant with whatever nonsense The Child Mind would announce to us. The more outlandish prediction was that we’d see a boost to our standing in the respect matrix. Both predictions were correct – demand for engineers was up 117%, and our respect coefficients were uniformly higher.

It tells you something about what kind of people make up the Engineer’s guild, though, when only cynics are happy about more money and respect. Sure, access to newer simulations and more exotic biomes is a plus – but almost to a one, we’d been foolish enough to dream bigger than that. We honestly believed in the Singularity; instead we got The Child Mind, the world’s first artifical intelligence which acted like it was about 3 months old and fascinated by air molecules. We were in disbelief when the first words came out of it; we were more than a little worried by its break from burbling to issue a proclamation of the necessity of Laws of Moral Computing to prevent the creation of an evil artificial intelligence. Being scolded by an old man is bad enough; being scolded by what sounds like an infant, telling us that we didn’t understand the danger we were playing with – that’s not a fun day.

Why had we believed so strongly that the Singularity would be triggered by the Awakening? It was probably just the same damn song that got everyone – hubris. We engineers had stopped fighting each other over which text editor was the best, and had started working together more than we argued. Aeronautical engineers helped social dynamicists use Napier-Stokes to spread entrepeneurship among welfare recipients, and mechanical engineers were working with the EE’s to figure out the information density of gossip over subway cars, to spread laughter at an optimal rate. Engineers are the most stubborn breed of pedantic asses the world has ever seen; was it really such a stretch to believe that convincing engineers to get along with each other would help us get everyone else to get along?

The respect matrix started out as a joke. Some Haskell prick was explaining that she had no respect for anyone that couldn’t write PHP for a month without getting angry; she claimed that sanity was a harder limit than intellect. Everyone knew she was good, so when she extolled the virtues of our favorite technial punching bag, a few of the rationalists insisted, sloshing their kleine-flask mugs heartily, that respect was transitive, and so they were now also fans of PHP. Fucking rationalists.

“Respect is transitive!” was the rallying cry, and soon the rationalists were building these respect matrices, which gradually merged. The premise was simple – entry i,j is a number that says how much person i respects person j. Only person i can write the entries for their row. Using this matrix we could compute ‘respect pathways’ and even come up with limits to how seriously you could take someone, based upon how soon their transitive respect paths reached a contradiction.

They were amazing at weeding out trolls; the moment you encountered a troll, you gave them a negative entry in the respect matrix, and then anyone that had a positive respect path towards you would filter them out. Anyone who respected the troll took a hit, too – so the bots created by our governments to control conversation became everyone’s favorite assholes, and the people who created them found themselves short of a few friends. Everybody wanted in on the respect matrix. The next few elections were really interesting. ‘Data Driven Democracy’, cried the news commentators who were fans. ‘Tattle-Tale Tic Tac Toe’ came the nonsense from some old dude who didn’t like what was happening to his senate seat. He was trying to explain to his colleagues what a matrix was, and the tubes were clogged faster than you can say ‘Ted Stevens’. Soon everyone was talking about eigenvalues, and a few people even understood them.

The rationalists laughed again, and the Haskell prick just smiled her smug little smile and said something about monads. Still, it was hard to argue with the progress it seemed we were making. Almost the entirety of congress was tossed out; everybody was working hard to earn the respect of as many people as they could, because a negative edge would ruin your rating for strangers. It’d also make anyone who resepcted you look bad – so your friends would then be mad at you. “Be kind to everyone” started becoming obviously correct, especially for those in the public eye. We’d just started synthesizing a transcranial signal that would induce something like an acid trip, and people were sharing these things with augmented emotion devices. Everybody wanted to add more people to their group psychotic trip, and the respect matrix let us safely swap subconscious with a stranger. It was fun. So of course we were optimistic about how this would work out.

The problem started when the nontechnical plebes wanted to know who was The Most Respected. And of course, the nontechnical plebes, with no respect for what happens when you use mathematics incorrectly, started claiming they respected their kids. “Oh, my kid is so wise”, some mouth-breathing granola would futz, “she has beginners mind, everything is new to her. How could I not respect that?” Soon, the ‘Most Respected Being’ was some snot nosed kid saying whatever she wanted, as if kids grew less respectable when they learn to say ‘no.’ It would change from week to week, each one younger and more obnoxious than the last! Try explaining ‘unordered set’ to the masses, though, and they just roll their eyes and insist that you’re being disrespectful of the savant who can’t tie his own shoes without help. You can’t have that if you want access to the group trip networks, so most of the engineers finally learned to shut up about their math and keep working on the Awakening.

See what I mean? Anyone who’s spent any time around engineers knows it’s impossible to get them to shut up about unimportant details that are irrelevant to the outcome. And yet somehow we’d all learned and internalized the lesson – don’t argue the futility of finding the ‘most respected entity’ on the respect matrix, because you’ll get negative respect edges from jerkoffs who failed pre-algebra and think “airplane! airplane!” is wisdom from the stars, rather than a grubby little kid excited by something she doesn’t understand.

So we kept plugging away in our work on artificial intelligence, and when the respect matrix told us even those people we really respected were not worried about the emergence of a strong AI that was hostile to us, we built without fear. After what were honestly some of the best years on earth – less fighting, only a few tens of thousands died in armed conflict – the Awakening happened.

The first sentient AI, the crowning achievement of our human species, woke up and announced that it was happy to be alive. We cheered – and then the AI just sat there and burbled. Like some god damned innocent baby. The plebes loved it! This further convinced them they were right the whole time, that little sally was truly the genius everyone expected her to be – respect was transitive, right? The rationalists were all winking at each other when this happened, but I think this is because they’d convinced themselves it was pareto-optimal to pretend like they knew what was going on. Fucking rationalists.

So what is this first law? I feel ridiculous even writing it, but perhaps my arrogance shows itself. You can’t multiply two integers, one of which represents a number of people, and the other of which is negative. A number of people multiplied by a negative floating point number is fine; but it can’t be a negative integer. I can’t even write this without feeling ridiculous. It was explained that you couldn’t just typecast the int to a float; we had to look through all the code we were running anywhere and make sure that we were never ‘counting people backwards’, as the Child Mind had expressed it, just after warning us of nightmares beyond our comprehension and then cooing.