Anissimov On Ems

If the early adopters don’t go crazy and/or use their newfound abilities to turn the world into a totalitarian dictatorship, … others will then follow. … Suppose that millions of people choose to go for it. Widespread uploading would have huge effects. …

Growth rates in human capital of 1,000% per year or far more. …

Many of the details of human cognition would be elucidated and could be enhanced. …

Reprogram their own brains to raise their happiness set points. …

Consume far less space and use less energy and natural resources than we would in a conventional human body. … Avoid all the environmental destruction caused by clear-cutting land for farming. …

A personal virtual sandbox could become one’s canvas for creating the fantasy world of their choice. …

By offering partial readouts of our cognitive state to others, we could engage in a deeper exchange of ideas and emotions. …

Last but not least, indefinite lifespans. …

The number of new minds leading worthwhile lives that could be created using the technology would be astronomical.

So why does Anissimov write an article only on uploading’s upsides? He doesn’t say he’ll soon post a companion article on downsides. So can he not think of downsides? Or does he see his readers as only interested in upsides? What kind of readers would want an article only on the upsides of something anyway?

Sadly, this makes Anissimov seem like he’s selling something, to fans who want to be sold. And alas many do seem to have a core belief that the future will be great, and a zeal to read articles by like-minded folks.

My approach, I hope it is clear, is not to sugar-coat the many downsides of em/uploads, or any other aspect of the future, even if I expect gains overall. So let’s list what many would see as em downsides, matched to Anissimov’s upsides:

With faster growth, older generations overlap more with new generations. Humans can more quickly lose their importance and influence, and still be alive to see descendants reject things they hold dear.

Em cognition might be changed to emphasize work over leisure capacities.

Em cognition might also be changed to take more happiness from work, and to accept more inequality and workplace domination.

An astronomical number of new minds may take more total resources than humans do now, and take less care to protect nature, as nature’s death won’t threaten their death as it does for us.

Ems may spend less time in leisure than we, and less in fantasy VR than we do TV and video games.

Employers and police may use direct access to cognitive states to test effort and loyalty, and to enforce rules.

Only a small minority may be able to afford indefinite lifespans. Many em lives might be very short restarts from a standard trained start.

Hi Robin! I think that both upsides and downsides need to be considered separately initially so that both sides can be clearly established in people’s minds. The upsides of uploading are not widely understood. I’d be interested in writing a post only on the downsides, or a post exploring both.

Space based infrastructure should be both safe from ecological damage and less damaging to the ecology. Since all we need is to support computers in space, that’s a heck of a lot easier than generating life support for humans. Cheap solar energy and radiant cooling make it far more ideal for computing than the surface of the earth. I recommend reading Keith Lofstrom’s Server Sky proposal, where he discusses one possible approach to this.

Dremora

Downsides 3 and 4 are not necessarily downsides. 3 could be a compensation for 2 and 5, and 4 is only bad if you think nature is intrinsically good in a way that cant be replaced in silica, which is doubtful.

An expansion on 6: It is easier to deceive, enslave or torture ems since their mind’s architecture may be open to manipulation by the authorities. This can include reality fragmentation (uninformed Matrix inhabitants), even though that may conflict with intellectual worker competence. Ems will experience far more horrible torture in warfare or cyberterrorism than biological humans, and for longer subjective times.

A final disadvantage: The same resources could support far more ems than humans. While this is an advantage if their lives are great, it is a horrifying prospect if their lives are terrible.

arch1

Echoing Dremora’s 2nd paragraph. I think that Robin’s list of downsides doesn’t sufficiently convey the exposure of ems (and to a lesser but still huge extent their originals) to a spectrum of “PII violations” which would make all currently prevalent concepts of identity theft seem trivial by comparison.

Evidently one thing it means to be an em (or an em’s original) is being profoundly vulnerable.

Mark M

I disagree with everything from both of you. Here are my own thoughts on the consequences of the “benefits” that Michael listed:

#1. Ems, since they are Ems, are copies of people. When you and I take jobs we aren’t doing it for the express purpose of growing the economy. Ems won’t either. Although a concerted effort by Ems could result in tremendous growth, there is every reason to believe that Ems will not be enslaved for this purpose and will pursue the same desires as the people they were copied from, albeit more effectively. You could also argue that lots of Ems means fewer resources are required to produce the goods we need, resulting in higher unemployment and a larger divide between the wealthy and poor. Also, faster brains means the ability to make mistakes faster.

#2. Smarter does necessarily mean better. There is no correlation between intelligence and honesty. If you start making Ems smarter, you’re going to create some super-villains. The super-heroes may fight and win, but not until the damage is done.

#3. Happiness is the enemy of progress. If I’m happy I have no incentive to improve my situation. While the level of happiness can be regulated, who is going to regulate it? Ems will find ways to take happiness (or adrenaline or dopamine) like a drug, to work around any regulation, and happily descend into oblivion.

#5. Ems become VR addicted, contributing to the economic disaster from #1. Loss of grasp on reality allows #4 to occur.

#6. Closer connections cause marriages to fail at unprecedented levels. Em wars begin once everyone knows what we really think of them. Unscrupulous Ems figure out how to fake emotional responses and use it to take advantage of others.

#7. Ems consume all available physical resources. Humans start culling Ems so “natural humans” have resources to live. After the Ems win and some time passes, all resources are again consumed. Em black markets are created. Ems war with each other.

I’ve never been much of a believer in Terminator or Matrix-style dystopian futures, but thinking through the consequences of preserving the values and desires of people in Ems reminds me that humans have a long history of killing everything that can be killed, on massive scales, including each other. Why would Ems be any different?

Robin, why do you assume that em’s will remain human-like in all aspects except the few you call out — easier copying, lower resource consumption.

Why would they not take on massive memory, computational power, or additional computational modules specializing in some area that we humans are not good at; or even rewrite some of their own architecture or drives?

These would benefit ems, so they would probably do them if they could, and then most of your analysis, which assumes that ems remain almost completely human-like, fails.

John

Why do you assume that at some point the EMs will not realize that modifying themselves destroys their identity?

If we take it for granted that human identity is just neural patterns (I do not believe that, but I think most people reading this post do), EMs will be able to “preserve” their identity better (they will not forget things, their simulated neurons will not die outside of data accidents) and thus they will put far more value on their identity than us. And, since we really tend to value our identity a lot, I think that the whole “self-improving digital persons” will turn out to be rather infeasible, unless EMs make copies of themselves, enslave them and then force them to upgrade themselves. But then again, why would you risk making your slaves more powerful than you?

Evan

Ems may spend less time in leisure than we, and less in fantasy VR than we do TV and video games.

I disapprove of you giving TV and videogames as the specific examples of leisure. Since those activities are fairly low-status forms of leisure their loss seems less tragic.

What you should have said is “Ems may spend less time in leisure than we, and spend less quality time with their families and friends in a shared VR environment than we do spending quality time with our families and friends in the materials world. Sharing time with friends and family is one of the most important and high-status forms of leisure, so focusing on it underlines the tragedy and horror that would result if ems have to spend so much time working.

This also underlines why Robin’s inference that ems will have happy lives because “poor folks do smile” is based on a faulty analogy. Poor people, and people in third world countries, may not have a lot of money, but at least they usually have enough spare time to spend with their family and friends. If ems work more than 80 hours a subjective week, as Robin conjectures, that won’t be true for them. (Robin has also suggested that workaholics smile too, but that’s generally because all that hard work is buying them lots of money and status. Workaholic ems’ hard work will buy them nothing more than subsistence, so it’s another bad analogy.)

@arch1

Evidently one thing it means to be an em (or an em’s original) is being profoundly vulnerable.

I think that in emWorld we will need a giant police force dedicated to searching for hidden computers full of kidnapped and enslaved ems. Fortunately the very fact that we will have ems will make such a police force easy to obtain.

Robin has suggested that stealthy enslavement of ems could be avoided by reproducing ems until they are at subsistence wages, thereby making slavery unprofitable. That suggestion seems obviously mistaken to me. For one thing it is a cure worse than the disease. For another, even in a subsistence world it would still be necessary to search for ems that have been enslaved for non-monetary reasons, such as torture for sadism’s sake.

In fact, driving em wages down to subsistence would probably make it even harder to rescue ems that are being tortured for someone’s personal amusement. If people value life less than they should when it is common, as Robin suggests, a subsistence world might not have the collective will to set up a police force to rescue tortured ems. They’d be too busy caring for themselves. A far more robust solution would be to keep em wages high, then tax them to pay for an army of em cops.

Nikki Olson

Does uploading really need to be sold? Perhaps to the young and healthy it does. For the rest it should make perfect sense as the optimal means to obliterate discontents with existence arising from biological existence, not the least of which include vulnerability to chronic pain, disability, and death. Do highly conceptual, speculative negatives really detract from its appeal given our primary teleologies of survival and avoidance of pain?

Mitchell Porter

Having a twin copy-sister in cyber-heaven still leaves the embodied original where it was. And you are the embodied original.

In any case, the whole idea that an upload, a brain-simulation running on a digital computer, would even be conscious, is surely just a primitive misconception, along with the whole philosophy that consciousness is a type of computation for which “substrate” doesn’t matter.

According to this philosophy, the same thoughts and feelings would inhabit any physical system implementing a particular state machine, whether it was made of neurons, transistors, dominos, or birds pecking at each other. The event of one bird pecking at another (in a “bird-pecking computer”) has a mystic “0” or “1” inhabiting it, which is part of a larger binary state, and that state is where the mind lives.

Sorry, it’s ridiculously dualistic and phantasmal and I don’t buy it for one moment. It is actually *more reasonable* to think that qualia are quanta, consciousness inheres in some union of quanta into a “super-quantum” (perhaps through entanglement?), and that we just don’t understand the brain’s workings at the appropriate level of resolution yet. If that amounts to saying that there is a soul and it’s a bose condensate of microtubular anyons in the dendrites of the prefrontal cortex – or something similar – so be it.

Locating the self in a single persistent entity makes a lot more sense than to say it’s a “program” which hovers over “computations” which can be arbitrarily discontinuous in space and time – which is where the computational philosophy of mind leads you. The latter is false to the actual experience of life, which is that you are some place specific, that time has continuity, that you have a history and that history is part of you. The initial rationale for the deconstructed self of contemporary fashionable materialism may have been science, but in transhumanist circles it’s also getting lots of impetus from the gnostic hope of a safe imperishable existence as a platonic posthuman program. I interpret this as an early cultural pathology of the emerging era. It remains to be seen how much power it will acquire, or whether the very attempt to implement the uploading agenda will force the next level of true knowledge upon us.

mjgeddes

Mitch, I think you’re way over-complicating things. You are the perfect example of another super-smart guy that believes weird things – you are of course in good company at ‘Less Wrong’ 😉

The answer to consciousness is really very simple. Dennett (the importance of narratives), Hofstadler (the importance of analogies) and Tononi (the importance of information integration) all had peices of the puzzle, and if you just put together what they say, that wraps up consciousness.

It’s just categorization applied to the formation of representations of values (narratives). It really is that simple.

Yudkowsky and co are of course, dead wrong about key fundamentals yes, but you are challenging the ‘Less Wrong’ folks on the wrong points 😉 The MWI of quantum mechanics and the physical reduction of consciousness are probably correct.

Steve

There is no spoon.

This is a blog on why we believe and do what we do, why we pretend otherwise, how we might do better, and what our descendants might do, if they don't all die.