Monthly Archives: April 2012

In this article, I’ve focused on ideas that are just over the horizon for most of us. So green roofs, LED lights, genetically modified crops, 3D printers and geo-engineering are important, but improvements are likely to be steady rather than seismic. Instead, I’ve chosen “disruptive ideas” that could radically affect the way our society operates. … 1. Driverless electric cars … 2. Space elevators … 3. Nanotechnology … 4. Ubiquitous Data … 5. Machine Intelligence …

He has many thoughtful policy comments on the first four topics, but when he gets to machine intelligence, he throws up his hands:

A machine that can emulate the human brain would challenge all occupations, from hairdressers to architects. In the case of this science breakthrough, it’s hard to even begin to think how policymakers would respond. Do we limit how many times you can replicate yourself? If we have a machine that contains your memories and can think like you, shall we treat it like a slave or pay it a wage? Do you have the right to turn off copies of yourself? Will this breakthrough cause wages to fall? If so, how do we make sure that everyone has some capital to get by? After thinking about Hanson’s work for a few weeks, I’ve decided that this is one breakthrough for which I don’t want to be around.

Leigh’s attitude makes sense to me. After spending years getting expert in thinking about good policy for this our industrial era, Leigh can see that ems are a whole new era where policy must be re-thought, starting back from basics. He doesn’t want to do that – he’d rather build on the expertise he has acquired to attack our many important industrial era problems. I hope he succeeds at that.

I also hope that when people like me do think through em policy more carefully, people with Leigh’s good sense and connections will listen.

Morality should exist; … there should exist creatures who know what is moral, and who act on that.

Many commenters disagreed, yet today I will go further:

Morality should be adaptive; it should help groups survive.

Humans evolved moral feelings as an adaptive response to difficult coordination problems in forager communal living. Culture tweaked those feelings to better fit farming life. Related feelings in other animals evolved for related reasons. So morality evolved to help us survive, and it has been intricately but not infinitely matched to that purpose. If, after a sudden unexpected change in our environment, we apply that morality in such a way as to make ourselves go extinct, that seems a rather dysfunctional broken application of such morality!

Our moral feelings are crude and imprecise – they can have error. Given how complex is our world and crude our minds, and given how weird is our modern world relative to our evolved expectations, we should expect a lot of error. We should not blindly follow our moral intuitions, but should instead correct them as best we can whenever we can estimate a non-zero net error. And if your intuitions suggest that people like you should go extinct, well seems like a pretty damn big clue of error. Of a BIG error. Correct!

Added 4p: The evolutionary context of our moral intuitions gives a rich detailed framework for defining and estimating moral error. If you reject that framework, the question is what other framework will you substitute? How do you otherwise define and estimate the error in your specific moral intuitions?

In the New Yorker and on NPR, Bryan Caplan’s views on kids have recently been contrasted with ethical arguments against having kids – that the possibility kids might suffer outweighs all their likely joys and benefit to others. Now while I could understand some obscure academics or oddball activists taking this position, I find it bizarre to see it taken seriously in the mainstream media.

I mean, really, the whole human race should go extinct to avoid the risk that some future kid might suffer at some point?! And since the same argument applies to non-humans, all life should go extinct?! How could that ever be a remotely acceptable mainstream position? Cryonics is silly, and that is not?!

Yes I know I cannot refute this claim with just an incredulous stare, so let me suggest a moral axiom with apparently very strong intuitive support, no matter what your concept of morality: morality should exist. That is, there should exist creatures who know what is moral, and who act on that. So if your moral theory implies that in ordinary circumstances moral creatures should exterminate themselves, leaving only immoral creatures, or no creatures at all, well that seems a sufficient reductio to solidly reject your moral theory. I’m not saying I can’t imagine any possible circumstances where moral creatures shouldn’t die off, but I am saying that those are not ordinary circumstances.

If the early adopters don’t go crazy and/or use their newfound abilities to turn the world into a totalitarian dictatorship, … others will then follow. … Suppose that millions of people choose to go for it. Widespread uploading would have huge effects. …

Growth rates in human capital of 1,000% per year or far more. …

Many of the details of human cognition would be elucidated and could be enhanced. …

Reprogram their own brains to raise their happiness set points. …

Consume far less space and use less energy and natural resources than we would in a conventional human body. … Avoid all the environmental destruction caused by clear-cutting land for farming. …

A personal virtual sandbox could become one’s canvas for creating the fantasy world of their choice. …

By offering partial readouts of our cognitive state to others, we could engage in a deeper exchange of ideas and emotions. …

Last but not least, indefinite lifespans. …

The number of new minds leading worthwhile lives that could be created using the technology would be astronomical.

So why does Anissimov write an article only on uploading’s upsides? He doesn’t say he’ll soon post a companion article on downsides. So can he not think of downsides? Or does he see his readers as only interested in upsides? What kind of readers would want an article only on the upsides of something anyway?

Sadly, this makes Anissimov seem like he’s selling something, to fans who want to be sold. And alas many do seem to have a core belief that the future will be great, and a zeal to read articles by like-minded folks.

My approach, I hope it is clear, is not to sugar-coat the many downsides of em/uploads, or any other aspect of the future, even if I expect gains overall. So let’s list what many would see as em downsides, matched to Anissimov’s upsides:

With faster growth, older generations overlap more with new generations. Humans can more quickly lose their importance and influence, and still be alive to see descendants reject things they hold dear.

Em cognition might be changed to emphasize work over leisure capacities.

Em cognition might also be changed to take more happiness from work, and to accept more inequality and workplace domination.

An astronomical number of new minds may take more total resources than humans do now, and take less care to protect nature, as nature’s death won’t threaten their death as it does for us.

Ems may spend less time in leisure than we, and less in fantasy VR than we do TV and video games.

Employers and police may use direct access to cognitive states to test effort and loyalty, and to enforce rules.

Only a small minority may be able to afford indefinite lifespans. Many em lives might be very short restarts from a standard trained start.

A lot of the press on Tyler’s new book has focused on his suggestion to avoid restaurants with pretty women:

Beware the Beautiful, Laughing Women

When I’m out looking for food, and I come across a restaurant where the patrons are laughing and smiling and appear very sociable, I become wary. … Many restaurants, especially in downtown urban areas, fill seats—and charge high prices—by creating social scenes for drinking, dating, and carousing. They’re not using the food to draw in their customers. The food in most of these places is “not bad,” because the restaurant needs to maintain a trendy image. … I also start to worry if many women in a restaurant are beautiful in a trendy or stylish way. The point is not that beautiful women have bad taste in food. Instead, the problem is that they will attract a lot of men to the restaurant, whether or not the place serves excellent food. And that allows the restaurant to cut back on the quality of the food. … When you enter a restaurant, you don’t want to see expressions of disgust on the diners’ faces, but you do want to see a certain seriousness of purpose. … This review on Zagat.com says it all:

One of my favorite places in DC—awesome lounge, great decor, and food is delicious.

Matt thought he disagreed, but Tyler clarified. It seems to me that people focus on this issue because it is a veiled insult. Chuck Rudd says it more directly:

Initially, I rebelled against Cowen’s implication that men have unrefined palates or that they just don’t care about food quality. I don’t want to make some sort of gender issue out of it, but his argument implies that these trend-seeking women’s palates are unrefined as well. (more)

Notice that the claim is that places with more pretty women cut back on food quality, but not on decor, location, or service quality. So it isn’t that places just generically slack off when they are more popular. It must instead be that pretty trendy people, compared to other people, can less distinguish or less care about food quality, relative to other types of quality. And since food quality seems harder to observe that decor, service, etc. quality, the implication is that pretty trendy people are more shallow, i.e., less discerning about or interested in harder to observe qualities.

Sounds plausible, though, since I don’t get many offers to hang out with pretty trendy people, I don’t have first hand evidence one way or the other. I’m open to chances to collect evidence though. You know, in case any of you pretty trendy people have a slot open …

Note that Tyler probably got more attention for a veiled insult than if he had insulted directly. Homo hypocritus delights in indirectly jockeying for status and support.

The Cambrian explosion … was the relatively rapid appearance, around 530 million years ago, of most major animal phyla, as demonstrated in the fossil record, accompanied by major diversification of organisms including animals, phytoplankton, and calcimicrobes. Before about 580 million years ago, most organisms were simple, composed of individual cells occasionally organized into colonies. Over the following 70 or 80 million years the rate of evolution accelerated by an order of magnitude (as defined in terms of the extinction and origination rate of species[4]) and the diversity of life began to resemble that of today. (more)

Now that humans have pioneered powerful innovations in law, tech, and organization, the obvious long-term future to expect is a diversity in use of those innovations: our descendants will radiate out in feature space to fill a wide range of niches, not only on Earth, but under and above it as well. Just as Cambrian explosion descendants shared common cell tech and structures, our descendants may mostly preserve some key features of human minds and societies far into the coming explosion. But that still leaves room for a vast diversity.

Since our society tends to give lip service to celebrating diversity, it can also give lip service to celebrating this future diversity. But humans also tend to be wary of inequality. Foragers were especially vigilant to prevent some of them from overtly dominating others, and while farming and industry cultures have led us to tolerate more inequalities, we aren’t especially happy about it.

This is a problem because it is very hard to imagine a Cambrian explosion level of diversity among our descendants without a lot more inequality. For plants or animals today, pick most any dimension along which you want to call some “better” than others, and you’ll find a wide variation — some are a lot better than others. Pretty much the only dimension in which all existing species are near “equal” is survival – all have survived. But of course they are a tiny fraction of winners vastly outnumbered by all the dead loser species.

Thus our descendants are likely to differ greatly from one another on most all imaginable dimensions, including dimensions of value, where some are called “better”. The only ways to prevent that is either to destroy all descendants, or for a central power to seize control of this process and impose a concept of equality favored by those who control it. And if you supported an attempt to seize central control on this basis, you’d risk folks with other agendas seizing control of this central power base.

While such a central control attempt might make sense someday, when we have learned better ways to coordinate, for now I think we have to accept that the future will come with both great diversity and great inequality – and that we can’t really have much diversity without a lot of inequality as well.

When we look at a picture, we see the far away landscape more in far mode, and the central nearby objects more in near mode. This suggests that our peripheral vision tends more to see things in far mode:

It seems that we get information on what type of scene we are looking at from our peripheral vision. We process the “gist” of what we are looking at from our peripheral vision. The researchers at Kansas State showed people photographs of common scenes, for example a photograph of a kitchen or a living room. In some of the photographs the outside of the image was obscured, and in others the central part of the images were obscured. The images were shown for very short amounts of time. Then they asked the research participants what they were looking at. Peripheral vision was more important – What they found is that if the central part of the photo was missing people could still identify what they were looking at. But when the peripheral part of the image was missing then they couldn’t say whether it was a living room or a kitchen. (more)

I recently found fault with Keith Henson’s assumption that sexual competition would induce ems to run as fast as physically possible. So how fast do I think ems would run? Here is my current analysis:

Em speeds should intersect supply and demand. Speed supply comes from how em hardware (e.g., device, energy, and cooling) costs vary with speed. Since human brains use a very parallel design with cells whose signals change far slower than electronic circuits, the cost of em hardware should be roughly linear in em speed across a wide range, to a very fast max, perhaps a million times faster than humans. In this range, thinking twice as fast costs about twice as much.

Above that linear regime, a 1% speedup will add more than 1% to costs, with this speed premium approaching infinity at a maximum feasible speedup, perhaps a factor of a billion. Very slow ems should also suffer a cost premium, as they’d still need to store a mental state.

With compatible hardware, brief speed increases might be cheap if em brains have substantial heat capacity. Longer but still temporary speed changes might be made by swapping into different brain hardware, though this could have substantial switching costs.

On the demand for em speed, I see seven relevant factors:

When physical systems have natural resonance periods, managing those systems suggets em response times near the shortest of those periods. For example, since small moveable human body parts have resonance periods of a fraction of a second, human brains have reaction times on that time scale – reacting faster might help sometimes, but costs too much. Ems with smaller human-like bodies would want faster brains to match their shorter periods.

Ems that talk often would benefit from having similar mind speeds. This would create a tendency for em speeds to clump at common standard speeds. Ems that talk often to humans would have near human speeds. Ems with highly mismatched speeds could talk naturally if the slow one temporarily moved to faster mental hardware.

It is awkward for ems to talk when there are substantial communication delays. For any given distance to em conversation partners, there is some max speed above which delays are noticeable and hence costly.

It is tempting to use faster ems to speed up any project whose duration might take a substantial fraction of the economy’s doubling time, or where there is a race with competing projects. Of course project durations may be limited by factors other than em thinking speeds.

The more important is a negotiation or argument between ems, the more private gains can come from having a faster em mind, to out-think the other ems. So in hierarchical organizations, higher level leaders would have faster minds.

When it is useful to coordinate two different tasks, one could either have two ems do the two tasks and talk periodically, or have a single faster em do both tasks. A single em doing both tasks probably has skills less well matched to those tasks, and would pay extra costs to switch between tasks. But when task coordination is important enough, these can be prices worth paying.

When it is important to minimize the time a worker is away from their tasks at leisure and sleep, it will be tempting to run those non-work activities very fast. This could allow near continuous time coverage of a task.

Thus while some ems will have speeds to match the physical systems they manage, and ems would be faster at sleep, leisure, on thinking-dominated projects, and at high organization levels. The speed of other ems would be set more by how important is coordination for their tasks, and em speeds would tend to clump.

Coordination seems especially important in key design tasks, and in management. For example, it would be especially tempting to have all the parts of a large intricate software project written by the same very fast em. It would also be tempting to have the top thousand or more manager roles in a big organizations all be filled by a single very fast em.

Faster ems would naturally tend to be richer ems, if nothing else because they’d have some discretion in how they used their time, and that time is worth more. Thus a single very fast boss could afford to own more of a firm, reducing owner vs. manager conflicts.

If faster ems tend to be richer, win arguments, and fill key design and management roles, they would naturally be treated as higher status, at least by our status cues. Ems would also likely see them as higher status.

Social roles can often be usefully divided into roles that deal more with insiders, vs. roles that deal more with outsiders. For example, in a family, childcare is an inside role, while working for money is an outside role. In a hierarchical organization, managers have a more outside role – they deal more with outsiders. We care more about openness and helpfulness in inside roles, but more about opacity and toughness in outside roles.

When ems of different speeds meet, the slower em would naturally be more transparent and the faster one more opaque. It seems that faster ems would tend more to take on outside roles, which will be associated with higher status. In hierarchical organizations, subordinates might be expected to be open, such as via allowing direct hardware access to their emotional expressions, while bosses might typically hide their feelings from subordinates.

The overall picture here seems to be of even more inequality than I had imagined when I just considered wealth inequality among a larger future population whose lifespans vary more. Each em firm may have one very fast rich dominant boss who personally owns a lot of the firm. All front line managers might report to this one super boss, in meetings where they temporarily run at boss speeds, and are expected to be emotionally open to boss inspection. Sir, yes sir!

All else equal, an increase in the spatial extent of a firm or city would tend to reduce the speed of ems that might notice substantial communication delays. If em firms and cities tend to naturally grow larger over time, they’d also tend to naturally become slower, at least at their peak speeds. The gains that the would have otherwise achieved from faster speeds would be compensated by being able to interact naturally with a wider range of ems.

Remember the Netflix Prize? Turns out, Netflix didn’t make must use of the winning method, because the prize was based on dvd rental data, and their customers now stream movies more; dvds tend to be chosen more in far mode, while streaming movies are chosen more in near mode:

Netflix launched an instant streaming service in 2007, one year after the Netflix Prize began. Streaming has not only changed the way our members interact with the service, but also the type of data available to use in our algorithms. For DVDs our goal is to help people fill their queue with titles to receive in the mail over the coming days and weeks; selection is distant in time from viewing, people select carefully because exchanging a DVD for another takes more than a day, and we get no feedback during viewing. For streaming members are looking for something great to watch right now; they can sample a few videos before settling on one, they can consume several in one session, and we can observe viewing statistics such as whether a video was watched fully or only partially. …

when people rent a movie that won’t arrive for a few days, they’re making a bet on what they want at some future point. And, people tend to have a more… optimistic viewpoint of their future selves. That is, they may be willing to rent, say, an “artsy” movie that won’t show up for a few days, feeling that they’ll be in the mood to watch it a few days (weeks?) in the future, knowing they’re not in the mood immediately. But when the choice is immediate, they deal with their present selves, and that choice can be quite different. (more; HT Carl Shulman)

Physicist Tom Murphy says he argued with “an established economics professor from a prestigious institution,” on whether economic growth can continue forever. They both agreed to assume Earth-bound economies, and quickly also agreed that total energy usage must reach an upper bound within centuries, because of Earth’s limited ability to discard waste heat via radiation.

Murphy then argued that economic growth cannot grow exponentially if any small but necessary part of the economy is fails to grow, or if any small fraction of people fail to give value to the things that do grow:

Not everyone will want to live this virtual existence. … Many would prefer the smell of real flowers. … You might be able to simulate all these things, but not everyone will want to live an artificial life. And as long as there are any holdouts, the plan of squeezing energy requirements to some arbitrarily low level fails. …

Energy today is roughly 10% of GDP. Let’s say we cap the physical amount available each year at some level, but allow GDP to keep growing. … Then in order to have real GDP growth on top of flat energy, the fractional cost of energy goes down relative to the GDP as a whole. … But if energy became arbitrarily cheap, someone could buy all of it. … There will be a floor to how low energy prices can go as a fraction of GDP. … So once our fixed annual energy costs 1% of GDP, the 99% remaining will find itself stuck. If it tries to grow, energy prices must grow in proportion and we have monetary inflation, but no real growth. …

Chefs will continue to innovate. Imagine a preparation/presentation 400 years from now that would blow your mind. … No more energy, no more ingredients, yet of increased value to society. … [But] Keith plopped the tuna onto the bread in an inverted container-shaped lump, then put the other piece of bread on top without first spreading the tuna. … I asked if he intended to spread the tuna before eating it. He looked at me quizzically, and said—memorably, “It all goes in the same place.” My point is that the stunning presentation of desserts will not have universal value to society. It all goes in the same place, after all. (more; HT Amara Graps)

While I agree with Murphy’s conclusion that the utility an average human-like mind gains from their life cannot increase exponentially forever, Murphy’s arguments for that conclusion are wrong. In particular, if only a fixed non-zero fraction of such minds could increase their utility exponentially, the average utility would also increase exponentially.

Also, the standard power law (Cobb-Douglas) functional form for how utility depends on several inputs says that utility can grow without bound when one sector of the economy grows without bound, even when another needed sector does not grow at all and takes a fixed fraction of income. For example, if utility U is given by U = Ea N1-a, where E is energy and N is non-energy, then at competitive prices the fraction of income going to the energy sector is fixed at a, no matter how big N gets. So N can grow without bound, making U grow without bound, while E is fixed.

My skepticism on exponential growth is instead based on an expectation of strongly diminishing returns to everything, including improved designs:

Imagine that … over the last million years they’ve also been searching the space of enjoyable virtual reality designs. From the very beginning they had designs offering people vast galaxies of fascinating exotic places to visit, and vast numbers of subjects to command. (Of course most of that wasn’t computed in much detail until the person interacted with related things.) For a million years they have searched for possible story lines to create engaging and satisfying experiences in such vast places, without requiring more computational resources behind the scenes to manage.

Now in this context, imagine what it means for “imagination” to improve by 4% per year. That is a factor of a billion every 529 years. If we are talking about utility gains, this means that you’d be indifferent between keeping a current virtual reality design, or taking a one in a two billion chance to get a virtual reality design from 529 years later. If you lose this gamble, you have to take a half-utility design, which gives you only half of the utility of the design you started with. …

It may be possible to create creatures who have such strong preferences for subtle differences, differences that can only be found after a million or trillion years of a vast galactic or larger civilization searching the space of possible designs. But humans do not seem remotely like such creatures. (more)