If you imagine religions, governments, and criminals not getting too far out of control, and a basically capitalist world, then your main future fears are probably going to be about for-profit firms, especially regarding how they treat workers. You’ll fear firms enslaving workers, or drugging them into submission, or just tricking them with ideology.

Because of this, I’m not so surprised by the deep terror many non-economists hold of future competition. For example, Scott Alexander (see also his review):

I agree with Robin Hanson. This is the dream time .. where we are unusually safe from multipolar traps, and as such weird things like art and science and philosophy and love can flourish. As technological advance increases, .. new opportunities to throw values under the bus for increased competitiveness will arise. .. Capitalism and democracy, previously our protectors, will figure out ways to route around their inconvenient dependence on human values. And our coordination power will not be nearly up to the task, assuming something much more powerful than all of us combined doesn’t show up and crush our combined efforts with a wave of its paw.

But I was honestly surprised to see my libertarian economist colleague Bryan Caplan also holding a similarly dark view of competition. As you may recall, Caplan had many complaints about my language and emphasis in my book, but in terms of the key evaluation criteria that I care about, namely how well I applied standard academic consensus to my scenario assumptions, he had three main points.

First, he called my estimate of an em economic growth doubling time of one month my “single craziest claim.” He seems to agree that standard economic growth models can predict far faster growth when substitutes for human labor can be made in factories, and that we have twice before seen economic growth rates jump by more than a factor of fifty in a less than previous doubling time. Even so, he can’t see economic growth rates even doubling, because of “bottlenecks”:

Politically, something as simple as zoning could do the trick. .. the most favorable political environments on earth still have plenty of regulatory hurdles .. we should expect bottlenecks for key natural resources, location, and so on. .. Personally, I’d be amazed if an em economy doubled the global economy’s annual growth rate.

His other two points are that competition would lead to ems being very docile slaves. I responded that slavery has been rare in history, and that docility and slavery aren’t especially productive today. But he called the example of Soviet nuclear scientists “powerful” even though “Soviet and Nazi slaves’ productivity was normally low.” He rejected the relevance of our large literatures on productivity correlates and how to motive workers, as little of that explicitly includes slaves. He concluded:

If, as I’ve argued, we would copy the most robot-like people and treat them as slaves, at least 90% of Robin’s details are wrong.

As I didn’t think the docility of ems mattered that much for most of my book, I challenged him to audit five random pages. He reported “Robin’s only 80% wrong”, though I count only 63% from his particulars, and half of those come from his seeing ems as very literally “robot-like”. For example, he says ems are not disturbed by “life events”, only by disappointing their masters. They only group, identify, and organize as commanded, not as they prefer or choose. They have no personality “in a human sense.” They never disagree with each other, and never need to make excuses for anything.

Caplan offered no citations with specific support for these claims, instead pointing me to the literature on the economics of slavery. So I took the time to read up on that and posted a 1600 summary, concluding:

I still can’t find a rationale for Bryan Caplan’s claim that all ems would be fully slaves. .. even less .. that they would be so docile and “robot-like” as to not even have human-like personalities.

Yesterday, he briefly “clarified” his reasoning. He says ems would start out as slaves since few humans see them as having moral value:

1. Most human beings wouldn’t see ems as “human,” so neither would their legal systems. .. 2. At the dawn of the Age of Em, humans will initially control (a) which brains they copy, and (b) the circumstances into which these copies emerge. In the absence of moral or legal barriers, pure self-interest will guide creators’ choices – and slavery will be an available option.

Now I’ve repeatedly pointed out that the first scans would be destructive, so either the first scanned humans see ems as “human” and expect to not be treated badly, or they are killed against their will. But I want to focus instead on the core issue: like Scott Alexander and many others, Caplan sees a robust tendency of future competition to devolve into hell, held at bay only by contingent circumstances such as strong moral feelings. Today the very limited supply of substitutes for human workers keeps wages high, but if that supply were to greatly increase then Caplan expects that without strong moral resistance capitalist competition eventually turns everyone into docile inhuman slaves, because that arrangment robustly wins productivity competitions.

Bryan Caplan made strong, and to me incredible, claims that econ consensus predicts all ems would be fully slaves with no human personality. As he won’texplain his reasoning, but just says to read the slavery literature, I’ve done a quick lit review, which I now summarize, and then apply quickly to the future in general, and to ems in particular.

The ability to control your pain, actions, and income are distinct property rights. When someone else owns them all, you are said to be a slave, especially if they allocate these rights via something close to a “full control” package. In this package, you have little control over assets or actions. Pain is usually threatened, and often implemented, to force specific disliked but demanded actions. (Pain was used more on children than adults.) Think of rowing for a galley ship, digging up silver in a mine, picking cotton, or advancing on a simple war front line.

A second “mixed control” package allocates these rights by letting you retain control over many action details, only rarely causing pain, and letting you earn a residual income or status. This scenario was more common for domestic slaves, for slaves with better options for sabotage or escape, and for complex jobs where motivation matters more, via worker discretion, responsibility, attentiveness, pleasantness, intelligence, or creativity. By collecting a residual income, slaves might eventually buy their freedom. Free people have often sold this package of rights for short durations in traditional jobs. The main difference is your ease of changing jobs; the harder it is to change jobs, the more like this kind of slave you are.

In a third “debt” package, you must pay off a loan but are otherwise mostly free to choose your own job, location, and living arrangements. The option to impose pain is reserved for rare situations. Closely related is “share cropping” wherein the owner demands a percentage of income earned. Some combination of a fixed payment plus a percentage of income was a common scenario for slaves in southern US cities. This is also the usual way state rulers extort the locals they “own” via taxation. Many people voluntarily choose to go into debt, and sell percentages of their business income, and most legal systems reserve the right to impose pain in rare situations, a situation most people are okay with.

A fourth “ransom” approach sells these rights back to some combination of you and your associates. Often this converts these rights into debt held by someone who is better able to motivate and monitor you.

Many considerations influence the efficiency of these allocations, including costs of monitoring and restraint, losses from theft, rebellion, escape, and sabotage, individual preferences for pain, status, autonomy, and work style, effects of pain, status, and control on motivation and focus, information rents from workers being better aware of work details, complementary investments in training and capital, who knows better and has better incentives to use control rights, and signaling status, productivity, etc. to outsiders.

Historically, even when slaves were common, they were usually a minority of the population. (Beware, the term “slave” is used in different ways.) About 10% in the Roman Empire and US south. Foragers didn’t do slaves at all. About 0.3% of the world is in slavery today, mostly in forms of debt bondage.

The common existence of slavery that wasn’t converted immediately into debt or ransom does suggest that it was sometimes locally efficient as a resource allocation, ignoring larger social externalities, even given substantial costs of monitoring, enforcement, and worse motivation and allocation of skills.

Sometimes during hard times people would sell themselves or their children into slavery; better to be fed than dead. Sometimes slaves were created as collateral for loans, and freed when the loan was paid. Sometimes slavery was the contractual result of a failure to pay loans. Sometimes people sold themselves into slavery for a limited time, as with apprenticeships and indentured servitude.

But historically, slaves were mostly created in war. Drafted soldiers are slave-like. When a winning side didn’t expect to hold the territory, and feared leaving the vanquished to recover then retaliate, their remaining options were death or slavery. But slaves were only valuable when delivered to a useable location. So the worse treatment of slaves has been in transit immediately after capture.

Slave populations usually dwindled until replenished by war, probably because through most of history interest rates were too high to justify the long term investment of raising human children. Domesticated crops and animals grow much quicker. This same short term focus also often induced slave owners to work their slaves to death. A short term focus was often increased by distant ownership, as local manager’ incentives were tied more to immediate production. Workings slaves to death induced more slave revolts.

The US south was unusual in that it grew long-lived slaves from birth. Interest rates were unusually low, peace lasted long, and once US law forbad importing slaves, owners were highly motivated to preserve their big plantation industry. Slaves weren’t converted into debt perhaps because of credit market failures, or more plausibly because the full control approach was especially productive on plantations. (The sex story is overrated, as only 1-2% of slave babies were fathered by white men.)

That is, on plantations slaves plausibly produced more when threatened with pain, even if their utility was lower. The fact that humans can feel strongly disliked pain while living a long productive life and successfully reproducing does suggest that our pain signals are biologically maladaptive. But given how different is the modern world from the one where our pain signals evolved, we should expect this sort of thing sometimes.

Data on US south slave prices tells us what was valued in slaves then. For adults, age was bad, as were slaves from distant places within the US, and slaves that the owner chose to sell, as opposed to being forced to sell. New slaves imported from overseas were no more or less valued. It was good to be male, light-skinned, have artisan skills, and be guaranteed not to be sick or run away.

I didn’t find any data on slaves and docility, though I did find how docility fits into the standard five factor personality framework. Docility is lumped with “submissive, dependent, pliant” as part of “passivity”, which correlates most strongly and positively with neuroticism, but also positively with agreeableness and negatively with openness. In general only the agreeable part suggests more productivity in most jobs today; neurotic people are less productive, and the effect of openness depends more on job type.

What is there to dislike about slavery? The war and theft that cause slavery are clearly lamentable. And the possibility of slavery increases the range of possible inequality, at least if you ignore the dead. But the full control allocation package seems the main reason to dislike slavery. Other packages seem much closer to those resulting from free choices, and when they result from free choices they don’t seem strongly objectionable.

Today slavery, especially full control slavery, is discouraged not only via moral censure and political coordination, but also by stronger nation-states, few wars, better credit markets, increasing wealth, increasing vulnerability to sabotage, more automation, and more complex jobs. The only contrary factors I can think of are easier monitoring and preventing escape. If all these trends continue in the same relative proportions, we should expect a continued decline in slavery.

In the world of my book, The Age of Em, many of these trends continue. Nation-states and credit markets get stronger, and war remains rare. Automation advances, and jobs get even more complex, with motivation and sabotage mattering even more. Monitoring and preventing escape also get easier.

Individual em incomes do fall, which gives a thicker lower tail of outcomes, and in traditional societies that allowed slavery this low tail was often filled with slaves. However, ems can fall via running slower while remaining free, and this option would reduce the fraction that fall into slavery, even if slavery were allowed.

Ems are initially created via destructive scanning of high income human volunteers at the peak of their careers, in a world that forbids slavery. Soon after they are destructive scans of the most promising young children. So these volunteers do not expect to become slaves, and the world around them, being like ours, initially tries to discourage that transition.

However, since a lot changes we can’t offer much assurance that attitudes toward slavery don’t change. Also, labor supply factors matter a lot less; if even one productive em is enslaved, and slavery is allowed, then copies of it could fill a whole slave sector. What matters far more is demand, i.e., what are the more efficient ways to allocate labor? If allowed, there are probably some jobs where full control slavery is more efficient; the em world is big, with many corners. But most jobs are complex, where the full control scenario is inefficient. And the debt or mixed control allocations that are more efficient for typical jobs are probably not substantially more efficient under slavery, as slavery hurts motivation. Debt should be good enough.

So, bottom line, after a quick review of the econ of slavery literature, I still can’t find a rationale for Bryan Caplan’s claim that all ems would be fully slaves. Ancient society never got close to that state of affairs. And I see even less rationale for his claim that they would be so docile and “robot-like” as to not even have human-like personalities. Which is his main reason for saying 80% of my book is wrong. Neither the literatures on choosing employees today, nor that on choosing slaves in the past, put much emphasis on docility. And even if they did, the idea that they’d emphasize it so much as to eliminate human personality, that just sounds crazy.

So Bryan, how about actually giving an argument, instead of waving your hands in the general direction of the literature?

There is a difference between predicting the weather, and predicting climate. If you know many details on current air pressures, wind speeds, etc, you can predict the weather nearby a few days forward, but after weeks to months at most you basically only know an overall distribution. However, if there is some fundamental change in the environment, such as via carbon emissions, you might predict how that distribution will change as a result far into the future; that is predicting climate.

Henry Farrell, at Crooked Timber, seems to disagree with Age of Em because he thinks we can only predict social weather, not social climate:

Tyler Cowen says .. Age of Em .. won’t happen. I agree. I enjoyed the book. .. First – the book makes a strong claim for the value of social science in extrapolating likely futures. I am a lot more skeptical that social science can help you make predictions. .. Hanson’s arguments seem to me to rely on a specific combination of (a) an application of evolutionary theory to social development with (b) the notion that evolutionary solutions will rapidly converge on globally efficient outcomes. This is a common set of assumptions among economists with evolutionary predilections, but it seems to me to be implausible. In actually existing markets, we see some limited convergence in the short term on e.g. forms of organization, but this is plausibly driven at least as much by homophily and politics as by the actual identification of efficient solutions. Evolutionary forces may indeed lead to the discovery of new equilibria, but haltingly, and in unexpected ways. .. This suggests an approach to social science which doesn’t aim at specific predictions a la Hanson, so much as at identifying the underlying forces which interact (often in unpredictable ways) to shape and constrain the range of possible futures. ..

In the end, much science fiction is doing the same kind of thing as Hanson ends up doing – trying in a reasonably systematic way to think through the social, economic and political consequences of certain trends, should they develop in particular ways. The aims of extrapolationistas and science fiction writers aims may be different – prediction versus constrained fiction writing but their end result – enriching our sense of the range of possible futures that might be out there – are pretty close to each other. .. it is the reason I got value from his book. ..

So Hanson’s extrapolated future seems to me to reflect an economist’s perspective in which markets have priority, and hierarchy is either subordinated to the market or pushed aside altogether. The work of Hannu Rajaniemi provides a rich, detailed, alternative account of the future in which something like the opposite is true .. [with] vast and distributed hierarchies of exploitation. .. Rajaniemi’s books .. provide a rich counter-extrapolation of what a profoundly different society might look like. .. I don’t know what the future will look like, but I suspect it will be weird in ways that echo Rajaniemi’s way of thinking (which generates complexities) rather than Hanson’s (which breaks them down).

If we can only see forces that shape and constrain the future, but not the distribution of future outcomes, what is the point of looking at samples from the “range of possibilities”? That only seems useful if in fact you can learn things about that range. In which case you are learning about the overall distribution. Isn’t Farrell’s claim about more future “hierarchies of exploitation” relative to “markets” just the sort of overall outcome he claims we can’t know? (Rajaniemi blurbed and likes my book, so I don’t think he sees it as such a polar opposite. And how does hierarchy “generate complexities” while markets “break them down”?) Is Farrell really claiming that there is no overall tendency toward more efficient practices and institutions, making moves away from them just as likely as moves toward them? Are all the insights economic historians think they have gained using efficiency to understand history illusory?

My more charitable interpretation is that Farrell sees me as making forecasts much more confidently than I intend. While I’ve constructed a point prediction, my uncertainty is widely distributed around that point, while Farrell sees me as claiming more concentration. I’ll bet Farrell does in fact see a tendency toward efficiency, and he thinks looking at cases does teach us about distributions. And he probably even thinks supply and demand is often a reasonable first cut approximation. So I’m guessing that, with the right caveat about confidence, he actually thinks my point prediction makes a useful contribution to our understanding of the future.

One clarification. Farrell writes:

One of the unresolved tensions .. Are [ems] free agents, or are they slaves? I don’t think that Hanson’s answer is entirely consistent (or at least I wasn’t able to follow the thread of the consistent argument if it was). Sometimes he seems to suggest that they will have successful means of figuring out if they have been enslaved, and refusing to cooperate, hence leading to a likely convergence on free-ish market relations. Other times, he seems to suggest that it doesn’t make much difference to his broad predictive argument whether they are or are not slaves.

Much of the book doesn’t depend on if ems are slaves, but some parts do, such as the part on how ems might try to detect if they’ve been unwittingly enslaved.

Do I think Robin Hanson’s “Age of Em” actually will happen? A reader has been asking me this question, and my answer is…no! Don’t get me wrong, I still think it is a stimulating and wonderful book. .. But it is best not read as a predictive text, much as Robin might disagree with that assessment. Why not? I have three main reasons, all of which are a sort of punting, nonetheless on topics outside one’s areas of expertise deference is very often the correct response. Here goes: 1. I know a few people who have expertise in neuroscience, and they have never mentioned to me that things might turn out this way.

Now at GMU econ we often have academics who visit for lunch and take the common academic stance of reluctance to state opinions which they can’t back up with academic evidence. Tyler is usually impatient with that, and pushes such visitors to make best estimates. Yet here it is Tyler who shows reluctance. I hypothesize that he is following this common principle:

One does not express serious opinions on topics not yet authorized by the proper prestigious people.

Once a topic has been authorized, then unless a topic has a moral coloring it is usually okay to express a wide range of opinions on it; it is even often expected that clever people will often take contrarian or complex positions, sometimes outside their areas of expertise. But unless the right serious people have authorized a topic, that topic remains “silly”, and can only be discussed in a silly mode.

Now sometimes a topic remains unauthorized because serious people think everything about it has a low probability. But there are many other causes for topics to be seen as silly. For example, sex was long seen as a topic serious people didn’t discuss, even though we were quite sure sex exists. And even though most everyone is pretty sure aliens must exist out there somewhere, aliens remain a relatively silly subject.

In the case of ems, I interpret Tyler above as noting that the people who seem to him the proper authorities have not yet authorized serious discussion of ems. That is what he means by pointing to experts, saying “no reason” and “scientific consensus,” and yet being unwilling to state a probability, or even clarify which claim he rejects, even though I argued a 1% chance is enough. It explains his initial emphasis on treating my book metaphorically. This is less about probabilities, and more about topic authorization.

Compare the topic of ems to the topic of super-intelligence, wherein a single hand-coded AI quickly improves itself so fast that it can take over the world. As this topic seems recently endorsed by Elon Musk, Bill Gates, and Steven Hawking, it is now seen more as an authorized topic. Even though, if you are inclined to be skeptical, we have far more reasons to doubt we will eventually know how to hand-code software as broadly smart as humans, or vastly better than the entire rest of the world put together at improving itself. Our reason for thinking ems eventually feasible is far more solid.

Yet I predict Tyler would more easily accept an invitation to write or speak on super-intelligence, compared to ems. And I conclude many readers see my book primarily as a bid to put ems on the list of serious topics, and they doubt enough proper prestigious people will endorse that bid. And yes, while if we could talk probabilities I think I have a pretty good case, even my list of prestigious book blurters probably aren’t enough. Until someone of the rank of Musk, Gates, or Hawking endorses it, my topic remains silly.

There are smart intellectuals out there think economics is all hogwash, and who resent economists continuing on while their concerns have not been adequately addressed. Similarly, people in philosophy of religion and philosophy of mind resent cosmologists and brain scientists continuing on as if one could just model cosmology without a god, or reduce the mind to physical interactions of brain cells. But in my mind such debates have become so stuck that there is little point in waiting until they are resolved; some of us should just get on with assuming particular positions, especially positions that seem so very reasonable, even obvious, and seeing where they lead.

Similarly, I have heard people debate the feasibility of ems for many decades, and such debates have similarly become stuck, making little progress. Instead of getting mired in that debate, I thought it better to explore the consequences of what seems to me the very reasonable positions that ems will eventually be possible. Alas, that mud pit has strong suction. For example, Tyler Cowen:

Do I think Robin Hanson’s “Age of Em” actually will happen? … my answer is…no! .. Don’t get me wrong, I still think it is a stimulating and wonderful book. And if you don’t believe me, here is The Wall Street Journal:

Mr. Hanson’s book is comprehensive and not put-downable.

But it is best not read as a predictive text, much as Robin might disagree with that assessment. Why not? I have three main reasons, all of which are a sort of punting, nonetheless on topics outside one’s areas of expertise deference is very often the correct response. Here goes:

1. I know a few people who have expertise in neuroscience, and they have never mentioned to me that things might turn out this way (brain scans uploaded into computers to create actual beings and furthermore as the dominant form of civilization). Maybe they’re just holding back, but I don’t think so. The neuroscience profession as a whole seems to be unconvinced and for the most part not even pondering this scenario. ..

3. Robin seems to think the age of Em could come about reasonably soon. … Yet I don’t see any sign of such a radical transformation in market prices. .. There are for instance a variety of 100-year bonds, but Em scenarios do not seem to be a factor in their pricing.

But the author of that Wall Street Journal review, Daniel J. Levitin, is a neuroscientist! You’d think that if his colleagues thought the very idea of ems iffy, he might have mentioned caveats in his review. But no, he worries only about timing:

The only weak point I find in the argument is that it seems to me that if we were as close to emulating human brains as we would need to be for Mr. Hanson’s predictions to come true, you’d think that by now we’d already have emulated ant brains, or Venus fly traps or even tree bark.

Because readers kept asking, in the book I give a concrete estimate of “within roughly a century or so.” But the book really doesn’t depend much on that estimate. What it mainly depends on is ems initiating the next huge disruption on the scale of the farming or industrial revolutions. Also, if the future is important enough to have a hundred books exploring scenarios, it can be worth having books on scenarios with only a 1% chance of happening, and taking those books seriously as real possibilities.

Tyler has spent too much time around media pundits if he thinks he should be hearing a buzz about anything big that might happen in the next few centuries! Should he have expected to hear about cell phones in 1960, or smart phones in 1980, from a typical phone expert then, even without asking directly about such things? Both of these were reasonable foreseen many decades in advance, yet you’d find it hard to see signs of them several decades before they took off in casual conversations with phone experts, or in phone firm stock prices. (Betting markets directly on these topics would have seen them. Alas we still don’t have such things.)

I’m happy to accept neuroscientist expertise, but mainly on in how hard it is to scan brain cells and model them on computers. This isn’t going to come up in casual conversation, but if asked neuroscientists will pretty much all agree that it should eventually be be possible to create computer models of brain cells that capture their key signal processing behavior, i.e., the part that matters for signals received by the rest of the body. They will say it is a matter of when, not if. (Remember, we’ve already done this for the key signal processing behaviors of eyes and ears.)

Many neuroscientists won’t be familiar with computer modeling of brain cell activity, so they won’t have much of an idea of how much computing power is needed. But for those familiar with computer modeling, the key question is: once we understand brain cells well, what are plausible ranges for 1) the number of bits required store the current state of each inactive brain cell, and 2) how many computer processing steps (or gate operations) per second are needed to mimic an active cell’s signal processing.

Once you have those numbers, you’ll need to talk to people familiar with computing cost projections to translate these computing requirements into dates when they can be met cheaply. And then you’d need to talk to economists (like me) to understand how that might influence the economy. You shouldn’t remotely expect typical neuroscientists to have good estimates there. And finally, you’ll have to talk to people who think about other potential big future disruptions to see how plausible it is that ems will be the first big upcoming disruption on the scale of the farming or industrial revolutions.

Age of Em is the “book of the day” today at the Guardian newspaper, the 5th most widely read one in the world. Reviewer Steven Poole hates the em world:

The Age of Em is a fanatically serious attempt .. to use economic and social science to forecast in fine detail how this world (if it is even possible) will actually work. The future it portrays is very strange and, in the end, quite horrific for everyone involved. .. This hellish cyberworld is quite cool to think about in a dystopian Matrixy way, although the book is much drier than fiction.

I’m fine with people not liking the em world, if they understand it. But disliking the world also seems to translate into disliking my analysis. My point by point responses:

Hanson says it reads more like an encyclopedia. But if it’s an encyclopedia, what are its sources?

References take 31 pages, others have complained of too many cites, and you complain of dry text. Yet you really wanted more cites & references?

“Today,” he complains, “we take far more effort to study the past than the future, even though we can’t change the past.” Yes, you might respond: that is because we literally cannot “study” the future – because either it doesn’t exist or (in the block-universe model of time) it does exist but is completely inaccessible to us.

We infer theories from data on the present and past. The whole reason for theory is to help us infer things where we don’t have data. Like the future. That is what theorists do. So we can study the future by applying our best theories, as I tried to do in the book.

Given that, the book’s confidence in its own brilliantly weird extrapolations is both impressive and quite peculiar. Hanson describes his approach as that of “using basic social theory, in addition to common sense and trend projection, to forecast future societies”. The casual use of “common sense” there should, as always, ring alarm bells. And a lot of the book’s sense is arguably quite uncommon.

Here you insinuate that much is wrong, but you don’t actually point out anything specific as wrong.

The governing tone is strikingly misanthropic, despairing of current humans’ “maladaptation” to the environment.

How is it remotely “hating” of people to see recent behavior as more evolutionarily maladaptive?

And there is an unargued assumption throughout that social patterns and institutions are more likely to revert to pre-industrial norms in the future.

I argue explicitly in some detail for some attitudes reverting to those more typical of poor farmers, when ems get poor. But the only institutions that might revert would be those driven mainly by attitudes, such as perhaps democracy.

Hanson .. erects a large edifice of sociological speculation on how the liberal use of em copies and backups will change attitudes to sex, law, death and pretty much everything else. But .. if someone announces they will upload my consciousness into a robot and then destroy my existing body, I will take this as a threat of murder. .. So ems – the first of whom are, by definition, going to have minds identical to those of humans – may very well exhibit the same kind of reaction, in which case a lot of Hanson’s more thrillingly bizarre social developments will not happen.

Yes, you feel strongly, but everyone need not share your feelings. Yes, the first brain scans will be destructive, but out of a world population of billions it only takes a few biological humans willing to be scanned this way to fill the em world. And if there were only a few of them, they’d each earn trillions.

But then, the rather underwhelming upshot of this project is that fast-living and super-clever ems will probably crack the problem of proper AI – actual intelligent machines – within a year or so of ordinary human time.

I didn’t say “probably” here; I gave that as one identifiable possibility.

Given that this future is so gloomy for just about everyone, one does end up wondering why Hanson wants to wake up in it – he reveals in the book that he has arranged to be cryogenically frozen on his death. I suppose it is at least possible that, one day, he could open his eyes and have the last laugh, as he surveys the appalling future he foresaw so long ago.

Because I describe a world you don’t like I must be a people hater pleased to see everyone suffer? Really?! For the record, I don’t now see the em world as appalling, and if I changed my mind on that upon seeing it up close, I’d be quite disappointed.

A very different—indeed startling—vision of the future .. What is remarkable about Mr. Hanson’s book is not just the detail with which he imagines this future but the way he situates it within a perceptive analysis of our human past and present. .. His is a dyspeptic-topia. It looks grim. ..

Mr. Hanson’s book is comprehensive and not put-downable. The author has thought of everything. He’s anticipated every one of my objections, including the manifestly unscientific one of how creepy this all sounds. He admirably explains the assumptions he’s making and the limitations. ..

The only weak point I find in the argument is that it seems to me that if we were as close to emulating human brains as we would need to be for Mr. Hanson’s predictions to come true, you’d think that by now we’d already have emulated ant brains, or Venus fly traps or even tree bark. ..

For my own part, I hope that the ems come soon. .. Even if you aren’t interested in the future, “The Age of Em” provides a wonderful overview of the current social psychology of productivity. .. For readers of this newspaper, a particularly interesting section discusses how free-market forces will change economic behaviors, negotiations, price-setting and fee structures. Mr. Hanson is an amiable narrator and guide to all these topics and more. We could use a few more of him.

It may be, too, that we should look with some trepidation toward the transitional period—that strange era in which our real-world ways will be disrupted by the introduction of new and bizarre simulated life forms. In “The Age of Em,” a nonfiction work of social-science speculation published earlier this year, the economist and futurist Robin Hanson describes a time in which researchers haven’t yet cracked artificial intelligence but have learned to copy themselves into their computers, creating “ems,” or emulated people, who quickly come to outnumber the real ones. Unlike Bostrom, who supposes that our descendants will create simulated worlds for curiosity’s sake, Hanson sees the business case for simulating people: instead of struggling to find a team of programmers, a company will be able to hire a single, brilliant em and then replicate her a million times. An enterprising em might gladly replicate herself to work many jobs at once; after she completes a job, a copied em might choose to delete herself, or “end.” (An em contemplating ending won’t ask “Do I want to die?,” Hanson writes, since other copies will live on; instead, she’ll ask, “Do I want to remember this?”) An em might be copied right after a vacation, so that whenever she is pasted into the simulated workplace, she is cheerful, rested, and ready to work. She might also be run on computer hardware that is more powerful than a human brain, and so think (and live) at a speed millions or even trillions of times faster than an ordinary human being.

Hanson doesn’t think that ems must necessarily live unhappy lives. On the contrary, they may thrive, fall in love, and find fulfillment in their competitive, flexible, high-speed world. Non-simulated people, meanwhile, may retire on the proceeds from their investments in the accelerated and increasingly autonomous em economy—a pleasant vantage point from which to observe the twilight of non-emulated civilization. Many people have imagined that technology will free us from the burden of work; if Hanson is right, that freedom could come through the virtualization of the human race.

This was in an article about the simulation argument. Two years ago I compared em and sim conversations, noting that in both cases many discuss using them as fiction settings, the chances that they are true, clues for inferring if they are true, and what they imply for identity, consciousness, physics, etc. But few discuss social consequences, such as how to live in a simulation or what a em world is like socially.

So according to Jones, we can’t trust anthropologists to describe foragers they’ve met, we can’t trust economics when tech changes society, and familiar design principles fail for understanding brains and tiny chemical systems. Apparently only his field, physics, can be trusted well outside current experience. In reply, I say I’d rather rely on experts in each field, relative to his generic skepticism. Brain scientists see familiar design principles as applying to brains, even when designed by evolution, economists see economics as applying to past and distant societies with different tech, and anthropologists think they can understand cultures they visit.

Jones complained on twitter that I “prefer to argue from authority rather than engage with their substance.” I replied “There can’t be much specific response to generic skepticism,” to which he replied, “Well, there’s more than 4000 words of quite technical argument on the mind uploading question in the post I reference.” He’s right that he wrote 4400 words. But let me explain why I see them more as generic skepticism than technical argument.

For context, note that there arewholefields of biological engineering, wherein standard engineering principles are used to understand the engineering of biological systems. These include the design of many specific systems with organisms, such as lungs, blood, muscles, bone, and skin, and also specific subsystems within cells, and also standard behaviors, such as gait rhythms and foraging patterns. Standard design principles are also used to understand why cells are split into different modules that perform distinct functions, instead of having each cell try to contribute to all functions, and why only a few degrees of freedom for each cell matters for that cell’s contribution to its system. Such design principles can also be used to understand why systems are abstract, in the sense of as having only one main type of muscle, for creating forces used for many purposes, one main type of blood system, to move most everything around, or only one main fast signal system, for sending signals of many types.

Our models of the function of many key organs have in fact often enabled us to create functional replacements for them. In addition, we already have good models of, and successful physical emulations of, key parts of the brain’s input and out, such, as input from eyes and ears, and output to arms and legs.

Okay, now here are Jones’ key words:

This separation between the physical and the digital in an integrated circuit isn’t an accident or something pre-ordained – it happens because we’ve designed it to be that way. For those of us who don’t accept the idea of intelligent design in biology, that’s not true for brains. There is no clean “digital abstraction layer” in a brain – why should there be, unless someone designed it that way?

But evolution does design, and its designs do respect standard design principles. Evolution has gained by using both abstraction and modularity. Organs exist. Humans may be better in some ways than evolution at searching large design spaces, but biology definitely designs.

In a brain, for example, the digital is continually remodelling the physical – we see changes in connectivity and changes in synaptic strength as a consequence of the information being processed, changes, that as we see, are the manifestation of substantial physical changes, at the molecular level, in the neurons and synapses.

Underlying all these phenomena are processes of macromolecular shape change in response to a changing local environment. .. This emphasizes that the fundamental unit of biological information processing is not the neuron or the synapse, it’s the molecule.

But you could make that same sort of argument about all organs, such as bones, muscles, lungs, blood, etc., and say we also can’t understand or emulate them without measuring and modeling them them in molecular detail. Similarly for the brain input/output systems that we have already emulated.

Determining the location and connectivity of individual neurons .. is necessary, but far from sufficient condition for specifying the informational state of the brain. .. The molecular basis of biological computation means that it isn’t deterministic, it’s stochastic, it’s random.

Randomness is quite easy to emulate, and most who see ems as possible expect to need brain scans with substantial chemical, in addition to spatial, resolution.

And that’s it, that is Jones’ “technical” critique. Since biological systems are made by evolution human design principles don’t apply, and since they are made of molecules one can’t emulate them without measuring and modeling at the molecular level. Never mind that we have actually seen design principles apply, and emulated while ignoring molecules. That’s what I call “generic skepticism”.

In contrast, I say brains are signal processing systems, and applying standard design principles to such systems tells us:

To manage its intended input-output relation, a signal processor simply must be designed to minimize the coupling between its designed input, output, and internal channels, and all of its other “extra” physical degrees of freedom. .. To emulate a biological signal processor, one need only identify its key internal signal dimensions and their internal mappings – how input signals are mapped to output signals for each part of the system. These key dimensions are typically a tiny fraction of its physical degrees of freedom. Reproducing such dimensions and mappings with sufficient accuracy will reproduce the function of the system. This is proven daily by the 200,000 people with artificial ears, and will be proven soon when artificial eyes are fielded.

As many have noted, ours in an era of ideological polarization. On topics where there are strong emotions, we tend to gravitate to extremes, and are less interest in intermediate positions. Which is a problem for my book; while most see it as too weird, others mostly see it as not weird enough. A tech futurism minority expects to soon see very rapid progress in artificial intelligence and machine learning, and so see brain emulations as too slow and inefficient compared to the super-intelligence they foresee. And the majority to whom that seems pretty crazy also seem brain emulations as similarly crazy; they don’t care much if ems seem a bit less crazy.

At least future tech enthusiasts who think my book not weird enough are willing to write reviews to say so. But those who think my book too weird mostly stay silent; I’ve heard privately of many who were going to cover the book before they fully realized what it is about. So I thank my college Bryan Caplan for being willing to say what others won’t, in his critical review. His review is long, with ten criticisms. This response will also be long, going point by point.