Alexander on Age of Em

If I ever have an executioner, I want him to be Scott Alexander. Alexander has such a winning way with words that I and his many fans enjoy him even when we disagree. I’d hardly notice my destination as his pleasing patter entranced me while we took the long way around to the gallows.

So I am honored that Alexander wrote a long review of Age of Em (9K words, 6% as long as the book), wherein he not only likes and recommends it, he also accepts pretty much all its claims within its main focus. That is, I present my book as being expert on the topic of what would actually happen if cheap ems were our next huge social change. Where Alexander disagrees is on two auxiliary topics, which I mention but on which I claim less expertise, namely how likely is this key scenario assumption, and how valuable is the resulting civilization I describe.

On the subject of value, Alexander leans forager (i.e., liberal) on the forager vs. farmer scale. He dislikes civilization evolving away from the behaviors and values of our forager ancestors, and today he partly blames this on capitalism. He doesn’t see our increase in numbers, comfort, and lifespan as sufficient compensation. (I think he’d like the book Against Civilization.) He says:

[Nick Land’s Ascended Economy] seems to me the natural end of the economic system. Right now it needs humans only as laborers, investors, and consumers. But robot laborers are potentially more efficient, companies based around algorithmic trading are already pushing out human investors, and most consumers already aren’t individuals – they’re companies and governments and organizations. At each step you can gain efficiency by eliminating humans, until finally humans aren’t involved anywhere. .. The Age of Em is an economy in the early stages of such a transformation. Instead of being able to replace everything with literal robots, it replaces them with humans who have had some aspects of their humanity stripped away. Biological bodies. The desire and ability to have children normally. ..

I envision a spectrum between the current world of humans and Nick Land’s Ascended Economy. Somewhere on the spectrum we have ems who get leisure time. A little further on the spectrum we have ems who don’t get leisure time. But we can go further. .. I expect [greatly reduced sex desire] would happen about ten minutes after the advent of the Age of Em .. Combine that with the stimulant use mentioned above, and you can have people who will never have nor want to have any thought about anything other than working on the precise task at which they are supposed to be working at any given time. ..

I see almost no interesting difference between an em world with full use of these tweaks and an Ascended Economy world. Yes, there are things that look vaguely human in outline laboring in the one and not the other, but it’s not like there will be different thought processes or different results. I’m not even sure what it would mean for the ems to be conscious in a world like this – they’re not doing anything interesting with the consciousness. .. If we get ems after all, I expect them to be lobotomized and drugged until they become effectively inhuman, cogs in the Ascended Economy that would no more fall in love than an automobile would eat hay and whinny.

Alexander seems to strongly endorse the usual forager value of leisure over work, so much so that he can’t see people focused on their work as human, conscious, or of any moral value. Creatures only seem valuable to him to the extent that they have sex, leisure time, minds wandering away from work, and desires to do things other than work.

This seems ironic because Scott Alexander is one of the most human and productive workers I know. He has a full time job as a psychiatrist, an especially demanding job, and in addition finds time to write frequent long careful analyses of many topics. I find it hard to see where he has that much time for leisure, and doubt he would in fact be substantially more productive overall if he took drugs to make him forget sex, mentally wander less, and focus more on his immediate tasks. He is exactly the sort of person an em economy would want many copies of, pretty much just as he is. Yet if we are to believe him, he only sees value in his brief leisure hours.

I see Alexander as having too little respect for the functionality of human behaviors and mind design. Yes, maximally competitive em-era behaviors and minds won’t be exactly like current ones. But that doesn’t necessarily mean one wants to throw out most existing behaviors and brain modules wholesale and start over from scratch. As these behaviors and modules all arose because they helped our ancestors be more competitive in some prior context, it makes more sense to try to repair, reform, and repurpose them.

For example, the robust productivity gains observed from workers who take breaks don’t seem to depend much on worker motivation. Breaks aren’t just about motivation; they are a deeply entrenched part of being productive. Similarly, wandering minds may take away from the current immediate task, but they help one to search for hidden problems and opportunities. Also, workers today who focus on just doing immediate tasks often lose out to others who attend more to building and managing social relations, as well as office politics. Love and sex can be very helpful in forming and maintaining relations.

Of course I’m not trying to offer any long term assurances, and it is quite reasonable to worry about what we will lose along with what we will gain. But since today most of the people we most respect and celebrate tend to be workaholics, I just can’t buy the claim that most of us today can’t find value in similarly productive and work-focused ems. And I just can’t see thoughtless workers being the most productive in the early em era of my book.

I find it strange that you mention my writing, since that’s something I do in my leisure time as a stimulating kind of self-expression, exactly the sort of thing I’m worried the em world wouldn’t have enough of.

I’m blessed to have a really great day job which I love. But I think that if you offered me the choice between a quiet oblivion, or having to work that day job 24-7 (minus sleep time) with no leisure or vacation ever again, I would probably take the oblivion. I’ve sometimes had to work 100 hour weeks and think I would have preferred oblivion to that experience extended forever. On the one hand I realize I may be atypical in that. On the other I think I have a much better day job than most people and that eg coal miners might feel this way even more strongly.

I mentioned that since you brought me up as a personal example, but I agree with your argument in the book that ems will be created from people unlike me, who genuinely like work and would be happy to do it forever.

But suppose we genetically engineered (or wireheaded) a group of people until they really enjoyed digging ditches and filling them in again. Not because of any particular unusual feature of the ditch-digging; it’s not that it takes creative work to align the ditches, or that they bond with their ditch-digging partner – they just think ditch-digging-and-filling-in is really great, full stop. Such a world seems to me to have no more (or less) value than a pure wireheading scenario, where everyone just sits in their individual cubicle deliriously happy because we’re stimulating their pleasure centers directly.

I’m not saying that the em world as you present it is that bad. I think that even an em world with no leisure would have some moral value. But I would place most of that moral value in the things that are most like injections of typical leisure activities into the work experience – getting to bond with coworkers, getting to solve interesting problems, any projects that require creative self-expression. Absent those, ems’ enjoyment of raw drudgery seems close to the ditch-digging/wireheading scenario.

What I said wouldn’t have *any* (or much) moral value was the full Ascended Economy scenario. But I categorize that as ems being fully reduced to coincidentally neuromorphic cogs in a system, such that almost all human qualities have been taken out of them. I realize we’ve talked about this possibility before and you disagree that it can happen (or at least think it would be prohibitively hard). I hope you’re right, but if it did happen I wouldn’t find it any more morally valuable than the wireheading universe or the ditch-digging universe.

Yes, you are only talking about losing all value in a limit, and you grant that it isn’t obvious this limit would be achieved during the era I discuss. My book is not about a limit, but is instead focused on a next era.

There is a literature on maximum productive work hours that I cite in the book, and the max is far less than all hours besides sleep. This is probably deeply embedded feature, and so can’t be changed with tweaks. So tweaked ems still have many non-sleep non-work hours.

I’m not sure you appreciate the degree to which cultural plasticity has always enabled humans to become reasonably content with a pretty wide range of environments, and jobs. Including digging ditches. Jobs as tedious as digging ditches in the past did lead to lives worth living. No need for fancy wire-heading; culture can and has done the job. The main reason we look on them with horror today is that they are low status in our world, and we all look with horror on being low status.

J Storrs Hall

And why, we must ask, do we look with horror at being low status? It’s because the pecking order cutoff heuristic doesn’t work otherwise. In an edge-of-subsistence world, when times get hard, the low end has to accept starving as its due, and the high end accept keeping the food to themselves. Otherwise when there is half enough food for everyone, instead of half the people getting enough and half dying, everybody will get half enough and die. Without the psychological mechanisms of status, the solution would break down as the losers decided to fight and the winners tried to be altruistic.
One wonders if status mechanisms might not be strongly selected for in an em world on the edge of subsistence.

> I find it strange that you mention my writing, since that’s something I do in my leisure time as a stimulating kind of self-expression, exactly the sort of thing I’m worried the em world wouldn’t have enough of.

But that’s kind of the point, isn’t it? You’re exceptionally productive at writing thoughtful and well-researched essays, doing the kind of work that somebody else might consider tedious, because it doesn’t feel like work to you. It feels like leisure, the kind of enjoyable thing that you do in order to relax and which you’re happy to do even though you don’t get paid for it. Which is exactly what “fully enjoying your work and being happy to spend most of your time on it” feels like from the inside!

In the em-copying scenario, that’s exactly how a lot of ems would view their existence – they would think of themselves as being awfully lucky to have the opportunity to spend lots of time doing something that they view as enjoyable leisure. Because that’s the kind of a mind who’d be the most motivated to work hard on something – one that didn’t view their work as work.

Viliam

I guess it’s about autonomy.

When in job, Yvain must do what he is told, when he is told, and any of his inputs can be rejected even for nonsensical reasons. When writing a blog, Yvain chooses the topic, the time, the length, and the style. Also, he is free to stop writing the blog, if he chooses so.

Now imagine that Yvain would be updated to computer and told: “You must keep writing the blog every day, and if you weekly fail to attract at least 0.000001 of the total online readers, you will be deleted.” He probably wouldn’t like the scenario, and would probably start hating the blog itself.

So, maybe I am projecting here, but I guess Yvain values doing things where he is not under threat of punishment if he fails to reach some quota. The fact that he reaches the quota anyway is a different issue.

We could reach the equivalent situation by lying to the Yvain-em, by telling him that his work is completely voluntary, but then deleting him anyway if he fails to reach some quota. (And then replacing him with some old backup, and making up some cover story for why he doesn’t remember writing his recent articles.) In this scenario, Yvain would probably object against being lied to, especially in things that impact his survival.

My analysis says ems are the most productive, it doesn’t say they are told what to do. If not being told what to do makes you more productive, then that is how ems will be.

Sniffnoy

I’m confused. Doesn’t “productive” in an economists sense normally mean something that someone is paying for? And — apologies if this is answered in the book, which I haven’t read — but if all you mean is that ems would do productive things with their leisure time (or “free time”, if you insist that productive time does not count as leisure), how would this come about? My understanding is that ems would be constantly productive — in the sense of doing paid work — because otherwise they won’t have the resources to stay on. How does whether one does productive or unproductive in one’s free time (like, writing blog posts vs watching TV) have any effect on that?

“productive” here is in an evolutionary sense of leading to more descendants (weighted by resources like speed).

Sniffnoy

Hm. I’m still unclear though as to what reason is there then to think that ems will have any “leisure time”, in the sense of time to work on whatever they choose to, regardless of whether they can find a buyer for it.

So what value do you assign to the lives of typical residents of the farming era? They had near subsistence wages in a very competitive world, and their jobs were so much less mentally challenging than the typical forager job that brain sizes decreased over the farming era. Were typical farmer lives only valuable because they eventually led to lives like yours?

But if farming was a very competitive world, and farmers had substantial leisure time, they why wouldn’t the competitive em world also have substantial leisure time? I’m questioning your inference of low leisure from the key fact of competition.

Jns

We get the word zombie from the somewhat substantiated rumours about slaves who were drugged to keep them docile and working mindlessly, tirelessly and with superhuman endurance.

For the people in charge without access to infinite amounts of cheap drugs leisure time was a concession weighed against riots and rebellions.

A process is more efficient and cheaper without superfluous components and the most efficient use of an em is to poke at it until it does what you want and only what you want forever. If not everybody would be on board with such a thing then the market would select for those who are.

I had not heard of a connection between zombies and plantation slavery. I guess it could make sense, I just thought its roots went further back to west African religious practice.

Jns

We know of slaves that were drugged and we know that the original description of how a zombi was made was making the victim inhale a powder that made them servile and took them into a state between living and dead but it’s difficult to know whether that was the original meaning of the word or if it’s an explanation that got applied retroactively.

I hadn’t heard about slaves being drugged. Usually social control is indicated as being sufficient.

Jns

At scale it’s probably far superior than having to maintain constant supply and oversee a large workforce. Still it’s not like this practice ever went away as drugs are still used today to keep people in positions they would rather escape from from Hollywood actors (Robin Williams notably talked about him and other talent being kept on a leash with drugs as a matter of course), sex workers to scandals like the one in Bulls-Hit, Hastings.

I’m skeptical about the state of pharmacology in the past (drugs existed, but I expect most lowered productivity) and the willingness of overseers to bother. Robin Williams wasn’t a slave and was personally consuming cocaine for the same reasons most people do.

Could you give some links regarding the substantiation of those rumors?

and lot of creative people in the 70 and 80 were introduced to drugs under similar reasoning by their producers, a single reliable provider making it difficult to bargain for a better position or leave

as for the link between that and zombies is far more tenuous Felicia Felix-Mentor, Clairvius Narcisse are the two most famous cases

The linked infographic begins with the “late 1800s”. The Civil War ends in 1865, so there’s a limit to how late it can be and still part of slavery. The alkaloid was first synthesized in 1855 and used as an anaesthetic, but this research took place in Europe. Perhaps coca leaves were being imported to the United States before then and given to slaves, but I’m skeptical going off a citation-free reference to slavery on that page.

Jns

Perhaps not but you specifically mentioned cocaine and being sceptical of the idea that drugs were used in the past to increase productivity.
It’s doubtful that cocaine in particular would have any connection to that period.

As for whether or not plantation slaves in the US were ever kept in line with drugs I wouldn’t hazard to guess. Rumours about Haitian zombie were theorised to have more to do with puffer fish toxin or some unknown psychosomatic substance.

I was skeptical about cocaine because it wasn’t especially popular that far back, like I said it wasn’t synthesized until 1855 and that’s near the end of slavery in the US. Haiti of course abolished slavery earlier. The pufferfish toxin is supposed to cause a coma or the appearance of it, which you’d expect to cut down on productivity.

Jns

It would at that, wouldn’t it? Datura was also a suggestion.
It’s hard to know whether we’re overfitting a convenient theory in retrospect since the evidence that exists isn’t very good. Personal accounts are unreliable and even if it was practised the lack of actual records means we’re still just dealing in rumours.

Samuel

A key difference between farmer and em words is the enormous amount of dependence farmers had on natural processes. Things like available daylight and the speed the crops grew. From my history study while during certain times of year farmers did indeed work basically every available hour during other times they simply wasn’t any productive work to be done. Especially during winter when there was nothing to plant nothing to harvest and it was dark half the day anyway (at least in Britain). An em civilization would not I think suffer from such lulls and even if it did could simply pause the ems.

It is just silly to claim there was nothing useful farmers could do in the winter. Working less then was a choice.

Yvain

I think farmers had lots of leisure because if you only have an acre of land and you want to grow corn on it, after planting the corn, watering the corn, and so on there’s not that much extra you can do. Getting more land wasn’t an option because most farmland was already occupied early in the farming era, and because there were different cultural norms around capitalism and reinvesting your proceeds back into making more money.
There were also long fallow seasons when you couldn’t grow crops even if you wanted to.

Ever played the board game Agricola? There are SO many useful tasks & chores one can do in a farming society, even in winter.

jhertzli

So… When workers can keep more they work more?

Ronfar

Subsistence farmers have downtime in which to rest and socialize. The intensity of work tends to be seasonal; in ancient Egypt, there was enough surplus labor during the off season to build the Pyramids. So there would have been plenty of time for people to add value to their lives through non-“work” activities.

With life being cheap in the em-world, the horror of war greatly reduces. The penalty of autarky also greatly reduces with people being able to train their equivalents of 18 year old selves on practically any task. What prevents a war of clans against other clans in this scenario?

Life being cheap doesn’t actually make war more likely; wars of attrition are feasible and expensive. But I can’t assure you there won’t be war.

Lord

I see it quite differently. Ems as our virtual selves and while they/we may be obsessed with tasks that may be called work, and some lower tasks necessary to accomplish their higher ends, it is these higher ends that will dominate, among these will be creating their virtual worlds as rich as ours, including interfaces to ours. They will be bound to drudgery only to the extent necessary to automate and remove it from their need to do it. This is why an em/ai dichotomy is misleading. One of their main aims will be to produce ais to handle all that and ais would evolve as fast as ems, and their world would evolve as much as they do. Their first task would be to understand themselves. Thus, there won’t be endless ems, but a city with a mix of ems and ais growing in complexity.

I tried to present the situation as something that you might consider to be a problem, but that while my analysis should enable better problem solving, I’ve personally focused on just describing this situation. Mixing up normative and positive discussions risks the positive being overshadowed by the normative, and positive claims seeming less reliable when mixed up with more disputable normative claims.

One commenter even asked whether Robin intended to write a follow-up book addressing normative issues. But it seems that this book treats normative issues freely, which makes me wonder how Robin took his own advice to separate them.

—————

On another topic: is there a scientific consensus that ems are theoretically possible? This is an area where I can’t claim the slightest expertise, but I have to wonder about the certainty that the causal relationships underlying behavior can be completely expressed in code. It would seem that the lines of code needed to express a causal relationship in the real world would be infinite in number – similar to the problem of reducing theoretical statements to an observation language. Perhaps a finite code could be practically sufficient, but is this guaranteed? And even if it is, what grounds are there for thinking that the amount of code required for practical usefulness isn’t prohibitively huge?

I think if you actually look at the book you will see it is very light on normative discussion.

Marc Geddes

The Em scenario is nonsense of course. Upload technology is not easier than AGI, it is much much harder …the human brain is the most complex object in the known universe. There will be uploads one day, but that will be *after* super-intelligence, so human uploads won’t be workers.

Makes no sense at all to use uploads as workers. The deep learning techniques we have today are probably sufficient to replace 40% of the workforce alone (you don’t even need full AGI to replace workers) , so why on Earth would an employer want to use the far more expensive and messy human mind for work purposes?

Listen, don’t be too hard on yourself though Robin, your intellectual contemporaries (Bostrom and Yudkowsky) have also got just about everything wrong on the topic of superintelligence, so you’re in good company 😉 The three of you make a truly hilarious trio for future historians to mull over.

Alexander seems to strongly endorse the usual forager value of leisure over work, so much so that he can’t see people focused on their work as human, conscious, or of any moral value. Creatures only seem valuable to him to the extent that they have sex, leisure time, minds wandering away from work, and desires to do things other than work.

Would you say that the Marxist criticism of “alienated labor” is forager? The classic Marxist view is that work should be changed to make it desired in itself – not that it be abolished in favor of leisure.

It seems that em society solves the problem of alienated labor – but not by changing labor’s nature, but by changing man’s nature. A hyper-intense selection process creates a humanity enamored of its work – pretty much whatever that work happens to be.

Yes, since foragers live in a world they were evolved for, they find their work feels more natural. It is farmers who first needed to get used to and accept less-natural work, which required more self-control and social pressures to get people to do them. So wanting to return to that first situation is forager-like.

Vamair

Wouldn’t there be companies that create more productive modified ems? It may be profitable, for example if any such em is paying the company some percent of the em’s gains. The results would be the optimization of an em.for its particular area of expertise and cutting down on all the other processes to improve the rate of money earned to the processor-time. (I would start with cutting, it’s probably easier). In time such ems turn into optimized algorithms for its job. And somehow I don’t think a specialized contract-drafting algorithm has an intrinsic value.

Joe

Depends on the cost of modifying ems. If ems are expensive to modify, doing so won’t be worth it.

Regarding your point on value, I’m curious: do you think specialized hunter-gathering algorithms have intrinsic value? Or in other words, what is it about humans that you think gives them value, that you don’t expect to see in other intelligent entities?

Vamair

Ems are data, why should they be expensive to modify? Even if they are, the gains could be extremely large given their numbers. I don’t in general care much about individual ants, so hunter-gatherer algorithms aren’t in general too valuable either. And no, I don’t really know why I value humans and not fruit flies.

Today we have many legacy software systems that are hard to usefully change, even though we can change any bits we want to. Hard to figure out which bit changes would help.

Vamair

We aren’t living in the conditions where we can profit from every CPU operation we’ve freed and doing that is a matter of life and death. A corporation can easily experiment on ems, and if they’re able to get rid of a half of an em’s brain without it affecting work performance more than twice, they’ll do it as fast as they can. It’s even possible to propagate some changes, given that human brains are similar. And we aren’t even talking about writing specialized algorithms from scratch and then replacing ems with them.

Joe

I think this ignores opportunity costs. It might well be possible to gain some extra performance from ems by spending lots of time and money trying to modify their brains. But in order to conclude that this will be done you also need some reason to expect such experimentation to be a better use of those resources than any of the other possible ways they could be used.

Additionally, since research is largely a public good, I think you would expect a sub-optimal amount of it to be done even in a world of ems that are better at coordinating than we are.

Vamair

You only need to optimize a single em a little to let it and its copies take over a whole industry. There’s no reason a company would not take an em that does the same work but uses 10% less processing power. If the modified ems pay 5% of their wages back to the researchers because of the licensing, then the researches get to keep a 5% of the industry’s worth of wages while there’s no better em. And if they’re the only such researchers, that would continue indefinetely. Which means such research is lucrative. Which means it would be done by a lot of people and would be much faster if not so lucrative.

Peter David Jones

The background assumption is that we will be emulating ehole brains becase we don’t know how to engineer AGIs out of components, so we have to So why would we know how to re engineer brains, which is a harder problem, since they are not cleanly designed.

IMASBA

Well maybe in the em-world there will be a market for Scott Alexander copies who alk to other ems just before those ems get scrapped/executed…