Layers of Delusion

Humans have many delusions, i.e., mistaken beliefs they are especially reluctant to correct. To understand such delusions, it helps to understand the many layers that make up the human mind. In the following I outline my best guesses, ordered roughly by time/layer. (I’m especially interested in what I’ve missed or mis-classified.)

Animal – Animals have core mental processes to manage desires for food, warmth, sex, and to avoid harms. These processes can misfire, creating errors that other levels are reluctant to override. For example, we can be terrified of heights, even when we “know” we are safe. Animal minds are organized by level of abstraction (near vs. far), and so expect things near (far) in some ways to be near (far) in other ways. Animals usually act as if things not directly in view don’t exist. Embedded in brain architecture, these mistakes are hard to correct.

Socialite – Animals that pair-bond have bigger brains, to deal with trust issues in long-term relations. Social animals can betray one another, and have relative status, and so can be mistaken about status and partner loyalty. Social distance adds to near/far. Social mammals use a standard stress response for social stress; being disliked hurts health as if others had psychic powers.

Primate – Very social animals have meta-beliefs about who thinks what about who, and so on. So they can be mistaken about meta-beliefs. Primates can often gain from errors via favorably influencing others’ meta-beliefs. For example, overconfidence in one’s ability or loyalty induces confidence in others. This creates selection pressures for delusions. In very social primates, power and status depend less on individual abilities and more on political coalitions. Primates can thus be deluded about who supports which coalition, and how strong are coalitions.

Talker – With language, we can say more things, and so can lie, be mistaken, and be deluded about more things. We find it hard to appreciate how language changes our thought, and so are often deluded to think reality divides neatly according to our word categories. We want our words to be believed, and to seem confident they will be believed. So we are deluded to think we reason more to find truth than to win arguments, and to think reality constraints shared beliefs more than it does.

Forager – Using language, foragers coordinate to enforce social norms against overt non-family dominance, bragging, sub-band coalition, and band-harming selfishness. Norms add to near/far as far goals. But since norms can only limit commonly-visible behavior, foragers violate norms covertly. Since conscious thoughts are more visible, they dominate, brag, ally, and self-serve unconsciously, via hard-to-verify eye contact, body motions, tone of voice, word double-meanings, etc. Consciously homo hypocritus foragers are deluded, especially in far mode, to less see the unflattering functions of their acts, and their bowing to social pressure. For example, foragers and their descendants have diverse styles (dress, body, music, food, language, stories, etc.) and are biased toward seeing :

Changing styles as just better, vs. show gossip ties, social savvy, loyal.

Local styles as just better, vs. show local ties.

Attraction from shared values, vs. impressed by features, loyal.

Medicine, charity as help, vs. show loyal, wealth.

Gossip as curiosity vs. collusion, status moves.

Politics as help group, vs. show values, loyal.

Far talk as curious, info share, vs. dominate, show smarts, ties.

Laughter as due to funny events, vs. show comfort, loyal.

Stories as social practice, “fun” vs. show values, discernment.

Art as a pursuit of beauty, insight, function, vs. bond, show off.

Sport as healthy, “fun”, vs. show off.

Farmer – Higher forager density led to trade and rapid innovation, which led to herding, farming, marriage, war, and social classes. Social norms expanded, to induce behavior in conflict with forager inclinations. These included norms of long hard work hours, fair trade with strangers, life-long marriage, deference to elite classes, and of patriotic devotion in war. Frequently invaded regions evolved pro-community over pro-family norms. Added farmer self-control (i.e., norm adherence) came from norms encouraging far-thinking, persecuting deviants, just-world-delusions,and religion, i.e., submission to supernatural moralizers. Farmers were deluded on norm origins, and on high costs of environment alienation and from repressing forager-desires.

City – Many folks living close is a productive, if alien, lifestyle whose anonymity can reduce norm pressures that come from non-work social monitoring. This weakened non-work norms like marriage, family, and religion. It also made social status depend less on informal reputation and more on clear signals like wealth, degrees, fame, etc. City folk seem deluded to think informal reputation and social monitoring are stronger than they are; e.g., they credit confidence more than they should. Cheap surveillance, however, may soon strengthen social monitoring.

Rich – Industry has recently made non-elites rich enough to afford to reduce alienation, e.g. more greenery, and to please their inner forager by reducing non-work self-control. Such folk return toward forager levels of sexual promiscuity, though via cheating and serial monogamy instead of polygamy, and less social monitoring in cities. Rich folk get less religious and patriotic, and seek political forms to mimic forager-style democracy, deliberation, food sharing, and sick-helping. They deludedly justify such policies in other ways, e.g., medical market failures. Coordination remains harder than most realize.

Stimulant – Industry has devised hyper-stimulating food, art, stories, sport, games, drugs, etc., which rich low-self-control folks eagerly consume, at the expense of work, kids, and work-like-hobbies. Hyper-status-seeking can compensate, inducing more work and hobbies, but not more kids. Most are deluded to think this a stable situation; if allowed, gene and culture selection would rapidly cut such waste. Most also have the addict’s delusion, “I can quit anytime I want.”

Emulation – The next big change is likely whole brain emulations (i.e., ems), within a century or so. Profit-seeking investors may make trillions of copies of dozens of most suitable humans, and further select among trillions of ways to tweak each em. This will allow enormous selection for the most adaptive em minds. Adaptive behavior in the early em era has high work coordination, accepts more alien bodies and environments, has little interest in kids or hyper-stimuli, and accepts death, high trainee failure rates, long work hours, and near-subsistence wages. Some of this may be achieved via genuine preference changes, but initially most will be achieved via strong delusory self-control.

Stability – More big eras may appear after ems, but soon rapid change will end. We now live in the brief few-millenia “dreamtime” when people are poorly adapted to their environment. Within a few millennia, and then for trillions of years thereafter, economic growth and innovation rates will slow to a near halt, and people will once again be well adapted to their stable slow-grow world – as were foragers for millions of years. As with foragers, what they do will mostly be adaptive, even if they are deluded about why they do it. And they may well be much less deluded, due to better academic knowledge, more mental transparency, ubiquitous documentation, and more prediction markets.

I note that the Primate and Talker layers may encourage people to think they have successfully identified others’ illusions, and be over-confident in their own assesments of how other people are deluded.

Interestingly, in your aristocrat point you assert that elites need certain beliefs, so to achieve these farming elites pioneered certain things, including schools that aim to achieve them. However, you have never presented any good evidence that schools do actually achieve those results, as opposed to intending to achieve them and I have produced a number of counter-examples indicating that they haven’t. Why do you keep restating this idea about schools then without mentioning any caveats? Are you perhaps falling subject to the Primate and Talker layers?

Another alternative is that I am so deluded that I am missing a bunch of empirical evidence supporting the argument that schools do achieve their intended results. 🙂

anon

I’m not getting your point. Aristocrat schools were very different from modern mass education; it’s quite possible that these schools were more focused on promoting desired values than sharing useful info, especially since ancient elites had nothing comparable to modern academic knowledge.

Tracy W

I’m talking about modern schools, which were introduced by the people in political power at the time. Robin linked to a previous post of his about public schools (state schools for the British) so I presumed he was continuing that line of thought.

http://www.twitter.com/tdr_dmorgan Daniel M

I doubt this works so neatly chronologically. Deference and war are found in wolf packs. Perhaps I’m misinterpreting the categories.

For example, overconfidence in one’s abilities induces confidence in others.

I read this as: if I wrongly believe I’m competent, others may (correctly) see that I believe it. That wouldn’t be an example of inducing a mistaken meta-belief in others; like this is: I know that I’m incompetent, but I convince others that I think I’m competent.

Of course I realize that other-deception is most convincing when the telltale parts of the liar’s brain are self-deceived.

Minor point, but the invasion concept doesn’t make that much sense to me when contrasting the nuclear family and open access institutional framework of Anglo nations versus the clan and closed access framework of societies along the arid Afrasian belt.

Perhaps I’m seeing a conflict that doesn’t exist, but societies and regions vulnerable to invasion seem clannish in the extreme. How is this not “pro-family norms” rather than “pro-community norms”?

http://graehl.posterous.com Jonathan Graehl

Magnificent storytelling. I love the insinuation that the liberal rich are juicing forager-era emotional rewards.

I agree that people are often unaware of the real explanation for their (often hypocritical) beliefs, actions, and desires. (of course, I doubt that Robin Hanson’s stories constitute exactly the “real explanation”).

I wondered if it’s right to call all of these “people are wrong about why they are/believe/choose …” errors “bias”. What sort of actual errors in predicting the world and its response to our choices do these wrong self-beliefs cause? It’s not obvious for all of them.

Or, put another way, do self-only biases matter? Suppose I perfectly plan and predict my environment, and model other people fine; I just have a blind spot about myself. If I have a wrong belief about what will truly satisfy me, does knowing that help me make better plans? If I’m wrong about what I’ll really be willing and able to do, can I only make plans that I will actually execute? I think so.

Evan

While I generally like this series of posts I think a few times Robin has been skirting dangerously close to the common mistake of conflating the metaphorical “purposes” and “desires” of our selfish genes with our own subconscious, like those people who think we have a subconscious desire to spread our genes when really we are just programmed to like sex and think babies are cute without really knowing why (note, that I am giving this specific mistake as an example to clarify what I am saying, not saying Robin makes it).

For instance, in the “Aristocrat” section he talks about how aristocrats support chivalry to placate non-elites. Isn’t it equally possible that aristocrats and elites are simply programmed to like chivalrous norms without knowing why and the fact that it helps keep them in power is a result of genetic and cultural selection (aristocrats who lack genes coding for a love of chivalry get overthrown).

Similarly, in the Rich section, it is proposed that medical helping norms serve to show wealth, loyalty and caring, and the idea that they are motivated by helping is a delusion. Isn’t evolution more subtle than that? Another possible explanation could be that there is simply a switch (metaphorically speaking) in our brains that gets activated when we feel financially secure and induces us to display far-level helping behavior, without any need for any delusory process.

Maybe I’m just reading this wrong, but it occasionally sounds like Robin is proposing that we are programmed by evolution to subconsciously seek status and wealth while subconsciously deluding ourselves that we have nobler motives, when it seems equally possible that evolution programmed us to have genuinely noble motives that happen to signal positive attributes to others, but also included akrasia to make sure we didn’t go too far and harm our fitness.

A sort of natural experiment might occur in the future if we invent some sort of cheap and effective drug that inhibits akrasia, some sort of super-adderol with less side effects. Would we see more focus on work and work-like hobbies?

Also, I have great difficulty seeing the ems accepting the sort of work conditions Robin implies will exist. , I doubt ems of humans would either. More likely they’d work hard enough to survive, and to raise enough money to build a big VR playground for themselves and other ems to pursue work-like hobbies and super-stimuli. This is especially true if they keep the anti-domination instincts humans have during the transition to ems.

Evan

Made a mistake in that last paragraph, I meant to add a link to Robin’s “Self Control is Slavery” post to emphasize my point (since the point of that post is that farmers wouldn’t voluntarily work long hours unless they were paid a whole lot more than subsistence wages) but I don’t think I quite understand how to insert links. I’ll just add the whole thing http://www.overcomingbias.com/2010/06/self-control-is-slavery.html

http://timtyler.org/ Tim Tyler

It isn’t clear that there will be more prediction markets in the far future. We may see fewer marketplaces. Markets are ways of resolving resource disputes among competing agents. In the future we may see universal cooperation – in which case markets will not be needed, and other methods are likely to be used to allocate resources.

Matthew Fallshaw

Prediction markets are computers, rapidly working out which other algorithms most accurately calculate the values of things. If Robin is right that the rate of change will slow, then the dominant algorithms markets are using will become obvious, and the markets will stop adding value.
Central planning fails today due mainly to information and incentive problems. I think Robin is describing a world in which central planning would work better than markets.

http://kazart.blogspot.com Mike

“Animals usually act as if things not directly in view don’t exist. ” My poodle-retriever mix will look for tennis balls for quite a long time when it has been thrown or put out of her sight. If she has a tasty bone we have forced her to leave outside, she will head straight for where she left it the next time she is allowed out.

If dogs do it, I’d be surprised if primates didn’t do it.

SymbolicalHead

Yes, as good an argument as it is about people that live with their heads in the sand (and there are a lot) it is a poor argument to explain people that spend their lives worrying over imaginary problems (and there are a lot of those).

Anonim

@Mike: Is your dog capable of paying back monetary debts? 😉

http://www.alfin2500.blogspot.com Alice Finkel

Robin’s terminal expectation of stability is clearly delusional. Why Robin’s subconscious feels such a strong need for this expectational belief is not clear. No doubt a number of clustered delusions underlie the belief. 😉

Steven Schreiber

Everything is always so dystopian in Robin’s world.

Harmonious Jim

There seems to be a thread running through many of these Historical Modes of Delusion (each with their Means of Delusion and Relations of Delusion): the common thread is that people tend to think that they are more noble, knowledgeable, helpful, sharing, truth-seeking, and generally good than they really are.

Missing delusions: 1) males and females each have delusions specific to their own sex — concerning sex. 2) In agraria we get specialists in violence (warriors), ritual and ideas (priests), and production — each has their own delusions. But surely the priests’ are the greatest, since their trade was delusion.

mjgeddes

Humans have many delusions, i.e., mistaken beliefs they are especially reluctant to correct

.

Indeed. I fear that many postings of the LW/OB crowd will confirm this for future historians all too well.

The ‘Overcoming Bias’ postings seem to be dangerously close to conflating what is evolutionary adaptive with what is good. Personality, I don’t give a toss about evolution or ‘reproductive fitness’ and neither will the posthumans.

Great! Personality I find reality dull by comparison. Super-stimuli are good because they form the basis of positive conscious experience, I say the more super-stimuli the better (within certain limits). I’m not saying we should retreat from reality – certain contraints will always tie us to reality to some extent, but these contraints are ultimately weak – the only real limit I would set is that consumption of superstimuli should not compromise our individual survivial – but this condition is not the same as the evolutionary one (genetic fitness).

More likely, evolution will be overthrow by a centralized singleton, the future will be saturated with superstimuli, far mode activities (arts) will come to dominate, and economic activities or evolutionary psychology won’t have much relevence, because all the near-mode stuff will have been automated, and we will have modified many of our preferences.

All the OB/LW visions are off track. There can be no reflective decision theory without sentience (conscious experience). In fact, your pututative ‘reflective decision theory’ is equivalent to information theory. Bayes is not the foundation of rationality (it is merely a special case of categorization). CEV is nonsense (you can’t perform these extrapolations without conscious deliberations). There will be no ems explosion. Wages won’t decline to the survival level. This is because economics and evolutionary psychology will fade into insignficance, as sentient super-intelligences based not on CEV but on an information-theoretic definition of beauty and aesthetic-preference take centralized control, stop evolution in its tracks, and saturate the future with superstimuli.

I look forward to my kind of world, where the need for tedious work has been dispensed with, annoying kids and sullen teenagers are few and far between, the Bayesian paradigm has been overthrown, markets and evolution have been trampled under the jack-boots of a centralized Singleton, and I can indulge in superstimuli to my hearts content.

*sigh* All that is saturating OB/LW at present is bad thinking, sloppy suppositions and useless AI designs.

http://new-savanna.blogspot.com/ bill benzon

Hi Robin,

You might want to take a look at an old essay that David Hays and I published in the old Journal of Social and Biological Structures: Principles and Development of Natural Intelligence (download here): Here’s the abstract:

The phenomena of natural intelligence can be grouped into five classes, and a specific principle of information processing, implemented in neural tissue, produces each class of phenomena. (1) The modal principle subserves feeling and is implemented in the reticular formation. (2) The diagonalization principle subserves coherence and is the basic principle, implemented in neocortex. (3) Action is subserved by the decision principle, which involves interlinked positive and negative feedback loops, and resides in modally differentiated cortex. (4) The problem of finitization resolves into a figural principle, implemented in secondary cortical areas; figurality resolves the conflict between pro-positional and Gestalt accounts of mental representations. (5) Finally, the phenomena of analysis reflect the action of the indexing principle, which is implemented through the neural mechanisms of language.

These principles have an intrinsic ordering (as given above) such that implementation of each principle presupposes the prior implementation of its predecessor. This ordering is preserved in phylogeny: (1) mode, vertebrates; (2) diagonalization, reptiles; (3) decision, mammals; (4) figural, primates; (5) indexing. Homo sapiens sapiens. The same ordering appears in human ontogeny and corresponds to Piaget’s stages of intellectual development, and to stages of language acquisition.

That’ll only give you “five” layers, but then your scheme includes cultural developments. I’ve got an old paper that does some of that in terms of narrative and personality: The Evolution of Narrative and the Self (download here). Abstract:

Narratives bring a range of disparate behavioral modes before the conscious self. Preliterate narratives consist of a loose string of episodes where each episode, or small group of episodes, displays a single mode. With literacy comes the ability to construct long narratives in which the episodes are tightly structured so as to exhibit a character’s essential nature. Complex strands of episodes are woven together into a single narrative, with flashbacks being common. The emergence of the novel makes it possible to depict personal growth and change. Intimacy, a private sphere of sociality, emerges as both a mode of experience depicted within novels and as a mode in which people read novels. The novelist constructs a narrator to structure experience for reorganization.

http://www.hopeanon.typepad.com Hopefully Anonymous

Nice and ambitious post.

http://adrianmonck.com Adrian Monck

Surely the biggest problem we face is that much of the enjoyment we gain from life is down to satisfaction of basic animal impulses (social interaction, physical activity, sustenance, procreation etc.) and that fulfilment of those skews much of our economic organization and therefore resource use.

To move “beyond” that into your transhuman phase would not be very fulfilling for human beings… as Keynes so rightly said, we have seen the future already and it is the rich. He wasn’t being optimistic…

(Incidentally, ironic that an activity like hunting could remain an “aristocratic” or elite activity for millennia past its functional expiry date.)

Regirock

“fun” eh?

I take it you don’t believe that there is any human endeavor taken for entirely personal reasons or reasons that are only indirectly connected to signaling?

The explanation of signaling does not map well with our intuitions about human behavior, our internal dialogue on human behavior, nor entirely with formal study of human behavior.

I’m not saying that our psychology isn’t built on tribal evolution but our mental states today are not simple signaling mechanisms, they are complex emergent states that no longer map perfectly to their simplistic roots.

In other words I think you have taken your eliminative reductionism too far.

http://actualfreedom.com.au/ John L

The next step in human “evolution” is already available, and it actually requires intentional altruistic mutation– done now– rather than arm-chair philosophizing, technology, or anything of the like. Such a mutation is not just the next step, by the way, but the final as well; it spells the end of human evolution (because perfection cannot evolve). It is with great delight that I am able to say with empirical proof that said “next step” is possible for anyone able to read and understand the contents of this web-site:

> For example, we can be terrified
> of heights, even when we “know”
> we are safe.

Putting the word “know” in quotation marks suggests you don’t intend it to be taken seriously, i.e. we think we know, we say we know, but maybe we don’t really. Is that what you meant? Quotation marks tend to de-emphasize a word.

mjgeddes

I have realized that superstimuli play an absolutely central role in conscious experience – to wit, we are all under the control of superstimuli! Evolutionary psychology may point to the central importance of sex, but remember, the female form itself (sex symbols) becomes a superstimuli when make-up is applied (blusher, lip-stick etc. etc).

And that’s just human created superstimuli. Imagine what superstimuli a transhuman could create. Actually we probably couldn’t imagine. Suffice it say the prols could be easily controlled, without any actual coercion or even without people realizing they are under control.

The propoganda techniques first discovered accidently by Nazi Germany actually had their origins in the arts – Hilter was a failed artist and Goebbels was an arts aficionado. Arts are based on the principles of superstimuli. Consider that the ability to weld superstimuli effectively literally did enable these evil folks to conquer half of Europe.

The amount of consumption of superstimuli and the power they weild seems to be massive and out of all proportion ( sports, drugs and pop culture are only the beginning, consider again political propganda, and the example given above)…

Something is going on here that even the Hansons and Bostroms have missed, no?.

Considering the power of superstimuli and its association with arts and the cognitive skill of categorization should give Bayesian fan-boys very serious pause as to whether theirs is the ultimate cognitive power. Black swan warning sirens are wailing somewhere in the distance. Think it through folks.

Eric

Does it frustrate anybody else to try to read lists that are composed of words/phrases of different parts of speech?

Animal: “Animals usually act as if things not directly in view don’t exist. Embedded in brain architecture, these mistakes are hard to correct.” This is an odd comment as applied to human psychology. We are constantly, routinely aware of the existence of things not currently visible.

Socialite: “Social mammals use a standard stress response for social stress; being disliked hurts health as if others had psychic powers.” I suppose the cognitive error in this is actual belief in the psychic powers of other people. This has been widely overcome in the civilized world. Of course, you are claiming only that overcoming it was *hard*, not that it was *impossible*. The vagueness of your claim makes it hard to evaluate.

Primate: “Primates can thus be deluded about who supports which coalition, and how strong are coalitions.” We are also prey to many other sorts of overconfidence. But some people are chronic self-doubters; where do they fit your scheme?

Talker: “We . . . are often deluded to think reality divides neatly according to our word categories. . . . We . . . are deluded to think we reason more to find truth than to win arguments, and to think reality constrain[]s shared beliefs more than it does.” These seem like philosophical errors, of little practical importance.

Interesting stuff, even if not wholly convincing.

Patrick McCann

To make sure you haven’t missed any, you should make sure all the entries over at the self-delusion blog, http://youarenotsosmart.com/, fit into your groupings.

That’s a good question. I’d guess it arose with farming, since it correlates with farming-era values: “people who … believe in a just world also tend to be more religious, more authoritarian, more conservative, more likely to admire political leaders and existing social institutions.” To strengthen the power of social norms, we came to believe those who follow social norms are rewarded, those who do not are punished, more so than they actually are.

To understand such delusions, it helps to understand the many layers that make up the human mind.

Does it? For example, I can trace my father’s belief that the Black Panthers are going to take over society in a civil war to the primate layer: “Primates can thus be deluded about who supports which coalition, and how strong are coalitions.” But how does that help me understand it?