A novice rationalist studying under the master Ougi was rebuked by a friend who said, "You spend all this time listening to your master, and talking of 'rational' this and 'rational' that—you have fallen into a cult!" The novice was deeply disturbed; he heard the words, "You have fallen into a cult!" resounding in his ears as he lay in bed that night, and even in his dreams. The next day, the novice approached Ougi and related the events, and said, "Master, I am constantly consumed by worry that this is all really a cult, and that your teachings are only dogma." Ougi replied, "If you find a hammer lying in the road and sell it, you may ask a low price or a high one. But if you keep the hammer and use it to drive nails, who can doubt its worth?" The novice said, "See, now that's just the sort of thing I worry about—your mysterious Zen replies." Ougi said, "Fine, then, I will speak more plainly, and lay out perfectly reasonable arguments which demonstrate that you have not fallen into a cult. But first you have to wear this silly hat."

Ougi gave the novice a huge brown ten-gallon cowboy hat. "Er, master..." said the novice. "When I have explained everything to you," said Ougi, "you will see why this was necessary. Or otherwise, you can continue to lie awake nights, wondering whether this is a cult." The novice put on the cowboy hat. Ougi said, "How long will you repeat my words and ignore the meaning? Disordered thoughts begin as feelings of attachment to preferred conclusions. You are too anxious about your self-image as a rationalist. You came to me to seek reassurance. If you had been truly curious, not knowing one way or the other, you would have thought of ways to resolve your doubts. Because you needed to resolve your cognitive dissonance, you were willing to put on a silly hat. If I had been an evil man, I could have made you pay a hundred silver coins. When you concentrate on a real-world question, the worth or worthlessness of your understanding will soon become apparent. You are like a swordsman who keeps glancing away to see if anyone might be laughing at him—" "All right," said the novice. "You asked for the long version," said Ougi. This novice later succeeded Ougi and became known as Ni no Tachi. Ever after, he would not allow his students to cite his words in their debates, saying, "Use the techniques and do not mention them."

A novice rationalist approached the master Ougi and said, "Master, I worry that our rationality dojo is... well... a little cultish." "That is a grave concern," said Ougi. The novice waited a time, but Ougi said nothing more. So the novice spoke up again: "I mean, I'm sorry, but having to wear these robes, and the hood—it just seems like we're the bloody Freemasons or something." "Ah," said Ougi, "the robes and trappings." "Well, yes the robes and trappings," said the novice. "It just seems terribly irrational." "I will address all your concerns," said the master, "but first you must put on this silly hat." And Ougi drew out a wizard's hat, embroidered with crescents and stars. The novice took the hat, looked at it, and then burst out in frustration: "How can this possibly help?" "Since you are so concerned about the interactions of clothing with probability theory," Ougi said, "it should not surprise you that you must wear a special hat to understand." When the novice attained the rank of grad student, he took the name Bouzo and would only discuss rationality while wearing a clown suit.

If I am concerned that the group I belong to is becoming cultish, the thing to do is to ask what is a cult and what is not a cult, and see if the definition applies to your group. The second koan reminds us not to use extraneous details like uniforms. More non-cultists than cultists use uniforms. In general P(Category|Feature) != P(Feature|Category).

Maybe... but here's the thing: Uniforms actually ARE a very cultish thing. They are one of the quantitative traits that can add up into driving you into the cult attractor. The proper rationalist response is actually "I will not wear the hat, because I don't want to and you've not given me a reason to."

The only sort of "uniform" that is rationally justifiable is something like body armor, or a hazmat suit, or a labcoat; sure, it's all the same, because it SERVES A PURPOSE---there's a reason soldiers wear Kevlar instead of tissue paper. But if you can't actually justify the uniform (like a nurse hat, or epaulettes, or the Pope's miter), then it really IS a bad sign that you are slipping into irrationality.

For example, is a nurse's uniform as "rationally justifiable" as a hazmat suit? No. But it does serve a useful purpose for nurses, in that it frequently makes patients more likely to treat them as authority figures.

Now, you might ask why patients do that, but in some sense that doesn't matter. Even if patients are irrational to do that, it is still pragmatically useful for nurses to wear the uniform if it reliably obtains that benefit.

But in fact it isn't a senseless thing for patients to do, either, in that wearing a nurse's uniform is a more costly signal if I'm not a nurse than simply saying "I'm a nurse" (since other nurses might see me wearing the uniform and punish me), and therefore more reliable than simply saying that.

More generally: uniforms are one way humans signal a certain kind of social status, and status signaling is a valuable function.

Tiiba, do you mean to imply that if your group is cultish, the group I belong to is safe by the tu quoque principle? ;)

In all seriousness, I think the second point is a little more complicated than you described. Specifically, I think Mr. Yudkowsky is trying to point out that Bouzo had plenty of evidence more closely entangled with cult/non-cult status than whether or not they had uniforms. He had experienced Ougi's teachings directly. In the absence of that evidence, of course, the uniforms would be more important evidence.

But Mr. Yudkowski, why shouldn't we be concerned with such apparently superficial signs of cultishness? Shouldn't we -- especially the less experienced rationalists among us -- be on the lookout for objective clues to bias? After all, as has been pointed out already on Overcoming Bias, situations often look different from the inside, in ways that generate inaccuracies. (People tend to underestimate how biased their beliefs are. It's easier to show that someone's wearing uniform clothes than to show that they're wearing irrationally uniform beliefs.)

Well, I must confess I was initially arguing in gamelike fashion -- just to see what your next move might be -- but I fell into my own trap door, and I'm really unsure that
1) this isn't a cult,
and that
2) that would be a bad thing.

That is, who cares if we have mutually reinforcing behaviors etc.? What matters is whether these are good mutually reinforcing behaviors, and that we evaluate their goodness from a non-tautological (i.e. external) perspective. (That is, I'd distinguish between circles and vicious circles.)

I don't know whether or not that's part of your point. I suppose I shouldn't care. But it's certainly what the idealized Eliezer Yudkowsky in my head thinks. I certainly hope you're as smart as he is. ;)

Ian C.,

If I had a hammer that seemed to me to work really well, but no one was willing to pay me the going rate for hammers of that quality, would it really be ridiculous for me to seriously question the accuracy of my perception of the hammer?

"If I had a hammer that seemed to me to work really well, but no one was willing to pay me the going rate for hammers of that quality, would it really be ridiculous for me to seriously question the accuracy of my perception of the hammer?"

Yes, in the case of a hammer I think it would be [ridiculous to doubt yourself]. In the case of something more complicated, like Wine, you might start to question whether there is some subtle difference between your wine and theirs that you're not detecting, but there is no such thing as a hammer connoisseur.

I've occasionally met people who are surprised and even incredulous to learn that the tinwhistle, recorder, and piccolo are different instruments, though no one who has ever spent time getting good with any of those instruments would think of them as hard to distinguish, even through double blind listening tests. It's all a matter of what you pay attention to.

I think I'll side with the novices against Ougi here. The novices deserve a clearer answer than "think of ways to resolve your doubts" and "all will be clear when you try to use this stuff." Cults usually tell people many things that are actually useful, and confronting leaders seems a reasonable way to resolve doubts. As I said before, the word "cult" is a bit too easy a word to throw around - I'd prefer a clearer description of what it means exactly and how to recognize one.

- People are put in physically or emotionally distressing situations;
- Their problems are reduced to one simple explanation, which is repeatedly emphasized;
- They receive what seems to be unconditional love, acceptance, and attention from a charismatic leader or group;
- They get a new identity based on the group;
- They are subject to entrapment (isolation from friends, relatives and the mainstream culture)

Cults also seem to involve the adulation of charismatic leaders, who become corrupted by the power the situation brings them. The most common complaints against cults involve sexual abuse of members. Cults have also been known to cause harm through getting members to forego medical care.

Nope. (Well, not intentionally, anyway... does the basilisk count? People did have nightmares from that...)

Their problems are reduced to one simple explanation, which is repeatedly emphasized;

...Sort of? I mean, "rationality" is trumpeted as the Winning Way, but I'd say that's justified in that it is the Winning Way. Besides, the category of "rationality" is rather broad, which might mean it doesn't satisfy the definition of "one simple explanation"... eh. Tentatively going with "yes" for now.

They receive what seems to be unconditional love, acceptance, and attention from a charismatic leader or group;

Uh, yeah, not even close.

They get a new identity based on the group;

Pretty clear that this one gets a "yes". People self-identify as "LWers" all the time.

They are subject to entrapment (isolation from friends, relatives and the mainstream culture)

Nope.

So, LessWrong meets two of the five criteria, and does not satisfy the remaining three. Does that make it a cult, or not?

Robin, transmission of expertise in non-rational domains has to rely on authority rather than argument, so is more susceptible to slide into abuse of authority than transmission in rational domains. The original post here is strange in that it supposes such a type of transmission in the field of rational teaching. The definition of cult in the field of master / disciple relationships has to start with an examination of whether authority is being abused by, for example, being exercised in areas unrelated to the teaching. Don't take sweets from philosophers.

Ian C., do you know for sure, before evaluating this kind of evidence, whether the problem is simple like you seem to think a hammer is, or complicated like you seem to think wine is? I had thought that wine is simpler as its functionality is a matter of taste, while hammers' functionality is something that exists in some sense outside my first impression of it. (It may be hammering in the nails all crooked-like or something, and perhaps I don't notice bur everyone else does.) What mistake am I making?

Botogol, why not say that I have misestimated either the going rate, or what quality a hammer it is. (If I think it is steel, but it is cast-iron, that doesn't mean I have misestimated the going rate for steel hammers; it means I don't have one.)

Caledonian, I have no idea what that means, but I'm sure it's very profound. I would greatly appreciate it if you or anyone else would explain it.

I had a friend in college who was a philosophy major; he'd been raised a fundamentalist Christian, and turned from that into some sort of chaotic evil deist. I used to enjoy arguing philosophy and ethics with him. But the further he got into his study, the more his arguments turned into half-understood quotations of Wittgenstein and Kant, and his debating technique turned into sophistries and trying to name and call out others' logical fallacies, sometimes correctly and sometimes not -- the same techniques he grew up with, only without having to wake up early on Sunday. I don't argue with him anymore...

I think you are artificially restricting your knowledge to direct perception of the objects. Don't you also know
that there are entire shops that sell only wine, that they have many varieties, at many prices, and they have competitions, experts etc?

It is *this* knowledge that leads you to conclude that wine is probably a complicated business and you may very well have made a mistake.

"Since you are so concerned about the interactions of clothing with probability theory," Ougi said, "it should not surprise you that you must wear a special hat to understand."

But isn't this almost the exact opposite of what the student was saying? Questioning the robes indicates to me that the student felt there was not any interaction between learning probability theory and clothing, and that therefore it served some other purpose, presumably differentiating between an in group and an out group.

Or am I just nuts for trying to argue with you about the internal thoughts of your own fictional characters?

Do not attempt to deduce the thought processes of fictional characters; that is impossible. Seek only to recognize the truth: that what the author says is objective, has implications on its own, and does not necessarily have anything to do with what he meant.

A few of you touched on the point I got out of this, but no one explained it very well. In the first koan, Ougi says two things. The clearer one is tangential to rationality, but important for self-doubting cultists. "You are like a swordsman who keeps glancing away to see if anyone might be laughing at him".

The more important point was that the teachings are valuable if they are useful. (This is applicable to the sword fighter because allowing yourself to be distracted is an immediate danger.)

The importance of the parable about hammers doesn't relate to prices, but to usefulness. "Use the hammer to drive nails" in a discussion about rationality is metaphoric for using the techniques to make better decisions. If Ougi's teachings help you make better decisions in your life, then they are valuable. If they merely bind you more tightly to Ougi, then you are a cultist.

Bouzo didn't learn anything that helped him make decisions, he was merely cowed into following Ougi more closely. Ni no Tachi learned to "concentrate on a real-world question", so "the worth ... of his understanding [became] apparent."

Ni no Tachi figured out how to use the hammer, but Bouzo only sold them without understanding their value.

Chris, for a while I was a member of what most people would consider a "cult", and from my experience "cults" usually teach people useful things. So it is a mistake to assure yourself you are not in a cult because you see that you are learning useful things.

Because we've read too many comics, manga, and sci-fi/fantasy books that romanticize a particular aesthetic. Robes are cool, silly. So is writing ideas in Japanese, makes the wisdom sound wiser, but what's especially important is that it's fun. Who wants to be part of a boring revolution?

Robin, you're right, most people do think economics is a cult, even though there may be a small proportion of usefulness in the teachings... characteristics are, cult members cut off from contact with non-cult members (in this case by the ignorance of the non-cult members, of course), devotion to the cult leader (Keynes ! Friedman ! the Gourd! the Sandal!), proclamations of infallibility (the market is infallible), progressive alienation (this is a science, I can believe six impossible things before breakfast), and ending in total learned helplessness (for instance, when a team of six beauticians, or whoever it happens to be this week, outperform the nation's best fund managers yet again...).
Only teasing, but I was just reading some very old threads and came across one where you professed surprised at relative levels of acceptance of announcements in economics and physics, and am still suffering from vertigo. Happy Christmas !

Because it confuses them into thinking that maybe the clothes have something to do with the meat of the instruction, and it takes longer for them to learn what they're being taught. It obscures the the real teaching, hides it among irrelevant rituals.

Ni no Tachi figured out how to use the hammer, but Bouzo only sold them without understanding their value.

"A bird in the hand is worth what you can get for it." --Ambrose Bierce

Fiction is fiction, but it seems to me that that if student objects to wearing silly clothes and his master responds by ordering him to wear yet sillier clothes, it's a lot more plausible that the student will conclude his master is a quack and drop out than that he'll decide to extend his master's teaching by taking silly clothes to a whole new level.

Maybe the whole point of this exercise is to remind us that one can't come to reliable conclusions from fictional evidence? If so, well, maybe I haven't learned anything... but at least I've learned I haven't learned anything.

as someone who is easily irritated I do not read the comments because they usually irritate me if it were not for the fact that i have nothing better to do today i would have missed this and continued on being greatly confused.

Thanks TGGP. That's a better suggestion than I came up with. Since Robin seemed to be responding to me (rather than to "Chris"), I guessed he was referring to the "cultish" experience he shared with me. But your suggestions are more likely.

Robin: "It is a mistake to assure yourself you are not in a cult because you see that you are learning useful things." But isn't the opposite conclusion safe? If you can't figure out how to use the hammer, but you nonetheless convince younger students of its importance, aren't you likely to be in a cult? Or more likely, if you can see that many of the other teachers are teaching material they don't understand?

If you can't figure out how to use the hammer, but you nonetheless convince younger students of its importance, aren't you likely to be in a cult? Or more likely, if you can see that many of the other teachers are teaching material they don't understand?

Eliezer Yudkowsky wrote of ideas one can't see the value of, and teachers who don't seem to understand their teachings, "Sounds like either a cult or a college."

I dunno, at least for many technical fields and for some other endeavors too (like learning to communicate effectively in writing) one can see that many of the teachers can do some handy hard-to-fake real-world stuff, and that the students emerging through the pipeline tend to be able to do it too. When I was an undergraduate, the EEs in my residence hall traditionally maintained a little hand-made custom-programmed telephone PBX which ran from the two college official phone jacks in the lobby to a motley collection of old salvaged telephones in most of the other rooms. I, at least, was impressed. If you're in an organization where the initiates routinely levitate out their windows to go to lunch, and levitate some more whenever they have trouble finding a convenient chair, is it a mystical cult because levitation or funny hats or even confusing explanations are involved, or might it be unusually successful pragmatic applied philosophy?

Once stretched to cover everything from incompetent posers to arrogant weird competent people (like Isaac Newton at the hypercompetent extreme, or various academics in a less extreme way), a concept like "cult" may not be all that valuable. Perhaps there is value in reminding us that part of the reason the posers can gull people with their behavior is that it's not so uncommon for non-posers to act in some similar way. But there is also value in to reminding people that part of the reason speculative bubbles can happen is that price moves based on fundamentals can look similar enough to gull speculators into mistaking a bubble for one one. That doesn't mean we should think of every big price move as a "bubble" (or as being bubble-ish, or whatever). We might say "every big price move wants to be a bubble," but saying a market situation where the fundamentals don't make sense "sounds like a bubble or an ordinary market" would seem to me to be missing a point.

In the Second Scroll of Wen the Eternally Surprised a story is written concerning one day when the apprentice Clodpool, in a rebellious mood, approached Wen and spake thusly:

"Master, what is the difference between a humanistic, monastic system of belief in which wisdom is sought by means of an apparently nonsensical system of questions and answers, and a lot of mystic gibberish made up on the spur of the moment?"

Teacher (after thinking for a moment): If I take a lamp and shine it toward a wall, a bright spot will appear on the wall. The lamp is our search for truth, for understanding. Too often, we assume that the light on the wall is God. But the light is not the goal of the search. It is the result of the search. The more intense the search, the brighter the light on the wall. The brighter the light on the wall, the greater the sense of revelation upon seeing it.

[All the students are looking on with blank faces]

Teacher: Similarly, someone who does not search, who does not bring a lantern with him...sees nothing. What we perceive as God is the byproduct of our search for God. It may simply be an appreciation of the light, pure and unblemished, not understanding that it comes from us. Sometimes we stand in front of the light and assume that we are the center of the universe. God looks astonishingly like we do. Or we turn to look at our shadow and assume that all is darkness. If we allow ourselves to get in the way, we defeat the purpose, which is to use the light of our search to illuminate the wall in all its beauty... and all its flaws... and in so doing better understand the world around us.

Student: Ah...yes...but... What is truth? And what is God?

Teacher (rolls eyes): Truth is... a river.

[amazed looks and murmurs from the students]

Student: And god?

Teacher: God is the mouth of the river.

[pleased expressions and excited exclamations among the students, Teacher facepalms]

Oy. I just glanced through the last couple weeks of posts. Hence the lack of a loud sigh on this one before. So consider this the loud-sigh of the confirmedly anti-koan, the person who thinks that metaphor and other such non-expository modes of speech have aesthetic value only, and that if one cannot speak of an idea in clear language, well, one ought to keep silent about it. (I can see Wittgenstein glaring at me...)

Or: what's the point of rationalist koan* exactly?

* It also irks me like crazy to see people taking the Japanese word "koan" and sticking an s on the end to pluralize it. You don't do that in Japanese.

It seems to me that some people take the notion "Never accept anything uncritically!" (or equivalently, "There are no certainties in science!") too far.

The core tenets of logic as set out by Mill, at least, must be accepted uncritically and never doubted, or the whole conversation in which they are doubted disintegrates into fallacy and nonsense, and thus becomes useless (except to a dishonest speaker who might use it to manipulate irrational people). There are other beliefs which are similarly necessary (for instance, mathematics) if the discussion extends to topics where they apply.

Well, if the **point** of the hammer is to build a house, you have a problem, because both 2 and 3 are options. Number 2 is silly as hell, regardless of whether you glued all the junk on it yourself, or you claim to have found it in the lost tomb of Jesus. It may still hammer nails, but no one in their right mind would use it, except out of necessity of there being no other hammers available. Option #1 is completely useless, gold being too soft to hammer much of anything, and again, it doesn't become a more valuable **hammer** if you claim that "it" was found in the lost tomb of the great Illuminati or some similar gibberish. A cult is a cult because it either contains nothing but gold hammers, or it insists that everything **must** be done using hammers that have lots of idiotic junk glued onto them, which don't help make the hammer any more useful. Someone intent on building a house can as easily use the second option as the third, but the moment you start thinking that you "have to" have all the silly BS glued to the hammer for it to work, it becomes a cult. Witness pretty much all religions, which nearly universally insist that its impossible to invent a hammer, without the guidance of high order of hammer makers, as to what sort of bangles and filigree one has to add to it before the magic spirits will make it work right. ;) Some are though tend to be satisfied with just the hammer, but insist the handle must be red, or something, to make it lucky (which might be the approach of the few relatively "rational" religions in the world), of which most are not.

In general, the more rational your starting framework, the least silly the necessary robes/clown suits are, but even the most rational groups, if they don't take a critical look at certain assumptions, can get cultish. The key difference being if they opt to make those choices for distinction, or because they have actually come to *believe* that some detail of their world won't work without the silly hat.

Oh, and anyone know the answer to the original question that got me here? lol

The first novice, Ni no Tachi, was not a cultist. The key was Ougi's statement: "How long will you repeat my words and ignore the meaning?"

Ni no Tachi learned the meanings and became a true rationalist - he understood that the clothes had nothing at all to do with it. You could kind of think of them as a way of promoting group identity while weeding out those destined to be forever irrational.

Bouzo never even got to the key statement. He stopped questioning as soon as Ougi implied that the silly hat was necessary. He learned all the teachings of Ougi, but he never understood their meanings. Eventually he would only discuss rationality while wearing a clown suit, because he believed without any evidence that silly clothes were related to probability theory, which is completely irrational.

(Unlurking and creating an account for use from this point onwards; どうかお手柔らかにお願いします。)

Something I found curious in the reading of the comments for this article is the perception that Bouzo took away the conclusion that clothing was in fact important for probability.

Airing my initial impression for possible contrast (/as an indication of my uncertainty): When I read the last sentence, I imagined an unwritten 'And in that moment the novice was enlightened', mirroring the structure of certain koans I once glanced through.

My interpretation is/was that those words of Ougi's were what caused the novice to realise his error (in focusing on the clothing rather than the teachings when considering likelihood of cultishness), the absurdity of a hat worn affecting one's understanding revealing the absurdity the clothing worn by those in the dojo inherently for their rationality (though arguments could be made about indirect advantages and/or disadvantages in both directions?).

From that, the clown suit could be taken as a result of him being humbled by this lesson, it making such a deep impression on him that, taking it to heart, he established something to remind him (and others?) of it as part of his daily behaviour.

(This would be that, rather than thinking a clown suit really could make him more rational, him wearing a clown suit when discussing rationality to dramatically demonstrate the lack of importance of what he wore for what he said. I'm also reminded of the concept of abnormality being going to school in a clown suit rather than in all black, though this is arguably not directly relevant.)

I am as yet unfamiliar with the architecture of this location, and so do not know if anyone will automatically know about it nor in fact if it will ever be read, but it nevertheless is a significant weight off my mind to speak about my impressions alongside those of others. I thank you all.

I am as yet unfamiliar with the architecture of this location, and so do not know if anyone will automatically know about it nor in fact if it will ever be read, but it nevertheless is a significant weight off my mind to speak about my impressions alongside those of others. I thank you all.

The "Recent Comments" section on the right hand side displays the five most recent comments. So even comments on old posts have a chance to catch the eye one who is browsing. Some even subscribe to all the comments using RSS,