Every once in a while, I come across something that I just can't share with any real people, so I have to post it here.

Today is that day. Apparently, there is a forum of "rationalists". The rationalists are very concerned with what happens after the singularity (Artificial Intelligence (AI) created with such intelligence that man is no longer king). Because conventional forms of rationality give surprising results when faced with some weird corner cases, these folks invented a new definition of rationality. Then they invented a new philosophical decision theory (Timeless Decision Theory), defining how one should behave to meet the requirements for rationality.

All perfectly reasonable; at least no less reasonable than posting here. But then someone (Roko, specifically) took all those developments (new rationality, new decision theory, etc.) and crafted a thought experiment along the lines of Pascal's Wager (since the cost is low, and the benefit high, one should believe in god), except with AI in the place of God. The though experiment includes the AI creating simulations of you, and torturing those simulations, which is apparently supposed to be just as bad as torturing you. And when the guy who runs the forum found out, he freaked out. Because apparently, if one had never heard of the thought experiment, he couldn't be punished for not believing* in the AI. But once it was posted, anyone who read it was at risk for having simulations of themselves tortured in the future by the AI because the person in the past (today) didn't behave the way the (not yet created) AI wanted. So the forum mod deleted the posts, and banned any discussion of the topic, termed Roko's Basilisk.

* It is not actually believing, but devoting ones life to the service of the AI, but close enough, and simpler to write.

So on the one hand, I love that these guys are taking ideas seriously. But on the other hand, this is really more along the lines of the Raelians than a serious group.

Given your description, I don't see how someone "invented" god. Within context of the forum, I suppose the topic is worthy of discussion, but I don't understand a reaction, never mind the reaction of banning and locking the thread. Did the moderator believe someone was actually being tortured, or there was some "credible" threat of torture? _________________lolgov. 'cause where we're going, you don't have civil liberties.

Given your description, I don't see how someone "invented" god. Within context of the forum, I suppose the topic is worthy of discussion, but I don't understand a reaction, never mind the reaction of banning and locking the thread. Did the moderator believe someone was actually being tortured, or there was some "credible" threat of torture?

The moderator has since stated that (1) he shouldn't have banned the post, (2) that he doesn't believe that the thought experiment will actually lead to torture, but (3) he does think that similar thought experiments along the same lines could lead to torture by a to-be-created AI-God. Also note that it is not just torture, but eternal torture, since the AI first creates a version of your brain in something more permanent than your body, then proceeds to torture the immortal you.

he does think that similar thought experiments along the same lines could lead to torture by a to-be-created AI-God. Also note that it is not just torture, but eternal torture, since the AI first creates a version of your brain in something more permanent than your body, then proceeds to torture the immortal you.

Well, it wouldn't be me. That aside, wow. Even if it were some day possible, I can't imagine the record of those posts or discussions would survive to be reference material for the inventor._________________lolgov. 'cause where we're going, you don't have civil liberties.

I think they don't like Roko's basilisk because it shows them the futility of their efforts and beliefs. No wonder some got their heads spinning and minds worrying. Losing faith and succumbing into fatalism just tends to work that way.

I think that for them there is no need to discuss Roko's basilisk - the Christians don't talk about the possibility of God being evil either, nor do the muslims or probably any religion. "The rokoists" or "the basiliskists" or whatever are the equivalent of a medieval Christian self-flagellant movement. Not constructive at all, since the flagellants thought that the end is so nigh (Roko's basilisk will catch them anyway in the future) that nothing can be done about it - except damage self immediately to show subservience or something.

if one had never heard of the thought experiment, he couldn't be punished for not believing* in the AI. But once it was posted, anyone who read it was at risk for having simulations of themselves tortured in the future by the AI because the person in the past (today) didn't behave the way the (not yet created) AI wanted.

ZOMG! Quick, kill all the butterflies, so that one of them can never cause the collapse of the Brooklyn Bridge!!!

If only Ushpia hadn't conjectured back in 2,300 BCE about putting out people's eyes to make them loyal to him -- or if somebody had smashed that clay tablet, eradicating the Babylonian cuneiform record of the thought -- then the Spanish Inquisition, Witch Hunts, and Holocaust never would have happened!

I think this guy's brain has a bug in its loop optimization algorithms.