What question would you ask to identify whether or not you were chatting with a well developed software or a person?

Imagine an experiment where you are asked to chat with one hundred people online, no sound or image, just text. Three of them are actually not real, they an extremely good automated response systems. Your task is to identify those three. You are allowed to ask only one and same question from everyone. People on the other end are specifically chosen such that none of them have similar personality. Programs are also given a unique personality. Only trick is, while you ask questions, programs observe responses of everybody else and may or may not change behavior based on that. What would your question be?

P.S. If you would like to be sure how good is 'extremely good' automated response system in the though experiment above, you may consider it to be the best of such systems you think is possible.

Closing Statement from Farrukh Yakubov

Now that the conversation is over I would like to leave you with more thoughts.

Imagine, this experiment took place and you asked your question, and indicated three of the participants as programs. What if this experiment was not what you thought it was, and after the experiment you were told that 100 participants were all human or all programs, or even a single person answering 100 different ways? What if the purpose of the experiment was not about the capabilities of programs, but about the people - to see how people percieve an intelligent software? Did you think about this possibility?

On the other hand, if the experiment was to test the programs, how effective do you thinki it would be to use this same question of the experiment? i.e. asking "What question would you ask to identify whether or not you were chatting with a well developed software or a person?" from each of the 100 participants.

It is up to you to chose the post experinment scenario, and you would be correct. Because, the experiment can work both ways wether you decide to look at this experiment as an attemp to test programs, or a way of understanding peoples' understanding of programs.

Jan 27 2014:
Yes, machines on this experiment are not constrained. They could lie, but telling a lie while making it sound like truth is not simpler than telling the truth. Unless its a yes/no answer, it does not matter if its the truth or lie, any answer has some logic incorporated into it. Human participants could also lie, and it would have same effect as if they told the truth. Because, no prior information on the participants is provided.

Jan 27 2014:
No such magical question can exist. There is no single question that can determine whether or not a "conversational partner" is human. There is no perfectly logical way to determine this. One must, instead, rely upon the illogical presumption that there is and must be a flaw in the masquerade that is never, under any circumstances, to be replicated by human-to-human misunderstanding or human variation. If we're dealing with ca. 1985 "AI", maybe, but we are not limiting ourselves to 1985, 2014, or the limits of any year. Thus, we can posit near-infinite databases, decision trees, and expert systems for the machine participants. The puzzle is posed in a way that cannot be solved.

Jan 28 2014:
This is a thought experiment and the question mentions the audience to imagine this scenario. The question is not about whether or not a single question can differenciate a software from human, it is about what question one would ask trying to see the difference, given they are limited to a single question. For this experiment it does not matter if such a question exists or not, it's what people come up with as such a question. :)

Jan 28 2014:
If no such question exists, then there is no point in coming up with any such question, which has been my contention from the start. It's as nonsensical a task as saying "What incantation will take you to the moon without any means other than the power of magic engendered from your voice?"

Jan 23 2014:
From each participant I would demand an association chain starting with 'witch' and ending of 'blue dog' with a minimum of 42 freely chosen, different, yet related steps in between them, by which each step has to alternate between a subject, or an object and a transitional descriptive adjective which both have in common and relate them to one another in the development direction from left to right.

Jan 23 2014:
I think I"m not understanding the question, because I think the answer would always be: "What makes the dog colored is the adjective 'blue' before the word 'dog', as specified in the question." That would not seem to depend on any precedents in the association chain, and it doesn't depend on the immediate precedent "smelly".

In your question when you ask what made the dog colored, are you assuming that "blue dog" refers to a real dog that is physically colored blue? You could instead be referring to artist George Rodrigue's famous "blue dog" cartoon, or any of a number of other "blue dog" references. If you're referring to Rodrigue's blue dog, here is his answer to this question:

Rodrigue answers the title question by explaining that Blue Dog's color depends on what the artist is doing: when Rodrigue goes fishing, he paints the dog a salmon color; when Rodrigue wants a hot dog, he paints the dog mustard yellow, and so on. Would that be a valid answer?

Jan 23 2014:
If that was your answer and you a participant in that test, I would have you on the 'humanoid' pile in first selection. :o)

As Farrukh virtually installed 'extremely good automated response systems' in his thought experiment, about which I have no clue what 'extremely good' means, because my one and only experience with this type of software was a pretty boring ELIZA derivative, I have to assume the worst, which for this response systems would be 'really damn good'.

The approach I chose to gain sufficient information is based on complexity, uncertainty, ambiguities and creativity which are likely to confuse both, humans and programs, whereas the focus in the analysis would be the underlying approach of each given answer.

When you think the answer would always be 'the adjective 'blue' before the word 'dog'', you assume, that all participants cut of the whole end of my question - as you did - which clearly asked for a relation to the 24th object in the association chain (if there is any, as also subjects were allowed) by which the dog and not the witch was colored. And as 'blue' is not an object, this answer returns a contradiction to my question.

The uncertainty here is, if the 'blue' in 'blue dog' already represents the preceding adjective to 'dog' at the end of the chain or is seen as a closed entity which would allow for two descriptive adjectives at end, which is not forbidden. This is a choice everyone and the programs has to make, it is confusing on purpose to provoke uncertainty to some degree, because if you look closely at the example I gave, you'll find a separation arrow (->) in between smelly and blue dog, which holds some hint in itself.

I didn't know about George Rodrigue's artwork 9 hours ago and I didn't have to, to find you in the situation which was desired. Here is another uncertainty, even multifold, as 'blue' in the English language can also be interpreted as 'sad' or 'melancholic'., which is another ambiguous degree.

Jan 23 2014:
And although Rodrigue's answer why his dog is blue is irrelevant to my question, as he is a subject and can therefore never be the 24th object of an association chain, the information you returned is valuable by its creativity and testifies your knowledge in, at least, his work.

But Farrukh installed 'extremely good automated response systems' in this experiment by which I have to assume, that they are programmed in a way to gather 'knowledge' in real-time if necessary for a task or question, that is why knowledge alone would not be a good enough filter to spot them.

It is also to assume that those programs are designed to mimic human imperfection, because after all that is what makes us special, in a way, but as far as I am aware of, this is tricky to cast into machine code... I mean, the intended ones :o)

If I would take your comment as a 'valid' response in this test, I would consider the fact, that you spend almost half of all your words on the artist Rodrigue, which in proportion would be so way off my only and initial question, that you became a potential candidate for being a human in first selection.

Whether you are highly interested in arts in general or just in this artist or randomly just knew about him is secondary behind the quantity you spent on it. But as quantity disproportions could also be 'simulated', it qualifies you humanoid 'only' in first degree, by which the number of selection steps itself would depend on the overall tendency of answers and their 'quality' and therefore 'spot-ability' of the programs.

What I have to avoid in my final decision for each answer is a clear decision matrix which could and would be programmed in advance by smart programmers, so at the end it has to be my good old gut feeling I rely on.

And not to render all of my pitfalls and approaches I stated here obsolete, please don't tell me, that you already are 'an extremely good automated response system' installed on TED to keep conversations going here ... ;o

Jan 23 2014:
The Turing test, which this question is related to, is one of my favorite thought experiments. I believe that we are nearing a time when there is no way to tell the difference with one question. My question would be a bit recursive:

What one question could I ask you to determine whether you're a human or software?

Jan 22 2014:
You are in a prison with two other people, one always lies and the other always tells the truth. There are two doors in the prison, one leads to sudden death, the other to feedom. Both people know what is behind each door. You may ask one question to either person and walk out to freedom, what is the question?

Jan 23 2014:
"What door would the other person point to, if I asked which one leads to freedom?" The door that was not pointed is the one to go through. Does not depend on which one of them answers this.

Jan 23 2014:
The comment above was just how I would answer to Keith's question if I was one of the 100 on the other end of the network. But if you mean how would I judge if I was the one asking questions, then I would not expect everyone to answer the right way, because there might not be single correct way to answer. Instead, one way I could judge is to get all 100 answers, then compare them.

Jan 23 2014:
You and I have very similar interests and knowledge backgrounds I see, my guess is less than 1 in a billion can solve that problem without looking it up on the internet. That one was easy, would you like to try this one? Can you tell me how to sort data without moving it? That took IBM's best over thirty years, I did it over a weekend 46 years ago. If you get that one I will give you a really hard one about quantum physics. I am curious to see if you have any limitations.

Jan 23 2014:
Why do you think a liar never says a single truth? What if they play tricks on you becasuse they know you don't trust them? Do you think an intelligence test like this can help you to find a real person you like or has much in common with you in your real life?

Jan 23 2014:
Concise way of explaining this, is that those two (people in the question) behave like quantum entangled particles. Longer verbal explanation is below:

Hi Yoka, I think Keith said it's wrong because just asking any one of them about the safe way out does not provide sufficient information to identify where the doors lead. You may just get lucky and ask the person that knows the truth, or it may turn out otherwise. The trick is to assume the person you are going to talk to could be both of them. If you asked any one of them which way the other one would point to as safe, they could either lie or tell the truth. But if they lie, then the other person would tell the truth, and vice versa. While having only two possible answer choices, opposite of lie is the truth, of truth is a lie.

Therefore, no matter who you ask, you either get truth about the lie, or a lie about the truth. Thus you get a lie. Now you can be sure about where the doors lead.

Jan 23 2014:
“"What door would the other person point to, if I asked which one leads to freedom?" The door that was not pointed is the one to go through. Does not depend on which one of them answers this. ”
Why this answer can't be programmed in a smart robot?

Jan 23 2014:
Thank you for your elaboration. I think I understand it. But I meant to take all of us three to get out of the prison. I can just let them open the door and follow them to go out. So if the liar wants to survive, he has to tell the truth.

And actually, I don't think this kind of question can help me judge a person on the internet in our real life. I'd be too lazy to answer it and pass my attention to chat with other people.

Jan 23 2014:
Thanks for your patience Yoka, I figured Farrukh could help better than I could, he is a smart and gentle guy. The question is a brain teaser and not at all easy to solve but you just plowed into it anyway and I give you a thumbs up for trying. I enjoy your comments and think you have a lot to offer so hang in there and fire away anytime you like.

Jan 23 2014:
The heap sort is a comparative sort still an incredibly slow sort compared to mine. I'll give you a hint, my sort does not sort anything. It operates as fast as the records can be read, no data movement and near zero cpu time. It was ingenious 46 years ago and as far as I know it is still the fastest sort in the world. Some Professors at Stanford challanged me to beat their sort version because their's is the fastest sort ever published. My sort has never been published and aside from my Professor a retired Air Force Mathematician no one has ever seen my code. It was my first program, a simple assignment for class and it was supposed to be written in COBOL, however I wrote it in Fortran which I taught myself and he did not understand the code.
By the way I had a good laugh about your "quantum entangled particles" explanation. By the way if you have not seen Princess Bride by all means watch it some time.. 3 min. part on logic- https://www.youtube.com/watch?v=0sPVEBAtwmg

Jan 24 2014:
The first time I thought about this I assumed no data movement meant using any memory (other than where data already resides) for structures regarding the sorting information is not allowed. Also, I'm assuming linear complexity when u say "zero cpu time". Please let me know if you meant something else other than the above. Also, does your design work with any type of data with same efficiency? From what you describe it sounds as if its a method of accessing data as if it was sorted, while order of data entries remain unchanged.
If the purpose is just to provide the sorted index of a requested entry, Selection algorithm to find kth smallest item from the set has a linear complexity. But it is not ideal if random kth items are being continuously accessed.
Thus I have a solution in mind, that modifies, reuses and combines existing methods to create generic non-comparative sorting that works with a set of data (let size of the set be 'n'), where each item has arbitrary length, and does so in linear time.

Edit: I don't expect it to be same or similar to what you have in mind, its just another way of doing things.

Algorithm is explained on the next comment. This is going to be divided into few chunks due to limits of this conversation platform.

It does not modify the original data set, but produces an array of pointers (referred as the map) of length n. Other memory that will be used is of size 256 integers (referred as the workspace), which is no longer required after completion of the algorithm. I'm going to start describing it from the lowest component to highest. Also, I'll use C notation to avoid wordy sentences.

First component takes advantage of pointer manipulation and underlying architecture.It is a partial Counting sort. This stage takes in only set of bytes.
1.Reset workspace to zeros.
2.for each item e in the input set, perform workspace[e]++ //offset of each entry in workspace represents a value of an item; value at the offset represents the # of items in the set that are equal to the item.
3.for i=1 to 'size of workspace', perform workspace[i]+=workspace[i-1] //value of each entry in workspace represents #of items in the set that are less than or equal to the item with value 'offset'.
4.First component does not proceed with constructing sorted array, but instead provides a way find index of each item as if the set was sorted. Index of 'someItem' from the input set in a sorted set would be workspace[someItem]. Higer level component will obtain index for each item exacly once.

Second component is a radix sort, but bytes will be used for grouping instead of bits. The map is initialized such that map[i] contains adress of set[i]. At each iteration, the first component is used to divide each subsequent set up to 256 groups, until they no longer need sorting, i.e. is of length 1. Also, actual items in the set will not be moved around, instead only the pointers in the map are modified such that map[i] is the index of ith item in a "sorted" set.

Counting sort(first subcomponent) has complexity O(n+k), k is maximum possible value of each integer item (256 in this case), n is length of the current subset. This is a stable non-comparative sort.

Radix sort using stable non-comparative sort has execution time of Θ(d(n+k)), d is length of items in the set. n is size of the set. For arbitrary length items, upper bound should be O(p(n+k)). p is an avarage length of items in set. p.s. items of length less than p, will no longer be in subsets of size larger than 1 after p iterations.

I may not use the same method if the nature of the input is known beforehand.

Final comment, in the process I discovered this platform does not include anything after 'less than' symbol.

Jan 25 2014:
Farrukh - can you tell me if I'm not understanding your algorithm properly? I believe your algorithm is n-log-n and not linear, because the original question placed no constraints on the size of each input element.

If each input element were allowed to be a random 64-bit integer, the size of your work space would be 16 quintillion bytes, which would be an issue.

Jan 23 2014:
I'm assuming you don't know which is which. It took me some time, but I think I figured it out. You ask either of them: what will the other guy say is the door to sudden death? The person will indicate a door, that's the one you want to take. Alternatively, you can ask which will the other guy say is the door to freedom, and take the door not indicated.

Jan 23 2014:
Very good but Farrukh posted the answer 25 minutes earlier. Did you look it up or figure it out?
Another way to phrase it is: Which door will the other guy tell me go through? and then go through the other one.
Good work out Farrukh, Yoka and Timo... remember it is the journey that is most important and all of you took the same journey. Because you got different answers should in no way spoil your journey because there "is" no destination, the destination is an illusion. Buddha put it this way:

"Nirvana is this moment seen directly. There is no where else than here. The only gate is now. The only doorway is your own body and mind. There’s nowhere to go. There’s nothing else to be. There’s no destination. It’s not something to aim for in the afterlife. It’s simply the quality of this moment."

Jan 23 2014:
The question doesn't matter.
If i choose life i'll go to the door that the lier will show me, no matter what i asked.

edited

Freedom is a lie, the lier will show me the door to freedom, if i ask for it. If i ask the door to sudden death, he will show me the same door, to freedom , because he is a lier.
If ask the person who always tells truth, where is the door to freedom , he will show me the door to sudden death, because , it's a real freedom. If i ask him where is the door to sudden death , he will show me the door to sudden death, because he always tells truth.

Jan 23 2014:
Maybe you've pushed the wrong reply button ?:)
It's me,not Farrukh.
I've experienced the beauty of logic on the way to..., but now i see the flaws.
Frankly, i don't see any version of explanation that can eliminate the uncertainty.
In case, there is such and you know it, please share !

Jan 24 2014:
Interesting way of putting things together. If I were to further analyze this, under the above explained conditions, choosing a random door would be as good as talking to anyone. However, under this concept of the world, there is still a solution that leads to certainty. It's actually more efficient that the one in standard concept. If you ask anyone of them which way the other one would point to to freedom, they will always point to freedom. Its guaranteed by the design of the preconditions, no post thinking to be done like as in other case. Since a lier always points to freedom, truthful person would not alter lier's decision. On the other hand, the lier would point to any door other than the truthful person believes to be freedom. What I find amazing is that formulating a different preconditions allows formulating a logic that does not contradict with those of different setups.

Jan 25 2014:
"What do you think is the biggest problem in the world today...?"

This must be one of the hardest questions to answer, due to many problems and not all of them can be compared. Perhaps this would be a good question of choice for the above thought experiment. :)

Science is key to move everything forward, and computer science seems to be the beating heart of the current era. I am not sure what I would want to tackle first, but I would let my interests lead the way.

Jan 24 2014:
Farrukh I copied your last response to a word file and will go over it next week, I'm not as sharp as I used to be so it will take me a while to figure out your method. I tried to email you but the link did not work for me. Here is my email (keithwhenline@gmail.com) drop me a line and I will tell you as best I can remember how my sort works for your information and you can do whatever you like with it. I am curious about your background I assume you spent time or was raised in the Kazakhstan area and moved to the US to further your education. Also wondering what kind of impact you want to have in the world, with your knowledge you obviously have a wide range of possiblities. What do you think is the biggest problem in the world today and are you willing to tackle it?

Jan 25 2014:
Natasha you are right of course, I have no right to give anyone any more attention than someone else and I apologize for offending you. It was totally my fault. The riddle I purposed was my way of telling if I was speaking to a bot or a very smart person and was another version of his original Turning type suggestions. Upon reading Farrukh's background which is very similar to mine I wanted to see how deep the rabbit hole goes and I found it has no bottom to my delight. I got caught up in that as you witnessed and forgot my manners and you have every right to call me on it, thank you. I hope you can forgive me and I will try not to every do that again.

Jan 25 2014:
No worries, you don't have any chance to offend me !
I mean, my ego is thin enough :)
Your riddle and that episode from " The Princes Bride " gave me an aha moment and i am grateful for that. Actually, those two are in perfect congruence. Probably i was a bit upset that there seemed to be nobody who was interested, but on the other hand, it's not easy to language what i've got, so it's OK anyway.

Jan 23 2014:
The question about a 100% liar and 100% truth teller assumes that you can find two people, such that one always tells the truth and one always lies. I don't think that ever happens in reality, and it's not a premise of the original question, so I can't see how this classic logic puzzle is a solution to this Turing test.

Jan 25 2014:
I agree, if the two persons lie at the same time(not one lies, another must be honest )......I think making them go through the door first could help to get all the people free from the prison in reality. But if they're terrorists who want to kill you with any cost......:)

Jan 29 2014:
Most, if not all, proposed answers in this thread have ignored the following clause.

"Only trick is, while you ask questions, programs observe responses of everybody else and may or may not change behavior based on that. "

It seems to me that, unless the computer application is first to provide an answer, a natural language processing (NLP) facility based on machine learning and/or statistical methods could be devised which could infer a convincing response based on those previously observed. Obviously if the computer is the 100th to answer, it is much more likely to be convincing than if it is first.

Given that we're limited to text only, that stops any sort of perception test of the Captcha form. If it weren't for the facility to observe previous responses you could try some sort of text based emoticon, but it would only have to repeat a previous answer to pass that particular test.

The nearest I can get is some sort of internally referenced query constructed entirely on the mechanics of the test itself. This would depend on people answering sequentially and knowing which position they held in the sequence.

Something like.

Using the numeric characters which represent your position in the sequence of responses (from 1 to 100) and additional punctuation marks found on a standard keyboard, create an original emoticon and explain what you intend it to mean. eg. if you are the eighth person to answer you could respond with "8=) means happy face" (this example may not be replicated in the test).

Truth is though that while SMTP is a text based protocol, it has been extended by MIME and other RFC standards to allow encoding of the inline multimedia and attachments that we see today.

For these to work it is necessary that server and client software understand the encoding standards and provide functionality by which the encoded data is presented in the intended format.

I guess our assumption when responding to your question, was that you were talking about the resulting/formatted communicated content (what a recipient might actually see and/or percieve) rather than the underlying character set processed by the communications protocol.

Jan 27 2014:
I have concluded that this is a loaded question. It is meant to illustrate the point that we are no different to the ideal synthetic brain!
The ideal robotic replica of a human brain can logically only exist if it mirrors/replicates the human mind to perfection.
This means that there is no question that can exist to differentiate between the two. Even though Farrukh does not specifically say it must be an ideal software he does say it must be highly developed which implies perfect or near perfect replication of human thinking.
Or maybe Farrukh is a TEDbot and we are being played like lab rats!
Lol.

Jan 26 2014:
I would ask: "Considering the rumor that your mother sleeps around a lot, how many sweaty gardeners did she sleep with yesterday if she slept with 3 at midday and 2 at 5 pm?"
...
If your answer is 5 you're speaking to a robot.

Jan 27 2014:
Lol.
Yes. The promiscuous mother taunt is clearly a diversion.
While the robot is processing a 'human' response to the emotive question, it will conclude it does not have a mother and will be so overcome with programmed emotion that a calculation error is bound to occur.