OK, I am going to share my situation in hopes that someone out there can answer some of my questions. I will post this on several boards.

Here is what I have: two propellers 8 core cpus, three basicstampIIs with different speeds, 11 small robots, one arduino, one arduino mega, several single core computers and one dual core and one quad core. One laptop and one net-book. One robot head. Broadband internet wired and wireless, linksys WRT54GL router. I have a somewhat entry level experience on Basic and C++.

I have mastered the art of obstacle avoidance. But, when I try to do more, everything gets so slow it becomes not practical.

I have two web-cams that can recognize faces, tell who they are, track moving objects, my voice is recognized and I can hear the answer in a human voice. It can recognize colors and flesh tones.

There is a data base that can interact and learn from me speaking to it. I have not tried OCR yet, but I am sure it will work. I have not tried object recognition, but I am sure it will work. I have used chatterboxes and Open-CV, and many more open source stuff.

Now, the problem is, all of this is scattered out over many robots and computers. How would one go about tying all of these items together into ONE machine? I am lost of how to do this. Where do I start? I have a deep desire to experience AI in a machine. Not to give it orders or commands , but to communicate with it and have it have it's own free agency. To watch it learn and grow and become more than just the sum of it's parts.

Oh, BTW, my wife says I have NO MORE money to dump into this stupid project. What would you do if you were in my shoes. Where would you start?

I hope there is an Engineer out there or some hobbyist that can answer that question for me.

You've acquired yourself some tech there, The system all together can give quiet some punch on computing power.Its hard to think of anything (that would give an ultimate design you assembled lots of hardware to thinker and build with. Most important what is going to be build and what do you need.

I would start with getting an overview from all that stuff(You did that already).Then making a list with property's I would like to have like interaction, path-finding etc things that assist the goal.Then which hardware does some property's the best what worked on which device U could maintain and connect.Further what kind off information is most important to relay to a system and what does that information contains.(like if you have robots how much did they travelled around, the location they bump which left a pattern of the enviromentanything able mapped into a database.

If you gradually select all of these features which software module does this to which hardware or which hardware gives information about that, gives a picture starting with information acquisition to store in a database.All the devices available could relay this information to a central application(think of this such as a domotica system gettingstatus information around the house). But not necessarily it could help to build yourself a server system which cope with theinformation.

I understand your main goal is communicating with the system, Probably meaning status information sort of where, what and when.All these system have an input / output for instance , The robots travel around -> information (How much did it got forward , left , right , back: After going X steps forward it bumped went left got forward again) theres a pattern in this mapping an area based on how the robot laid a path. A program that response to the mapping which stores the info in a database could also relay's this on other system like tracking with the web cam, locations tracking with voice for speech recognition. The system could learn how to gather information to get an optimum enviroment trackingas an example. The reverse is also possible using the other system to instruct a device or track a path towards something learning or working switching from assisted investigation mode to exploration.

There are lots of way to confront the A.I problem building ones own framework. Discovering new data simplifying that for a certain purpose.

I'm curious about this parallax propeller chip how is it to work with them, Its the first time I've read about this eight cores processor.How did you applied them what for. Found the site but though hearing it from someone who already worked with it could give valuable back ground of these chips they are amazingly cheap considering the cores it have .

I started working with Parallax about three years ago. Their service is second to none. But, I was going to build a boebot and I told them I wanted to do a lot of stuff. They suggested the propeller. It was brand new and had eight cores. This sounded fantastic to me so I purchaced a boebot with the propeller. When I went to program it, there where several bits and pieces of code to make it work. Some was ASM, some Basic and some SPIN. Everytime I find a decent computer, it is either SLOW or like the Prop, it has only a small amount of Ram to work with each cpu core. Why?? I was rusty and could never figure it out much, finally, I bought a second boebot with the BasicStampII. It was much more easier to work with. But, after I mastered the Navigation part, the machine got slow everytime I would add a sensor and such until it was so slow, I could not use it. I ordered several stamp versions that were faster,but they were not compatible, so I was stuck with either not doing what I wanted to do and it work, or it was going to be slow. So, I moved the the Arduino platform. It also had good support and I got it up and running much faster. But, again after I mastered the navigation part, it slowed down because I added eye brows for emotions. After that, I saw Zeno, he did about what I wanted to do without all of the hassle. So, I waited for him to come out, but he never did. (promised in 2009) Plus, the price kept inching higher until it is now going to be out of my price range. I may decide to liquidate all and get him. Then I saw Aiko. The robot in itself is nothing that great, but the B.R.A.I.N.S. software is doing EXACTLY what I wanted to do with my bots. I started to build a robot head. I have a frame with two webcams and a mouth. I am awaiting to have the funds to put servos and such and make it sort of like Kismet the robot. It looks like Mr.Le doesn't want to share his hard works and efforts. I guess I can't blame him. But, I was hoping he would make it available as a beta, or Opensource or such. It seems to do what I needed to realize my dream of having a robot that would have A.I. (I have been dreaming this since 1961 and building robots since 1975. But, I do not have Mr. Le's gift for doing so. I have watched the computer evolve since 1975 to now and personal robots are about to Explode in the Marketplace. What we are waiting for now is a pratical Robot D.O.S. and an application that would be equivalent to a word processor or speadsheet program on the personal computers. I think that the B.R.A.I.N.S. software would have a chance to be the D.O.S. and something like WolframAlpha could be the other app. But, others are working hard to do the same thing. Just, by the time it really happens, I may be way to old to enjoy it. Sadly but true. The BEST product does not always win. I remember that NorthStar had the best O.S., but ucky!ucky!ucky! CP/M and MP/M became the standard before IBM and Microsoft did the windows. I am trying to build my own with OpenSource which has a million ways to do things and I may or may not be capable. Wish me luck.

I have tried all of the OpenSource stuff. It seems to be way over my head. There is Alice, Rebecca, George,Anna,Urbi,Yarp,AnswerPad,Mavis,Marie, and many others. Each have about 12 directorys full of stuff that I do not understand. It is just over my head. :-(

I guess I will just have to be patient and wait until someone releases something more simple. I am a computer tech and I work with complicated stuff everyday, but this stuff is greek to me.

Yup thats why we need functional programming just like Alan Kay suggested many years ago.Could be just what we need to encourage non-programmers start for example building robot function code by symbols.Same go's for geometric structures as well like circles , boxes etc translating them in geometric computations.Its a paradigm of the past but can revolutionize future programming adaptations for things like robotics.

It does not make it less harder rewriting code for your own project or open source software which need similar approach at recoding stuffto fit. So I guess your right, There are no programs that fit everything together All recognition parts are separated voice ,vision and sense thats the problem thats why using open source hard to figure out connecting together.

Somehow all these alices, rebbecca have to be build into a package with a simple interface and a logic behind it in automating response without figuring out yourself how the program does that as long if the output can be configured.

Sounds like a prototype project to test something out you got my interest.