Friday, June 09, 2006

1) I agree that direct reward has to be in-built(into brain / AI system).2) I don't see why direct reward cannot be used for rewarding mentalachievements. I think that this "direct rewarding mechanism" ispreprogrammed in genes and cannot be used directly by mind.This mechanism probably can be cheated to the certain extend by themind. For example mind can claim that there is mental achievement whenactually there is none.That possibility of cheating with rewards is definitely a problem.I think this problem is solved (in human brain) by using only smalldozes of "mental rewards".For example, you can get small positive mental rewards by cheating yourmind to like finding solutions to "1+1=2" problem.However, if you do it too often you'll eventually get hungry and wouldget huge negative reward. This negative reward would not just stop youdoing "1+1=2" operation over and over, it would also re-setup yourjudgement mechanism, so you will not consider "1+1=2" problem as anachievement anymore.

Also, we all familiar with what "boring" is.When you solve a problem once - it's boring to solve it again.I guess that that is another genetically programmed mechanism withprevents cheating with mental rewards.

3) Indirect rewarding mechanisms definitely work too, but they are notsufficient for bootstrapping strong-AI capable system.Consider a baby. She doesn't know why it's good to play (alone or withothers). Indirect reward from "childhood playing" will come years laterfrom professional success. Baby cannot understand human language yet, so she cannot envision thissuccess.AI system would face the same problem.

Back to real baby: typically nobody explains to baby that it's good to play.But somehow babies/children like to play.My conclusion: there are direct reward mechanisms in humans even forthings which are not directly beneficial to the system (like mentalachievements, speech, physical activity).

Richard Loosemore (rpwl at lightlink.com):All thinking systems do have a motivation system of some sort (what you were talking about below as "rewards"), but people's ideas about the design of that motivational system vary widely from the implicit and confused to the detailed and convoluted (but not necessarily less confused).===