Comments 0

Document transcript

Artificial Life, Vol. 1, Nol 1/2, 1994.

Toward Synthesizing Artificial Neural Networksthat Exhibit Cooperative Intelligent Behavior:Some Open Issues in Artificial LifeMichael G. DyerComputer Science DepartmentUCLA, Los Angeles CA 90024(dyer@cs.ucla.edu)AbstractThe tasks that animals perform require a high degree of intelligence.Animals forage for food, migrate, navigate, court mates,rear offspring, defend against predators, construct nests, and so on.These tasks commonly require social interaction/cooperationand are accomplished by animal nervous systems, which are the result of billions of years of evolution and complexdevelopmental/learning processes.The Artificial Life approach to synthesizing intelligent behavior is guided by this biologicalperspective.In this paper we examine some of the numerous open problems in synthesizing intelligent animal behavior(especially cooperative behavior involving communication) that face the field of Artificial Life, a discipline still in its infancy.1. IntroductionA major goal of Artificial Life (AL) research is to gain insight into both life as it is and life as it might have been (Langton 1989; Langton et al.1992).As such, AL issues span potentially a very broad range, from the role of chaos and prebiotic chemistry in allowing intercellular processesto have come about, to the emergence of human intelligence and organizations.In this paper we focus on the research goal of understanding thenature of intelligence from an AL perspective -- i.e. the evolution and development of complex nervous systems, supporting both animal andhuman cooperative behavior.We are interested in (a) how artificial neural networks (ANNs) support animal and human cognitive processes and(b) how intelligence might be distributed within groups or populations of individuals, with a special focus on the role of communication insurvival strategies requiring cooperation.2. AI vs AL Approach to CognitionThe traditional approach to synthesizing intelligence has been developed within the field of Artificial Intelligence (AI) and has focussed, for themost part, on modeling both everyday and expert-level knowledge and reasoning in humans -- e.g. problem solving, planning, languagecomprehension, learning, and invention.In the field of AI, such models are realized in terms of computational systems that manipulate symbolicstructures via rules of inference (Nilsson 1980; Rich and Knight 1991; Pylyshyn 1984; Firebaugh 1988).Although there are growing fields ofDistributed AI (DAI) (Bond and Gasser 1988; Gasser and Huhns 1989), situated agents (Agre and Chapman 1987), and artificialneural/connectionist networks (Churchland and Sejnowski 1992; Bechtel and Abrahamsen 1991; Smolensky 1988), the emphasis in AI has beenon modeling cognitive tasks involving a single individual, via logic and list-processing procedures.In contrast, the field of AL brings a decidedly biological perspective to the study of intelligent behavior.Table 1 illustrates how AL and AIparadigms differ.Artificial Intelligence (AI)

AI vs AL ParadigmsMajor differences between the two paradigms are mainly due to AL's biological perspective, which emphasizes the evolution and developmentof animal cognition within interacting, multi-species populations over many generations.Here, any strategy must be complementary to thosethat aid survival.All tasks must be situated -- i.e. occurring within an environment of interacting agents.As a result, the sensing and motorcapabilities of each individual must be taken into account and integrated with any kind of planning or problem-solving strategy.The time-spanis also quite different, with AL agents living out lifetimes, giving "birth" to offspring, raising their young and finally dying.Typical tasks in AIinclude medical diagnosis, story comprehension and question answering, chess playing and I.Q. tests while typical AL tasks involve predatoravoidance, food foraging, mate finding, nest building, and so on.Thus, if nothing else, what might be considered benchmark or touchstone tasksdiffer markedly for these two synthetic approaches toward understanding intelligence.One attractive feature of the AL approach is that the fundamental unit of manipulation is thepopulation.Genetic operators are applied topopulations and produce populations.This approach is in direct contrast to AI approaches.For example, in distributed artificial intelligence(DAI), researchers take theindividualto be the fundamental unit, and then attempt to engineer individual agents who will interact correctlysolve group-oriented tasks.In genetic systems, however, no individual can be central, especially since individuals are replaced by offspring thatvary from their parents.In contrast to the DAI point of view, the evolutionary approach of AL is novel in another way-- organisms not onlytransmit information via some form of language or communication, but also one can view mating between individuals as the 'communication' ofsuccessful hardware (with variation) to their offspring.Thus, both neural architectures (i.e. innate components) and distributed representations(i.e. learned components) can be engineered within a single, all-encompassing framework.The AL modeling approach involves specifying:(a)environments-- simulated worlds whose conditions match, at some level of abstraction, those selectional pressures in which a variety ofanimal behaviors may evolve or develop (Meyer and Wilson 1991),(b)processes of genetic expression-- mappings from artificial genomes to phenotypes which control behavior (Goldberg 1989; Goldberg andHolland 1988).A genotype is commonly a bit string; a phenotype is often some type of artificial neural network (ANN) or connectionistarchitecture that controls the artificial organism's behavior, through the simulation of sensory/motor neurons and interneurons .(c)learning and development-- methods under genetic control for modifying or growing the nervous systems of artificial animals during theirlifetimes, e.g. (Rumelhart and McClelland 1986; Nolfi and Parisi 1991), and(d)evolution-- recombination and mutation of parental genomesduring mating to produce variation in their offspring (Holland 1992).The evolutionary approach allows one to study the sociobiological aspects of intelligence.In addition, it supplies an alternative to engineeringgroup interaction by hand; instead, one need only set up appropriate environments, such that there is an evolutionary path to the desired groupbehavior.Of course, many issues arise.What environments will lead to desired forms of behavior?How can the combinatorially explosivesearch for desirable organisms be reduced?For example, with genomes of just 1000 bits, one is searching in an immense space of 21000organisms.How should organisms be represented?What are useful mappings from genotype to phenotype?What simplifications are acceptable,i.e. will lead to desired results, but without compromising fundamental issues?A very long-term goal of AL is to gain insight ultimately into the evolution and nature of human intelligence, through modeling the evolution ofcommunication and cooperative behavior in lower life forms.Such models might someday give us insights concerning hominid or proto-humanlanguage and intelligence, for which there is little or no fossil record.Such a goal is truly a daunting one;there are so many parameters(environmental, genetic, developmental) that are potentially relevant -- creating an enormous search space of research models/theories.Consequently, ways must be found to impose constraints on model and theory construction.Constraints can be obtained from multiple source:neuroscientific data, comparative cognitive science and computational analysis -- e.g. analysis of the size of a given genetic or developmentalsearch space and the computations required to find a given solution within them.Other obvious constraints come from the biological sciences --i.e. evolutionary and population biology, genetics, ecology and ethology.3. Animal Intelligence: Open Questions in Artificial LifeOf the approximately 1.5 million life forms on earth today, about 1/3 are plants and 2/3 are animals.Of the animal species, around 4k aremammals, 9k birds, 6k reptiles, 3k amphibians, 20k fish and about 900k arthropods, with the remaining 80k consisting of lower life forms, suchas corals, jellyfish and worms (Morris 1990).Over the last 3 billion years, evolution has produced animals with complex nervous systems thatsupport extremely sophisticated individual and social patterns of behavior.Each general strategy (or even elementary unit of behavior) requiresthe integration and control of complex sensory/motor information.Each task is accomplished via the activity of the nervous system -- anextremely powerful, massively parallel computational mechanism which is the product of both evolutionary and developmental processes.Thebrains of insects alone employ up to a million nerve cells (Menzel and Erber 1978) with thousands of times more synapses.Avian brains containaround a billion neurons and mammalian brains on the order of tens of billions of neurons.In the next section we briefly examine some representative animal behaviors.Open research questions in AL include synthesizing, with respectto any of these behaviors, any one or more of the following:(a) a group of interacting ANN-controlled agents (or single ANN) capable of demonstrating the relevant behavior(s),(b) a developmental process, e.g. gene expression in ontology (Edelman 1987), adaptive learning or a combination of both, capable ofautomatically creating/modifying ANNs, via interaction with the environment, that result in ANN-controlled agents able to perform the relevanttask(s),(c) an evolutionary process (with or without learning) by which genetic operations (e.g. recombination and mutation), combined with selectionpressures from the environment, lead to ANN-controlled agents capable of exhibiting the relevant behavior(s) -- i.e. synthesis of the evolution ofnervous systems e.g. (Ebbesson 1984).3.1. Common Behaviors in Animals(1)Social Grouping-- Animals form a wide range of social groups (Crook 1970; Tinbergen 1953), useful for protection and that enhancecooperation for such tasks as nesting, parenting, hunting, etc.Nesting fish, e.g. many cichlids, and the majority of birds form pairs and share theduties of parenting equally.Deer, fur seals and Hamadryas baboons form harems, with a single dominant male serving as leader and protectorof numerous females.Elephants form a matriarchy where the dominant elephant of the group is female and the big bull elephants live solitarylives.Some birds form all male groups, e.g., during breeding season, black grouse males cluster in an arena or lek so that the females may selecta mate.During migrations, there form huge swarms of insects, flocks of birds and herds of antelopes.(2)Specialization of Labor-- Social insects have evolved extremely complex caste systems, in which each class performs a distinct function,e.g. soldier, queen, drone, forager, and nest builder (Wilson 1980).In mammals, tasks are differentiated based on age, gender, strength; forexample, lionesses hunt while males protect the pride.(3)Food Finding, Preparation and Storage-- There are many food-finding strategies (Owen 1980), including grazing by herbivores, browsing ontrees by elephants, koalas, etc.; flower-finding and feeding by bees and hummingbirds (Frisch 1967), and foraging by ants, who produce andfollow pheromone trails to food sources (Wilson 1971).Many carnivores also have scavenging strategies in addition to hunting.For example,jackals often follow hunting lions and steal small portions of the kill even as the lions are feeding (Estes 1991).After an animal has obtained its food, it may execute complex food preparation behaviors.For example, the agouti (a rodent of South andCentral America) carefully peals its food before eating it.Wild cats pluck the feathers of birds and shake them off before eating (Leyhausen1979) with New World cats plucking and shaking out the feathers in a manner different from Old World cats.Predators that kill large prey (and thus leave a carcass they cannot consume in one sitting) have developed food storage strategies (Lyall-Watson 1964).Wolves will dig holes and drop large pieces of meat into them.Shaking the meat removes the dirt when the meat is dug up.Leopards will carry a carcass up into a tree for safe keeping.Animals store food for times when it will become scarce.Arctic foxes have beenknown to collect over time and hoard, in a single cache, over 40 bird eggs.The mole will create a hoard of up to a thousand worms, with eachworm immobilized by a bite to its front end.Some animals, e.g. jay birds and squirrels, instead of creating a single store, will scatter their foodabout in multiple small caches, in order to make it difficult of hoard-raiders to find them.A single nutcracker bird will hide over 10,000 cachesof 2-5 pine seeds each.Experiments (in which additional seeds are buried by the experimenters) have shown that such birds retrieve their cachesbased on memory and/or pattern recognition (vs. smell).This capability is quite impressive, since retrieval is performed months later, afterwhich the foliage landscape has changed dramatically, e.g. covered with snow (Vander Wall 1982; Shettleworth 1983).(4)Symbiotic Behavior-- Distinct species will sometimes develop a symbiotic relationship (Perry 1983).For example, the Egyptian plover birdis allowed to enter the crocodiles mouth in order to eat parasites and food remnants from between its teeth.Large coral fish will form a queue,waiting to be cleaned by smaller, cleaning wrasse fish, who also clean about and within the mouth of the client fish.Grouper fish that are beingserviced by gobies will partially shut the gobies in their mouths if there is any danger to the gobies, thus protecting their cleaners.Impala herdsand baboon troops often stay together; the impalas serving as watchdogs while the baboons provide protection.The honey badger relies on abird called the honeyguide to lead it to a source of honey.After the honey badger has destroyed the bee's hive and eaten its fill of honey, thehoneyguide can then dine.In some cases, one species makes use of another without "mutual consent".For instance, Amazon ants raid the nests of other species of ants andsteal their eggs.When the larvae mature, they become "slave" worker ants who help maintain their masters' nest.Some species of ants havebecome so successful in this strategy that they can no longer feed themselves, relying instead on their slaves (Wilson 1975, 1980).(5)Dominance, Combat

and Territoriality-- Many animals establish dominance hierarchies, with the more dominant males having greateraccess to females and food, but also serving the roles of protector and leader.In the case of bees, for example, stinging duels among daughterqueens determine which one will become queen of a new colony as the old colony splits in two.Dominance and combat may determine the sizeor location of a territory for nesting.Territorial animals stake out territories through a combination of acoustic, chemical and visual messages.Many birds announce their areas vocally while some deer mark their territory with scent.Animals may issue threat displays to those enteringtheir territory, or they may attack the intruder.Dominance is often established by stylized forms of combat, in which both combatants showtremendous restraint; as a result, combatants are rarely seriously injured (Carthy and Ebling 1964; Southwick 1970).Male kangaroos box oneanother; Bengal monitor lizards wrestle upright; giraffes push one another with their necks and deer spar with their antlers.Dominance structurescan be as simple as a straight-line ranking from most to least dominant, or, as in the case of chimpanzees, it can involve complex networks ofshifting coalitions, collective leadership, and bargaining in the overthrow of bullies (De Waal 1982).(6)Mate Selection and Mating-- Animals must find, attract and/or choose mates before the actual act of mating (Bastock 1967; Halliday 1980;Thornhill and Alcock 1983).For example, the river bullhead fish finds a cavity under a stone and enlarges it by digging with his mouth.When afemale passes nearby, he drags her into the cave with his mouth and then blocks the entrance so that she cannot escape.A ripe female will thensearch for the best egg-laying site within the cave.After spawning, the male remains to guard to eggs while the female departs.Among non-pairing birds, females will arrive at a courtship site (i.e. a lek or arena), possibly containing many males.Each male will try to attract a femaleby performing elaborate displays, involving vocalizations, strutting behaviors and feather erections.For example, the Australian malebowerbird, called also the stagemaker, completely clears (i.e. by removing every root, twig, leaf etc.) a circular area of ground 8 feet in diameter.He then searches for a set of special articles that he places on display within this area.These special objects are often large, fresh leaves, turnedwith the underside up.Numerous leaves are arranged in what the males hope will be an exciting pattern to the female.When the female appears,he leaps about, picking up the leaves and using them as props in his song and dance courtship routine (Gilliard 1969).Bower birds will alsoemploy shells, feathers and human artifacts as courtship props (Marshall 1954).The mating act itself requirescontrol of, or coordination with, the other partner.For example, the male lion gently bites the lioness' neck,causing her to act as a passive cub, from the time when she was carried by her mother (Morris 1990).(7)Nesting-- Many animals build complex structures (Frisch 1974).For example, during mating birds construct single and communal nests inthe form of bowls, domes, and tubes.One of the most sophisticated is that produced by the male weaver-birds.Weavers wrap tough, long grassaround the fork of a branch.Several of these grasses are joined to form a circle, with a male weaving in the middle, similar to the way a humanweaves a basket..The male makes use of two other types of stitches.When knotting, he pulls, pushes and twists with his beak while holding thegrass with his feet.When twining, he threads the grass in and out, like someone sewing with a thread.The weaver bird can tie slip knots, half-hitches and overhand knots.Social weavers will build apartment complexes with up to a hundred distinct entrances to individual nests (Morris1990).In addition to the complex behaviors involved in the gathering and placement of materials in nest construction, there is also nestmaintenance.Bowerbirds, for example, will keep their nests clean, for example, by replacing flowers (originally brought to decorate the nest)that have wilted (Marshall 1954).Nest also need constant repair.Most animals, instead of rebuilding a nest from scratch when part of it isdamaged, have successful strategies of local repair, even though there are innumerable distinct ways in which nests can become damaged..Forexample, beavers will patch those sections of their dam where they hear the sound of running water (Wilsson 1971).Ants and termites alsosomehow repair their tunnels and chambers as they undergo varying forms of damage (Wilson 1971).In addition the site for nesting must be selected.For instance, honeybees in cold climates send out scouts to forage for a good new nest sitewhen the colony is going to split into two (Seeley 1982).The scouts chosen are the oldest bees in the swarm since they have had the most nectarforaging experience and thus know the terrain the best.Once the swarm (of about 30,000 bees or 1/2 of the hive) splits off, it lands on someobject and the scout bees are sent off in search of a new nesting site.The scouts search within a 10 kilometer area for a nest.Studies reveal thatthe preferred nest is more than three meters off the ground (to make defense against predators easier); its entrance is less than 60 squarecentimeters and is located at the bottom of the nesting hollow and facing Southward (in order to control the nest's microclimate and warm thebees as they take off).Like the dance performed by nectar-foraging bees (Frisch 1967), each nest-scout performs a dance for the other scouts inwhich the orientation and amount of waggling of their abdomens indicate the location and preferability of the nesting site encountered.If a scoutB1 encounters another scout B2 whose dance is more energetic, then B1 will fly off to check out B2's proposed site.Only after B1 hasexamined the alternate site and concurs with B2 will she alter her waggle dance to conform to that of B2 (Lindauer 1971).As a result, overtime all the scouts will reach consensus, which may take several days.After consensus is reached, the scouts somehow inform the swarm, whichthen takes flight.The scouts then guide the swarm to the new nesting site and construction begins.(8)Parenting-- Animals protect, feed, clean, teach and in general care for their young (Rheingold 1963).Care of young after birth requiresrecognizing one's offspring as one's own.Sheep require 20 minutes of licking and cleaning their babies in order to create/store a memory tracesufficiently strong for later discriminating them from other lambs.Keeping the nest clean is also important.For instance, parent birds will pickup and carry the droppings of their young away from the nest.Parents must also supply food and drink to their young.For example, theSandgrouse male dips his lower feathers into water until they are soaked.When he returns to the nest, his chicks push against his feathers andsuck the water from his breast (Goodwin 1970).Other birds carry water back in their beaks and dribble it into the open mouths of their chicks.Some mammals and birds will eat food and then regurgitate it upon return to the nest.In the case of mammals, female parents must decide whenand for how long to recline so that their offspring can nurse.Parents must also decide when to eat the food themselves and when to deny themselves in order to share food with their brood.For example,blackbirds will normally eat the first few insects they catch, including dragonflies.However, when feeding their young they will immediatelytake the dragonfly back to the nest (Orians 1980).Blackbirds will often carry several insects at once when returning to the nest.Although moreefficient, this collection task is itself complex, since insects that have dropped from the bird's beak must be found and picked up again withoutdropping those currently being held.Upon leaving the nest, the young may be carried by their parents.Rats, dogs and cats pick up their cubs in their jaws by the scruff of the neck,carrying them one by one to new sites.Anteaters and monkeys carry their young on their backs, as do scorpions and some spiders.Parents alsoemploy strategies designed to teach their children survival skills.For example, as cubs grow older, the lioness will bring back live animals sothat the cubs gain practice at hunting, catching and killing prey (Leyhausen 1979).Mammals are not always kind to the young of their species.Male lions that take over a pride will sometimes kill and eat the cubs that are nottheir own.After mating with the lionesses, they are good parents to their own offspring (Cloudsley-Thompson 1965; Fox 1968).In somespecies, however, orphaned young may be adopted by other adults (Morris 1990).(9)Predation Strategies-- Predators have developed numerous strategies, from those that are largely instinctual in lower forms. e.g. (Ewert1987), to those requiring learning in mammals.Angler fish and certain amphibians and snakes make use of parts of their bodies as a bait to luretheir prey to them (Wickler 1968).Bears will scoop up migrating fish in rivers.Herons will stand very still and then at the opportune moment,rapidly stab the fish.Cats stalk their prey, crouching down and inching along until within striking distance.Humpback whales, for instance, let out a stream of air bubbles in a circular pattern around small fish, who are reluctant to pass through thesebubbles and thus become concentrated and easier for the whale to consume (Herman 1980).Some predators, such as lions, hunt cooperatively(Schaller 1972).Griffin (1984), for instance, describes observing four lioness slowly spreading out into a U-shaped formation to surround apack of wildebeast.Two of the lionesses approached and sat very conspicuously at sites near the herd while the third lioness crawled low to theground toward the herd.Suddenly, the four lion, who had circled way about, rushed out from within a nearby forest and drove the herd towardthe crouched lioness, who then leaped out and grabbed a wildebeest.The other two lionesses then very leisurely strolled toward the one whohad made the kill; whereupon they shared it.Griffin concludes that it appears that the two conspicuous lionesses were intentionally drawingattention of the wildebeasts away from the stalking lionesses.(10)Predator Avoidance and Defense-- Prey employ different strategies in the face of predation (Edmunds 1974).Hares will freeze until apredator is quite near before springing away.Squirrels dart to the blind side of a tree and then freeze.During flight, animals vary their speedand direction of flight dramatically, based on the behavior of their pursuer.Many prey species flee to the safety of trees, water or burrows.Flightstrategies include combinations of startle displays (e.g. birds displaying a bright spot of color; the frilled lizard extending the flap of skin aroundits face), dashing, freezing and hiding.Startling a predator gives the prey a chance to escape.For example, a cornered cat will hiss and spit;some insects will assume the posture of poisonous insects in order to scare off an attacker.In the case of flight, the type of safety zone a preywill seek often depends on the type of predator in pursuit.A rooster will climb upon a rock in the face of a ground predator while hiding underbushes when spotting a hawk from above (Marler, 1967; Marler and Hamilton 1966).Prey that are cornered (or too fatigued to flee) may turn and attack the predator.Many species of small birds (e.g. warblers, finches, thrushes andeven hummingbirds) will mob a bird of prey.For instance, if an owl remains stationary (perhaps due to illness), the prey birds may beginmobbing it.They dive down, buzzing within inches of its head; even clawing it.This mobbing behavior attracts more birds to join in until theowl is routed.Ground squirrels will also mob a snake, rushing it and trying to bite it or throw sand in its eyes.A group of baboons orchimpanzees may also mob a leopard (Morris 1990).Predator avoidance is quite complex because it involves continual monitoring, by the prey, of any predator dispositions.For instance, gazelleswill actually follow predators in order to keep watch on their movements (Walther 1969).In order to avoid needless expenditure of energy, preymust continuously assess whether or not predators are about to strike or are simply passing by.(11)Dissembling Behaviors--Many animals, including various species of insects, frogs, snakes, birds and small mammals (e.g. squirrels andopossums) will feign death (Matthews, 1969; Mertens 1960).They will remain immobile while being mauled.Once the predator is convincedthat the prey is dead, the predator may leave the prey alone, thus giving it an opportunity to escape.Many ground-nesting birds will pretend tobe wounded by dragging an extended a wing and/or struggling to walk; thus distracting the predator away from the nest (Armstrong 1947).Other dissembling strategies in ground-nesting birds includes false incubation, in which the bird pretends to be sitting on nest of eggs at a siteaway from the actual eggs (Sordahl 1981).Such behavior is quite complex.The dissembling bird continuously must monitor the predator.If thepredator appears to lose interest and/or fails to follow, then the bird recognizes this and temporarily abandons its dissembling behavior to flyback toward to predator.It then reinitiates its dissembling strategy.(12)Primitive Tool Use and Culture-- Wild chimpanzees make use of sticks in order to extract termites from their nests (Goodall 1971).Themongoose throws hard-shelled eggs at nearby rocks in order to smash them open.The sea otter swims on its back with a flat stone on itsstomachIt then smashes clams against this stone to break them open (Griffin 1984). The Egyptian vulture picks up a stone in its beak andthrows it down at an ostrich egg in order to crack the egg open (Ewer 1968).Herring gulls and some crows break open clams by dropping themwhen in flight onto hard, rocky surfaces (Beck 1980).Studies show that such birds select shells based on weight (indicating a live/edible vs.dead mollusk within) and vary the height of release based on the perceived hardness of the ground below (Zach 1978).Insects also use tools.Adult weaver ants roll up leaves and fix the edges with silk in order to make nests.But since the adults cannot producesilk themselves, they pick up their silk-producing ant-larvae and squeeze them to force out a drop of liquid silk.The adults thus use the passivelarvae as tubes of glue (Holldobler and Wilson 1977).Leaf-cutter ants have developed a form of agriculture, in which they carry undigestibleleaves back to their colony.They chew the leaves and add fecal material, which then serves as food for a fungus that they grow in specialchambers within the nest.The ants then eat the fungus.When a reproductive female leaves to form a new colony, she takes with her a small bitof the fungus, as starting material for her new fungus garden (Wilson 1971).Higher animals, e.g. primates, exhibit aspects of culture.For instance, knowledge for how to clean dirt-covered sweet-potatoes (by washingthem in water) was seen to spread, via primate learning, observation and mimicry, through a troop of Japanese macaques (Kawamura 1962;Kawai 1965).This skill was later passed down from parent to offspring.Only the original, older males never acquired the behavior.(13)Other Complex Behaviors-- The behaviors above by no means exhaust the types of complex strategies employed for survival.Otherbehaviors include:migration and navigation,strategies to protect oneself while sleeping or ill, grooming behaviors to avoid parasites anddisease, and among mammals, behaviors involving play (Tizard and Harvey 1977; Smith 1982), in which the young try out and learn aspects ofparental roles.These behaviors (taken either individually or, more realistically, in a social context and thus requiring integration of multiple coordinated andcarefully sequenced sensory/motor behaviors) constitute a wealth of benchmark tasks for any synthesis-oriented research program.Theexecution of any of these behaviors requires a high degree of intelligence and computation.For example, food finding requires complex patternrecognition and generalization capabilities.Generalization is required because no two forms of food, whether stationary or moving, will lookidentical; likewise, any two acts within the same functional category (e.g defense, submission, feeding, etc.) will not be identical.Differentialresponses to distinct members of one's own species or kin, and to distinct kinds of predators and prey, requires categorization of objects andactions in the environment.Animal decision making is also complex and state dependent.For instance, predator avoidance requires planning in the face of multiple andpossibly conflicting constraints, as in the case, for instance, when an animal is both hungry and thirsty but food is at one location and water atanother;a potential predator is in between the animal and one food source but possibly looks tired or satiated; the animal's own cubs are atanother site and must be fed soon; it is getting dark, and so on.While the nature or even existence of conscious thought is highly controversialin animals (Griffin 1984; Walker 1983), it is clear that the tasks performed by animals require enormous amounts of computation.For example,consider cooperative hunting carnivores or nesting site selection and group nest construction by bees, weaver-birds or, among mammals, thedams built by beavers (Wilsson 1968). The sensing, locomotion, manipulative and social skills exhibited by such animals in performing thesetasks completely eclipse any kind of robotic behavior produced so far within the field of AI.3.2.Animal Cooperation via CommunicationMost forms of cooperation require communication, which can be accomplished by visual, tactile, acoustic and/or olfactory means (Estes 1991).These signals (e.g. facial expressions, scent marks, body postures/movements, vocalizations, displays, etc.) need not be "intentional" toconstitute communication.What is necessary is that the behavior (or resulting state caused by the behavior of the sender) be perceived and bringabout differential behavior on the part of the perceivers that is in some way ultimately advantageous to the sender or the sender's group.Thisresulting behavior may then itself be perceived by the original sender, thus possibly causing differential behaviors in response.Communicationmay occur both within and across species.(1) Insect Communication.Insects commonly communicate via chemicals, such as pheromones.They also employ tactile motions and, onoccasion, visually perceived displays.For instance, bees communicate to their comrades the location and amount of nectar found at a site bymeans of a waggle dance.The angle, amount of waggling, and wing-fanning display indicates, respectively,the direction, distance, and amountof nectar (Frisch 1967).Weaver ants make use of different pheromones, excreted from rectal and sternal glands, to recruit other weaver ants forforaging, for emigration to a new nest, or in defense against attackers.These chemical signals are combined with tactile signals, such astouching with antennae, head-waving, mandible-use, and jerking of the body (Holldobler and Wilson 1977).Acoustic signals are also commonly employed by insects.For example, the Douglas-fir beetle employs a half-dozen distinct acoustic signals.The males employ distinct chirps, indicating:(a) approach to the egg gallery entrance containing a female, (b) imminent copulation duringcourtship, (c) rivalry with other males, and (d) situations of stress.The female generates a distinct chirp when constructing and guarding her egggallery (Ryker 1984).Fireflies use flashing lights to signal and attract mates.The male firefly indicates his gender and species by his pattern of flashing.The femalethen signals back with a distinct set of pulses.The timing of her flashing, with respect to the males, indicates her gender/species.Predatoryfemale fireflies have evolved mimicry of the response timings of females of other species and thus can attract the corresponding males to them,which the predatory females then devour (Lloyd 1981).Physical behaviors can also serve as signals.For example, the male black-tippedhangingfly presents the prey it has caught to the female which the female then examines to determine whether or not she will mate with him(Thornhill 1980).(2)Avian Communication.Birds commonly communicate (Lorenz 1971), both acoustically and visually, through the production of songs andelaborate dances and displays (e.g. lekking).Many birds give off specific vocalizations when they encounter food or sense danger (Marler,1977).Domestic chickens can communicate information about the quality of available food (Marler et al. 1986) while roosters give off differentvocalizations for warning of raccoons versus hawks.What is a particular interest is that the rooster only issues these vocal warning signals whenthe rooster is aware of a nearby hen of the same species (Ristau 1991; Marler et al.1986).If no hen is near, then the rooster intelligently avoidspossible predator awareness and attack by remaining silent.Duet-singing birds, such as the African shrike, produce complex antiphonal (i.e. interspersed notes) and polyphonic (interspersed andsimultaneous) music.Mating pairs (who often mate for life) discriminate their mates from other shrikes by coordinating musical pitch, harmonicand melodic intervals, duration and timing in an extremely precise manner (Thorpe 1973).Such duets also function in recognizing species/kinmembership, maintaining social contact in dense foliage, and establishing territory.For example, females will not answer the songs of othermales and when the male is absent, during that time the female sings not only its own part, but also that of its mate.It is believed that thisrehearsal allows one bird to continue remembering its mate's unique "signature" for later recognition upon return of the mate.(3)Mammalian Communication.This can take many forms.For instance, tactile communication among Duiker antelopes includes sociallicking/nibbling of the head, neck and shoulders, with male/female pairs rubbing together the glands near their eyes for maintaining pairbonding.Two Duiker males will do this also, but as a prelude to combat (Ralls and Kranz 1984).Signals are used also for warning of predators.Prairie dogs not only issue distinct calls for different predators (hawks, snakes, coyotes, humans,etc.) but there is evidence that they can, in their signal, specify distinct individual predators who have become familiar to them from within agiven predator group (Slobodchikoff et. al., 1986, 1991).Many mammals have a wide repertoire of behaviors that act as signals to others.Consider the cat family (Leyhausen 1979).Felids communicateby body posture, action, facial expression, scent (urine spraying and rubbing), vocalizations and licking/nuzzling/pawing actions.Facially, themovement of ears (relaxed, pricked up, flattened, twitching, rotated) mean different things; for example, cats rotate their ears upward whenangry.In general, facial and body postures indicate predisposition to attack, defense, or flight.Vocal signals include meowing, yowling,growling, grunting, snarling, purring, spitting, hissing, roaring, etc.For instance, mother felids employ a special call to attract their cubs whenbringing back live prey.The intensity of this call is different, depending upon whether the live prey is small and harmless or large andpotentially dangerous.In the latter case the cubs respond more carefully.Older, mobile cubs are warned to take cover by the mother bothslapping and hissing at them. When family members meet, they greet each other by rubbing noses, then heads and sides.Vocalizations andpostures also differentiate defensive vs. offensive roles and dominance vs. submission.(4)Primate Communication.Vervet monkeys also distinguish predator classes (leopards, eagles, snakes) in their warning calls.Each distinct callcauses a different type of behavior.When hearing a leopard-warning sound, vervet monkeys run into the trees.Snake alarms cause them to riseup on their hind legs and look in the grass while eagle warning sounds cause them to look at the sky or duck under bushes (Struhsaker 1967).Primates also appear to actively attempt to deceive one another (Whiten and Byrne 1988).In general, apes appear to have the greatest capacityfor abstract thought among primates (Premack and Premack 1983) and the largest vocal repertoires (Byrne and Whiten 1988; Cheney andSeyfarth 1990; Ristau 1991) with Goodall (1986) cataloging over 30 chimpanzee calls used in the wild and over 200 signs (either visual symbolpatterns or sign language gestures) learned in human-trained gorillas and chimpanzees (Gardner and Gardner 1969; Terrace 1979; Patterson andLinden 1981; Ristau and Robbins 1982).(5) Cross-Species Communication.For instance, the honeyguide bird signals to the honey badger that there is honey available by chattering andmaking displays.The honey badger then replies with a special hissing chuckle.Thereupon the honeyguide leads the honeybadger to the honeysite.The honeyguide never gets more than 20 feet ahead of the honeybadger, in order to keep it in visual contact (Perry 1983).Communicationacross species commonly occurs in predator-prey interactions.Prey animals with defensive systems (e.g. the skunk) will often first warn apotential predator before actually employing its defensive system; thus possibly avoiding having to expend energy using it.3.3. Animal Development and LearningAnimal behaviors are the result of complex interactions between genetic and developmental factors.In general, genetic effects are morenoticeable and tend to dominate in lower life forms.For example, although each species of cricket has a unique set of distinct chirp patterns, ithas been shown that these patterns are not learned, but are genetically determined (Bentley and Hoy 1974).However, there is strong evidencethat nearly all animals, even slugs and insects, are capable of learning (Griffin 1984).For example, bees are able to quickly learn and remember,over many days, the odor and color of flowers that yield a given pollen or nectar (Menzel and Erber 1978).Experiments in which bees aremoved to distant sites during the night support the theory that bees construct mental maps of their environments, which they use for navigation(Gould and Marler 1987).Bees go through different occupational stages, based on their age and experience.The youngest bees clean thehoneycombs;older bees help build the combs; later these bees "graduate" from comb construction and leave the hive to forage for nectar(Lindauer 1971).In birds, many behaviors (that might at first be considered instinctual) have been shown to be learned.For instance, European blackbirds willlearn to mob a harmless (non-predator) honeycreeper bird if it is made to think that other birds are mobbing that bird (Morris 1990).In thewhite-crowned sparrow, the learning of songs depends upon both a sensitive/critical period and access to examples of appropriate, species-specific songs.The young songbird makes use of species-specific songs it hears and, in a trial-and-error manner, develops it's own song version(Gould and Marler 1987).It has been demonstrated that pigeons can learn to discriminate underwater photographs of fish (Herrnstein and deVillers 1980) -- a task for which there has been no evolutionary selectional pressure.An interesting interaction between genetics and development is that of imprinting (Lorenz 1970).For instance, what a duckling imprints upondepends on what is currently in the environment during a critical period.At the same time, the range over which imprinting operates isgenetically determined and highly constrained (e.g. to mobile objects within a given size range; otherwise, baby ducklings would imprint onrocks, insects, or even their own siblings).A complex form of learning is that of learning through observation and subsequent imitation (Griffin 1984).Most birds and mammals exhibitthis kind of learning.Young lions, for instance, learn how to hunt properly by observing their mother hunt.4. Synthesizing Animal Intelligence via Evolution and LearningRecent computer models used in biological, ethological and ecological studies have employed game theory (Maynard-Smith 1982), optimizationtechniques and dynamic state-variable models (Mangel and Clark 1988; Clark 1991).The AL approach extends such modelling techniques byincluding simulations of artificial nervous systems (at some level of abstraction) which control sensing/acting artificial agents that can moveabout in artificial environments that havespatial structure (usually 2-D).In addition, artificial genomes are specified that encode ANNs (eitherdirectly, or indirectly, through some ontogenetic process).Genomes are modified, over many generations, via a variety of genetic algorithms.During an artificial animal's lifetime, its ANN may also be modified via adaptive learning algorithms.Thus, AL models offer a wide range ofmore realistic modeling techniques, accessible for the first time as the result of great increases in computational power and reduction in the costof computation and memory storage.Although the field of AL is still in its infancy, a number of simulation models have been developed, in which ANN-controlled life forms evolveand/or develop survival behaviors within simulated environments.Some of these are briefly mentioned below.4.1. Evolution/Learning of Food DiscriminationTodd and Miller (1991) set up an abstract, simulated "aquatic" environment containing two distinct patches of "plant material" -- a red patch andgreen patch.Within one patch the red plant serves as "food" for the evolving creatures while the green plants act as "poison".In the other patchthe color roles are reversed.Each creature remains, during its lifetime, in a given patch; however, its offspring at birth may be placed in theopposite patch.If a creature eats food its metabolism is increased (thus improving reproductive success) but if it consumes poison itsmetabolism is reduced.Creatures are immobile but food material (or poison) "floats" past them and they must decide whether to eat it or ignoreit.In addition, food always smells "sweet" while poison always smells "sour" (i.e. no matter what their color assignments happen to be within agiven patch).Finally, there is "turbulence" in the "water".Increased turbulence decreases the accuracy of smell.For example, a red plant maymistakenly smell sweet even though, in that patch, it is always poisonous.The behavior of Todd and Miller's creatures is controlled by neural networks containing (a) color and scent sensory neurons, (b) an ingest motorneuron and (c) an interneuron.The genome of each creature directly codes excitatory/inhibitory connections between these neurons.In addition,connections can be either fixed or learnable.Learnable connections are modified, during a creature's lifetime, via a Hebbian rule, in whichcorrelated firing of connected neurons increases the strength of their connections.Since what is poisonous can change color from generation to generation, there is no advantage to hardwiring connections for poison avoidancein terms of color.However, within an individual's lifetime, color does serve as a poison/food discriminator that (depending on the amount ofturbulence) is more accurate than smell.Over several hundreds of generations, creatures evolve with a hardwired connection between the smelland motor (eating) neurons but with a learnable connections between the color and motor neurons.This connection is then modified over thelife a a given creature, based on which type of food patch it ended up in.4.2. Evolution of Foraging and Trail LayingCollins (1992) performed a series of experiments (labelled AntFarm I through AntFarm V) in which he attempted to evolve colonies of ANN-controlled artificial ants that both forage for food and lay down pheromone trails to guide others to food sites.Most of the experiments consistedof 32K colonies with 128 ants per colonies being simulated on a massively parallel computer.He varied numerous parameters, including thegenome size (from 25,590 bits in AntFarm I to 487 bits in AntFarm V) and ANN encoding schemes (e.g. symmetric vs. non-symmetricconnections; excitory only vs. incorporation of inhibitory connections; interneurons varying from 21 bits to 0 bits).In the earlier experiments,food foraging behaviors evolved, but they were non-ant like, e.g. ants all walked in large circles or semi-circles.Collins only succeeded inevolving ant-like behavior in AntFarm V.These ants exhibit mostly forward movement with random turns until a food site is found, then theytransport some food directly back to the nest while laying pheromones.In this experiment, Collins forced ants to involuntarily lay pheromonesfor generations 1000 -2000 and then returned pheromone-release control back to each ant.At generation 2001 there was a large decrease in theamount of pheromone being released butthe ants had evolved to both lay and follow trails by generation 2100.Collins theorizes that, beforeants were forced to release pheromones, trail following could not evolve (and so trail laying could not evolve either).However, once ants hadfinally evolved to follow trails (i.e. follow those trails created during the generations in which forced pheromone release occurred), then any(initially inadvertent) pheromone release upon return from a food site would lead other ants to that site and thus increase survivability of thecolony.As a result, trail laying and trail following could now co-evolve.4.3. Evolution of CommunicationMacLennan (1991) reasons that communication emerges only when (a) what he terms "simorgs" (simulated organisms) can perceive, in theirlocal environments, information unavailable but advantageous to other simorgs and (b) the signalers are able to produce changes in theenvironment that the receivers can detect.MacLennan also argues for incorporating overlapping generations into Holland's genetic algorithm(GA), so that simorg offspring can learn from prior generations.In MacLennan's experiments, each simorg's genotype is represented by atransition table; likewise its phenotype.In addition to local environments, there is a shared, global environment of symbols and each simorg,based on its transition table, can match and/or post a symbol to the global environment.Whenever a simorg's action matches that of the mostrecent symbol posted (representing information about the local environment of another simorg), both matching simorgs receive a credit.Simorgs with higher credit have a greater chance of being selected for mating.MacLennan's experiments indicate that average fitness of thepopulation increases much more rapidly when communication is allowed than when it is repressed.He also compares the relative influence ofcommunication with learning enabled/disabled and shows that concurrent learning and communication result in the highest average fitness.In (Werner and Dyer 1991) we evolved simple communication protocols for mate finding between ANN-controlled artificial animals in a 2Dtoroidal, grid environment.Females received, as sensory inputs, the location and orientation of any male within a 5x5 grid sensory areasurrounding the female.The motor output of a female was interpreted as a signal that was then copied as input to any male within the female's25-cell sensory area.In contrast, the male's motor output caused him to move left, right, forward, or stand still.Thus, females were immobile butcould sense and signal nearby males.Males were "blind" but could "hear" the signals of nearby females and move about in the environment.Mating occurred whenever a male landed on a grid cell occupied by a female.As a result of mating, a new male and female offspring (viarecombination and mutation of parental genes) were produced and placed at random locations in the environment.At the same time, a randommale and female were removed from the environment, thus keeping the overall population size stable.Starting with a population of randomgenomes, the simulation resulted in a progression of generations of ANNs that exhibited increasingly effective mate finding strategies.In effect,the females evolved to inform nearby males how to move in order to find the females and mate.The males simultaneously co-evolved tointerpret female signals in order to land on female-occupied squares.In addition, a number of distinct subspecies (i.e. groups with differentsignaling protocols or "dialects") evolved and competed in the environment.Experiments with physical barriers in the environment were alsoperformed.A partially permeable barrier allowed a separate subspecies to evolve and survive for indefinite periods of time, in spite of occasionalmigration and contact from members of other subspecies.Like in MacLennan's model, we have overlapping generations.However, in our model, organisms are not selected for mating, but must findtheir own mates.We believe that it is important not to directly select for communication.Instead, communication should evolve because itenhances the ability of organisms to perform tasks that are directly related to survivability, such as finding mates.4.4. Evolution of Predation and Predator AvoidanceIn (Werner and Dyer in press-a, b) we have extended our 2-D environment to include simulated "smells" and "sounds" of different types andintensities, that diffuse away from their sources, thus creating complex sensory gradients.Multiple species of life forms (termed "biots") interactin the environment (termed "BioLand"), which also contains simple physical objects, such as plants, trees and holes.Plants serve as food forsome biot species while trees and holes serve as landmarks, and/or safety zones for nesting and/or escape from predator biots.Biots produceinvoluntary smells, which serve to indicate their species/gender membership and can be extended to indicate other physical properties, such asage, receptivity to mating, etc.Biots also produce involuntary sounds, which become louder the faster they move and thus diffuse over greaterdistances.Sounds can also be producedvoluntarily -- that is, under biot neuro-motor control.Each biot can generate a variety of distinctvoluntary sounds (termed "frequencies").Different frequencies allow biots to communicate and/or discriminate different sounds and soundcombinations that they receive through their bilateral sensory neurons.Biots are capable of smoothly turning and altering their speeds.They possess a metabolism which drops as they perform actions and riseswhenever they eat food (i.e., food consists of prey biots for carnivores; of plants for herbivores).Biots can grasp/release objects and also containa mating motor neuron.Mating and offspring occur whenever two same-species biots (each with sufficient metabolism) come near one anotherand simultaneously fire their mating motor neurons.In such cases, variant offspring are produced via mutation and recombination of parentalgenomes.Biot genes encode both standard axo-dendritic connections and also higher-order, axo-axonal gating connections.In one experiment,herbivore biots were termed prairie dogs while two distinct predator species were termed snakes and hawks.Prairie dogsevolved to run away from snakes and hawks while these predators evolved to chase the prairie dogs.Prairie dog biots also evolved to form herdsfor protection from predators.In ongoing experiments we are evolving differential predator warning signals among prairie dogs, who will thenseek the appropriate shelter (tree vs. hole) based on the nature of the warning signal.4.5. Toward the Synthesis of Proto-Human IntelligenceANNs capable of exhibiting complex animal behaviors (e.g. nesting, parenting) have not yet been engineered, evolved, or designed throughlearning; therefore, it may be premature to discuss the synthesis of proto-human forms of intelligence (such as human-like language andthought) via AL techniques.It may be possible, however, that sophisticated ANNs, capable of aspects of human thinking and language could befirst engineered and then a population of them placed in a simulated environment.The population could then undergo evolution, for instance, tosee if such engineered ANNs could survive, interact, cooperate, communicate and adapt.There is currently an intense, ongoing debate over therelative roles of innate brain structures, development/learning processes and adaptation in the evolution of human language (Bates et al. 1989;Lieberman 1989; Piattelli-Palmarini 1989; Pinker and Bloom 1990; Premack 1990).This debate cannot be resolved either through examiningthe fossil record or engaging in thought experiments.We believe that the AL simulation approach proposed here has the long-term potential ofproviding important insights in this area of inquiry.At this point, a number of ANNs, that exhibit aspects of human-level inferencing, languageacquisition and comprehension have been engineered.In my own lab, for example, we have designed:(a) ANNs that learn the meanings of wordsequences by observing the motions of simulated objects on an artificial retina (Nenov and Dyer in press), (b) ANNs that automaticallydisambiguate word meanings via plan analysis (e.g. "...washed the pot in the dishwasher" vs. "...hid the pot in the dishwasher to keep fromgetting busted") (Lange and Dyer 1989), and (c) ANNs that acquire both world knowledge (i.e. scripts -- stereotypic action sequences), wordmeanings and infer unstated events when generating story paraphrases (Miikkulainen and Dyer 1991).However, we are still years away fromplacing even small groups of such complex neural networks into evolutionary or adaptive environments.5. Other Research Issues and Methodological PrinciplesThe AL approach to synthesizing intelligent behavior described here has its roots in the fields of parallel distributed processing (Rumelhart andMcClelland 1986), connectionism (Touretzky 1989; Horgan and Tienson 1991), computational neuroscience (Anderson and Rosenfeld 1988;Schwartz 1990), neuroethology (Camhi 1984; Ewert 1987), comparative psychology (Roitblat 1982, 1987), genetic algorithms (Belew andBooker 1991), biological cybernetics (Braitenberg 1984), situated agents (Travers 1989) and reactive micro-robotics (Brooks1991).As a result,the AL approach to synthetic cognition carries along with it other general research issues, that are related to each of these fields.These generalissues involve: (a) The relationship between learning and evolution.For example, Nolfi et al. (1990) and Belew (1990) and colleagues (Belew etal. 1992) have performed experiments in which evolving ANNs are augmented with backpropagation learning during their lifetimes.(b) HowANNs encode knowledge of their environments (Roitblat 1982; Roitblat et al. 1983; Churchland and Sejnowski 1992) and respond adaptively tothem through feedback (Beer 1990).(c) How genomes and artificial organisms are best encoded (Collins and Jefferson 1991) and expressedontogenetically (Nolfi and Parisi 1991).(d) How to address outstanding problems in population biology, ecology and ethology, such as theevolution of altruism (Axelrod 1984).(e) How to encode environments that are computationally tractable but rich enough to supply thenecessary selectional pressures (Werner and Dyer in press-a).The AL approach toward synthesizing intelligence can be computationally extremely expensive, since it involves simulating many generations ofentire populations of the individual lifetimes of sensing/acting ANNs.Consequently, models must be simplified or made more abstract in someway, in order to remain tractable.Below we offer the following methodological principles as possible guidelines in simulation design.(1)Principle of Hypothesis-Driven Abstraction

Hierarchies-- Any given task can be represented at many different levels of abstraction.Forexample, the task of building a bird nest can be modeled at a very abstract level (e.g. a single nest-building motor neuron fires and, presto!, acomplete nest is created in the environment) or at an extremely detailed level (e.g. millions of neurons fire, to control a robot bird with hundredsof muscles/joints to simulate complex beak, claw, torso, head movements during nest construction sensory/motor actions).Which level ofabstraction is selected depends greatly on where theoretical claims are being made.A single nest-making motor neuron might be sufficient for amodel whose hypothesis involves the placement of a nest in a site selection model; however, it would be insufficient for a model concerning theevolution/learning of nest-construction behavior.(2)Principle of Minimal Effective Embodiment-- For the task of synthesizing intelligent cooperative behaviors, it is important to concentrate onthe evolution of the structure of nervous systems while avoiding, wherever possible, the inclusion of physical properties of the organisms.Forexample, real animals vary in size.However, if one is trying to evolve, for instance, predator avoidance strategies, then the modeling of sizemay be detrimental, and result simply in an evolutionary "size arms race" in which artificial animal populations avoid predation by simplyevolving to be larger.Again, what physical characteristics of an organism to include depends on the research hypothesis being explored.Forexample, if one is evolving predator escape behavior in a landscape of burrows of varying sizes, then size (of both prey and predator) might beimportant.However, even in this case one would want to hold the size attribute fixed over evolutionary time, so that one can concentrate on theevolution of the behavior and the structure of the ANNs that produce it.Another form of minimality is to reduce the scale or granularity of theinterface between the organism and its environment as much as possible; for instance, instead of 1 million rods/cones for an artificial eye,employing a 10 x 10 matrix of sensory neurons.(3)Principle of Midpoint Entry-- Because the search space (of possible agent populations, interactions, and environments) is so vast and broadranging,it is acceptable to start one's model somewhere in the middle.This principle supports the mixing of bottom-up and top-down researchstrategies.The top-down approach involves starting out with a specific goal in mind and then engineering a system to achieve that goal, usingwhatever constraints are available to guide model construction.The bottom-up approach involves starting out with a set of tools or buildingblocks and then tinkering with these in systematic ways to see what results.In general, the AL approach tends to be bottom-up in nature, sinceboth evolution and adaptive learning are mainly bottom-up.However, given that the research space is so vast, it appears reasonable to use top-down techniques to design organisms at a given level of sophistication, that are then placed in environments in which evolutionary and adaptivelearning processes are then applied.This mixed approach allows one to start, for instance, with a population of pre-engineered ANNs withsophisticated capabilities and then see if such ANNs can improve (or even maintain) these capabilities within an evolutionary and learningenvironment.In fact, every researcher who specifies the sensory/motor capabilities of a given artificial agent is initially using a top-downapproach.(4)Principle of Indirectness-- Whenever possible, it is important to avoid setting up an objective function that selects directly for the desiredbehavior.Instead, the desired behavior should arise indirectly, as a side-effect of increased survivability (due to the development or evolution ofthat behavior).For example, instead of directly selecting artificial animals for reproduction who communicate, in (Werner and Dyer 1991) onlyanimals who found their own mates could reproduce.Communication then evolved because it improved the ability of animals to find their matesand thus produce offspring(5)Principle of Naturalness-- Since there are potentially so many parameters to adjust in AL systems, it is important to impose constraints thatare intuitive.A major source of intuitions comes from constraints that already exist for real animals in Nature.Whenever these intuitions areviolated, they should be carefully noted.For example, in (Werner and Dyer 1991) when mating occurred we placed the offspring at distant sitesfrom their parents.This kind of event does not occur in Nature.Actually, we did it because our ANNs had no way of discriminating their kin norreason to avoid committing incest, andparents paid no metabolic cost in generating offspring.As a result, parents and offspring, if allowed tostay in the same spot after mating, would simply remain there, forever mating with each other.Thus, this initial lack of "naturalness" led to amodel that became more and more difficult to extend.To solve this problem, we decided to add new features to the environment (see section4.4) in order to make the environment conform more to natural environments.Finally, this paper has mainly concentrated on issues in software simulation.Clearly, there is a corresponding (albeit distinct) entire researchprogram in synthesizing intelligent hardware (i.e. robots).An AL-based robotics research program poses distinctive challenges, only two ofwhich I will mention here:(1) With physical robots it is very difficult to have large populations, thus greatly reducing the effectiveness ofgenetic algorithms.Even a small population of 100 micro-robots would be very difficult to control and time-consuming to set up for repeatedexperimentation.Perhaps a mixed approach can be developed, in which large populations of ANNs are evolved in simulated environments andthen a small subset of these are down-loaded (as software control) into the actual robots, which are then tested in the real world.(2) Models ofontogenetic development will also be extremely difficult, because one cannot easily modify the morphology of a physical robot.6. ConclusionsAlthough animals lack human-level thought and language, they still exhibit a high degree of intelligence.Animals must satisfy multiple, oftenconflicting constraints in order to survive.Animals hide from predators, forage and scavenge, fight, court, mate, establish dominance, climb,burrow, build elaborate nests, teach and care for their offspring, groom, establish and defend territories, migrate, navigate, use primitive tools,cooperate and communicate.Such tasks require complex sensory analysis and motor integration.They also learn via observation and adjust theirbehaviors to constantly changing environments.They categorize and manipulate their environments and generalize from experience (Gould1982,; Griffin 1984).They form social organizations through communication and cooperation.One major goal of the field of Artificial Life is tosynthesize these complex behaviors by constructing artificial nervous systems, via developmental and evolutionary modeling techniques, andthus gain insight into the computations underlying the cognitive capabilities of both animals and man.AcknowledgmentResearch by author on evolving communication and predation was supported in part by NSF grant IRI-910730.ReferencesAgre, P. E. and Chapman, D. (1987).Pengi: an implementation of a theory of situated action.InProceedings of Sixth National Conference onArtificial Intelligence, AAAI/Morgan Kaufmann Publ., Los Altos, CA.Anderson, J. A. and Rosenfeld, E. eds. (1988).Neurocomputing: Foundations of Research.MIT Press, Cambridge MA.Armstrong, E. A. (1947).Bird Display and Behaviour, Drummond, London.Axelrod, R.The Evolution of Cooperation. Basic Books, NY, 1984.Bastock, M. (1967).Courtship: a Zoological Study.Heinemann, London.Bates, D., Thal, D. and Marchman, V. (1989).Symbols and syntax:A Darwinian Approach to Language Development.In:N. Krasnegor, D.Rumbaugh, M. Studdert-Kennedy and R. Schiefelbusch (eds.),The Biological Foundations of Language Development..Oxford University Press.Bechtel, W. and Abrahamsen, A. (1991).Connectionism and the Mind. Blackwell Publ., Cambridge MA.Beck, B. B. (1980).Animal Tool Behavior.Garland STPM Press, NY.Beer, R. D. (1990).Intelligence as Adaptive Behavior: An Experiment in Computational Neuroethology. Academic Press, NY.Belew, R. K.Evolution, learning and culture:computational metaphors for adaptive search.Complex Systems, 4(1), 1990.Belew, R. K. and Booker, L. B. eds., (1991).Proceedings of the Fourth International conference on Genetic Algorithms.Morgan Kaufmann,San Mateo, CA.Belew, R. K, McInerney, J. and N. N. Schraudolph. (1992). Evolving Networks:Using the Genetic Algorithm with Connectionist Learning.InLangton, C. G., Taylor, C., Farmer, J. D. and S. Rasmussen, eds.Artificial Life II, Addison-Wesley.Bentley, D. and Hoy, R. R. (1974).The Neurobiology of Cricket Song.Scientific American. 231: 2, 34-44.Bond, A. and L. Gasser (eds.). (1988).Readings in Distributed Artificial Intelligence.

31, 1-44.Pinker, S. and Bloom, P.(1990).Natural language and natural selection.Behavioral and Brain Sciences13, 707-727.Premack, D.(1990).On the coevolution of language and social competence.Behavioral and Brain Sciences