Weembodiednetworksofculturedbiologicalneuronsinsimulationandinrobotics.Thisisanewresearchparadigmtostudylearning,memory,andinfor-mationprocessinginrealtime:theNeurally-ControlledAnimat.Neuralactivitywassubjecttodetailedelectricalandopticalobservationusingmulti-electrodearraysandmicroscopyinordertoaccesstheneuralcorrelatesofanimatbehavior.Neurobiol-ogyhasgiveninspirationtoAIsincetheadventoftheperceptronandconsequentartificialneuralnetworks,developedusinglocalpropertiesofindividualneurons.Wewishtocontinuethistrendbystudyingthenetworkprocessingofensemblesofliving neurons that lead to higher-level cognition and intelligent behavior.1 IntroductionWepresentanewparadigmforstudyingtheimportanceofinteractionsbetweenanor-ganismanditsenvironmentusingacombinationofbiologyandtechnology:embodyingculturedlivingneuronsviarobotics.Fromthisplatform,explanationsoftheemergentneuralnetworkpropertiesleadingtocognitionaresoughtthroughdetailedopticalandelectricalobservationofneuralactivity.Abetterunderstandingoftheprocessesleadingtobiologicalcognitioncan,inturn,facilitateprogressinunderstandingneuralpathologies,designingneuralprosthetics,andcreatingfundamentallydifferenttypesofartificialintel-ligence.ThePottergroupisoneofsevenintheLaboratoryforNeuroengineering(Neuro-lab1)attheGeorgiaInstituteofTechnology,allworkingattheinterfacebetweenneural

1

http://www.ece.gatech.edu/research/neuro/EMBODIED ARTIFICIAL INTELLIGENCE(pre-print)Springer-Verlag2tissueandengineeredsystems.Weenvisionafutureinwhichmechanismsemployedbybrainstoachieveintelligentbehaviorarealsousedinartificialsystems;weoverviewthreepreliminaryexamplesoftheNeurally-ControlledAnimatsapproachbelow.Byusingbiol-ogy directly, we hope to remove some of the 'A' from AI.

processing.Inparticular,weanalyzedhowthepropertiesofneuronslead to real-time control and adaptation to novel environments.2.1 Living Neurons Control a Simulated AnimalThefirstNeurally-ControlledAnimat[16]comprisedasystemfordetectingspatio-temporalpatternsofneuralactivity,whichdirectedexploratorymovementofasimulatedanimalinrealtime(Fig.4).Neuralfiringswereintegratedovertimetoproduceanactiv-ityvectorevery200ms,representingthecurrentactivitypattern,andrecurringpatternswereclusteredinactivityspace.Eachclusterwasassignedadirectionofmovement(left,right,forward,backward).Proprioceptiveandexteroceptivefeedbackviaelectricalstimulationwasprovidedtotheneuralcultureforeachmovementandforcollisionswithwallsandbarriers.Thestimulationinducedneuralactivitythat,inturn,wasdetectedthroughtheactivityvectorsandusedascommandsforsubsequentmovements.Wecre-atedthesoftwareandhardwarenecessarytoenablea15-mssensory-motorfeedbackla-tency,sincewefeelitisimportantthatatightconnectionbetweentheneuralsystemandits environment is likely to be crucial to adaptive control and learning.Withinthisreal-timefeedbackloop,bothspontaneousandstimulatedneuralactivitypatternswereobserved.Thesepatternsemergedoverthecourseoftheexperiment,some-timesassemblingintoarecurrentsequenceofpatternsoverseveralseconds,orthedevel-opmentofnewpatterns,asthesystemevolved.Theoveralleffectofthefeedbacklooponneuralactivitywasobservedfromthepathoftheanimat'smovementthroughoutitsenvi-ronment(Fig.4).Astheneuralnetworkmoveditsartificialbody,itreceivedfeedbackandinturnproducedmoremovement.Thebehavioraloutputwasadirectresultofbothspon-taneousactivitywithinthenetworkaswellasactivityproducedbyfeedbackduetothenetworksinteractionwithitsvirtualenvironment.Hencethepathoftheanimatwasin-dicativeofcurrentactivityaswellastheeffectsoffeedback.Analyzingthechangeinbe-havioroftheneurally-controlledanimatprovidedasimplebehavioraltooltostudyshiftsinthestatesofneuralactivity.However,thisfirstNeurally-ControlledAnimatdidnotEMBODIED ARTIFICIAL INTELLIGENCE(pre-print)Springer-Verlag6demonstratenoticeablegoal-directedbehavior,whichthenextexampleaddressesexplic-itly.Fig. 4.Animat setup and activity. Above: neural signals are used to control the movement of ananimat, whose 'brain' is exposed to microscopic imaging; feedback from the environment determinessubsequent electrical stimulation of the living neuronal network in an MEA. Below: One hour of theanimat's path (curved lines), as it moves about within its environment under neural control, withfeedback. The white boxes represent various environmental obstacles.Embodied Cultured NetworksBakkum et al.72.2 Living Neurons Control a Mobile RobotFig.5.

Livingneuronscontrolamobilerobot.Neuralfiringsinresponsetopairedelectricalstimu-lationsatvariousinter-stimulusintervals(ISI)areplotted.Intheexperiments,theISIwaspropor-tionaltothedistancebetweentheneurallycontrolledapproachinganimatanditstargetobject.Itwasconsideredpositiveifthetargetwaslocatedtotherightoftheanimatandnegativeifleftoftherobot.Theneuralresponsedeterminedthemagnitudeofsubsequentanimatmovement;thedirectionofmovementwasdeterminedfromwhichquadranttheISIfellinto(seethearrowsandmovementkey,bottom).Inset:theneurallycontrolledanimat'strajectory(Koalarobot,representedbythetri-angle).Thetargetobject(Kheperarobot,representedbythesquare)washeldstationaryuntilthero-bot approached, and then it was moved continuously (down and to the right in the figure).Oneofthesimplestformsof‘intelligent’behavioristhatofapproachandavoidance.ThegoalofthesecondsystemwastocreateaneuralinterfacebetweenneuronandrobotthatEMBODIED ARTIFICIAL INTELLIGENCE(pre-print)Springer-Verlag8wouldapproachatargetobjectbutnotcollidewithit,maintainingadesireddistancefromthetarget.Ifagivenneuralreactionisrepeatablewithlowvariance,thentheresponsemaybeusedtocontrolarobottohandleaspecifictask.Usingoneoftheseresponseprop-erties, we created a system that could achieve the goal [26].Networksstimulatedwithpairsofelectricalstimuliappliedatdifferentelectrodesre-liablyproduceanonlinearresponse,asafunctionofinter-stimulusinterval(ISI).Figure5showsaveragedfiringrateoverall60electrodesfollowingtwostimulationsseparatedbyatimeinterval.AtshortISI's,theresponseofthenetworkfollowingstimulationwasen-hanced;atlongerintervals,theresponsewasdepressed.Furthermore,thevarianceofthedataforeachISIwasrelativelysmall,indicatingtheeffectisrobustandthusqualifiesasagood candidate for an input/output mapping to perform computation.BymappingtheneuralresponsetoagivenISIasatransformationofdistancetoanobject,wecreatedarobotthatreactstoenvironmentalstimuli(inthiscasesensoryinfor-mationaboutdistancefromanobject)byapproachingandavoidingthattarget.Tocon-structour"approachandfollow"hybrot,sensoryinformation(thelocationofareferenceobjectwithrespecttotherobot)wasencodedinanISIstimulationasfollows:theclosertherobotistotheobject,thesmallertheISI.Theresponseoftheneuronstoastimulationpair,measuredasanaveragedfiringrateacrossallelectrodesfor100msafterthesecondstimulus,wasusedtocontroltherobot’smovements:alargerneuralresponsecorre-sponded to a longer movement (either forward or backward) of the robot.Whentherobotwasfarawayfromthereferenceobject,theISIofthestimulationpairwaslong,andtheneuralresponsewaslarge,movingtherobottowardstheobject(Fig.5,right).Astherobotmovedclosertotheobject,thestimulationintervaldecreaseduntilitreached150ms.Atthispoint,theneuralresponsewasminimal,andnomovementwascommanded.Inotherwords,therobotreacheditsdesiredlocationwithrespecttotheref-erenceobject.Iftherobotwasclosertotheobject,theneuralreactionwaslarger(averyshortISI),thistimedrivingtherobotawayfromtheobject.WedividedtheinputISIinto4quadrants(Fig.5,left).Eachofthe4quadrantscorrespondedtoadirectionalmove-ment:forward/right,forward/left,backward/right,andbackward/left.Then,apositiveISIcausedmovementinadirectionoppositethatforanegativeISI.GiventheneuralresponsetoanISIstimulation,wedecodedwhichquadranttheresponsebelongedtowithgoodac-curacy (>95%).WeusedtheKoalaandKheperarobots(manufacturedbyK-Team)toembodythecul-turednetwork,andtoprovideanenvironmentwithamovingobject.TheKoalarobotwasusedastheneurallycontrolledrobot,whiletheKheperaservedasthereferenceobject,movingatrandomundercomputercontrol.Underneuralcontrol,theKoalasuccessfullyapproachedtheKheperaandmaintainedadistancefromit,movingforwardiftheKheperamoved away, or backing up if the Khepera approachedInadditiontodemonstratingthecomputationalcapacityinherentinculturedneurons,thishybrotcanbeusedtostudylearninginculturedneuralnetworks.Inthiscase,learn-ingwouldbemanifestedthroughchangesintheneuralactivityandchangesatthebehav-ioralleveloftherobot.Preliminarystudiesindicatethatquantifiablebehavioraltraits,Embodied Cultured NetworksBakkum et al.9suchasthespeedwithwhichthehybrotapproachestheobject,maybemanipulatedthrough mechanisms of neural plasticity.2.3 Living Neurons Control a Drawing Arm

Fig.6.

Meart–TheSemi-LivingArtist.Left:Meart’sarmsusedmarkerstodrawonapieceofpa-per,underliveneuralcontrol.InthebackgroundwasaprojectionoftheMEAandculturednet,Meart's 'brain'. Right: one drawing created by Meart in an exhibition.Meart(Multi-ElectrodeArrayart)wasahybrotbornfromcollaborationwiththeSymbi-oticAResearchGroup2.The'brain'ofdissociatedratneuronsinculturewasgrownonanMEAinourlabinAtlantawhilethegeographicallydetached'body'residedinPerth.Thebodyconsistedofpneumaticallyactuatedroboticarmsmovingpensonapieceofpaper(Fig.6).Acameralocatedabovetheworkspacecapturedtheprogressofdrawingscreatedbytheneurally-controlledmovementofthearms.Thevisualdatatheninstructedstimula-tionfrequenciesforthe60electrodesontheMEA.Thebrainandbodyinteractedthroughtheinternet(TCP/IP)inrealtimeprovidingclosedloopcommunicationforaneurallycontrolled'semi-livingartist'.Weseethisasamediumfromwhichtoaddressvarioussci-entific, philosophical, and artistic questions.Mearthasbroughtneurobiologyresearchtotwoartisticevents:BiennaleofElectronicArtsPerth

SymbioticA:theArtandScienceCollaborativeResearchLaboratory(http://www.fishandchips.uwa.edu.au/),basedintheSchoolofAnatomyandHumanBiologyattheUniversityofWestern Australia in Perth.EMBODIED ARTIFICIAL INTELLIGENCE(pre-print)Springer-Verlag10ceivedsignalsfromAtlanta.AnoverviewofhowMeartworkedmaybestbedescribedbytheartisticconceptionbehindtheArtbotspresentation:portraitdrawing.First,ablankpieceofpaperwasplacedbeneaththearm'send-effectorandadigitalphotographwastakenofanaudiencemember.Then,communicationbetweenthearmandtheneuronswasbegun.TheneuralstimulationviatheMEAwasdeterminedbyacomparisonoftheactualdrawing,foundusingavideocameratakingimagesofthedrawingpaper,tothetargetimageofaperson'sphotograph.Boththeactualimageandthetargetimagewerereducedto60pixels,correspondingtotheMEAelectrodes,andthegrayscaleintensityofeachpixelwasfound.Similartohowanartistcontinuallycomparesherworktohersub-ject,thegrayscalepercentagesforcorrespondingpixelsonthetwoimageswerecontinu-ouslycompared,inthiscasesubtractedtoproduceamatrixoferrorvalues.The60errorvaluesdeterminedinreal-timethestimulationfrequencyperelectrodeusingacustomstimulationcircuitbuiltbyThomasDeMarse.Armmovementwasdeterminedbythere-cordedneuralactivity,usingaveragedfiringratesoftheinducedandspontaneousactivityperstimulation.Stimulationaffectedthisneuralactivity,andsothecommunicationformed a loop, with a loop time of approximately one second.Inthepriorexample,thesensory-motormappingsusedastableneuralpropertytore-liablycontroltherobot.WithMeart,thesensory-motormappingsarelesswelldefined,inthehopeofdemonstratingamicro-scaleversionofthebrain'screativeprocesses.Thebe-havioralresponseoftherobotshedslightonthepropertiesoftheneuralnetworkanddi-rectsfurtherencodingrefinements.Thus,Meartisa'workinprogress'withthesensory-motorencodingcontinuouslybeingimprovedtodemonstratelearningprocesses.Anex-ampledrawingisshowninFigure6.Thedrawingschangedthroughoutthelifeofcultures(andweredifferentfordifferentcultures)demonstratingneuralplasticity,however,themechanisms are still under investigation.3 Discussion3.1 Embodying Cultures: TheoryABlankSlate.

Thebiologicalbrainmakesassociationsbetweendifferentphenomenaob-servedthroughsensation,whetherbetweenvariousexternalstimuliorbetweentheactionsofabodyandtheirconsequences,andthencommandsmovementaccordingly.Ourmeth-odshavebeendevelopedtostudytheseprocessesinrealtimewithenoughresolutiontocapturethedynamicsoftheseinteractions.Theseprocessescanbeexpressedusingdy-namicalsystemstheory(DST),amathematicalframeworktodescribesystemsthatchangeintime.Forexample,theformationofcertainfunctionalstructures(oculardominancecolumns)inthevisualcortexhasbeendescribedusingAlanTuring'sreaction-diffusionequations[34].KuniyoshiandhisgroupexploreDSTtoconnectsensory-motorcontroltothecognitivelevel[35].Asappliedtocognition[34],DSTdescribesthemindwithasetofcomplex,recursivefilters.Thisopposestheclassicalcognitiveconceptofneuralproc-essingbeinganalogoustoadigitalcomputer,containingdistinctstorageandprocessingofsymbols[36],[37].DSTcontendsthatmultiplefeedbackloopsandtransmissiondelays,bothofwhicharewidespreadinthebrain,provideatimedimensiontoallowhigher-levelcognitiontoemergewithouttheneedforsymbolicprocessing[38].DSTisaframeworkcompatiblewithembodiedperspectives.Thedynamicalsystemsperspectivehastoooftenbeen neglected in neurobiology and cognitive sciences.Incontrasttoanintactbraininananimal,culturesofneuronsareisolatedbecausetheydonotcontaintheafferentsensoryinputsorefferentmotoroutputsabodywouldprovideandthereforenolongerhaveaworldwithwhichtoreferencetheiractivity.Undertheseconditions,whatassociationscanthenetworkmake,andwhatwouldthoseassociationsmean?Moreover,whatsymbolsareoperatedon?Becauseofthis,anyassociationsthataremademustconsequentlybeself-referentialorcircularandneuralactivitymaybemisleading.Thenetworkasasetofcomplex,recursivefiltershasnoexternalsignalstofilter,possiblyleadingtotheabnormalbarrageactivitydescribedbelow.Toaddressthismajorshortcomingofinvitrosystems,ourneuralculturesareembodiedwithsensoryfeedbacksystems,motorsystems,andsituatedinanenvironment,providinganewframeofreference.Newfindingsaboutthedynamicsoflivingneuralnetworksmightbeusedtodesign more biological, less artificial AI.IntelligenceandMeaning.

Byembodyingculturedneurons,the‘meaning’ofneuralac-tivityemerges,sincethisactivityaffectssubsequentstimulation.Nowthenetworkhasabodybehavingandproducingexperiences,allowingforthestudyofconceptssuchasin-EMBODIED ARTIFICIAL INTELLIGENCE(pre-print)Springer-Verlag12telligence.Wewilltakeabehavioraldefinitionofintelligenceasourstart:RodneyBrooksdescribesintelligenceintermsofhowsuccessfullyanagentinteractswithitsworldtoachievegoaldirectedbehavior[39].WilliamJamesstates,"Intelligentbeingsfindawaytoreachtheirgoal,evenifcircuitous,"[40].Neuronshaveinherentlocalgoals(totransmitsignals,integratesynapticinput,optimizesynapticstrengths,andmuchmore)thatprovidethefoundationtointelligentlyachievemeaningfulbehavioralgoals.Nodoubtthebasisforintelligenceisinherentatbirth,butaninteractionwithasufficientlycomplexenvironment (learning) is needed to develop it.Inourculturednetworks,thelocalgoalsofneuralinteractionaresubjecttodetailedopticalandelectricalobservation,whiletheexecutionofhigher-levelbehavioralgoalsareobservedthroughtheactivitiesoftheroboticbody.(Notethatthebehavioralgoalsarear-tificiallyconstrainedbythestimulationandrecordingtransformationschosen.)Wehopethiscombinationwillleadtoaclearerdefinitionandabetterunderstandingoftheneuro-logicalbasisofintelligence,inadditiontoexplanationsofotherpsychologicalterms:learning,memory,creativity,etc.NeurobiologyhasgiveninspirationtoAIsincethead-ventoftheperceptronandconsequentartificialneuralnetworks,whicharebasedonthelocalproperties(goals)ofindividualneurons.Wewishtocontinuethistrendbyfindingthe principles of network processing by multiple neurons that lead to higher-level goals.Network-wideBursting.

Theactivityofculturedneuronstendstowardstheformationofdish-wideglobalbursts(barrages)[8]:sweepsoffast,multipleneuralfiringsthroughoutthenetworklastingbetweenhundredsofmillisecondstosecondsinduration.Thesebar-rageshavebeenobservedofteninculturedneurons[41]butalsoincorticalslices[42]andincomputermodels[43].Barragesofactivityarereportedinthecortexinvivoduringearlydevelopment,duringepilepticseizures,whileasleep,andwhenunderanesthesia.Theseinvivoexamplesofbarragesoccuroverfiniteperiodsoftime.Incontrast,barragesinvitroarecontinuousoverthelifeoftheculture.Weconsiderthepossibilitythatatsomestage,dish-widebarragesofspikingactivityareabnormal,aconsequenceof'sensorydep-rivation' (manuscript in preparation), or a sign of arrested development [44].Forbothamodelsystem[43]andforculturedmousespinalneurons[45],ifmorethan30%oftheneuronsareendogenouslyactive,theneuronsfireatalowsteadyrateof1to5Hzperneuron,whileareductioninthefractionofendogenouslyactivecellsleadstobar-rageactivity.Endogenousactivityisfunctionallysimilartoactivityinducedbyafferentinput,suggestingembodimentwouldleadtolowsteadyfiringrates.Thehypothesisisthenthatthebarrageactivitymaybeduetothelackofanexternalenvironmentwithwhichtointeract.Wearedevelopinganimatmappingsinwhichcontinuoussensoryinputquietsbarrages,bringingthenetworkstoaless'sensory-deprived'statethatallowsmorecomplex, localized activity patterns.Embodied Cultured NetworksBakkum et al.133.2 The Importance of EmbodimentTheWorldandtheBrain.

Environmentaldeprivationleadstoabnormalbrainstructureandfunction,andenvironmentalexposureshapesneuraldevelopment.Similarly,pat-ternedstimulationsuppliedtoculturedneuronsmayleadtomorerobustnetworkstructureandfunctioningthanwithtrivialornostimulation.Themostdramaticexamplesoftheimportanceofembodimentcomefromstudiesduringdevelopment,whenthebrainismostmalleable.CognitivetestswereperformedoninstitutionalizedchildreninRomania,childrentypicallydeprivedofproperenvironmentalandsocialinteractionearlyinlife[46],[47].Comparedtopeers,thechildrenshowedseveredevelopmentalimpairmentthatimproved,however,aftertransplantationtoastablefamily.Thoseadoptedpriorto6monthsofageachievednearlycompletecognitivecatch-uptosimilarlyagedchildren,whilethoseadoptedafter6monthsofagehadsignificantbutincompletecatch-up.Like-wise,laboratoryratsraisedinenvironmentswithmazesandvariedvisualstimulihad30%greatercorticalsynapticdensitythanthoseraisedinminimalistenvironments,andper-formedbetterinvariouscognitiveexperiments[48],[49].Synapticmorphologyinadults[1]andadultneurogenesisisdependentonexternalcues[50]demonstratingthatenvi-ronmental interaction is important throughout life.Adisembodiedneuralculture,whoseactivityneverinfluencesfuturestimulation,willnotdevelopmeaningfulassociationstoaninput.Inthebrain,ifasensationisnotusefulininfluencingfuturebehavior(noassociationismadebetweenthetwo)theperceptofthesensationfades.Theenvironmenttriggersanenormousnumberofsensorysignals,andthebraindevelopstofilterouttheexcesswhileperceivingthebehaviorallyrelevant.Allone-month-oldinfantscandistinguishbetweentheEnglishLandRsounds.Fivemonthslater,JapaneseinfantslosetheabilitywhileAmericaninfantsmaintainit,becausethedistinc-tionisnotneededtounderstandtheJapaneselanguage[51].Japaneseadultsconsequentlyhavegreatdifficultydistinguishingthesesounds,butperceptionofthedistinctioncanbelearnedthroughtargetedinstruction.Thesestudiesfurtherdemonstratehowbrain(re)wiringdependsonenvironmentalcontextandoccursthroughoutlife:thebrainfocusesonperceivingtheportionsoftheenvironmentrelevanttoproduceameaningfulinterac-tion.TheBodyandtheBrain.

andhaveraisedanumberofis-suesaboutthevalidityoftraditional(disembodied)invitroneuralresearch.WehopethatotherswillmakeuseofthetoolswehavedevelopedsuchasourMeaBenchsoftware,4sealed-dishculturesystem[5],andmulti-sitestimulationtools[57],topursueawidevari-etyofquestionsabouthowneuralsystemsfunction.Weexpectthattheseinquirieswilllead to fundamentally different, more capable, and less artificial forms of AI.AcknowledgmentsWethanktheNIH(NINDS,NIBIB),theWhitakerFoundation,theNSFCenterforBehavioralNeuroscience,andArtsWesternAustraliaforfunding.WethankDanielWagenaar,RadhikaMadhavan,JohnBrumfield,ZenasChao,EnoEkong,GustavoPrado,BryanWilliams,PeterPas-saro, and Ian Sweetman for their many contributions to this work.

3

Manson,N(2004)"Brains,vats,andneurally-controlledanimats,"inStudiesintheHistoryandPhilosophy of Biology and the Biomedical Sciences, special issue on "The Brain in a Vat."4