The successful deployment of a multi-robot system (MRS) requires an effective methodof coordination to mediate the interactions among the robots and between the robots andthe task environment in order for a given system-level task to be performed. The designof coordination mechanisms has received increasing attention in recent years and hasincluded investigations into a wide variety of coordination mechanisms. A popular andsuccessful framework for the control of robots in coordinated MRS isbehavior-basedcontrol(1,2). Behavior-based control is a methodology in which robots are controlledthrough the principled integration of a set of interacting behaviors (e.g., wall following,collision avoidance, landmark recognition, etc.) in order to achieve desired system-levelbehavior. This chapter will describe, through explanation, discussion of demonstratedsimulated and physical mobile robots, and formal design and analysis, the range andcapabilities of behavior-based control applied to multi-robot coordination.

We begin by providing a brief overview of single-robot control philosophies andarchitectures, including behavior-based control, in Section 1. In Section 2 we move fromsingle robots to multi-robot systems (MRS) and discuss the additional challenges thistransition entails. In Section 3 we use empirical case studies to discuss and demonstratethree important ways in which robots can interact, and thus coordinate, their behavior. InSection 4 we discuss formal approaches to the design and analysis of MRS that are offundamental importance if the full potential of MRS is to beachieved. Finally, in Section5 we briefly discuss the future of coordinated behavior-based MRS and conclude thechapter.

1. Overview of Robot Control Architectures

In this section, we briefly discuss the most popular approaches and techniques for the

control of a single robot. In the next section we proceed with the fundamental principleof this chapter, the control of multiple robots, and how it is related to, and different from,the control of a single robot.

1.1 Single Robot Control

We define

robot controlas the process of mapping a robot’s sensory information intoactions in the real world. We do not consider entities that make no use of sensoryinformation in control decisions as robots, nor do we consider entities that do not performactions as robots, because neither category is truly interacting in the real world. Anyrobot must, in one manner or another, use incoming sensory information to makedecisions about what actions to execute. There are a number of control philosophiesdictatinghow this mapping from sensory information to actions should occur, each withits advantages and disadvantages. A continuum of approaches to robot control can bedescribed as a spectrum spanning from deliberative to reactive control.

Thedeliberativeapproach to robot control is usually computationally intensive due to theuse of explicit reasoning or planning using symbolic representations and world models(3). For the reasoning processes to be effective, complete and accurate models of theworld are required. In domains where such models are difficult to obtain, such as indynamic and fast-changing environments or situations with significant uncertainty in therobot’s sensing and action, it may be impossible for the robot to act in an appropriate ortimely manner using deliberative control (3,4).

In contrast to deliberative control, thereactiveapproach to robot control is characterizedby a tight coupling of sensing to action, typically involving no intervening reasoning(5,6). Reactive control doesnot require the acquisition or maintenance of world models,as it does not rely on the types of complex reasoning processes utilized in deliberativecontrol. Rather, simple rule-based methods involving a minimal amount of computation,internal representations, or knowledge of the world are typically used. This makesreactive control especially well suited to dynamic and unstructured worlds where havingaccess to a world model is not a realistic option. Furthermore, the minimal amount ofcomputation involvedmeans reactive systems are able to respond in a timely manner torapidly changing dynamics.

A middle ground between deliberative and reactive philosophies is found inhybridcontrol, exemplified by three-layered architectures (7,8). In this approach, asinglecontroller includes both reactive and deliberative components. The reactive part of thecontroller handles low-level control issues requiring fast response time, such as localobstacle avoidance. The deliberative part of the controller handles high-level issues on alonger time-scale, such as global path planning. A necessary third component of hybridcontrollers is a middle layer that interfaces the reactive and deliberative components.Three-layered architectures aim to harness the best of reactivecontrollers in the form ofdynamic and time-responsive control and the best of deliberative controllers in the formof globally efficient actions over a long time-scale. However, there are complex issuesinvolved in interfacing these fundamentally differing components and the manner inwhich their functionality should be partitioned is not yet well understood.

Behavior-basedcontrol, described in detail in the next section, offers an alternative tohybrid control. It can also include both deliberativeand reactive components, but unlikehybrid control, it is composed of a set of independent modular components that areexecuted in parallel (1,2).

The presented spectrum of control approaches is continuous and a precise classificationof a specific controller on the continuum may be difficult. The distinction betweendeliberative and reactive control, and hybrid and behavior-based control is often a matterof degree, based on the amount of computation performed and the response time of thesystem to relevant changes in the world. In a specific domain, the choice of controller isdependent on many factors, including how responsive the robot must be to changes in theworld, how accessible a world model is, and what level of efficiency or optimality isrequired.

1.2 Behavior-Based Control

The control methodology we focus on in this chapter is behavior-based (BB) control. TheBB approach to robot control must not be classified as strictly deliberative or reactive, asit can, and in many cases is, both. However, BB control is most closely identified (oftenincorrectly so) with the reactive side of the control spectrum, because primary importanceis placed on maintaining a tight, real-time coupling between sensing and action (7,8).

Fundamentally, a behavior-based controller is composed of a set of modular components,calledbehaviors, which are executed in parallel. Abehavioris a control law that clustersa set of constraints in order to achieve and maintain a goal (1,2). Each behavior receivesinputs fromsensors and/or other behaviors and provides outputs to the robot’s actuatorsor to other behaviors. For example, an obstacle avoidance behavior might send acommand to the robot’s wheels to turn left or right if the robot sensors detect the robot ismoving directly toward an obstacle. There is no centralized world representation or statein a BB system. Instead, individual behaviors and networks of behaviors maintain anymodels or state information.

Many different behaviors may independently receive input from the same sensors andoutput action commands to the same actuators. The issue of choosing a particular actiongiven inputs from potentially multiple sensors and behaviors is calledaction selection

(12). One of well-known mechanism for action selection is the use of a predefinedbehavior hierarchy, as in the Subsumption Architecture (10), in which commands fromthe highest-ranking active behavior are sent to the actuator and all others are ignored.(Note, however, that the Subsumption Architecture hasmost commonly been used in thecontext of reactive and not BB systems.) Numerous principled as well as ad hoc methodsfor addressing the action selection problem have been developed and demonstrated onrobotic systems. These include varieties of command fusion (13) and spreading ofactivation (14), among many others. For a comprehensive survey on action selectionmechanisms, see (15).

BB systems are varied, but there are two fundamental tenets all BB systems inherentlyadhere to: 1) the robot is embodied and 2) the robot is situated. A robot isembodiedinthe sense that it has a physical body and its behavior is limited by physical realities,uncertainties, and consequences of its actions, all of which may be hard to predict orsimulate. A robot issituatedin the sense that it is immersed in the real world and actsdirectly on the sensory information received from that world, not on abstract or processedrepresentations of the world.

BB control makes no assumptions on the availability of a complete world model;therefore, it is uncommon for a BB controller to perform extensive computation orreasoning relying on such a model. Instead, BB controllers maintain a tight coupling ofsensing and action, allowing them to act in a timely manner in response todynamic andfast-changing worlds. However, BB systems have also demonstrated elegant use ofdistributed representations enabling robot mapping and task learning (16,17,18).

This section has discussed approaches and philosophies to the control of a single robot,with a focus on the BB approach. In the next section, the scope is expanded to considerthe control of a coordinated group of multiple robots.

2. From Single Robot Control to Multi-Robot Control

In this section we discuss the advantages and additional issues involved in the control ofmulti-robot systems (MRS) as compared to the single-robot systems (SRS) discussed inthe previous section. AnMRSis a system composed of multiple, interacting robots.Thestudy of MRS has received increased attention in recent years. This is not surprising, ascontinually improving robustness, availability, and cost-effectiveness of roboticstechnology has made the deployment of MRS consisting of increasingly larger numbersof robots possible. With the growing interest in MRS comes the expectation that, at leastin some important respects, multiple robots will be superior to a single robot in achievinga given task. In this section we outline the benefits of a MRS over a SRS and introduceissues involved in MRS control and how they are similar and different to those of SRScontrol.

This chapter is focused on distributed MRS in which each robot operates independentlyunder local sensing and control.Distributed MRSstand in contrast tocentralized MRS, inwhich each robot’s actions are not completely determined locally, as they may bedetermined by an outside entity, such as another robot or by any type of externalcommand. In distributed MRS, each robot must make its own control decisions basedonly on limited, local, and noisy sensor information. We limit our consideration in thischapter to distributed MRS because they are the most appropriate for study with regard tosystems that are scalable and capable of performing in uncertain and unstructured real-world environments where uncertainties are inherent in the sensing and action of eachrobot. Furthermore, this chapter is centered on achieving system-level coordination in adistributed BB MRS. Strictly speaking, the issues in a centralized MRS are more akin toa scheduling or optimal assignment and less of a problem of coordination in a distributedsystem.

2.1 Advantages and Challenges of Multi-Robot Systems

Potential advantages of MRS over SRS include a reduction in total system cost byutilizing multiple simple and cheap robots as opposed to a single complex and expensiverobot. Also, multiple robots can increase system flexibility and robustness by takingadvantage of inherent parallelism and redundancy. Furthermore, the inherent complexityof some task environments may require the use of multiple robots, as the necessarycapabilities or resource requirements are too substantial to be met by a single robot.

However, the utilization of MRS poses potential disadvantages and additional challengesthat must be addressed if MRS are to present a viable and effective alternative to SRS. Apoorly designed MRS, with individual robots working toward opposing goals, can be lesseffective than a carefully designed SRS. A paramount challenge in the design of effectiveMRSis managing the complexity introduced by multiple, interacting robots. As such, inmost cases just taking a suitable SRS solution and scaling it up to multiple robots is notadequate.

2.2 Necessity of Coordination in Multi-Robot Systems

In order to maximize the effectiveness of a MRS, the robots' actions must be spatio-temporally coordinated and directed towards the achievement of a given system-leveltask or goal. Just having robots interact is not sufficient in itself to produce interesting orpractical system-level coordinated behavior. The design of MRS can be quite challengingbecause unexpected system-level behaviors may emerge due to unanticipatedramifications of the robots' local interactions. In order for the interacting robots toproduce coherent task-directed behavior, there must be some overarching coordinationmechanism that spatio-temporally organizes the interactions in a manner appropriate forthe task.

The design of such coordination mechanisms can be difficult; nonetheless, many eleganthandcrafted distributed MRS have been demonstrated, both in simulation and on physicalrobots (19,20,21). The methods by which these systems have achieved task-directedcoordination are diverse and the possibilities are seemingly limited only by the ingenuityof the designer. From a few robots performing a manipulation task (22,23), to tens ofrobots exploring a large indoor area (24,25), to potentially thousands of ecosystemmonitoring nanorobots (26,27), as the number of robots in the system increases, so doesthe necessity and importance of coordination. The next section examines mechanisms bywhich system-level coordination can be successfully achieved in a MRS.

3. From Local Interactions to Global Coordination

Given the importance of coordination in a MRS, we now address the issue of how toorganize the robots’ local interactions in a coherent manner in order to achieve system-level coordination. There are many mechanisms by which the interactions can beorganized. We classify them into three broad and often overlapping classes: interactionthrough the environment, interaction through sensing, and interaction throughcommunication. These classes are not mutually exclusive because MRS can, and oftendo, simultaneously utilize mechanisms from any or all of these classes to achieve system-level coordinated behavior.

In the following sections we describe each of these interaction classes in detail. Throughthe discussion of empirical case studies we demonstrate how each type of interaction canbe used to achieve system-level coordination in a MRS.

3.1 Interaction Through the Environment

The first mechanism for interaction is through the robots’ shared environment. This formof interaction isindirectin that it consists of no explicit communication or physicalinteraction between robots. Instead, the environment itself is used as a medium of indirectcommunication. This is a powerful approach that can be utilized by very simple robotswith no capability for complex reasoning or direct communication.

An example of interaction through the environment is demonstrated instigmergy, a formof interaction employed by a variety of insect societies. Originally introduced in thebiological sciences to explain some aspects of social insect nest-building behavior,stigmergy is defined as the process by which the coordination of tasks and the regulationof construction do not depend directly on the workers, but on the constructionsthemselves (28). This concept was first used to describe the nest-building behavior oftermites and ants (29). It was shown that coordination of building activity in a termitecolony was not inherent in the termites themselves. Instead, the coordination mechanismswere found to be regulated by the task environment, in that case the growing neststructure. A location on the growing nest stimulates a termite’s building behavior, therebytransforming the local nest structure, which in turn stimulates additional buildingbehavior of the same or another termite.

Through the careful design of robot sensing, actuation, and control features, it is possibleto utilize the concept of stigmergy in task-directed MRS. This powerful mechanism ofcoordination is attractive as it typically requires minimal capabilities of the individualrobots. Therobots do not require direct communication, unique recognition of otherrobots or even distinguishing other robots from miscellaneous objects in the environment,or the performance of computationally intensive reasoning or planning.

Stigmergy, and more generally interaction through the environment, has beensuccessfully demonstrated as a mechanism to coordinate robot actions in a number ofMRS. It has been demonstrated in an object manipulation domain (30) in which a largebox was transported to a goallocation through the coordinated pushing actions of a groupof robots. There was no globally agreed upon plan as to how or over what trajectory thebox should be moved; however, each robot could indirectly sense the pushing actions ofother robots throughthe motions of the box itself. Through simple rules, each robotdecided whether to push the box or move to another location based on the motions of thebox itself. As a large enough number of robots pushed in compatible directions, the boxmoved, which inturn encouraged other robots to push in the same direction.

Other examples of the use of stigmergy in MRS include distributed construction in whicha given structure was built in a specified construction sequence (31). The individualrobots were not capable of explicit communication and executed simple rule-basedcontrollers in which local sensory information was directly mapped to constructionactions. The construction actions of one robot altered the environment, and therefore thesubsequent sensoryinformation available for it and all other robots. This new sensoryinformation then activated future construction actions. In the following subsection wediscuss in detail how the concept of stigmergy was utilized in a MRS object clusteringtask domain(28).

3.2 Interaction Through the Environment Case Study: Object Clustering

We now describe an empirical case study in an object clustering task domain for whichinteraction through the environment was used to achieve system-level coordination. Theclustering task domain requires a group of objects, originally uniformly positioned in anenclosed environment, to be re-positioned by a group of robots into a single dense clusterof objects. There is noa prioritarget location for the cluster in the environment. Rather,the position of the cluster is to determined dynamically at the time of task execution.

The particular approach to the object clustering task we describe here is from workpresented in (28). There, the robots performing the task were extremely simple, capableonly of picking up and transporting and dropping a single object at a time. The robots hadvery limited local sensing and no explicitly communication, memory of past actions, orrecognition of other robots. Even with these highly limited capabilities, a homogeneousMRS composed of such robots was shown to be capable of successfully and robustolyperforming the object clustering task.

The robots in this task domain were able to coherently achieve system-level coordinationin the formation of a single cluster. The mechanism by which they achieved coordinationwas an example of interaction through the environment. The robots communicatedthrough their individual placement of objects over time, thus modifying the taskenvironment, and thereby indirectly influencing the future object-placement behaviors ofother robots and themselves. The location of the final cluster was not determined throughexplicit communication, negotiation, or planning on the part of the robots. Rather, it wasdetermined through a symmetry break in the initially uniform distribution of objects.Once a small cluster began to form, it was likely to grow larger. During the early stagesof task execution, several clusters were likely to be formed. However, over time, a singlelarge cluster resulted.

The robots in this work were designed in a manner that carefully exploited the physicaldynamics of interaction between the robots and their environment. Their hardware andrules were tuned so as to be probabilistically morelikely to pick up an object that is notphysically proximate to other objects (thus conserving clusters), to not drop objects nearboundaries (thus avoiding hard-to-find objects), and to be probabilistically more likely todeposit an object near other objects (thereby building up clusters). Together, thereproperties resulted in a form of positive feedback in which the larger a cluster of objectsbecame, the more likely it was to grow even larger.

Similar approaches employing stigmergy were also demonstrated in the physicalsegregation and sorting of a collection of object classes. Additional studies with physicalrobots have been conducted and, by making various changes in the robots and the taskenvironment, is has been demonstrated that one can influence the location of the finalcluster by initializing the initial distribution of objects in a non-uniform manner (28).

Given this specific example of system-level coordination achieved through the use ofinteraction through the environment, in the following subsection we move on to the nextmethod of organizing the robot’s interactions: interaction through sensing.

3.3 Interaction Through Sensing

The second mechanism for interaction among robots is through sensing. As described in(19), interaction through sensing ‘refers to local interactions that occur between robots asa result of sensing one another, but without explicit communication.’ As with interactionthrough the environment, interaction through sensing is alsoindirectas there is noexplicitcommunication between robots; however, it requires each robot to be able todistinguish other robots from miscellaneous objects in the environment. In someinstances, each robot may be required to uniquely identify all other robots, or classes ofother robots. In other instances, it may only be necessary to simply distinguish robotsfrom other objects in the environment.

Interaction through sensing can be used by a robot to model the behavior of other robotsor to determine what another robot is doing in order to make decisions and respondappropriately. For example, flocking birds use sensing to monitor the actions of otherbirds in their vicinity to make local corrections to their own motion. It has been shownthat effective flocking results from quite simple local rules followed by each birdresponding to the direction and speed of the local neighbors (32).

In the follow subsection we describe a case study in a formation marching domain inwhich interaction through sensing is used to achieve coordinatedgroup behavior. Otherdomains in which interaction through sensing has been utilized in MRS include flocking(33), in which each robot adjusts it motions according to the motions of locally observedrobots. Through this process, the robots can be made to move as a coherent flock throughan obstacle-laden and dynamic environment. Interaction through sensing has also beendemonstrated in an adaptive division of labor domain (34). In that domain, each robotdynamically changes the task it is executing based onthe observed actions of other robotsand the observed availability of tasks in the environment. Through this process, the groupof robots coherently divides the labor of the robots appropriately across a set of availabletasks.

3.4 Interaction Through Sensing Case Study: Formation Marching

In this section we describe an empirical case study of a formation marching task domainfor which interaction through sensing was used to achieve system-level coordination. Theformation marching task domain requiresa group of robots to achieve and maintainrelative positions to one another as the group moves through the environment n a globalformation. Each robot in the MRS operates under local sensing and control and is notaware of global information such as allother robot’s positions and headings. In someenvironments, the formation may need to be perturbed in order for the group to movethrough a constrained passage or around obstacles. In such cases, the formation needs tocorrectly re-align after the perturbation.

The approach to formation marching described here was presented in (35). The generalidea of the approach is that every robot in the MRS positioned itself relative to adesignated neighbor robot. This neighbor robot, in turn, positioned itself relative to itsown designated neighbor robot. As all robots are only concerned with their relativepositions with respect to their neighbor robot, no robot is aware of, or needs to be awareof, the global positions and headings of all robots in the formation.Each robot only needsto be capable of determining the distance and heading to its neighbor. The globalgeometry of the formation was then determined through the defined chain of neighbors.

A “leader” robot has no neighbors and independently determines the speed and headingof the entire formation. Therefore, as the leader robot moves forward, the robot(s) thathad the leader as their neighbor also move forward. This forward motion propagatesdown the chain of designated neighbors, causing the entire formation to move.

The formation could be dynamically changed by altering the structure of the localneighbor relationships. For example, if the desired formation is a line, each robot may bedesignated a neighbor robot to its left or right for which it desires to stay next to in orderto maintain the line formation. If a cue is given to all robots to change to a diamondformation, each robot may follow a new neighbor at a different relative position and theline formation would then be dynamically changed to adiamond.

In the following subsection we move to the next method of organizing the robot’sinteractions: interaction through communication.

3.5 Interaction Through Communication

The third mechanism for interaction among robots is through explicit communication.Unlike the first two forms of interaction, described above, which were indirect, ininteraction through communication robots may communicate with others directly. Suchrobot-directedcommunication can be used to request information or action from otherrobots or to respond to received requests.

Communication in physical robotics is not free or reliable and can be constrained bylimited bandwidth and range, and unpredictable interference. When utilizing it, one mustconsider how and toward whatend it is used. In some domains, such as the Internet,communication is reliable and of unlimited range; however, in physical robot systems,communication range and reliability are important factors in system design (2,36).

There are many types of communication. Communication could be direct from one robotto another, direct from one robot to a class of other robots, or broadcast from one robot toall others. Furthermore, the communication protocol can range from simple protocol-lessschemes to a complexnegotiation-based and communication-intensive schemes. Theinformation encoded in a communication may be state information contained by thecommunicating robot, a command to one or more other robots, or a request for additionalinformation from other robots, etc.

Communications may be task-related rather than robot-directed, in which case it is madeavailable to all (or a subset) of the robots in the MRS. A common task-relatedcommunication scheme ispublish/subscribe messaging. In publish/subscribe messaging,subscribing robots request to receive certain categories of messages, and publishingrobots supply messages to all appropriate subscribers.

In the next subsection, we describe a case study of the effective use of interaction throughcommunication.

The case study on interaction through communication in this section is focused on the useof explicit communication in a multi-target tracking task as discussed in (37). In multi-target tracking, the goal is to have a set of robots with limited sensing ranges position andorient themselves such they are able to acquire and track multiple objects moving throughtheir environment. The locations, trajectories, and number of targetsare not knownapriori. These difficulties are compounded in a distributed MRS, where the system mustdetermine which robot(s) should monitor which target(s). Robots redundantly trackingthe same target may be wasting resources and letting another targetremain untracked. Inthis domain, explicit communication between the robots has been shown to be capable ofeffectively achieving system-level coordination.

In the implementation described in (37), each robot had a limited sensing andcommunication range.Communication was used by each robot to transmit the positionand velocities of all targets within its sensing range to all other robots within itscommunication range. This simple communication scheme involved no handshaking ornegotiation.

Each robotwas constantly evaluating the importance of its current tracking activities andpossible changes in position that could increase the importance of its tracking activities.Communication was used to allow each robot to keep a local map of target movementswithin the robot’s communication range but outside its sensing range. As a result, thegroup as a whole effectively tracked a maximum number of targets with a minimumnumber of available robots.

This demonstration of the use of interaction through communication concludes thediscussion of MRS coordination mechanisms. As was mentioned above, any given MRSis likely to use any or all of the three mechanisms in varying degrees to achieve system-level coordination. Through an improved understanding of eachof these mechanisms ofcoordination, one is better positioned to design a MRS utilizing the most appropriatecombination of mechanisms for achieving a given task. In the next section we provide adiscussion on formal methods for the design and analysis of MRS that can provide aprincipled foundation upon which to base such design decisions.

4. Formal Design and Analysis of Multi-Robot Systems

The design of coordination mechanisms for multi-robot systems (MRS) has proven to bea difficult problem. In the last decade, the design of a variety of such mechanisms over awide range of task domains has been studied (19,20). Although the literature highlightssome elegant solutions, they are generally domain-specific and provide only indirectinsights into important questions such as how appropriate a given coordinationmechanism is for a particular domain, what performance characteristics one shouldexpect from it, how it is related to other coordination mechanisms, and how one canmodify it to improve systemperformance. These questions must be answered in aprincipled manner before one can quickly and efficiently produce an effective MRS for anew task domain. To fully utilize the power and potential of MRS and to move the designprocess closer to a science,principled design tools and methodologies. Such tools andmethodologies are needed for establishing a solid foundation upon which to constructincreasingly capable, robust, and efficient MRS.

The design of an effective task-directed MRS is often difficult because there is a lack ofunderstanding of the relationship between different design options and resulting taskperformance. In the common trial-and-error design process, the designer constructs aMRS and then tries it out either in simulation or on physical robots. Either way, theprocess is resource-intensive. Ideally, the designer should be equipped with an analyticaltool for the analysis of a potential MRS design. Such a tool would allow for efficientevaluation of different design options and thusresult in more effective and optimizedMRS designs.

The BB paradigm for multi-robot control is popular in MRS because it is robust to thedynamic interactions inherent in any MRS. Any MRS represents a highly non-linearsystem in which the actions of one robot are affected by the actions of all other robots.This makes any control approach that relies on complex reasoning or planning ineffectivebecause it is intractable to accurately predict future states of a non-trivial MRS. For thisreason, BB control is frequently used in MRS. The simplicity of the individual robotsalso confers the advantage of making the external analysis of predicted systemperformance on a given task feasible.

In the remainder of this section we discuss a variety of approaches tothe analysis andsynthesis of MRS.

4.1 Analysis of Multi-Robot Systems Using Macroscopic Models

Macroscopic models reason about the system-level MRS behavior without explicitconsideration of each individual robot in the system. As such, macroscopic models aregenerally more scalable and efficient in the calculation of system-level behaviors even asthe studied MRS consists of increasingly larger numbers of robots.

A macroscopic mathematical MRS model has been demonstrated in a foraging taskdomain (38). The model was used to study the effects of interference between robots, theresults of which could be used to modify individual robot control or determine theoptimal density of robots in order to maximize task performance. A macroscopicanalytical model has been applied to the study of the dynamics of collective behavior in acollaborative stick-pulling domain using a series of coupled differential equations (39).

A general macroscopic model for the study of adaptive multi-agent systems waspresentedin (40) and was applied to the analysis of a multi-robot adaptive task allocationdomain that was also addressed experimentally in (34). In this work, the robotsconstituting the MRS maintain a limited amount of persistent internal state to represent ashort history of past events but do not explicitly communicate with other robots.

4.2 Analysis of Multi-Robot Systems Using Microscopic Models

In contrast, microscopic modeling approaches directly consider each robot in the systemand may model individual robot interactions with other robots and with the taskenvironment in arbitrary detail, including simulating the exact behavior of each robot.However, most microscopic approaches model the behavior of each robot as a series ofstochastic events. Typically, the individual robot controllers are abstracted to somedegree and exact robot trajectories or interactions are not directly considered.

A microscopic probabilistic modeling methodology for the study of collective robotbehavior in a clustering task domain was presented in (41). The model was validatedthrough a largely quantitative agreement in the prediction of the evolution of cluster sizeswith embodied simulation experiments and with real-robot experiments. Theeffectiveness and accuracy of microscopic and macroscopic modeling techniquescompared to real robot experiments and embodied simulations was discussed in (42).Furthermore, a time-discrete, incremental methodology for modeling the dynamics ofcoordination in a distributed manipulation taskdomain was presented in (43).

4.3 Principled Synthesis of Multi-Robot System Controllers

One step beyond methodologies for the formal analysis of a given MRS design lie formalmethodologies for the synthesis of MRS controllers.Synthesisis the processofconstructing a MRS controller that meets design requirements such as achieving thedesired level of task performance while meeting constraints imposed by limited robotcapabilities. Being able to define a task domain and then have a formal method thatdesigns the MRS to accomplish the task while meeting the specified performance criteriais one of the long-term goals of the MRS community.

An important piece of work in the formal design of coordinated MRS was thedevelopment of information invariants, which aimed to define the informationrequirements of a given task and ways in which those requirements could be satisfied in arobot controller (44). Information invariants put the design of SRS and MRS on a formalfooting and began to identify how variousrobot sensors, actuators, and control strategiescould be used to satisfy task requirements. Furthermore, the work attempted to show howthese features were related and how one or more of these features could be formallydescribed in terms of a set of other features. The concept of information invariants wasexperimentally studied in a distributed manipulation task domain (45) and was extendedthrough the definition of equivalence classes among task definitions and robotcapabilities to assist in the choice of appropriate controller class in a given domain (46).

There has also been significant progress in the design of a formal design methodologybased on a MRS formalism that provides a principled framework for formally definingand reasoning about concepts relevant to MRS: the world, task definition, and capabilitiesof the robots themselves, including action selection, sensing, maintenance of local andpersistent internal state, and broadcast communication from one robot to all other robots(47). Based onthis formalism, the methodology utilizes an integrated set of MRSsynthesis and analysis methods. The methodology includes a suite of systematic MRSsynthesis methods, each of which takes as input the formal definitions of the world, task,and robotssans

controller and outputs a robot controller designed through a logic-induced procedure. Each of the synthesis methods is independent and produces acoordinated MRS through the use of a unique set of coordination mechanisms, includingthe use of internal state (48), inter-robot communication (34), and/or deterministic andprobabilistic action selection. Complimentary to the synthesis methods, this methodologyincorporates both macroscopic (47) and microscopic MRS modeling approaches.Together, the synthesis and analysis methods provide more than just pragmatic designtools. A defining feature of this design methodology is the integrated nature of thecontroller synthesis and analysis methods. The fact that they are integrated allows for thecapability to automatically and iteratively synthesize and analyze a large set of possibledesigns, thereby resulting in more optimal solutions and an improved understanding ofthe space of possible designs. This principled approach to MRS controller design hasbeen demonstrated in a sequentially constrained multi-robot construction task domain(34,47,48).

A theoretical framework for the design of control algorithms in a multi-robot objectclustering task domain has been developed (49). Issues addressed in this formalisminclude how to design control algorithms that result in a single final cluster, multipleclusters, and how to control the variance in cluster sizes.

Alternative approaches to the synthesis of MRS controllers can be found in evolutionarymethods (50) and learning methods (33,51). There also exist a number of MRS designenvironments, control architectures, and programming languages which assist in thedesign of coordinated MRS (52,53,54).

5. Conclusions and the Future of Multi-Robot Systems

Behavior-based(BB) control has been a popular paradigm of choice in the control ofmulti-robot systems (MRS). The BB control methodology represents a robust andeffective way to control individual as well as multiple robots. In a MRS, the taskenvironment is inherentlydynamic and non-linear as a result of the numerous types ofinteractions between the individual robots and between the robots and the taskenvironment. This makes complex control strategies relying on accurate world models toperform computationally complex reasoning or planning ineffective. BB control providesa tight coupling between sensing and action and does not rely on the acquisition of suchworld models. As such it is a very effective control methodology in the dynamic andunstructured environmentsin which MRS inherently operate.

BB MRS have been empirically demonstrated in a diverse array of task domains--fromforaging, to object clustering, to distributed manipulation, to construction. Each of thesetask domains requires some overarching mechanism by which to coordinate theinteractions of the individual robots such that the resulting system-level behavior isappropriate for the task. We have described and illustrated three different mechanisms toachieve this coordinated behavior: interactionthrough the environment, interactionthrough sensing, and interaction through communication. Each provides a coordinationscheme capable of organizing the individual robot’s behaviors toward system-level goals.

Another advantage of BB MRS is their amenability to formal analysis and synthesis. Dueto their rather straight-forward and direct coupling of sensing to action, formal methodsof synthesis and analysis become tractable and effective in producing and predicting thesystem-level behavior of a BB MRS.

The future possibilities and potentials of BB MRS are seemingly unlimited. Astechnology continues to improve and the nature and implications of different strategiesfor coordination are better understood, more task domains will become valid candidatesfor the application of MRS solutions.

Authors’ Biographies

Chris Jones is a Robotics Researcher at iRobot Corporation. He received his Ph.D. inComputer Science in May 2005 from the University of Southern California, where heworked on multi-robot coordination in the Interaction Lab. From 1999-2001, he was aMember of the Technical Staff in the Intelligent Systems and Robotics Center at SandiaNational Laboratories. He has worked in the areas of robotics motion planning, missionplanning, multi-robotcontrol and coordination, and formal methodologies for multi-robotsystem controller design.

MajaMatarićis an associate professor in theComputer Science Department andNeuroscience Program at theUniversity of Southern California, founding director ofUSC's interdisciplinary Center for Robotics and Embedded Systems (CRES) and co-director of the USC Robotics Research Lab. Prof. Mataric´ received her PhD inComputer Science and Artificial Intelligence from MIT in 1994, MS in ComputerScience from MIT in 1990, and BS in Computer Science from the University of Kansasin 1987. She is a recipient of theOkawa Foundation Award, the USC Viterbi School ofEngineering Service Award, the NSF Career Award, the MIT TR100 Innovation

Award,the IEEE Robotics and Automation Society Early Career Award, theUSC Viterbi Schoolof Engineering Junior Research Award, and the USC Provost's Center forInterdisciplinary Research Fellowship, and is featured in the Emmy Award-nominateddocumentary movie about scientists, "Me & Isaac Newton." She is an associate editor ofthree major journals: International Journal of Autonomous Agents and Multi-AgentSystems, International Journal of Humanoid Robotics, and Adaptive Behavior. She haspublished over 30 journal articles, 17 book chapters, 4 edited volumes, 94 conferencepapers, and 23 workshop papers, and has two books in the works with MIT Press. She isactive in educational outreach and is collaborating with K-12 teachers to develop hands-on robotics curricula for students at all levels as tools for promoting scienceandengineering topics and recruiting women and under-represented students. Her InteractionLab pursues research aimed at endowing robots with the ability to help people throughassistive interaction.