Chapter 6. Synthesis

Chapter 3 proposes and generally motivates an extensive set of design principles for synthetic ecosystems. Furthermore, it suggests, formalizes, and analyzes the pheromone infrastructure, an extension of runtime environments of software-agent systems that supports sign-based stigmergy. The Chapters 4 and 5 present the guided manufacturing control system, whose design follows the proposed principles and that makes use of the PI in its multi-agent coordination mechanisms.
The following chapter returns to all three major subjects. First, for each design principle the resulting characteristics of the GMC system are discussed systematically (Section 6.1). Then, the importance of the PI for the evaluation of emerging system-level behavior and modes of information sharing through the active agent environment are considered (Section 6.2). Finally, the operation of manufacturing control systems is discussed in abstract state space and conclusions for the GMC systems are drawn (Section 6.3).

6.1 Re-Visiting the Design Principles

The following discussion of the design principles further motivates their respective relevance for the design of large-scale self-organizing systems. The consequences and advantages for the GMC system and the applicability to manufacturing control in general are considered for each principle.

6.1.1 Single-Agent Principles

Things, not Functions (SE-Principle 1).-The reactive layer is the only part of the multi-agent system that has direct control of the physical environment. All other layers act only through the reactive layer. Thus, the advice to model agents according to real-world entities is only applicable to the design of the reactive layer. The other layers introduce agents on the basis of concepts like a strategy, a policy, or a user-focus. But it is never a function that is identified with an agent.

All agents in the reactive layer represent real-world entities. The resources in the manufacturing system are controlled by Loader-agents, Switch-agents, Processing-agents, and Unloader-agents. Workpieces are represented by Workpiece-agents. The composition of the agent system mirrors the current configuration of the manufacturing system.

One major advantage of the chosen way of modeling the reactive layer is the resulting simplicity of the change management. Whenever a resource in the manufacturing system is added or removed, the corresponding change is made in the agent system. Following the strict locality of the direct agent interactions (SE-Principle 9), such a change does not have to be told anyone else but the agents in the close neighborhood. Additionally, the automated information sharing mechanisms make sure that whenever information about resources are needed farther away, the change is automatically published.

What would have been the design of the reactive layer if the agents were modeled according to functions? Functions in the reactive layer are for instance transportation or

112

resource management. Assume there is an agent for transportation management. Such an agent would have to control the transport of every workpiece in the system simultaneously. While such an approach may be feasible in very small and simple transport systems, scaling up the system, for instance to the size of a real paint-shop, would require too much communication and too many decisions at the same time. No software system would be able to handle the load and hence, the transportation management agent would become a bottleneck in communication and computation.

Small Agents (SE-Principle 2).-Most agents in the GMC system are small in comparison to the whole system. A Ghost-agent is an example for small agents. The lifespan and the actual impact of the actions of one Ghost-agent are neglectable. They do not control anything and a single Ghost-agent by itself does not make anyone else do something. But the advisory system depends in its operation on the information they jointly generate.

In general, the agents are very simple to implement. Once an agent type is completely specified, it is just a matter of hours to implement it and test it for correctness according to the specification. Considering again the Ghost-agent as an example, agents of this type are programmed to handle two interactions consisting of one ask and one reply message altogether. Ghost-agents run just one process. They recognize two different pheromone types and they refresh a third type themselves. The decision process of a Ghost-agent is straightforward and easy to understand, even though it is one of the most complex processes in the system already. Other agent types run more than one process in parallel, but still they are all very simple software artifacts.

With the simplicity of implementation of the agent system, much more effort may be invested into design, tuning, and the thorough testing of the behavior of the whole system.

Diversity, Heterogeneity (SE-Principle 3).-A simple and intuitive way to design small agents is to diversify the agents in accordance to specific responsibilities. Each agent gets allotted only a small set of responsibilities. Agents of the same type share the same general responsibilities.

In the GMC system, a general division of responsibilities identifies the different sub-systems and layers. The control system, the interface layer, the advisory system, and the visualization system all fulfill different functions. The agents inside a layer are given specific responsibilities. For instance, it is the responsibility of a Tollbooth-agent to report all passing workpieces that have been completely and successfully processed. Tollbooth-agents are extremely simple, because they only report these workpieces but do not process the information any further.

Diversified according to their specific responsibilities, the agent types all require specific programming. But, as it is already shown in the example of the Ghost-agent, a small number of actual responsibilities lead to a simple implementation. Furthermore, maintenance of the control system requires less effort. Whenever elementary responsibilities change, the components that must be changed are small and easy to identify.

To reduce the complexity of the presentation of the GMC system, heterogeneous agent types have not been specified. But, there is still potential for heterogeneity. For instance, the Workpiece-agent type may be split into different families when normal orders and rush orders are handled separately. Then, all Workpiece-agents still engage in the same interactions and the exchanged messages are still the same, but the internal decision processes of the agents run differently.

113

Besides parallel heterogeneity represented in different families of the same agent type, sequential heterogeneity in generations of agents of the same type is another option in further versions of the GMC system. Consider again the Workpiece-agent. If the system operates for a long time in the same manufacturing system, Workpiece-agents could adapt their decision processes to the specific layout of the transport system or to the characteristics of the processing resources. The adaptation affects all Workpiece-agents in parallel, spawning new generations.

In general, agent types most prone to heterogeneity are those that have families with a distinctively different bias in their decision processes (e.g. normal order vs. rush order), or agents that have a comparatively short life cycle to change over generations.

Redundancy (SE-Principle 4).-The only component in the GMC system that includes redundant elements is the visualization layer. There are multiple Focus-agents, which wander through the system. They look for View-agents that consider their current documents sufficiently important to present them to the operator. The paths of the Focus-agents are not specified beforehand. They move randomly, guided by attracting pheromones sent out by the View-agents. Thus, redundancy makes sure that on average each View-agent is visited often enough to guarantee (statistically) a required response time of the overall visualization system.

But, redundancy may be incorporated into many other parts. For instance, each Workpiece-agent should be able to handle more than one workpiece at the same time to increase the stability of the system. Then, take-up mechanisms are included into the agents. They make sure that if a Workpiece-agent fails others pick up its tasks. Such redundancy is required when the system is actually fielded.

6.1.2 System-Architecture Principles

Decentralization (SE-Principle 5).-Most components in the GMC system are designed following the decentralization principle. Elements of centralization are only found in the higher layers where the manufacturing system is considered a single unit. There are for instance the Strategy-agents. Exactly one of them is active at a time, coordinating the evaluation of its strategy in its distributed implementation by the Policy-agents.

In the lower layers of the system, all agents operate decentralized. There is no global White Pages service required and local services are realized by the Place-agents of the PI. Teams are formed dynamically whenever tasks arise. The members of a team join in on their own accord.

Modularity (SE-Principle 6).-The GMC system is grown layer by layer. The layers and their respective sub-systems are identified according to major functionalities of the system.

The reactive layer is the core of the system. All by itself it realizes the completion of the incoming orders, making sure transport and production constraints are fulfilled. These constraints are set by the processing requirements of the products, by the layout, and by the capabilities of the manufacturing system. As specified in the SE-principle, the core part of the system is able to operate stand-alone.

The other layers add functionality to the global system behavior without compromising the correctness of the operation in the manufacturing environment. None but the reactive

114

layer is in direct control of the manufacturing system. All the agents of the other layers can do is advising the reactive layer in its operation. It is up to the agents of the reactive layer to heed or to ignore the advice. It is their responsibility to maintain a correct system behavior.

The advantage of ordering the components of the system into separate layers instead of arbitrary parts is to be found in the clear interdependencies of the layers. In general, each layer requires the availability of all layers below it. Thus, the start-up sequence of the system is clearly defined.

Modularization of complex software systems is not specific to the synthetic ecosystems approach. Modular systems design has been successfully used for many years now. Component-based approaches promote fast and efficient realization of large software systems through the combination of customized standard components, promising extended re-use and reduction of development costs. Thus, following the modularity principle integrates a successful general principle in software design into the synthetic ecosystems approach.

Parallelism (SE-Principle 7).-Parallelism is introduced by the manufacturing system itself. Manufacturing is inherently parallel. Transport and processing tasks arise throughout the system in an asynchronous manner. Thus, a system controlling the production has to match this parallelism.

Teams are formed dynamically in the reactive layer. There is always a Workpiece-agent and a Resource-agent teaming up to fulfill tasks that concern a workpiece. As long as a Workpiece-agent manages only one workpiece (see redundancy), the agent is engaged in one interaction at a time only. The Resource-agents, on the other hand, are designed to manipulate multiple workpieces. They join more than one team at a time and they interleave separate interactions with different Workpiece-agents.

At the same time Policy-agents pick up the pattern information generated by the Workpiece-agents and Ghost-agents and give their advice according to the local aspect of their global strategy.

Bottom Up Control (SE-Principle 8).-With a multitude of small agents interacting to control the underlying manufacturing system, the bottom-up-control principle is consequently followed in the GMC system. There is no single entity controlling the whole transport system or the whole manufacturing process by itself. No agent has global access or even the ability or knowledge to run the system.

Teams fulfilling small tasks are dynamically formed on the basis of the local expertise of the agents, their responsibilities, and their goals. As a consequence of these many activities, material flow patterns emerge. Agents at higher layers of the control system observe these patterns and base their decisions on them. The higher layers try to influence the lower layers. But, the actual control lies at the bottom - at the core of the GMC system.

6.1.3 Interaction Principles

Locality (SE-Principle 9).-Locality is strictly followed by all agents operating in the PI, which provides a spatial structuring of the agent system. The places in the PI map to elementary zones of the manufacturing system. A Resource-agent controls all elements in its zone. Agents that represent mobile entities (e.g., Workpiece-agent, Ghost-agent) move from place to place. But, wherever they are, they are only permitted access to pheromones

115

at their current location, and they only interact with agents sharing the current place. According to the current situation and local information the agents decide dynamically which direction to go to and what agents to interact with as they explore the system.

There is no agent that has global knowledge about the system. Even singular agents outside of the PI like Strategy-agents or the Incubator-agent know only specific aspects of the overall system.

Indirect Communication (SE-Principle 10).-Indirect communication among agents in the GMC system is used, wherever:

information generated by another agent at a different location is required for a decision of an agent, e.g.,

ability information of Processing-agents and Unloader-agents attracting the Workpiece-agents towards their locations

criticality information of View-agents attracting Focus-agents

a decision process of an agent requires information that is jointly generated by other agents at the same location, e.g.,

current load at the place generated by all Workpiece-agents present

the sender and the receiver of data operate on different time-scales, e.g.,

predicted load at the place generated by the Ghost-agents and processed by the Policy-agents

performance evaluation generated by the Policy-agents and processed by the Match-agents.

The PI is the channel for the indirect communication of data. For each transmission a pheromone type is specified. The provider of information refreshes selected pheromones in a specific way and thus the receiver may deduce the information from the sampled pheromone strength.

The design, evaluation, and the tuning of indirect communication mechanisms is supported by the results gained in the analysis of the formal model of the pheromone infrastructure (Section 3.2). For any given topology and any refresh behavior, the resulting local pheromone strength may be approximated. The more regular the refresh behavior is, the more accurate is the approximation.

Recursion, Self-Similarity (SE-Principle 11).-The regular refresh of pheromones is a generic principle used throughout the whole system to communicate and aggregate data in the PI. Regular refresh behavior is preferred since it allows a very accurate analytic prediction of the resulting pheromone patterns, reducing the error in the interpretation of the sampled data.

The use of pheromones for spatial coordination of movements of agents is a second mechanism found in different layers of the system. Examples are the attraction of Workpiece-agents and Ghost-agents by Processing-agents or Unloader-agents, or the attraction of Focus-agents by View-agents.

Structural recursion or recursion in mechanisms is not realized in the GMC system, because there is no singular agent at one level that represents an aggregation of multiple agents at another level. The only structure coming close is the representation of a strategy during implementation by the Policy-agents and during evaluation by the Strategy-agent. But then, completely different interactions occur at the different levels. More depth in

116

structure and more similarities among the different levels are required to make effective use of recursion.

Feedback, Reinforcement (SE-Principle 12).-Reinforcement and feedback play important roles in most coordination mechanisms of the GMC system. Reinforcement is used, for instance, to bias the system towards successful strategies in terms of goal fulfillment and to steer the system away from unsuccessful strategies (Section 5.1.2). The performance of the currently implemented strategy reinforces the bias towards or away from a strategy.

Feedback effects are included when the current strategy is implemented by its Policy-agents. Ghost-agents, regularly created by Workpiece-agents, generate a prediction of the local load pattern as they move downstream and approximate a possible future behavior of their Workpiece-agent. Each Policy-agent adapts its advice according to the local prediction. The advice is transmitted in pheromones that are propagated upstream. The path of a Ghost-agent is influenced by the pheromones it encounteres. Changed advice changes the predicted pattern, until the predicted pattern matches the goal pattern of the Policy-agent. There exists a closed loop of cause and effect, tuning the advice. The Workpiece-agents follow the advice in the same way their Ghost-agents do, making the prediction come true.

The adaptation of behavior on the basis of a repeated re-evaluation of a prediction is found in many coordination and control mechanisms in nature. Humans do it too, for instance when preparing to catch a ball.

Randomization (SE-Principle 13).-The application of feedback effects in synthetic ecosystems becomes very hard when no randomization in the internal decision processes of the agents is introduced at the same time. Consider again the adaptation of the advice of the Policy-agents, but now in a scenario with two Policy-agents at different places (x and y) acting on the same upstream place (z). The two Policy-agents both want some share of the flow of workpieces out of place z, but they both do not want the full flow.

Consider the decision process of Workpiece-agents and Ghost-agents at place z when choosing between the direction towards place x and towards place y but assume that the selection is not randomized and they always choose the direction from where the strongest attraction comes. For the sake of simplicity assume furthermore that the agents all go towards place x if the difference between the two competing forces is zero.

There is no simple and continuous solution to the presented problem. Either, the Workpiece-agents all go to place x, or they all go to y. Only when the Policy-agents change their advice very fast and synchronized, they may split the flow in the required manner. But then, they would have to operate on the same time-scale as the agents of the reactive layer do, since they are then primarily concerned with the single workpiece instead of a flow of workpieces.

The discussion already shows how randomization prevents thrashing effects, which make an intuitive, stable, and continuous control of the system hard or even impossible to achieve. Other examples are found in the routing decision of Workpiece-agents selecting processing resources in the reactive layer, the generation of the predicted pattern in the interface layer, or the evaluation of strategies. It remains an issue for future research, to assess if the randomization of the internal decision processes of the agents increases the entropy on the micro-level, and thus stabilizes the global operation of the system (macro-level), as proposed metaphorically in [Parunak, 1997].

117

Evolutionary Change (SE-Principle 14).-The adaptation of the advice of a Policy-agent is considered again. A Policy-agent perceives the difference between the current or the predicted local pattern and its goal pattern. The agent changes its advice by a small and fixed amount according to the perceived difference. Thus, this difference only determines the direction of the change but not its strength.

There are two primary motivations for the chosen design. First, the change in the advice takes some time for its effect on the patterns to become visible. And second, the perceived difference could be an abnormal perturbation.

The first reason, the time lag between the change and the effect, introduces a necessary delay into every complex adaptive process where the absolute effect of the change cannot be predicted beforehand. The Policy-agent can do nothing but wait to see how the changed advice influences the flow, especially, if there are other Policy-agents nearby that also influence the flow and that also change their advice continuously. In this case, abrupt changes lead only to thrashing and any sensible guidance is lost.

The nervousness of the agents is reduced when perturbations in the data is ignored. All data they perceive is noisy. Only lasting change in the material flow should be adapted to. Again, the Policy-agents should not change their advice with every workpiece that is running late. The Policy-agents operate on a larger time-scale. Their speed of change must be much slower for the single workpiece to lose itself in the aggregated flow. The use of pheromones is a means for such long-term aggregation. While they slowly evaporate, dozens or even hundreds of Workpiece-agents have left their tiny marks on the trail.

Information Sharing (SE-Principle 15).-Information sharing is encountered throughout the components of the GMC system and much more could be integrated yet. The use of pheromones to coordinate many small agents demonstrates system learning. Adaptation of the advice of a Policy-agent is an example for individual learning. Sharing information between two generations of agents is mentioned in the discussion of SE-principle 3 where adaptation of decision parameters of Workpiece-agents to the given layout of the manufacturing system is proposed.

System learning dominates especially in the design of emergent features in synthetic ecosystems made up of a multitude of simple agents. But, some form of global coordination is required to design emergent features. A very powerful tool for global coordination in synthetic ecosystems is the use of pheromones as shown in the GMC system.

Forgetting (SE-Principle 16).-Learning and adaptation in long-lived systems that act in a continuously changing environment improves performance only when combined with some form of forgetting. As long as no information is ever given up, the system and its components eventually suffocate under the load of accumulated data. It is necessary to lose old knowledge to make space for new.

The evaporation of pheromones in the pheromone infrastructure of the GMC system is the most obvious forgetting mechanism in information sharing. But it is only effective because most information used in system learning is contained in pheromones and the pheromones are used to bias otherwise random choices.

The eventual death of each adaptive agent is the second mechanism providing the system with forgetting. Agents permitted to live forever are the Resource-agents, which do not adapt. Most other agents potentially die. The agents with the shortest life expectancy are the Ghost-agents, followed by the Workpiece-agents. Policy-agents die

118

when their strategy is no longer implemented, and Strategy-agents die by the hand of the operator or they are killed by the Incubator-agent if they do not perform. With the potential death of each Strategy-agent, all data contained in the Match-agents is eventually replaced. Immortal agents like the View-agents, Focus-agents, and the Incubator-agent have no adaptive mechanism attached to them.

Multiple Goals (SE-Principle 17).-The GMC system is designed to suffice maintenance-goals. In the presented form there are two major goals:

completely process every incoming workpiece according to its processing needs

change the material flow towards improved global throughput

Other maintenance-goals may be integrated into the behavior of the system at any time. The suggested evaluation of the goal-fulfillment is open and intuitive. But, it is impossible for the system to fulfill a global achievement goal, primarily because the life of the system has no specified end-point where the fulfillment of the achievement-goal could be evaluated.

6.2 The PI - Agent System and Environment

Throughout the presentation in the Chapters 3 to 5, the pheromone infrastructure is considered as an extension of the runtime environment for software agents. In the following, the primary perspective is reversed and implications of an interpretation of the PI as an agent system are discussed.

Eventually, the agent system of the application returns into focus and the PI is only a part of the environment. Modes of information sharing among the agents are considered, presenting a broader and more abstract perspective than the previous chapters.

6.2.1 Analyzing Emergent Features

The PI is part of the runtime environment of agent applications and it provides the following generic and application-independent services (Section 3.3.2): Access and Topology, Local White Pages, Pheromones, and Notifications. The distributed implementation of the PI specifies the Place-agent type.

The community of Place-agents, which emulates an active pheromone environment for software agents, may be interpreted as a simple synthetic ecosystem: The Place-agents are small and simple; they only interact locally and share information. The information they share is encoded in pheromones, which evaporate and thus provide forgetting. Pheromones and pheromone patterns get their semantics in the context of the respective application, but the PI interprets a pheromone as what it originally is: a quantity of a chemical substance that is put into the environment to propagate and to evaporate.

There exists only a distributed specification of the system-level behavior of the PI that defines the behavior of the Place-agents and how they are linked. An evaluation of the system-level behavior requires an analysis of global features as they emerge from local interactions. For the time of the analysis of the PI (Section 3.2) the application agents (e.g., the GMC system) are considered the environment of the PI. Their behavior is only specified in terms of input patterns they generate in the PI. Furthermore, it is necessary to formally describe the internal Place-agent processes, the interactions among the agents of

119

the PI, and the influence of the outside environment (the application) to set up a formal model.

The PI is required to show global stability, which is a system-level feature. The formal model of the PI facilitates the proof of global stability. The model also permits the numerical prediction of emerging pheromone patterns in a wide range of input scenarios.

The ingredients required in the analysis of the features of the PI are also found in the synthetic ecosystem of an application that operates in the PI environment. The evaluation of the system-level behavior of the GMC system components follows a similar approach. To show a specific global feature, it represents the decision processes of all participating agents and their interactions among each other and with the PI in a formal model.

A numerical description of emerging multi-agent coordination is primarily supported by the use of indirect communication through the PI and the specification of many agent-internal decision processes based on probabilities determined by current pheromone patterns. The numerical description is further reduced in complexity when the SE-Principle 2 (Small Agents) is followed in the design. If many agents are required to make an impact, statistical or probabilistic abstractions may be used in the description.

The numerical description of emerging multi-agent coordination is demonstrated in the evaluation of the GMC system in Section 4.5. There is, for example, the emergence of PS and PP patterns in the operation of the interface layer. These patterns are predicted numerically and they are actually observed in the simulation.

6.2.2 Modes of Information Sharing

The PI supports sharing of information among agents in an active environment. The specific characteristics of the PI permit different agent behaviors in the sharing process that have consequences for the transmission of information. In general, all information put into the PI is lost over time to provide an automatic forgetting mechanism for the agent system. An information provider has to take the evaporation of pheromones into account. The provider refreshes the stored information regularly, because it does not know when the consumer accesses the data. As a consequence, the refresh mode, which results in a specific pheromone pattern, becomes a transmitter of information in its own right.

There are two basic modes of information sharing when using the PI. Either information is shared locally within the local context, or the information is shared globally, outside of the local context.

In local information sharing the provider and the consumer are located at the same place in the PI. The provider regularly refreshes the information encoded in pheromones while the consumer observes the resulting pheromone patterns. In local information sharing the input events to the pheromones are not intended to propagate. The analysis of the formal model of the PI provides the designer with numerical predictions of the resulting patterns and facilitates tuning and evaluation. The transmission of the change of the account of a strategy sent by a Policy-agent to its co-located Match-agent is an example of the local information-sharing mode (Section 5.1.2).

There are two approaches when sharing information globally: active or passive information sharing. If information is shared actively, agents move through the PI and refresh pheromones according to the local context. In passive sharing the provider remains stationary but its input events are propagated to neighboring locations.

120

Passive information sharing makes use of the propagation mechanisms provided by the PI. The information provider does not have to care about how the information is spread. As a consequence, the designer may keep the agent very simple, because it would not need any navigation mechanisms. The Processing-agents and the Unloader-agents spread their ability information upstream using passive information sharing (Section 4.2).

If information is shared passively, it is outside of the control of the provider how the information is adapted to the local context. In general, there is no adaptation at all, as it is the case in the attraction of Focus-agents to View-agents (Section 5.2). But the propagation may also be restricted to the context into which it is put. Such a restriction is set for the PA pheromones that do not propagate beyond the next upstream processing resource (Section 4.2.1). Finally, propagation filters may be specified to automatically translate information from one context to another. The GMC system defines very complex filters for the translation of advice (Section 4.4.2).

An agent may ignore the propagation mechanisms of the PI and spread its information actively. In this case, the agent remains in control of what information is put where. Active global information sharing is more selective and dynamic, because the reasoning process may incorporate the current situation and the current local context.

Active information sharing generates the predictive pattern of PP pheromones (Section 4.3.1). The Workpiece-agents regularly spawn new Ghost-agents, which simulate a possible future of their respective creator by emulating its transport decision process and by simulating processing activities. A refresh of a PP pheromone by a Ghost-agent includes its current simulated processing state and its location, which is a result of the emulated transport decisions. It would be close to impossible to create the same predictive pattern through passive information sharing with propagation filters. The PP pheromones would have to carry much more data and the propagation filters would be very complex. As a consequence, the effort spent in implementation, stabilization, and maintenance of the prediction mechanism would be much higher.

Finally, it remains to consider under what conditions direct agent interaction may be permissible or even required. When should the designer refrain from using pheromones? In general, direct communication should not be considered for global information sharing, except when the topology of the PI does not fit the interaction, when the data exchange might overload the processing or communication capabilities of the PI, or when the information cannot be encoded in pheromones. But, at least in the first case a redesign of the layout of the PI may still be a better alternative.

In local information sharing, when the provider and the consumer are at the same place and operate in the same time-scale, direct communication may be chosen for very simple interactions in which no aggregation of data from different sources is required. Also, joint decision-making resulting in a commitment by the agents might be better achieved in direct negotiations. Finally, when the speed of the interaction is critical for the correct operation of the whole system, the delay imposed by transmissions through the PI may be too long.

6.3 The GMC System in State Space

The following section re-visits the domain of manufacturing control systems. Taking a step back, the operation of a manufacturing system in state space is considered. Based on

121

the abstraction, reasoning modes available to a control system are sketched and referred to the GMC system.

6.3.1 Manufacturing Control in State Space

6.3.1.1 Parameters, State Space, and Trajectories

A manufacturing system, like other systems, has a set of system parameters, which influence the behavior of the system. Some of these parameters may be controlled by the attached manufacturing control system (e.g., material flow patterns, processing capacities). Other system parameters elude direct control (e.g., processing yield, availability of resources).

Figure 6.1. A System in State Space

There are system variables (e.g., buffer-levels, WIP, processing states of workpieces, etc.) whose respective domains are combined to span a state space of the (manufacturing) system. A setting of these system variables defines a state of the system. The manufacturing system has an initial state the moment it is started and it has a current state at some observation point in time (Figure 6.1).

The behavior of the manufacturing system as a whole follows its intrinsic dynamics (e.g., transport and processing of workpieces), which is influenced by the system parameters. As a result of the dynamics, the system variables change over time. Models in systems theory specify these intrinsic dynamics either in discrete transition functions or in continuous flow fields.

The path of the changing system variables over time in state space is called the trajectory of the system. For any moment at which the state is observed, there is exactly one trajectory starting at the initial state and ending at the current state. From the current moment on, there may be an infinite number of possible future trajectories the system may follow. The selection of one of these trajectories depends on control decisions and on changes and disturbances influencing the system parameters.

6.3.1.2 Performance-Evaluation

At any point in time there exists an evaluation of the previous operation of the manufacturing system. Such an evaluation may be one-dimensional even when there are

122

multiple production goals, because these goals have to be weighted in importance and thus they may be combined into one value for an evaluation.

An evaluation of the performance of a manufacturing system usually considers the fulfillment of maintenance goals. In this case, an evaluation may be based on a sequence of states over time (trajectory) instead of just evaluating the final state, as it is sufficient for achievement goals. Hence, the performance evaluation of a manufacturing system may be seen as a mapping from the space of system trajectories to the space of performance values. Usually only limited trajectories of a fixed length are considered. An example for an evaluation that is based on a limited trajectory is the throughput of a manufacturing system over a given period.

Consider a space of possible trajectories and its mapping to evaluations. It is the going concern of any control system to keep the evolving trajectory of a manufacturing system in a sub-space of the trajectory space where evaluations are high. Good control decisions reduce the set of future trajectories; cutting off sub-spaces of inferior evaluation and increasing the chance to access highly evaluated trajectories even though unforeseen events may occur.

6.3.1.3 Structures in State Space

A state space may have attractors, each with a basin of attraction. If the current state of the system falls into such a basin and random events do not push it out of the basin again, the intrinsic dynamics of the manufacturing system eventually take the state into the attractor (or arbitrarily close to it). Depending on the type of attractor, the state of the system does not change anymore (point attractor), it repeats itself infinitely (limit cycle attractor), or it runs on quasi-periodic or strange orbits.

How the intrinsic dynamics of the system change the system variables (and thus the state) depends on the current setting of the system parameters. As a consequence, the location, the size and the depth of the basin, and even the type of an attractor may change when the system parameters change.

Figure 6.2. Predicting an Attractor in State Space

What is the advantage of considering attractors in state space? If the current state is located in the basin of attraction, the control system may approximate the continued trajectory of the manufacturing system, which approaches the attractor following its own intrinsic dynamics. Thus, the control system is able to decide if it has to interfere or if it can leave the system to its own dynamics for a while, assuming the uncontrolled system parameters do not change the attractor. Figure 6.2 shows a scenario where at least one of the future possible trajectories leads into a limit cycle type of attractor.

123

A good control system should prefer attractors with many highly evaluated incoming trajectories that start at the current state of the system. Additionally, the basin of attraction of these attractors should be near the current state of the system in terms of control effort required to get into the basin. The basin should be large and deep too. Finally, the best of all attractors fulfilling the previous requirements are those that remain reasonably stable in the face of the most probable changes in the system parameters.

6.3.1.4 Levels of Sophistication in Control

The decision processes of the control system may operate on different levels of sophistication incorporating information of different complexity. In general, there may be four levels of sophistication identified: reactive, retrospective, predictive, and proactive decisions. All four levels of sophistication are found in the GMC system.

On the level of reactive control, every decision is based only on the current state of the manufacturing system. The reactive level of sophistication is already sufficient for the control of a manufacturing system if the set of trajectories remaining after a control decision was taken may be determined directly from the current state. Such a prediction may be based on heuristics. In the GMC system, the reactive layer implements this mode of reasoning. But, in a flexible flow shop, the reactive mode is insufficient to fulfill the production goals.

In the retrospective mode the reasoning is extended to incorporate past states and the trajectory leading up to the current state. At the retrospective level, system parameters defined over a period of time, such as local throughput values, are incorporated into the decision processes too. The GMC system realizes the retrospective level of reasoning in the advisory system. Information on the previous evolution of the system is encoded into the current pattern created by the interface layer. The strategy implementation layer and the strategy selection mechanisms incorporate information from the past and the present into their decision processes.

Whereas the retrospective mode considers the actual trajectory in the past, predictive reasoning explores the short-term effect of alternatives in the decision process. With the consequences of a decision made explicit, the appropriate sub-set of the available trajectories is selected. The predictive way of reasoning is found in the strategy implementation layer of the GMC system. A short-term prediction of the evolution of the system is generated under the assumption of continued operation (stable yield and advice) by the interface layer. The strategy implementation layer takes a prediction into account when tuning the advice.

Finally, on the proactive level the control system explicitly or implicitly tries to guide the state of the manufacturing system towards an area in trajectory space where most of the trajectories map to high evaluations and where pathologic states are unlikely. In such optimal areas even the effects of many possible changes and disturbances are not as devastating for the performance of the system as they may be in other areas. Learning processes in the strategy evaluation and generation layer bias the GMC system towards strategies that are successful in terms of the evaluation of the performance of the system. The resulting trend implements an implicit kind of proactiveness. Explicit proactive reasoning is not realized.

124

6.3.2 Observing the State Space

The control system must be enabled to actually perceive the state space and important system parameters to reason about previous or possible trajectories. There are at least two ways of accessing the required data: Either, the operation of a manufacturing system is monitored directly, or the trajectory is derived from pheromones inside the control system.

Direct monitoring is primarily the task of agents at the core of the control system that map to the physical entities in the manufacturing system. For instance, a Switch-agent could monitor the arrival pattern of workpieces at its entries. Observation activities amount to an additional task besides the fulfillment of routing requests. If agents in other components of the control system need access to the gathered information, the data has to be communicated indirectly. The use of direct communication would violate the design principles.

A direct monitoring of the state of the system has the advantage that it provides up-to-date and correct data. A disadvantage is the additional task given to the core part of the control system. But, if the decision processes inside the control system actually incorporate the trajectory information into their reasoning, it might be worthwhile to spend the additional effort in design and implementation.

On the other hand, the indirect approach might be considered if trajectory information is only required for observation purposes, for instance to study the general behavior of the system. Then, the task of information gathering should be given to the visualization system whose agents have access to all pheromones at all places in the control system. The conditions for deriving the characteristics of a trajectory from pheromones are considered in the following.

The GMC system applied in the paint-shop demonstration (Section 4.5) illustrates the observation problem. At the place of the Processing-agent P1 there are always a number of Workpiece-agents present. These agents regularly refresh one of the PS pheromones. Thus, in the pheromone strength the number of workpieces of a given processing state is transmitted to every agent located at P1.

Figure 6.3. Perceived Workpiece Count at Place P1 over Time

Depending on the evaporation parameter of PS, the pheromone strength takes a certain time to approximate the correct value. Therefore, the strength conveys a short-term average of the number of workpieces present. Figure 6.3 shows the averaging effect where one curve represents the actual number of workpieces present while the plots E=0.9,

125

E=0.99, and E=0.999 represent the workpiece count as perceived through PS for the respective evaporation parameter E.

Takens's Theorem [Takens, 1980] may provide an approach to analyze the characteristics of the trajectory of the manufacturing system. Informally, it states that if there is a time series generated from the observation of a state variable, a synthetic n-dimensional state vector may be constructed under specific conditions using successive n-tuples of the time series that is projected onto a n-dimensional sub-space but still captures the topology of the trajectory of the system in its native state space completely. One of the requirements set by the theorem is the existence of the second derivative of the state variable. It may be speculated that this requirement is more easily fulfilled in the indirect observation of the variable through some related pheromones.

In the demonstration, the number of workpieces present may be observed directly by observing the number of Workpiece-agents located at place P1. Or, it could be observed indirectly through the respective PS pheromone. If the indirect approach is chosen, it has to be guaranteed that the time series generated through the observation of the pheromone strength is strongly related to the actual workpiece count at P1. Thus, the sampling rate must be set in a way that most samples are taken when the pheromone strength is in equilibrium. Based on results gained in the analysis of the PI, the appropriate sampling rate may be selected when estimating the rate of change of the observed variables.

In specific scenarios an estimation of the quality of a time series generated from the observation of the pheromone strength may be given. For example, in the case of the number of workpieces at P1 it may be assumed that the workpiece count changes on average every tC time units by one workpiece. If a distance to the equilibrium of x percent of the average refresh per unit time is considered sufficiently close to the equilibrium, equation (3.29) may be used to determine if (on average) the equilibrium is reached at all. Hence, the time to equilibrium tE must be smaller than tC. Furthermore, the probability of sampling an equilibrium value is (tC-tE)/tC (percent).