Intuitive Definitions of Complex Systems

Complex system (complex
comesfrom Latin com-together + plecteretotwine or braid) is a system
composed from relatively many mutually related parts.

Complex systems are usually
(but not always) intricated - hard to describe or understand.

Examples of complex systems

Parts of human society:

Markets

Organizations

Language

Internet

Biology:

Cells

Organ – e.g. brain

Immune system

Organisms

Populations

Ecosystem

Physics:

Turbulence

Weather

Percolation

Sandpile

The world consists of many complex systems, ranging from our
own bodies to ecosystems to economic systems. Despite their diversity,
complex systems have many structural and functional features in
common that can be effectively simulated using powerful,
user-friendly software. As a result, virtually anyone can explore the
nature of complex systems and their dynamical behavior under a
range of assumptions and conditions. (M. Ruth, B Hannon,
Dynamic Modeling Series Preface)

Structural relations define which parts are connected together.

Functional relations define the behavior or dynamics of the
system - how does the change of state of one part influence the state of
other connected parts.

Structurally complex systemA system that can be analyzed into many components having
relatively many relations among them, so that the behavior of each
component can depend on the behavior of many others. (Herbert Simon)

Remember: The number of relationships could be much higher than the
number of components!

Dynamically complex systemA system that involves numerous interacting agents whose
aggregate behaviors are to be understood. Such aggregate activity is nonlinear, hence it cannot simply
be derived from summation of individual components behavior. (Jerome
Singer)

Example: Building of new highways could lead to more traffic jams,
because it initially decreases the waiting times and this increases
the desirability of car driving.

System dynamics

State space (phase space) is an abstract space in which all
possible states of a system are represented, with each possible state
of the system corresponding to one unique point in the state space.
Dimensions of state space represent all relevant parameters of the
system. For example state space of mechanical systems has six
dimensions and consists of all possible values of position and
momentum variables.

Dynamics of the system is the set of functions that
encode the movement of the system from one point in the state space to
another.

Trajectory of the system is the sequence of system
states.

Fixed point is a point in the state space where the system is
in equilibrium and does'nt change.

Attractor is a part of the state space where some
trajectories end.

Attractor types

Fixed point

Limit cycle

Strange attractor

Dynamical systems can often be modeled by differential equations
dx/dt=v(x), where x(t)=(x1(t), …, xn(t)) is a vector of state
variables, t is time, and v(x)=(v1(x), …, vn(x)) is a vector of
functions that encode the dynamics. For example, in a chemical
reaction, the state variables represent concentrations. The
differential equations represent the kinetic rate laws, which
usually involve nonlinear functions of the concentrations. Such
nonlinear equations are typically impossible to solve analytically,
but one can gain qualitative insight by imagining an abstract
n-dimensional state space with axes x1, …, xn. As the system
evolves, x(t) flows through state space, guided by the ‘velocity’
field dx/dt = v(x) like a speck carried along in a steady, viscous
fluid. Suppose x(t) eventually comes to rest at some point x*. Then
the velocity must be zero there, so we call x* a fixed point. It
corresponds to an equilibrium state of the physical system being
modeled. If all small disturbances away from x* damp out, x* is
called a stable fixed point — it acts as an attractor for states in
its vicinity. Another long-term possibility is that x(t) flows
towards a closed loop and eventually circulates around it forever.
Such a loop is called a limit cycle. It represents a self-sustained
oscillation of the physical system. A third possibility is that x(t)
might settle onto a strange attractor, a set of states on which it
wanders forever, never stopping or repeating. Such erratic,
aperiodic motion is considered chaotic if two nearby states flow
away from each other exponentially fast. Long-term prediction is
impossible in a real chaotic system because of this exponential
amplification of small uncertainties or measurement errors.
(Strogatz, 2001)

Example: The predator-prey model

We study the dynamics of mutually dependent population size of
predators and prey.

The model is supported by analyses of 100 year fur trapping
records of the Hudson's Bay Company.

Animation of the system dynamics and the two differential
equations governing the dynamics (Lotka-Volterra equations). X
represents the size of hare population and Y the size of lynx
population.

Formal definitions of complexity

Non suitable complexity measures for complex systems

There are many formal definitions of complexity available. Only a small
portion of them is suitable for description of complex systems. There are
two particular notions of complexity which are not suitable for
description of complex systems but have very good sense in other domains.

Computational complexity measures how much time or memory a
standard universal computer program needs for solving problems with
increasing amount of input data. The dependence of the amount of input
data and the required time or memory could be linaer, logarithmic,
polynomial, exponential, etc.

Algorithmic information content (AIC, sometimes also called
Kolmogorov complexity) of a string of bits is defined as the length of
the shortest program that will cause a standard universal computer to
print out the string of bits and then halt.
Gell-Mann (1995) writes „A random bit string has maximal AIC for its
length, since the shortest program that will cause the standard
computer to print it out and then halt is just the one that says PRINT
followed by the string. This property of AIC, which leads to its being
called, on occasion, "algorithmic randomness," reveals the
unsuitability of the quantity as a measure of complexity, since the
works of Shakespeare have a lower AIC than random gibberish of the
same length that would typically be typed by the proverbial roomful of
monkeys.“
The AIC is called monotonic complexity measure because with
increasing randomness it also increases.

Suitable complexity measures

Good measures of system complexity should measure the
amount of regularities in the system (and not its randomness). Such
measures should be low for both very simple systems (where is only one or
very few dominant regularities) and random systems (where are almost no
regularities). Such measures are called non-monotonic. We can say
that they are somewhere between order and randomness - on the „edge of
chaos“ (Langton, 1990).

Neural Complexity

Neural complexity (Sporns et al., 2000, 2002) is a measure inspired by
the cognitive processes in the brain. It measures how much the change of
activity in one part of he network changes the activity in other
parts. The authors described it shortly as a measure of
"the difference that makes difference". Neural complexity is one of
many complexity measures based on mutual information.

Mutual information between two parts of a system is defined:

MI(Xjk,, X-Xjk) =
H(Xjk) + H(X- Xjk) –
H(X)

There X is the system, Xjk is the j-th
permutation of a part of size k and X-Xjk
is the rest of the system.

Neural complexity is formally the sum of average mutual information
between subsets of the system and the rest of the system

The neural anatomy, neural activity, EEG signal and neural complexity of
the brain of an old (a), adult (b) and very young (c) cat. In the old cat
there are mostly local specialized connections present but the global
integrative connections are missing due to degenrative processes, the
result is rather random dynamics. In the adult cat there are both local
and global connections present, the result is complex dynamics. In the
young cat the local connections are not developed yet but the global
connections are already present, the result is regular dynamics (Edelman
& Tononi, 2000).

Statistical Complexity

The statistical complexity (Shallizi 2001, 2003, 2004) reflects the
intrinsic difficulty of predicting the future states of the system from
the system history. It is the amount of information needed about the
past of a given point in order to optimally predict its future. Systems
with a high degree of local statistical complexity are ones with intricate
spatio-temporal organization. Statistical complexity is low both for
highly disordered and trivially-ordered systems.

Self-organization and related concepts

Self-organization

Self-organization (First used by Ashby in 1948.). The
ability of the system to autonomously (without being guided or managed
by an outside source) increase its complexity.

If a local system is an open system receiving relatively stable and
appropriate amount of energy from its environment and the local
system is composed from sufficient number of parts which are able
to interact through positive and negative feedback, there could
(depending on some parameters) be established relatively stable network
of feedback loops. This process is called self-organization and the
established dynamic network is called self-organized system.

Examples of self-organisation:

In Physics:

Benard cells
- coherent motion of large number of molecules in heated liquid layers.

Belousov-Zhabotinsky
reactions - a specific "coctail" of chemical ingredients loops
through visualy discernable states when it receives thermal (heated) or
mechanical (stirred) energy.

In Biology:

Flocking
- a group of organism can self-organise in a relatively coherent whole
which is able to synchronously react on external stimuli.

Ants demo - increasing the length
of pheromone trace or the number of ants would lead to self-organization
of the food track (result of stigmergic interaction between agents
and environmnent).

An ecosystem or a whole biosphere - the
feedback loops between environment and organism could lead to
stabilisation (homeostatis) of some environmental parameters.

Emergence and self-organization

Traditional definition of emergence
The arising of characteristics of the whole which cannot be attributed
to the parts. There arise new qualitative and not only quantitative
changes. Very vaguely: the whole is more than sum of it parts (an
statement made already by Aristotle in Metaphysics).

Modern definition
Emergence is „the arising of novel and coherent
structures, patterns and properties during the process of
self-organization in complex systems." (Corning, 2002; Goldstein,
1999)

Common characteristics of emergence:

Radical novelty (features not previously observed in the system)

A global or macro “level” (i.e., there is some property of
“wholeness”)

Coherence or correlation (integrated wholes that maintain themselves
over some period of time)

It is the product of a dynamical process (it evolves)

It is meaningfull for us (i.e. has some pragmatic value for
us – we can use it).

Weak emergence: new properties arising in systems as a result
of the interactions at an elemental level. The causal conection between
the interactions of the parts and the properties of the whole can be
traced in great detail.

In many cases the relationship between parts and the whole
depends on large scales of space and time.

Domain

Elementary level

Global level

Geography

Flow of water

Shape of the river bed

Brain

Neuronal firing

Synaptical changes

Organism

Behaviour in specific situations

Ontogeny

Evolution

Life of an individual

Phylogeny

Language

Speech acts

Development of language

Economy

Activity of micro-economical subjects

Macro-economical properties

Strong emergence: the properties of the whole supervene on the
properties of the parts. Supervenience describes causal dependence
between sets of properties. If property B is causaly dependent on
property A, it means that one state of property B could be caused by
many states of property A, but one state of property A causes exactly
one state of property B.

Examples:

The relationship between physical body and conscious experience
(e.g. Mind-body or psycho-physical problem)

Problems

Some authors disagree with the above definition of emergence (De Wolf,
2005). Can there be emergence without self-organzation and vice versa?

What is the relation between quality and quantity? When does a new
quality arise?

Synergy

The combined (cooperative) effects that are produced by two or more
particles, elements, parts or organisms – effects that are not
otherwise attainable. (Corning, 2002)

Example: Lichen and other symbiotic organisms

Adaptability

Adaptability is the ability of a system to maintain its complexity
in changing environment. Often we can find a feed-back between system
and its environment.

System types

Constructed

Self-organized

Non-adaptable

Example: Classical machines

Example: Crystals

Adaptable

Example: Adaptable robots

Example: Living organisms

Daisy world – an example of self-organized
adaptive system

Aim: Proof of an biological hypothesis that biosphere as a whole can
regulate its own environment on a global scale. This hypothesis was
proposed by James Lovelock in the early 1970s and he called it the Gaia
hypothesis. (Gaia is the name of the Goddess of Earth in ancient Greek
mythology.)

Parts: black and white daisy flowers, environment

Structure of the environment: 2D matrix

Functions:

Black daisies increase and white daisies decrease the temperature
of their environment

The reproduction of daisies depends on the temperature of the
environment (feedback)

Cellular automatons (CA)

1970: An article in Scientific American (Gardner) about 2D CA called
Life (designed by Conway) provoked new interest in CA

1983 Wolfram published article about different classes of behavior in
CA

Characteristics

Bottom-up approach - simulation of very simple parts (cells)
interacting through very simple rules in a homogenous and
regular environment (matrix) could lead to extremely complex
behavior of the whole system

Precursor of Agent-Based models (ABM)

Discrete space and time

Discrete states - every cell has finite number of states, this number
is the same for all cells

Discrete dynamics - states of cells change synchronously

Local interaction in a homogenous spatial matrix - states of the cells
are dependent on states of neighboring cell

1D cellular automatons

Basic description of a simple cellular automaton (CA) as presented in
(Wolfram, 2002).
This book is available
on internet and represents a good introduction to cellular
automatons (and other simple automatons with complex behavior) but it
also gained bad reputation due its egocentric tone.

Basic types of rules

Elementary - the state of the cell is dependent on the
structure of neighboring cells (as in the above example)

If the cells could have k different states and the state of the
cell is dependent on n neighboring cells, then there exist k^(k^n)
rules. For example if each cell could be only in two possible states
(the CA is binary) and the state is dependent on three neighboring
cells (including itself) then there are 2^(2^3) = 256 rules.

More about CA dynamics

Self-replication

Self-replication was first investigated by von Neumann in 1940s. The von
Neumann self-reproducing automata is actually a universal constructor'
that constructs "any machine'' in its 29-state cellular space. In
particular, it is capable of Turing universal computation. It solves the
self-reproduction problem by reading a tape containing instructions on how
to build a copy of itself, provides the copy with a copy of its own input
tape, and then presses the ON button starting the copy in operation. In
the 1980s, C. Langton and then J. Byl showed that in fact much smaller
automata can in fact self-reproduce.

The DDLab manual by
Andy Wuensche with many information about CA, discrete dynamical networks
and their attractor basins.

Complex Networks

In CA there was interaction possible only between neighbouring cells in a
spatial matrix. But the interaction between active parts of a system could
be generally described by a network where the active components are
represented by nodes and the interactions by edges. Complex networks are a
subgroup of networks with "interesting" properties. Natural and social
networks are often complex.

Motivation for the study of complex networks:

Complex networks are almost everywhere

Many complex networks have similar properties

Structure of interactions affects the resulting dynamics

Examples of complex networks

Human society

Social networks

Economics

Epidemiology

Collaboration and Citation networks

Spreading of innovations

Electrical grid

Internet

The 1318 transnational corporations that form the
core of the economy. (Vitali S. et al., 2011)

Basic notions of Graph theory

Degree of a vertex ki is the number
of edges connected with the node vi

Degree distribution P(k) is the distribution of probabilities
that a random vertex has a degree k

Clustering coefficient for an undirected graph:

The clustering coefficient Ci for a vertex
vi is given by the number of links
between the vertices within its neighbourhood (ejk)
divided by the number of links that could possibly exist between
them (In a directed graph ejk
is distinct from ekj, and therefore for each
neighbourhood Ni there are ki(ki - 1) links that could exist among the
vertices within the neighbourhood (ki
is the total (in + out) degree of the vertex). In undirected graphs
eij and eji are
considered identical. Therefore, if a vertex vi
has ki neighbours, ki(ki - 1)/2 edges could exist among the
vertices within the neighbourhood.

Some properties of complex networks:

Short average path between vertices

Specific degree distribution

High clustering coefficient (in small world networks and some
scale-free networks)

Types of complex networks

Types of complex networks (Huang, 2005)

Random network

Small World network

Scale-free network

Random networks

First models of complex networks

Developed in 1960s by Erdos and Rényi

Model

Add edges between nodes with probability p

Properties

Length of paths close to log n (where n is the number of
nodes)

Small changes of p can lead to sudden emergence of new
characteristics

Giant component (the biggest connected part starts to grow
very fast when p comes to 1/n)

Poisson degree distribution

Different from most real complex networks

Clustering coefficient close to p

Lower then in most real complex networks

Small world networks

Model of human society

According to Milgram´s experiments in 1967 there are only six
people between any two people in the network of people personally
knowing each other (six degrees of separation)

High regularity with few irregular connections leads to low path
lengths

Model

Arrange n nodes in a circle and connect k
neighbors together

Then add new edges (or rewire the existing ones) with probability
p

Properties

Length of paths close to log n

Degree distribution similar to random networks

High clustering coefficient

Scale-free networks

Very common in real complex networks but not omnipresent

Model

Start with few nodes and edges

Add new nodes and connect them with higher probability to nodes
with higher degree (the rich gets richer)

After a while there will emerge nodes with a very high degree
(hubs)

Properties

Short path lengths

Scale-free degree distribution

Comparison between the degree distribution of scale-free networks
(circle) and random graphs (square) having the same number of
nodes and edges. For clarity the same two distributions are
plotted both on a linear (left) and logarithmic (right) scale. The
bell-shaped degree distribution of random graphs peaks at the
average degree and decreases fast for both smaller and larger
degrees, indicating that these graphs are statistically
homogeneous. By contrast, the degree distribution of the
scale-free network follows the power law, which appears as a
straight line on a logarithmic plot. The continuously decreasing
degree distribution indicates that low-degree nodes have the
highest frequencies; however, there is a broad degree range with
non-zero abundance of very highly connected nodes (hubs) as well.
Note that the nodes in a scale-free network do not fall into two
separable classes corresponding to low-degree nodes and hubs, but
every degree between these two limits appears with a frequency
given by P(k). (Albert, 2005)

Computer networks with scale-free architectures, such as the World
Wide Web, are highly resistant to accidental failures. But they are
very vulnerable to deliberate attacks on hubs.

Eradicating viruses, even known ones, from the Internet will be
effectively impossible.

Medicine

Vaccination campaigns against serious viruses, such as smallpox,
might be most effective if they concentrate on treating hubs--people
who have many connections to others. But identifying such individuals
can be difficult.

Mapping out the networks within the human cell could aid researchers
in uncovering and controlling the side effects of drugs. Furthermore,
identifying the hub molecules involved in certain diseases could lead
to new drugs that would target those hubs.

Business and politics

Understanding how companies, industries and economies are
interlinked could help researchers monitor and avoid cascading
financial failures.

Studying the spread of a contagion on a scale-free network could
offer new ways for marketers and politicians to propagate their
products and ideas.

Very short introduction to modeling methodology

Models are always “wrong” but sometimes could be useful! (Georg
E. P. Box)

All models are abstracted and simplified (like a map of a landscape).

We describe only those parts of reality which are important for us
and only to the extent allowed by our technical limits.

Useful models could help us to get insight into the structure and
behaviour of reality.

Bad models don't tell us anything new and only waste our time
(that’s the better alternative) or can lead to bad prediction about
the reality

Logic and modeling:

Deduction – we know the principles and try to predict the
system behavior. (Example: How will the electricity market behave
during the year?)

Induction - we know the behavior and search for
underlying fundamental principles of system dynamics. Is the model
robust? Does the model lead to the same or similar behavior for a
large range parameter values? (Example: How does the stability arise
in a predator-prey ecosystem?)

Abduction – we search for the best explanation (basic
assumptions and parameters) of specific interesting results.
(Example: When will the stock market tend to crash?)

A different look at logical relationship between a multiagent
model and reality:Axelrod (2003) points out: “like deduction model starts with a set
of explicit assumptions. But unlike deduction, it does not prove
theorems. Instead, a simulation generates data that can be analyzed
inductively”. Induction comes at the moment of explaining the behavior
of the model. It should be noted that although induction is used to
obtain knowledge about the behavior of a given model, the use of a model
to obtain knowledge about the behavior of the real world refers to the
logical process of abduction. Abduction, also called inference to the
best explanation, is amethod of reasoning in which one looks for the
hypothesis that would best explain the relevant evidence, as in the case
when the observation that the grass is wet allows one to suppose that it
rained. (Encyclopedia of Complexity)

Steps of modeling:

What exactly is our problem and what do we want to achieve with the
model?

Do we need a model at all?

Are there already similar models?

Choosing the scale

Space – what is the basic part of the system?

Time – what does represent the basic step of the simulation and
how far in the future we want to predict the behavior?

Choosing the aspects of the reality we want to model (abstraction)

Extensive boundaries: How many aspects of reality to include in
the system

Intensive boundaries: How detailed will be the description of
these aspects

Look for simplicity. Always start with simple models and gradually
add new features.

Remember: Without simplicity you will get stuck in tons of
data but too simple models can lose the connection with reality.

What are the key parts, processes and parameters of the model?

Choosing appropriate description and representation of the model

Choosing the modeling tools

Verification - check if the model does what we suppose it should
do.

Validation – check if the model behaves in accord to the reality

“Playing” with the model – repeated executing of the model, changing
parameters or other aspects and observing their effects on the model
behavior

For stochastic models statistical analysis of the results is
necessary!

This means running the models several times for the same
parameters, gathering the data and analyzing them in Excell, R or
other statistical package.

The amount of gathered data can be quite large

Agent-based models (ABM)

Agent

The term "agent" means an active, autonomous and situated unit.

Autonomous

Agents are not directed by an external central unit

Situated

Agents interact in and with some kind of shared environment (for
example movement is a type of interaction with environment)

Agents interact locally (in space or in a network)

Reactive agents could be relatively simple (without ability to
learn or sometimes even without memory) but they don't need to be
homogenous

Deliberative (or Intelligent) agents have memory and can have a
rich symbolical representation of the enivronment, they can use
classical metohods of artificial intelligence (machine learning,
neuronal networks, genetic algorithms) to make complex decisions, they
can adapt on the changing environment and actions of other agents

Relation between ABM and Multi-agent systems (MAS)

ABM are a subclass of Multi-agent systems (MAS). Typically in MAS agents
could be biological or artificial entitities situated in a real world like
a group of animals, group of cooperating robots or virtual entities
situated in a non-simulated environment like software agents acting in a
computer network. In ABM agents are typically software objects (inter)
acting in a simulated environment. ABM could be interpreted as models of
real-world MAS.

Characteristics of ABM:

Bottom-up approach (From basic parts to complex interactions - the
macro parameters are result of intaractions on the micro level.)

Time is discrete.

Basic building blocks are represented by agents (individuals).

Agents are defined by their parameters and recurrent functions
which define the behaviour of the agents.

The behavior is essentially the change of parameters in every step
of the model.

This change depends on the values of the parameters of the agent,
the parameters of other agents and on the local and global
parameters of the environment.

Agents could be adaptive - they change their behaviour in response to
the environmental change.

In some models agents could die and new agents could be introduced.

When to use ABM:

Complex, non-linear or discrete behavior and interaction of agents

Non-homogenous and boundedly rational population of agents

Interaction is local and dependent on some spatial or social structure

Which features of real complex systems can we better understand with the
help of ABM?

Self-organized behavior and decentralized management

Robustnes and phase transition

Logic and modeling

In every model there are present aspects of deduction, induction and
abduction. But according to the questions we ask the emphasis could
be on different types of logical reasoning.

Deduction – we know the principles and try to predict the
system behavior.

Induction - we know the global behavior and search for
underlying fundamental principles of system dynamics. Is the model
robust? Does the model lead to the same or similar behavior for a large
range parameter values?

The tournament of different playing strategies surprisingly
showed that the best strategy for IPD is very simple:Tit for Tat: If you cooperate I also cooperate, if you don't cooperate
I also don't cooperate.

If we encode the strategies into a simple genome and evolve
competing populations of these strategies the Tit for Tat strategy will
evolve spontaneously and become dominant for a reasonable range of
rewards.

Other examples:

The evolution of cooperative behavior in spatial environment, see Netlogo
model

ABM and micro-economical models share the bottom-up approach but in
other aspects they substantially differ.

In the process of formalizing a theory into mathematics it
is often the case that one or more — usually many! — assumptions
are made for purposes of simplification; representative agents
are introduced, or a single price vector is assumed to obtain in the
entire economy, or preferences are considered fixed, or the payoff
structure is exactly symmetrical, or common knowledge is postulated to
exist, and so on. It is rarely desirable to introduce such
assumptions, since they are not realistic and their effects on the
results are unknown a priori, but it is expedient to do so. ...
it is typically a relatively easy matter to relax such ‘heroic’
assumptions-of-simplification in agent-based computational models:
agents can be made diverse and heterogeneous prices can emerge,
payoffs may be noisy and all information can be local.(Axtel, 2000)

Huang, Chung-Yuan, Sun, Chuen-Tsai and Lin, Hsun-Cheng (2005). Influence
of Local Information on Social Simulations in Small-World Network Models.
Journal of Artificial Societies and Social Simulation 8(4)8
http://jasss.soc.surrey.ac.uk/8/4/8.html.