A RESEARCH BREAKTHROUGH REGARDING SHEEP??
A RECENT RESEARCH ANNOUNCEMENT ON BBC:
On 27th August 2014, a BBC news report claimed:
It seems the researchers had to use sophisticated technology to discover an
explanation that Dean Petters, a computationally informed Cognitive Science MSc
student, worked out simply by examining the problem (in 2001).

Note on Pre-SimAgent demos:
Items 10 and 12 below include demos using our pop11
tools before SimAgent was developed. Experience with those projects helped to
show the need for the SimAgent tools and the
RCLIB graphical
interface. If you would like to explore some pre-cursors to SimAgent take a
look at the demos near the end of this file. (Code for both still works.)

How the movies were produced

The first few movies were produced using techniques and scripts suggested by
Mike Lees at Nottingham University where he used SimAgent on a project
directed by Brian Logan. (Mike's
program, added to his SimAgent code, repeatedly paused and generated a
screendump of the current display. At the end of a run the image files were
merged to form a movie.)

Later movies were produced using the excellent Xvidcap
tool -- a far more convenient
method. All were generated using Poplog + the SimAgent toolkit running on either
a Sun computer or a PC with the linux operating system.

However, the .mpg movies produced originally don't seem to work with all modern
players, so webm versions have been added, all tested using the VLC player.

NOTE on uses of the mouse in the demos
These movies are non-interactive, though most record interactive
demonstrations.

When SimAgent is running on your machine you can move things in a demo with the
mouse. Most of the movies below were made from such runs, but you cannot
interact with the movie demos in the same way.

In some of the movies the mouse-movements produced by interacting with the
running program produce movements of picture objects without the mouse pointer
being visible. So the resulting movements may look mysterious.

Some of the later movies recorded with the xvidcap tool do show the mouse
pointer as well as the objects moved.

When the demos (apart from the first two, and the minder movies, produced before
RCLIB had been produced) are running "live", objects can be moved with the mouse
and their new locations in the simulated scenario (virtual world) will
immediately be sensed either by those objects or by others, or both, and some
behaviours may be changed as a result. All of this happens in a running virtual
machine, though the effects can be seen on a physical display that records what
happens in the virtual machine.

E.g. in the sheepdog demos, virtual sheep that have been put in the virtual pen
can be moved out using the mouse, and the virtual sheepdog will then notice and
try to get them back. The 'hybrid sheepdog' movies also include trees and sheep
moved with the mouse. Likewise some of the other movies.

However, since these are recordings, not 'live' demos, you will not be able to
use your own mouse to change anything!

All the graphical interaction depends on the use of the core SimAgent toolkit
augmented with the RCLIB graphical toolkit implemented as an extension
to Pop11 in Poplog. This handles not only the display of scenarios on the
screen, but also the interaction using mouse and keyboard.

The first two movies show the 'boids' (flocking) program and a tileworld
agent, provided by Mike Lees, at Nottingham, supervised by Brian Logan.

The flocking program shows agents 'flying' randomly at first then gradually
coming together in a flock. This may not seem obvious fora while because the
agents treat the display square as a toroid, i.e. the upper and lower edges are
joined, and the left and right edges are joined, so that an agent exiting across
an edge re-enters at the corresponding point on the opposite edge.

The tileworld movie (No 2) is harder to understand, but has some agents moving
around in a rectangular grid triggering various events.

A succession of sheepdog projects

Peter Waudby's "passive" sheepdog (not demonstrated here)

The third and fourth videos below, include a simulated sheepdog that makes use
of an earlier simulation, not shown here, in which the sheepdog was
mouse-controlled.

The earlier sheepdog was designed by Peter Waudby, a Physics graduate, for his
Cognitive Science MSc project, in 1996. The sheep are autonomous, and partly
unpredictable, unlike the dog. They react to the presence of the dog by trying
to move away from it. If they detect something in their path (another sheep, a
tree, or part of the pen) then tend to swerve to avoid it, a strategy that can
sometimes lead to a sheep getting jammed between two or three other things. The
sheep behaviours are mostly unchanged in all the later demonstrations, except
for one of the sheep in Dean Petters' program, described below.
That sheep has to "learn" to move away from the dog, as explained later.

The program remains available (slightly modified
by Aaron Sloman) as a demo included in the Poplog Simagent toolkit:
Pop11
SimAgent code and comments.
(Peter later went on to work for a (then) well known computer game company.)

The third and fourth demos use an extended version of Peter Waudby's
sheepdog scenario designed and implemented by Tom Carter, a philosophy graduate
with no programming experience prior to starting the Birmingham MSc in Cognitive
Science.

His project extended Peter Waudby's program by making the sheepdog autonomous,
but only able to herd one (randomly selected) sheep at a time, though a human
could use the mouse to move the dog, the sheep and the trees (obstacles) so as
to frustrate or help the dog in its task.

Tom's sheepdog has several sub-strategies, that get turned on an off
automatically according to the situation. It first selects a sheep, then, if not
already close to that sheep, moves until it is close to it, possibly avoiding
trees, the pen, or other sheep on the way -- or failing to avoid them!

When close to its chosen sheep it attempts to steer that sheep until it is
within the 'cone' of space emerging from the pen. It then changes strategy and
attempts to move the sheep into the pen. But if the sheep is moved out of the
cone, or the dog is moved away from the sheep, or a tree is moved to block the
path, the dog can (in many cases) change its behaviour, reverting to an earlier
stage if needed. The code for Tom Carter's sheepdog, slightly modified by Aaron
Sloman, is available
here, as part of the poplog teaching materials for SimAgent.

This sheepdog is purely reactive. Although it appears to be making plans and
carrying them out, it has no notion of any future state. Its behaviour at any
time is merely the result of following a rule that is triggered by (a) what it
perceives in the environment (b) what its current internal state is. Sometimes
what it perceives causes a switch to a new state, e.g. switching from a state in
which its actions move it towards a distant sheep to a new state in which it
steers a nearby sheep towards the pen. Tom's report gives more details. The next
sheepdog sometimes behaves reactively, sometimes deliberatively, making a
(possibly complex) plan to get to a new location. When carrying out the plan it
behaves reactively.

There is no learning in that sheepdog or in any of the others below. One
student attempted to use evolutionary computation to develop a successful
sheepdog, but the resulting animal was able only to heard one sheep if the gate
of the pen, the sheep and the dog lay on the same line in that order, initially.

The next sheepdog includes the reactive capabilities of the previous versions,
but adds deliberative capabilities that allow plans to be made several steps
ahead, even in a maze-like configuration of trees.

The fifth example, the hybrid deliberative/reactive sheepdog was an
introductory mini-project by Marek Kopicki done in November-December
2003, during his MSc course building on the earlier sheepdog programs.
Marek had done a degree in physics and began a PhD in physics, funded by part
time programming work. When he decided to switch to computing he came to
Birmingham to do our MSc. Although this was a degree in computer science, not
cognitive science, he was interested in learning some AI, so I supervised his
first term mini-project. This was the result.

Marek's program demonstrates a 'hybrid' deliberative/reactive version of Tom
Carter's
sheepdog presented earlier. The new sheepdog is able to find its way round far
more complex barriers than the purely reactive one can. It does this by
sometimes pausing to 'think' about where to go: i.e. it makes a plan, using the
clever "probabilistic roadmap" technique described below. Having embarked on a
plan, it does not blindly follow it: instead it can tell that something has
changed since the plan was made and there is a new better option (e.g. a short
cut) or a new obstacle. It sometimes proceeds by making a small adjustment to
the existing plan, and sometimes by making an entirely new plan.

So it alternates between simply reacting to what it senses while following an
existing plan and making a new plan: one of its internal reactive behaviours is
to switch to local plan-repair mode.

The lines shown on the display indicate the sheepdog's current plan, and while
it is moving it also displays its 'line of sight' to a later plan location. This
is used for smoothing the plan, and for detecting new short-cuts.

So the dog combines interleaved local re-planning and reactive plan execution
with minor (smoothing) adjustments, occasionally having to switch to global
re-planning because the current plan has met an obstacle.

NOTE:

This illustrates a type of deliberative competence. There are several
different types of deliberative competence some of them more sophisticated than
the behaviour shown here. For more information on the possible varieties see:

This shows the sheepdog fetching the sheep one at a time and steering them to the
pen. It repeatedly plans a route, displayed on the screen, then follows the route,
reactively adjusting its plan as new obstacles and opportunities are discovered,
resulting from objects (trees, sheep, dog) being moved with the mouse. Sometimes a
new obstacle turns up that requires global re-planning.

You can see the mouse moving things in the display. (The mouse pointer is a black
arrow.)

The dog may have to create a new plan when something changes, or modify its existing
plan, or modify how it executes the plan, e.g. taking a short cut that either did not
exist when the plan was created or was not noticed when the plan was created, because
the plan-creation process does not aim for an optimal plan, just a reasonable one.

Searching for an optimal plan would take very much longer. Notice that in some sense
the dog is conscious of where the sheep are, where the trees are, whether it has been
moved, whether a short cut is available. Its range of visibility extends over the
whole terrain (as if it could see through trees), but that does not mean it notices
every relationship between things it sees.

Another demonstration of the same program, this time showing the 'probabilistic
waypoints' used by the sheepdog to make its plans: their use hugely reduces the
space of possible plans. As a result the dog can find a plan very quickly by
searching the graph of non-obstructed connections between the waypoints. However
the plans thus found are often non-optimal and the sheepdog reactively discovers
opportunities for improving them by smoothing them as it acts on the plans.

The plan construction is very fast, using the technique of 'probabilistic waypoints'[*],
(also known as 'Probabilistic Road Maps').

The potential search space for paths is astronomically large because the
sheepdog has a location and heading expressed in 'real' numbers (i.e. decimals
-- the exact precision depending on the machine architecture used). The image
displayed is about 660x660, but the possible x and y coordinate values along a
horizontal or vertial line will be very much larger than that, resulting in a
truly huge space of possible paths.

But instead of searching in that huge space of all complete paths in the
terrain, the dog randomly generates a collection of 'waypoints' rejecting all
except those that are close to but not too close to other objects, and searches
for paths from the current location to the goal location going only through the
waypoints, using non-obstructed straight lines joining the waypoints. (The
number of waypoints selected is a parameter set by the programmer. So the
program randomly generates a pair of coordinates in the permitted range, and
discards the pair if the location is too close to one of the objects in the
scene, or too far from the nearest one. So it might do that until it has found
250 points, a parameter set by the programmer. It then considers paths linking
all pairs of points and rejects those that pass through or too close to
obstacles. The remaining set of points determines a not very large graph of
possible motion paths in which a planning program an search for an optimal
route.

This dramatically reduces the search space for the planner, compared with considering
all possible routes in the terrain. As a result, on a 1Ghz PC running linux poplog
most of the plans shown are found almost instantaneously -- though it is not
guaranteed to find the shortest plan and may miss some narrow gaps.

The movie shows the waypoints generated just before each plan is formed, in addition
to showing the selected plan. The plan may be somewhat jagged because the waypoints
are not necessarily optimal because they are randomly generated. These detours are
smoothed as the plans are followed. The movie shows how, while executing a plan the
sheepdog can reactively detect an unexpected obstacle or problem produced by the
sheep moving in an unexpected way. It can also detect a new opportunity for short
cuts e.g. because a tree has been moved out of the way.

In this movie some of the objects were moved using the mouse, sometimes making things
worse for the sheepdog, sometimes allowing new short-cuts. However, the mouse is not
shown.

As before, some of the newly detected problems and opportunities require only local
modifications to the current plan, whereas in some cases an unexpected obstacle (in
one case another sheep!) causes global re-planning.

Program code for the hybrid sheepdog

The code for the hybrid sheepdog, used to produce the video, is now included as a
demonstration library in the SimAgent package.

The toolkit, and the hybrid-sheepdog should work in any version of poplog running on
a linux/unix machine with the X window system, and also on a windows PC with
Vmware,, or using Poplog in combination
with Slackware7 on a bootable USB stick.

The sixth example shows a rather old 'Feelings' demo which is one of the early
tutorial examples in the SimAgent toolkit. (I think the original version was
available around 1996, or possibly a little later.)

Further examples come from Dean Petters' MSc project in 2001 (sheepdog herding
several sheep, one of which has to 'learn' to get away from the sheepdog -- not
shown here) and some exploratory programs written during his PhD modelling
'attachment' processes in infants.

This movie shows two "toy" emotional agents partly moving of their own accord (each
keeps trying to get to its 'target' of the same colour while avoiding obstacles), and
partly being moved with a mouse (not shown in the movies).

They each show, in faces at the bottom of the display, a set of 'emotional'
states, teasingly labelled 'glum', 'surprised', 'neutral', and 'happy'. Each
agent also prints out in the text stream below its face the results of its
introspections as to what it feels and why.

The details may be easier to take in if you can slow down the video player
(e.g. VLC allows this, at least on linux). Otherwise it may be worth repeatedly
using the pause button to check on what has just happened that could explain the
changes in emotional state. Unfortunately this recording was made without
displaying the mouse pointer that occasionally moves one of the agents,
obstacles, or targets, triggering a 'surprise' reaction.

Of course, this is just a shallow toy developed for tutorial purposes rather than
anything that could be said to have human-like emotions -- except insofar as some
(but not all) human emotions involve reacting to things that achieve or hinder goals
or desires. However the notion of 'surprise' in this demonstration is pure fake,
since these simple creatures have no expectations and therefore cannot be surprised.
But the change in facial expression is fun.

If you fetch and install the package on a machine on which poplog can run, e.g.
PC with linux, then you can run the program at varying speeds and play with it
-- frustrating or helping the little movers. For a student learning programming,
enriching the behaviours, e.g. by giving the movers richer minds, could be a
useful educational experience.

These agents differ from many so-called emotion simulations in that these creatures
have some self-knowledge so that they not only show their state (in facial expressions
and in movements) but also describe them (which humans cannot always do!)
This demo uses, in a simple way, features of
the SimAgent toolkit
that allow 'self monitoring' activities to run in parallel with others.

Some of the events are caused by objects moved with the mouse while the demonstration
was recorded. Unfortunately you will not be able to interact in the same way with the
movie of the program running.

Like the sheepdog this simple demonstration is potentially indefinitely extendable to
include more complex and sophisticated kinds of processes involving learning,
reasoning, communicating, etc. The whole package, including poplog is freely
available for anyone who wishes to use it for teaching, research or anything else.

There are many "more serious" papers and presentations concerning emotions, other kinds
of affect, and architectures in the Cognition and Affect web site, e.g.

The Simworld package by Matthias Scheutz, now at Tufts University, USA

After spending a year here in Birmingham in 2000-2001,
Matthias Scheutz used
the SimAgent toolkit for teaching and research while he was at The University of
Notre Dame (USA). (Now at Tufts University).

His Simworld package is
available here
as a gzipped tar file that can be run in a poplog environment. A paper on a
later development based on SimWorld is

Two movies show the sheepdog herding several sheep
simultaneously, using a program produced by Dean Petters in 2001, his MSc year.
One of the sheep has to learn to move away from the dog in order to avoid
'unpleasant emotions' that occur when it is close to the dog. The other sheep
instinctively move away from the dog. The dog is not able to herd all the sheep
until the "emotionally impoverished" one becomes normal! (This is not shown in
the movies here.)

This is (as far as I know) the
only sheepdog program produced so far with this toolkit that enables the dog to
move several sheep simultaneously.

In these recordings the mouse is used to move the sheep and the dog, sometimes
helping the dog, sometimes making its job harder.

http://www.theguardian.com/uk-news/2014/aug/27/sheepdogs-replaced-by-robots
Including the claim that what had been discovered could be used to program a
robot for herding sheep, humans or other robots:
.....
Writing in the Journal of the Royal Society Interface, the researchers
concluded: "Our approach should support efficient designs for herding
autonomous, interacting agents in a variety of contexts.

"Obvious cases are robot-assisted herding of livestock, and keeping animals away
from sensitive areas, but applications range from control of flocking robots,
cleaning up of environments and human crowd control."

It seems the researchers had to use sophisticated technology to discover an
explanation that Dean Petters, a computationally informed Cognitive Science MSc
student, worked out simply by examining the problem (in 2001).

Dean stayed on to do a PhD, jointly supervised by a clinical-developmental
psychologist, Dr Gill Harris, which included the work reported in the next
section.

A project modelling some aspects of 'attachment' in infants.

The next two movies (added 24th April 2004) made use of a still incomplete
program being developed by Dean
Petters as part of his PhD work on modelling
'attachment' processes in infancy.
(Petters, 2006)

The movies show fragments of a scenario with a carer and a baby, both moving
around in an environment containing food and other things, viewed by from above
by a downward-facing (simulated) camera. There is a dynamic histogram showing
the changing measured states of the baby. Everything is highly simplified of
course.

The mother needs food from time to time, and the baby needs to explore, and
to get attention from the mother. Sometimes the mother oscillates between going for
food and attending to the baby. Different 'personalities' can be given to the mother
to show how that affects the development of the baby.

Both movies show several aspects of the baby's internal state going up and down.
The column labelled 'a_sec' corresponds to insecurity, produced by the mother
being too far away or not looking at the baby. At the time this was recorded
this was still work in progress towards the PhD thesis completed (part time) a
few years later. The work was recognised by a leading attachment researcher in
the USA, who began a collaboration with Dean as a result.

This was originally proposed around 1986 by A.Sloman (while still at Sussex
Universit) as a framework for investigating architectural issues in complex
multi-functional intelligent agents with multiple, changing sources of
motivation, embedded in rich and dynamic, only partly known environments.

In 1994, while working towards his PhD, Ian Wright implemented some of the ideas
developed by Luc Beaudoin in a version of the nursemaid scenario referred to as
'Minder Version 0'. The program was shown in a BBC2 Television interview with
A.Sloman, broadcast on February 1997. Since then the program has had its 'cosmetics'
improved by A.Sloman, and the new version is used for the movie demos below.

The movies show the 'minder' indicated by a capital 'M' looking after variable
numbers of 'babies' labelled 'a', 'b', 'c', etc. whose charge state is indicated by
an '*' when fully charged and a decreasing single digit number as the charge
decreases. The babies move around at random, using up energy and reducing their
charge level. There is a recharge point (represented as two curved arrows) to which M
can carry babies when their charge gets dangerously low. They die if their charge
gets to 0. At the top and bottom edges of the nursery are ditches and if babies get
too close they risk falling in and dying. M notices when a baby's charge level is
low, when a baby is near a ditch, and when a baby is dead, acquiring new motives in
each case. Noticing that a baby is dead generates a motive to carry it to the
'disposal' location, marked with a skull and crossbones! A further motive can be
triggered in M whenever there are three or more individuals in a room: that makes the
'room too crowded' and M acquires the motive to carry a baby to another room. M's
motives are prioritised (e.g. how low a baby's charge-level is, how close it is to a
ditch, etc., with disposal of dead babies as having lowest priority, followed by
reducing crowding).

The nursery has several rooms and M's knowledge of the contents of the rooms is
constantly updated by a camera that scans the rooms in turn. It is possible for M to
act on information about what is happening in a room that is out of date because the
camera has not refreshed M's view of the room.

Thus as babies move around and their energy levels change, M's motives keep changing
and sometimes M's work on a motive has to be abandoned in favour of a new higher
priority motive. M has very little intelligence, and completely lacks deliberative
capabilities. Thus all its plans are stored reactive plans, activated by motive
generators, and reactively executed or over-ridden. M also has very little knowledge
about itself, and often behaves in stupid ways, e.g. attending to a distant baby
before a nearby one. There is no learning. When there are only 5 or 6 babies M may be
lucky and keep them alive for a long time, though sometimes even that situation
proves too much. The more babies there are, the more harassed M becomes and because
it has no way of filtering new motives, they all grab its attention and can cause it
to behave in a far from optimal fashion. Later work by Ian Wright developed the
suggestion, in Luc Beaudoin's thesis, of using a dynamically varying threshold for an
attention filter. (A movie of his later program will be added to this site
eventually.)

Here are several runs of the Minder program, the core of which is unchanged since
1994, though the graphics have been altered. (The original showed both the 'real'
world and the world as perceived by the minder, and used only the editor buffer for
textual output, whereas this version shows only the real world and uses graphical
'posters' in RCLIB to display changes in motivation.)

This program does not use SimAgent, as it is implemented directly in Pop-11. However
the experiences and problems arising out of this work helped to define requirements
both for the RCLIB package, and the SimAgent toolkit, both developed later. Ian
Wright's implementation of Minder 1 for his thesis used the toolkit.

Ian Wright's Minder-1 program

The program produced by Ian Wright as part as his PhD work should be represented
here. (When I get time.)

A 'Toy' conversational robot
modelled loosely on T. Winograd's SHRDLU

The Poplog 'RC_BLOCKS' package is used to produce the saved image called 'gblocks',
demonstrated below. This makes use of a 'toy' demonstration program, assembled for AI
teaching purposes by a collection of lecturers at Sussex University in the 1980s,
designed to be extendable by students -- though not absolute beginners.

The package includes a core that makes use of the Pop-11 GRAMMAR library, used here
only in input mode. (Other tutorials use it for generating sentences.) It also uses the Pop-11
2-D interactive graphics package RCLIB.

RC_BLOCKS is very much simpler than Winograd's program as it uses a very simple
grammar and parser, but it does illustrate how a system can use the combination of
syntactic, semantic, and 'world' knowledge in understanding potentially ambiguous
sentences and also shows what some of the libraries that come with Poplog (and
therefore with SimAgent) can do.

The two movies show how the package deals with only two different input sentences,
combining knowledge of grammar and a lexicon along with a database of information
about the current situation, in order to resolve ambiguities in input sentences.

The package can handle questions and instructions (constrained by a very simple
grammar, vocabulary and ontology). However its current resources are much too simple
for use in any practical robot, though the general ideas could be useful. The tools
used can also be useful for rapid prototyping of a simple but usable natural language
interface to a robot (or other machine).

Two formats are provided for each of the two movies, mpeg and webm. The latter is
more likely to work on modern machines.

These two demonstrations use only the RCLIB subset of the SimAgent Toolkit, along
with the GRAMMAR library and its semantic extension FACETS. They also show how the
XVed editor can be used as part of an interface for textual interaction. There is no
use of POPRULEBASE or SIM_AGENT, though extensions of the demonstration would benefit
from their use. However, the Pop-11 DATABASE package is used.

Brian arrived in 1995, after development of the toolkit, in collaboration with
PhD students and other colleagues, had begun. He was an extensive user and made
many suggestions during his time (1995-2001) here in Birmingham working closely
with me (mostly on an exploratory project funded by DERA at Malvern), and later
as a temporary lecturer, and joint proposer for a Leverhulme project on
evolveable architectures. His contributions to the SimAgent toolkit continued
after he moved to Nottingham university in 2001.

His papers in that period can be found on his
publications
page. He not only contributed to the design of the SimAgent
toolkit and also use it for teaching, and in a multi-agent planning project in
collaboration with Jeremy Baxter and colleagues at DERA, Malvern. Brian and
Jeremy both made significant contributions, as expert users, to the design of
SimAgent, and helped to make it bug-free (insofar as it is!). Unfortunately we
have no videos of Brian's multi-level planning program (partly because it used
3D terrain maps with copyright restrictions).