Welcome to our September issue. The main
feature this month continues August's AI-in-Python theme
with a look at Python for robotics and the Pyro
robot-control software. We also have a selection of
quotes,
and some computer-generated humour. As ever, comments and suggestions are welcome:
please mail popx@j-paine.org.

I came across a few of the quotes below while
looking up references for another article.
It's an article that
hasn't yet worked out, but I thought it
would be fun to use them to trace a path
through the past - and perhaps future - development of
AI. So here goes:

"Many smart people have been thinking about the AI problem for a long
time. There have been many ideas that have been pursued by
sophisticated research teams which turned out to be dead ends. This
includes all of the obvious ideas. Most grand solutions proposed
have been seen before (about 70% seem to be recapitulations of
Minsky proposals)."

"There has been a long-standing opposition within AI between 'neats' and 'scruffies' (I think the
terms were first invented in the late 70s by Roger Schank and/or Bob Abelson at Yale
University).
The neats regard it as a disgrace that many AI programs are complex, ill-structured, and so hard
to understand that it is not possible to explain or predict their behaviour, let alone prove that they
do what they are intended to do. John McCarthy in a televised debate in 1972 once complained
about the 'Look Ma no hands!' approach."

"Conrad Barski from Minneapolis sent me an action shot of the
John McCarthy Lisp t-shirt. He writes: '...
and since the portrait of John McCarthy is so uncanny, there was no need to explain the shirt to anyone in the audience.'"

"Lisp has jokingly been called 'the most intelligent
way to misuse a computer'. I think that description is a great
compliment because it transmits the full flavor of liberation:
it has assisted a number of our most gifted fellow humans
in thinking previously impossible thoughts."

"Elegance is unnatural, only achieveable at great expense. If you just do something,
it won't be elegant, but if you do it and then
see what might be more elegant, and do it again, you might, after an unknown number of
iterations, get something that is very elegant."

slashdot.org/article.pl?sid=01/11/03/1726251
From a Slashdot interview with
Lisp and Scheme implementor
Kent Pitman. He replies that
"Ironically it's non-Lisp languages that allow and
encourage you to put ()'s in any place you want,
as if there were no meaning to the introduction of
gratuitous paren groups."

"As the release of AutoCAD 2.1 loomed closer, we were
somewhat diffident about unleashing Lisp as our application language.
This was at the very peak of the hype-train about expert systems, artificial
intelligence, and Lisp machines,
and while we didn't mind the free publicity we'd gain from the choice of
Lisp, we were afraid that what was, in fact, a very simple macro language
embedded within AutoCAD would be perceived as requiring arcane and
specialised knowledge and thus frighten off the very application
developers for whom we implemented it.
In fact, when we first shipped AutoCAD 2.1, we didn't
use the word 'Lisp' at all - we called it the
'variables and expressions feature'. Only in
release 2.18, in which we provided the full
functional and iterative capabilities of Lisp, did we
introduce the term 'AutoLisp'."

"'AI winter' is the term first used in 1988 to describe the unfortunate commercial fate of AI.
From the late 1970’s and until the mid-1980’s, artificial intelligence was an important part of the
computer business - many companies were started with the then-abundant venture capital available
for high-tech start-ups. By 1988 it became clear to business analysts that AI would not experience
meteoric growth, and there was a backlash against AI and, with it, Lisp as a commercial concern.
AI companies started to have substantial financial difficulties, and so did the Lisp companies."

"The scruffies regard messy complexity as inevitable in intelligent systems and point to the failure
so far of all attempts to find workable clear and general mechanisms, or mathematical solutions
to any important AI problems. There are nice ideas in the General Problem Solver, logical
theorem provers, and suchlike but when confronted with non-toy problems they normally get
bogged down in combinatorial explosions. Messy complexity, according to scruffies, lies in the
nature of problem domains (e.g. our physical environment) and only by using large numbers of
ad-hoc special-purpose rules or heuristics, and specially tailored representational devices can
problems be solved in a reasonable time."

Must Intelligent Systems Be Scruffy?

"In rule-based, or expert systems, the programmer enters a
large number of rules. The problem here is that you cannot
anticipate every possible input. It is extremely tricky to be sure you have
rules that will cover everything. Thus these systems often break
down when some problems are presented; they are very 'brittle'. Connectionists
use learning rules in big networks of simple components - loosely inspired by
nerves in a brain. Connectionists take
pride in not understanding how a network solves a problem."

www.aaai.org/AITopics/html/reason.html
Marvin Minsky, from
Scientist on the Set: An Interview with Marvin Minsky, in Hal's
Legacy, edited by David Stork, 1996. Quoted on the AAAI
Reasoning page.

"Despite all the progress in neural networks the technology is still brittle and sometimes difficult to apply."

"All too soon, however, the hopes
kindled by AI's second age dimmed as well.
Using chips and computer programs, scientists
built artificial neural nets that mimicked the
information-processing techniques of the brain.
Some of these networks could learn to recognise
patterns, like words and faces. But the goal of a
broader, more comprehensive intelligence remained
far out of reach.
And so dawned the third age of AI.
Its boosters abandoned hopes of
designing the information-processing
protocols of intelligence, and tried
to evolve them instead. No one wrote
the program which controls the walking of
Aibo, a $1,500 robotic dog made by Sony.
Aibo's genetic algorithms
were grown - evolved through many generations of ancestral code in a Sony laboratory."

"GAs are a terrific approach to searching large,
ill-defined spaces, in this case the space of
'nice' melodic ideas. There is also an analogy to the
'population' of licks that most jazz players have in their heads.
These licks come and go over time in a manner similar to evolution;
ideas that were cool in the
past become overused or cliched, so I stop playing them."

"Dealing with ES is sometimes seen as 'strong tobacco', for
it takes a decent amount of probability theory and applied STATISTICS
to understand the inner workings of an ES, while it navigates through
the hyperspace of the usually n-dimensional problem space, by
throwing hyperellipses into the deep..."

www.faqs.org/faqs/ai-faq/genetic/part2/
From an account of the Technical University of Berlin's work on
Evolution Strategies, one of many detailed descriptions on evolutionary
algorithms in
part 2/6 of the comp.ai.genetic FAQ by
Joerg Heitkoetter and David Beasley, 1993-2001.

"It is raining instructions out there;
it's raining programs; it's raining
tree-growing, fluff-spreading, algorithms.
That is not a metaphor, it is the plain truth.
It couldn't be any plainer if it were raining floppy discs."

salmonriver.com/Books/dawblind.html
Quoted by
Naomi Sherer in her review of
The Blind Watchmaker: Why the Evidence of Evolution Reveals a Universe Without Design, Richard Dawkins, 1986.

"My optimism about the future of intelligent machines is based
partly on the evolutionary record. Nature holds the patents on high
intelligence. It invented it not once, but several times, as if to
demonstrate how easy it was. ...
The vertebrate retina has been studied extensively. Its 20
million neurons take signals from a million light sensors and combine
them in a series of simple operations to detect things like edges,
curvature and motion. Then image thus processed goes on to the much
bigger visual cortex in the brain.
Assuming the visual cortex does as much computing for its size
as the retina, we can estimate the total capability of the system.
The optic nerve has a million signal carrying fibers and the optical
cortex is a thousand times deeper than the neurons which do a basic
retinal operation. The eye can process ten images a second, so the
cortex handles the equivalent of 10,000 simple retinal operations a
second, or 3 million an hour."

It sometimes seems to me that the brain is actually a
very shitty computer. So why would you want to build a
computer out of slimy, wet, broken, slow, hungry, tired
neurons? I chose computer science over medical school
because I don't have the stomach for
those icky, bloody body parts. I prefer my technology
clean and dry, thank you. ...
The brain has to sleep, needs food, thinks about sex all the time. Useless!
I always say, if I wanted to build a computer from scratch, the
very last material I would choose to work with is meat. I'll
take transistors over meat any day. Human intelligence may even be a
poor kludge of the intelligence algorithm on an organ that is
basically a glorified animal eyeball."

"I claim that the soul, spirit, or consciousness may
exist, but for most people, most of the time, it is
almost infinitesimally small, compared with the robotic
machinery responsible for most of our thought and action. ...
That's not to say that some people can't be more enlightened than
others. But for the vast herd out there, on average, consciousness is
simply not a significant factor. Not even a second- or third-order effect.
Consciousness is marginal.
I say this with such confidence because of my experience
building robot brains over the past seven years. Almost
everything people ever say to our robot falls into one of
about 45,000 categories. Considering the astronomical number
of things people could say, if every sentence was an original
line of poetry, 45,000 is a very, very small number."

Richard Wallace.

"Asp, a Swedish researcher who once majored in industrial design, volunteered for the
fMRI probe. The scanner revealed a personality quite at odds with her own sense of self.
She searched the scanner's images for the excited neurons in her prefrontal cortex that
would reflect her enthusiasm for Prada and other high-fashion goods. Instead, the scanner
detected the agitation in brain areas associated with anxiety and pain, suggesting she found
it embarrassing to be seen in something insufficiently stylish.
It was fear, not admiration, that motivated her fashion sense."

"AI is much more likely to be a boon than a
threat to humans. In many ways one can best describe AI
technology as the development of what my colleague Ken
Ford calls 'cognitive prostheses': systems that people
can use to amplify their own intellectual capacities. Such tools
empower people and aid in removing social barriers. To
dramatize the point: about a hundred years
ago, rapid mental arithmetic was considered
an impressive intellectual talent, and people who could do it
received academic honors. Nowadays a high-school dropout at a
supermarket checkout can tell the customer the total charge in a
fraction of a second. A barcode scanner and a computer read-out
act as a mental amplifier enabling someone to perform a
task that, without it, would require greater mental capacity
than he could deploy unaided. True, we don't usually say that
the supermarket checkout clerk is using this machinery to think with; but ask yourself:
who is earning the wages, the human or the computer?"

Karel was introduced by Richard Pattis
in his book
Karel the Robot - A Gentle Introduction to the Art of Programming.
His book isn't on the Web, but I did find a Karel-based
course for C,
by Roland Untch. This
introduces us to
Karel, who lives in a grid of streets and walls.
Scattered throughout this grid are beepers, which Karel can
sense, pick up, and put down.
Students learn to program by instructing Karel to
perform assorted tasks, using
commands
such as TurnLeft() and
PickBeeper(). This highly imperative
style of programming is - I imagine - one
that students find easy to get started with.
However, the authors of
Avoiding the Karel-the-Robot
paradox assert that it eventually leads students to
a programming dead-end.
Similarly, they say, although inexpensive robots have made introductory
AI accessible to a wide range of
school and university students, they have led to a
robotics dead-end.

One problem is portability. There are many
robots on sale, but each tends to have its own programming
language and development tools, often very
different from those of other robots. This
make it difficult for students to transfer not
just code, but also programming techniques.

Also, many robot programming
systems are restricted in the sensors they support.
For example, many low-cost robots are often supplied
with infrared range sensors only. You might be
able to add something more
sophisticated such as a sonar or laser range sensor; but
even if your educational budget can afford this,
you may not be able to access the sensor from the software.

So,
widespread use of robots for teaching AI
needs not just cheap hardware, but also
control software that
can be ported to many different robots and make them
all look identical to the student.
That's Pyro's goal:
write-once/run-anywhere
robot programs. Then students can concentrate
on building robot brains.
Also, as they learn, they will be able to
gradually move up to
more and more sophisticated
robots. And such robots, if the software is
capable enough - and Pyro should be -
will be usable in research as well as teaching.

Pyro runs on Unix and Linux, but
according to the Pyro FAQ,
may also work with other operating systems. A
LiveCD is available; and
Zach Dodds has made a Windows implementation,
PyroWin.

The Pyro library includes modules for
various
robot control paradigms, robot learning,
robot vision, localization and mapping, and multiagent
robotics. The robot control paradigms include modules for
direct control, finite state
machines, subsumption architecture, fuzzy logic
control, and neural network control:
feedforward, recurrent, self-organizing
maps, other vector quantizing algorithms.
There are also genetic
algorithms and
genetic programming. The vision modules provide a
library of the most commonly used filters and vision algorithms
enabling students to concentrate on the uses
of vision in robot control. All this is open source:
it can be modified, and students can learn by looking
at the code. (The documentation is also open source,
available under a
Creative
Commons licence.) Modules planned for the future
include logic-based
reasoning and acting, classical planning, and path planning
and navigation.

One Farside cartoon depicts
two amoebae sitting in front of a television.
The female amoeba, sporting typical Larson nagging-wife upswept glasses,
is glaring
at the male amoeba and shouting
"Stimulus, response. Stimulus, response.
Don't you ever think!". If stimulus-response
control is low on the evolutionary ladder, it's
also easy to teach: let's start there, with an example that's
reprinted in several of the papers about Pyro, including
The Pyro
toolkit for AI and robotics:

Here, we're defining a robot "brain". These have to
be subclasses of class Brain. This
one is class Avoid: in Python, although
it might look like some kind of procedure call, the
code

class X(Y)

defines new class X to be a subclass
of Y.

Every Pyro
brain needs a step method,
which Pyro executes on every
control cycle. The one above
makes the robot continually wander, turning
as a direct response to its range sensor
if it has got too close to an obstacle on either side.

The authors emphasise that this program
does not depend on the robot or
range sensor.
it's also independent of the robot's length, since
Pyro translates sensory and motor data to multiples of length,
and
will avoid obstacles when they are within one robot length
of the front-left or front-right range sensors,
whatever that happens to be.

Let's move on to
a robot controlled by a finite-state machine.
The robot's job is a bit of simple recycling,
picking up and storing cans. The authors use
a simulated Pioneer robot
with gripper and "blob" camera, discussed in
the next section, on vision.
Cans are represented as randomly positioned red
pucks in a circular environment without obstacles. The
robot's goal is to collect all the red cans. Once
it has picked up a can, it stores it and looks for
more cans.

The finite-state controller has four states:
locateCan, approachCan,
grabCan, and done.
Each state corresponds to a particular
behaviour: it
is triggered by some condition in the environment, tries
to handle the condition, and may then move to
another state.

The controller starts in state
locateCan. In this state the robot rotates, looking for a blob
which
would mean a red can is
in sight. If it finds a can, the controller switches to
state approachCan to move the robot toward the closest
visible can. (If the robot loses sight of
the can, the controller returns to state locateCan.)
Once the robot has its gripper around a
can, the controller switches to state grabCan, making the
robot pick up and store the can. It then returns to
state locateCan to search for another can. This state
keeps track of how long it searches on each
activation of the state. If the robot has done a complete
rotation and not seen any cans, the controller switches to
state done and stops.

Here's the
locateCan state in Python.
As with the direct-control brain, each state
must
implement the
step method, called on every
control cycle. States use the
goto method to switch
to other states:

What about vision? As already mentioned,
Pyro has camera-interface and image-processing modules. Students
can write programs to implement vision algorithms,
such as colour histograms, motion detection, object
tracking, or edge detection.

For efficiency, the low-level vision library code
is written in C++,
but students can interactively use it to build
layers of filters in Pyro, calling the
computationally expensive C++ code while still having the benefits of
the high-level, interactive interface of Python.

The authors illustrate with
Aibo
looking at a ball and applying three filters to
the raw image:
colour matching, supercolour,
and blob segmentation. The colour matching
filter marks all pixels in an image that are within a
threshold of a given red/green/blue colour triplet. The
supercolour filter magnifies the differences between a
given colour and the others. For example, the supercolour
red filter makes reddish pixels more red, and the
others more black. Finally, the blob-segmentation filter
connects adjacent pixels of similar colour into regions,
computes a box completely surrounding the matching
pixels, and returns a list of these bounding boxes.
Students can use these filters without needing to worry about
the low-level image-processing details -
for example, detecting Aibo's ball by
finding the largest region matching its colour, then
drawing a bounding box around it. It's then easy
to program Aibo to move towards this region.

If you own an Aibo - surely the most popular of Pyro's robots -
why not consider Pyro as an alternative to
Sony's Open-R and other development tools?
As the examples from
Pyro's
Using the
Sony AIBO Robot page, commands are not difficult to write:

The getSensor gets data from one of Aibo's infra-red sensors, and the
setWalk
loads a gait.

Using the
Sony AIBO Robot
also mentions that two Aibo "brains" are available:
one for following a blob, and one which tries to kick a ball into
a goal. This indicates that, as one would expect, Aibo can be used with
Pyro's software for camera control and vision.

I suspect the ball-kicking brain is that described in
Ioana Butoi's dissertation
Find Kick Play: An Innate Behavior for the Aibo Robot.
This explains how Pyro was used to build Aibo software for recognising a ball
and a goal, and kicking one towards the other. Butoi
describes object-recognition
algorithms developed for the RoboCup competition, and also how to
stop Aibo falling over as it kicked the ball. Butoi had to devise
a stance in which Aibo could balance on three legs while kicking
with the fourth. A real dog might do that too (though in my experience,
it's more likely either to eat the ball or bite the experimenter); but a real dog would be
intelligent enough to constantly adjust its stance as its
fourth leg swings and kicks. Aibo isn't that clever, so
Butoi had to find a specially stable joint configuration for
it to balance on.

Tekkotsu
is an application development framework developed at CMU for
Aibo and other
intelligent robots. Like Pyro, it is intended for educational
use: how does it compare?

Pyro developer Douglas Blank
says in
Using the
Sony AIBO Robot and in
a posting
about the
Pyro-Tekkotsu
relationship that Aibo Pyro
actually uses part of Tekkotsu,
namely the Monitor - a set of servers
running on Aibo via which programs can transfer
sensor data, images, and motion commands.
That doesn't mean students need to learn Tekkotsu, though.
Blank goes on to say in his posting:

The main project of Tekkotsu offers a unique programming environment. If
I were going to land an Aibo on the moon, I'd probably use Tekkotsu to
control it. But for doing interactive teaching, and high-level scripting
and experiments in the lab, I'd use Pyro.
To give you an idea of the environments: In Tekkotsu, if you want to
change a line of code, you must recompile everything that depends on the
code (it is C++ code) using the provided cross-compiler. Then the code
is copied to the dog over ftp, the dog shuts down, and starts back up.
The whole process (compile + transfer + reboot) lasts at least a minute
on our machines. In Pyro, you simply press the "reload brain" button and
nearly instantly you are running the new code.

Pyro in general

www.cs.hmc.edu/~dodds/PyroWin/ -
PyroWin,
Pyro modified to run under Windows, by Zach Dodds.
"At some point, the official version of Pyro may run under
Windows out-of-the-box, and this page will disappear".

pyrorobotics.org/?page=PyroCurriculum -
The main Pyro Curriculum page. Links to course
notes on Pyro for behaviour-based control, neural nets, vision, and
other topics. Also links to two
slide presentations: the
AAAI 2005 overview (10 slides), and the
AAAI 2005 tutorial (118 slides).
These, particularly the tutorial, contain: examples of Python code, course topics,
and student projects; defects of Lego robotics;
diagrams of the Pyro architecture; pictures of the robots and simulators.

www.cs.pomona.edu/~marshall/papers/bringing_up_robot.pdf -
Bringing up robot: Fundamental mechanisms for creating a
self-motivated, self-organizing architecture, by
Douglas Blank, Deepak Kumar, Lisa Meeden, and James Marshall.
Interesting paper on self-organising maps for a hierarchical
control architecture, where each level "chunks"
sequences for use by the more abstract level above it.

Pyro versus Mindstorms and Tekkotsu

Quite by chance, I found the following in a book bought
second-hand from Oxfam some weeks ago:

A few years ago, Dr Graham Ritchie and Dr Kim Binsted
created a computer programme that could produce jokes.
We were keen to discover if computers were funnier than
humans, so entered five of the computer's best jokes into
LaughLab. Three of them received some of the lowest Joke
Scores in the entire database. Here are those failed puns:

What kind of contest can you drive on?
A duel carriageway.

What kind of line has sixteen balls?
A pool queue.

What kind of pig can you ignore at a party?
A wild bore.

However, two examples of computer comedy were surprisingly
successful and beat about 250 human jokes:

What do you call a ferocious nude?
A grizzly bare.

What kind of murderer has fibre?
A cereal killer.

So, jokes written by a computer are not particularly funny
to humans, but perhaps they would be hilarious to
other computers.

www.laughlab.co.uk/ -
LaughLab, created by Richard Wiseman, University of Hertfordshire,
in collaboration with the British Association for the Advancement of Science.

www.dcs.gla.ac.uk/~kimb/dai_version/dai_version.html -
A symbolic description of punning riddles and its computer implementation,
by
Kim Binsted and
Graeme Ritchie, 1994. Early paper, explaining the theory
behind such riddles as
"What do you give an elephant that's exhausted? Trunkquillizers",
and its embodiment
in the first
version of JAPE.

www.inf.ed.ac.uk/publications/online/0158.pdf -
The JAPE riddle generator: technical specification
by
Graeme Ritchie, 2003. The paper
contains formal definitions of JAPE-3's data structures, rules and procedures:
"the aim is to set out a formally precise, implementation-independent account of how
JAPE generates punning riddles. The reason for doing this is that experimental AI programs
are usually under-documented, making it difficult for other researchers to replicate
the work, or to know what theoretical claims are actually embodied in the implementation."

doc.utwente.nl/fid/1183 -
Humour Research: State of the Art, by
Matthijs Mulder and Anton Nijholt, Twente. A recent survey of humour theory and of
joke generators such as JAPE, the Light Bulb Joke Generator, and
Elmo, the Natural Language Robot. Includes a section
on resources such as WordNet.

groups.inf.ed.ac.uk/standup/papers/thepsychologist_0203omara.pdf -
What do you get when you cross a communication
aid with a riddle?, by Dave O’Mara and Annalu Waller.
Also in The Psychologist, volume 16, 2003,
this paper is published by
the STANDUP project (System To Augment Non-speakers Dialogue Using
Puns), which seeks to use humour to help
language-impaired children communicate.

www.aaai.org/AITopics/html/toons.html -
IAMAI's AI-toons. This AAAI cartoon page includes news on STANDUP
and other humour research. It explains that
"Kim Binsted had always had a love for making
people laugh and was part of the improvisational comedy team at school.
When her interest in physics and maths took her into artificial intelligence
she fell back on her comedy background to help her work on a few problems in computers.
Now, having created a programme where computers can generate their own puns, she works
on a system that uses comedy to help children learn a new language,
whilst still trying to fit a little improv in, in her spare time."