TDDD48 Automated planning (6 ECTS)

Running Planners

Introduction

A large number of planners, both from the Seventh International Planning
Competition (IPC-2011) and from other sources, are available for use in
the labs. We indicate in the lab instructions which planners we have
tested and verified to work for a variety of example domains. The
remainder of the planners have been included as an additional bonus in
case you want to experiment with them on your own, but these planners are
currently entirely untested and a few of them may not even run. This will
not affect your ability to perform the labs, as you can always fall back
on the recommended planners.

Please note that even the planners we have tested and verified are in
general state of the art research prototypes and there is no guarantee
that they are free from bugs or that they work perfectly in all
domains. In fact, labs seem to be an excellent way
of finding strange bugs in planners, since you as students tend
to have a greater variety in your domain models than the researchers
developing the planners, who tend to follow their own "patterns".

So if you encounter problems with one planner, first check
the list of common problems at the end. Then try
other planners as well – this helps you see whether the problem lies
in the planner or in your domain definition. Also, please ask us
about problems at the labs or by e-mail. Never spend hours or days
trying to solve a problem!

Satisficing and Optimizing Planners

We distinguish between satisficing and optimizing
planners. Both types are available.

An optimizing planner should always return a plan that is optimal
in terms of some cost function defined on plans, no matter how much time
it takes to generate such a plan. For example, it may return plans with
the smallest number of actions, or one may define costs for each action
and generate plans with minimal total cost.

We might say that the "opposite" of an optimizing planner is one that
returns a plan as quickly as possible, regardless of how bad the plan is.
However, this is seldom what we want: We rarely prefer a plan of cost
10000 over a plan of cost 20, even if the first one takes slightly less time to
produce.

Therefore, satisficing planners generally try to find a good
balance between time and quality. Some planners generate a single plan
using a built-in heuristic tuned for both criteria. Other planners
provide more flexibility by generating an initial plan but then continues
to search for better plans until explicitly interrupted by the user,
either because the current plan is "good enough" or because one cannot
wait any longer.

In fact, the satisficing track of the 2011 planning competition rewards
those planners that yield the highest quality plans within a time limit of
30 minutes. Several of the planners below will therefore continue trying
to generate better plans until interrupted, as discussed above. We will
show later how to run these planners.

Using Crabbofix

Many planner developers today use Linux, as do the international planning
competitions. The resulting planners are generally not easy to compile in
Solaris or Windows environments. We therefore use a separate machine to
actually run the planners: crabbofix, a dual-CPU 8-core AMD
Opteron machine. Should this turn out to be insufficient, please tell us
as soon as possible and we will discuss other alternatives.

To run a planner, you need to use SSH. Assuming you are logged in on a
SunRay, simply open a terminal window and type:

ssh crabbofix.ida.liu.se

You can log in with your ordinary IDA student account and password, and
you will have access to the files in your IDA account. Among other
things, this means that you can edit files on a SunRay machine and
immediately use them on crabbofix.

You can also log into crabbofix from another machine. If you are not on
the university network, this may require first using SSH to log
into remote-und.ida.liu.se (which is available "from the
outside") and then doing "ssh crabbofix" from
there. Putty is a free SSH client for
those of you who want to log in from a Windows computer.

Finally, if you want to compile and run the planners on your own Linux
machine, the entire set of planners can be checked out
from svn://svn@pleiades.plg.inf.uc3m.es/ipc2011/data/planners
using Subversion. (Note that some planners require additional software.
We cannot provide support for planners installed on your own computers
– if you are having problems, use Crabbofix.)

Running Planners

The instructions in this section are valid for the IPC-2011 planners.
Other planners will be discussed separately.

In preparation for running the planners, there are some things you may
have to do:

unsetenv PYTHONHOME # If you use the standard tcsh shell
unset PYTHONHOME # If you use bash

This line ensures you do not carry over a PYTHONHOME from the Sun machines
that is not valid on Crabbofix, which interferes with certain planners
partially or entirely written in Python. We may add further instructions
here if it turns out other Sun modules (that you use, but we do not use
ourselves) also interfere with the planners.

The planners from IPC-2011 are installed under
/home/TDDD48/planners/ipc2011, in several subdirectories:

The names of the individual planners are shown in the tables below. For
example, the winner of the sequential satisficing (seq-sat) track in 2011
was lama-2011, while fdss-1 won the sequential
optimizing (seq-opt) track. These two planners can then be executed as
follows:

Other planners follow the same pattern. All planners take the same three
arguments: A domain file, a problem file, and a solution file (which may be
generated by the planner or may be ignored, depending on the planner). For
example (in a single line):

Many planners write quite a bit of text to standard output, and it may be
difficult to see everything. This is particularly problematic for those
planners that continually search for better plans: You may miss the fact
that one or several plans have actually been generated already. In this
case you may let less buffer the output:

Simply press space to continue to the next screen of output, or use the
arrow keys to go up and down. Beware: When the planner has written a full
screen of output, it may block, waiting for you to press space until it can
go on. You can't simply leave the planner running piped
through less and go through the pages of output later,
unfortunately.

All of the following planners generate sequential plans and support STRIPS
expressivity. Many also support some extensions, but beware that this
support may be incomplete in certain ways. Entries in parentheses denote
cases where a PDDL feature is supported by the planner, but where inputs
not using the feature are preferred (this may mean that the planner is
slower if the feature is used, for example). More information can be found
in the
official
list of planners in IPC-2011.

Name

NegativeConditions

ADLConditions

ConditionalEffects

UniversalEffects

Comments

acoplan

–

–

–

–

Reports syntax errors in correct files

acoplan2

–

–

–

–

Reports syntax errors in correct files

arvand

(yes)

(yes)

(yes)

(yes)

brt

(yes)

(yes)

(yes)

(yes)

Seems quite slow

cbp

yes

yes

yes

yes

cbp2

yes

yes

yes

yes

cpt4

–

–

–

–

dae_yahsp

–

–

–

–

Plans are written to files but not to standard output, so you can't see
whether a plan has been generated yet. Use another shell to list files and
read solutions!

The planner can find one solution, try to find a better solution, and
then say "No solution found" which then means "No more solutions found"!

lama-2011

(yes)

(yes)

(yes)

(yes)

Like lama-2008

lamar

(yes)

(yes)

(yes)

(yes)

lprpgp

–

–

–

yes

Not compiled

madagascar

yes

yes

yes

yes

Only writes plans to output file, not to standard output

madagascar-p

yes

yes

yes

yes

popf2

–

–

–

yes

probe

yes

–

–

yes

Not compiled

randward

(yes)

(yes)

(yes)

(yes)

roamer

yes

yes

yes

yes

satplanlm-c

–

–

–

–

Crashes

sharaabi

–

–

–

–

yahsp2

–

–

–

–

One of the planners that generates multiple plans in separate output
files

yahsp2-mt

–

–

–

–

Disabled, as it uses multiple CPUs and therefore interferes too much
with other users on the same server

In the planning competition, the planners finished in this order (best
first): lama-2011, fdss-1, fdss-2, fd-autotune-1, roamer, fd-autotune-2,
forkuniform, probe, arvand, lama-2008, lamar, randward, brt, cbp2,
daeyahsp, yahsp2, yahsp2-mt, cbp, lprpgp, madagascar-p, popf2, madagascar,
cpt4, satplanlm-c, sharaabi, acoplan, acoplan2. Note that many planners
were quite close in performance, and that a planner that is better for one
domain is often worse for another!

All of the following planners generate sequential optimal plans and support
STRIPS expressivity. One also supports some extensions, but beware that
this support may be incomplete in certain ways. More information can be
found in the
official
list of planners in IPC-2011.

In the planning competition, the planners finished in this order (best
first): dae_yahsp, yahsp2-mt, popf2, yahsp2, lmtd, cpt4, charaabi, tlp-gp.
Note that performance differs across domains. For one particular domain
cpt4 was the best, for a couple of domains, popf2 and lmtd (third and
fifth place) did considerably better than the other planners, and so on.
Note also that results relate to the quality of the best plan generated
within 30 minutes.

FF is a very well-known forward-chaining heuristic search planner producing
sequential plans. Although it does not guarantee optimality, it tends to
find good solutions very quickly most of the time and parts of the planner
have been used as the basis of many other planners.

FF searches the state space forward from the initial state, using a variant
of hill-climbing search and a non-admissible heuristic. The heuristic is
computed using a simplified form of the same graph construction used by
Graphplan and IPP, but FF does not actually search for plans in this graph.

In ordinary hill-climbing search, the best (according to the heuristic)
neighboring node is chosen. The hill-climbing variant used by FF works such
that if all neighboring nodes are worse than the current node, a
breadth-first search is started outwards from the current node and continued
until a better node is found. FF also uses some cut-off techniques so that
it does not have to consider all neighboring nodes when searching for a
better node.

Before starting search, FF performs some analysis to try to simplify the
goal. If the goal simplifies to the constant FALSE, the problem is provably
unsolvable and the planner exits.

Note that FF requires all the arguments to an operator to be different,
i.e. an action such as (move A B A) is
considered impossible regardless of the domain definition.

Running FF

FF is compiled for Linux and available on Crabbofix. Please make sure you
have read the earlier instructions on this page and know how to log in!
Running FF is similar to running any of the other planners that are installed:

/home/TDDD48/planners/ff/plan [args]

All planners take the same three arguments: A domain file, a problem file,
and a solution file (which may be generated by the planner or may be
ignored, depending on the planner). For example (in a single line):

About IPP

Note that IPP, like FF, requires all the arguments to an operator to be
different, i.e. an action such as (move A B A) is
considered impossible regardless of the domain definition.

Running IPP

IPP is compiled for Linux and available on Crabbofix. Please make sure you
have read the earlier instructions on this page and know how to log in!
Running IPP is similar to running any of the other planners that are installed:

/home/TDDD48/planners/ipp/plan [args]

All planners take the same three arguments: A domain file, a problem file,
and a solution file (which may be generated by the planner or may be
ignored, depending on the planner). For example (in a single line):

In this you must do "unset PYTHONHOME" in the shell before you run the
planner. (The problem is caused by an environment variable that may be
valid on the Sun system but causes Python to look for libraries in the wrong
location on Crabbofix.)

Do you get the following or similar output when you try to run a planner?

A PNE is a PrimitiveNumericExpression. This most likely means that you are
writing a cost-based domain with costs defined by a function, and that the
function does not specify the costs for all combinations of arguments.
Maybe you missed the case where a UAV flies from a location to the same
location?