5 Actions, events, and changePlanning requires a representation of timeto express & reason about sequences of actionsto express the effects of actions on the worldPropositional Logicdoes not offer a representation for timeEach action description needs to be repeated for each stepSituation Calculus (AIMA Section 10.3)Is based on FOLEach time step is a ‘situation’Allows to represent plans and reason about actions & change

8 Situation Calculus Sequence of actions Projection task Planning taskResult([],s)=sResult([a|seq],s)=Result(seq,Result(a,s))Projection taskDeducing the outcome of a sequence of actionsPlanning taskFind a sequence of actions that achieves a desired effect

10 Importance of Situation CalculusHistorical noteSituation Calculus was the first attempt to formalizing planning in FOLOther formalisms include Event CalculusThe area of using logic for planning is informally called in the literature “Reasoning About Action & Change”Highlighted three important problemsFrame problemQualification problemRamification problem

11 ‘Famous’ Problems Frame problem Qualification problemRepresenting all things that stay the same from one situation to the nextInferential and representationalQualification problemDefining the circumstances under which an action is guaranteed to workExample: what if the gold is slippery or nailed down, etc.Ramification problemProliferation of implicit consequences of actions as actions may have secondary consequencesExamples: How about the dust on the gold?

13 Planning Languages Languages must represent.. Languages must be StatesGoalsActionsLanguages must beExpressive for ease of representationFlexible for manipulation by algorithms

14 State RepresentationA state is represented with a conjunction of positive literalsUsingLogical Propositions: Poor  UnknownFOL literals: At(Plane1,OMA)  At(Plan2,JFK)FOL literals must be ground & function-freeNot allowed: At(x,y) or At(Father(Fred),Sydney)Closed World AssumptionWhat is not stated are assumed false

15 Goal Representation Goal is a partially specified stateA proposition satisfies a goal if it contains all the atoms of the goal and possibly others..Example: Rich  Famous  Miserable satisfies the goal Rich  Famous

17 Applying an Action Find a substitution list  for the variablesof all the precondition literalswith (a subset of) the literals in the current state descriptionApply the substitution to the propositions in the effect listAdd the result to the current state description to generate the new stateExample:Current state: At(P1,JFK)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO)It satisfies the precondition with ={p/P1,from/JFK, to/SFO)Thus the action Fly(P1,JFK,SFO) is applicableThe new current state is: At(P1,SFO)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO)

25 State-Space Search (3)Remember that the language has no functions symbolsThus number of states is finiteAnd we can use any complete search algorithm (e.g., A*)We need an admissible heuristicThe solution is a path, a sequence of actions: total-order planningProblem: Space and time complexitySTRIPS-style planning is PSPACE-complete unless actions haveonly positive preconditions andonly one literal effect

27 Relevant Action An action is relevantIn Progression planning when its preconditions match a subset of the current stateIn Regression planning, when its effects match a subset of the current goal state

28 Consistent ActionThe purpose of applying an action is to ‘achieves a desired literal’We should be careful that the action does not undo a desired literal (as a side effect)A consistent action is an action that does not undo a desired literal

29 Backward State-Space SearchGivenA goal G descriptionAn action A that is relevant and consistentGenerate a predecessor state wherePositive effects (literals) of A in G are deletedPrecondition literals of A are added unless they already appearSubstituting any variables in A’s effects to match literals in GSubstituting any variables in A’s preconditions to match substitutions in A’s effectsRepeat until predecessor description matches initial state

30 Heuristic to Speed up SearchWe can use A*, but we need an admissible heuristicDivide-and-conquer: sub-goal independence assumptionProblem relaxation by removing… all preconditions… all preconditions and negative effects… negative effects only: Empty-Delete-List

31 1. Subgoal Independence AssumptionThe cost of solving a conjunction of subgoals is the sum of the costs of solving each subgoal independentlyOptimisticWhere subplans interact negativelyExample: one action in a subplan delete goal achieved by an action in another subplanPessimistic (not admissible)Redundant actions in subplans can be replaced by a single action in merged plan

32 2. Problem Relaxation: Removing PreconditionsRemove preconditions from action descriptionsAll actions are applicableEvery literal in the goal is achievable in one stepNumber of steps to achieve the conjunction of literals in the goal is equal to the number of unsatisfied literalsAlertSome actions may achieve several literalsSome action may remove the effect of another action

33 3. Remove Preconditions & Negative EffectsConsiders only positive interactions among actions to achieve multiple subgoalsThe minimum number of actions required is the sum of the union of the actions’ positive effects that satisfy the goalThe problem is reduced to a set cover problem, which is NP-hardApproximation by a greedy algorithm cannot guarantee an admissible heuristic

38 Components of a Plan A set of actions A set of ordering constraintsA p B reads “A before B” but not necessarily immediately before BAlert: caution to cycles A p B and B p AA set of causal links (protection intervals) between actionsA B reads “A achieves p for B” and p must remain true from the time A is applied to the time B is appliedExample “RightSock RightShoeA set of open preconditionsPlanners work to reduce the set of open preconditions to the empty set w/o introducing contradictionspRightSockOn

39 Consistent Plan (POP) Consistent plan is a plan that has SolutionNo cycle in the ordering constraintsNo conflicts with the causal linksSolutionIs a consistent plan with no open preconditionsTo solve a conflict between a causal link A B and an action C (that clobbers, threatens the causal link), we force C to occur outside the “protection interval” by addingthe constraint C p A (demoting C) orthe constraint B p C (promoting C)p

41 POP as a Search ProblemThe successor function arbitrarily picks one open precondition p on an action BFor every possible consistent action A that achieves pIt generates a successor plan adding the causal link A B and the ordering constraint A p BIf A was not in the plan, it adds Start p A and A p FinishIt resolves all conflicts betweenthe new causal link and all existing actionsbetween A and all existing causal linksThen it adds the successor states for combination of resolved conflictsIt repeats until no open precondition existsp

43 Add causal link between Remove(Spare,Trunk) and PutOn(Spare,Axle)Pick up At(Flat,Axle)There are 2 applicable actions: LeaveOvernight and Remove(Flat,Axle)Choose LeaveOvernightLeaveOvernight has effect At(Spare,Ground), which conflicts with the causal linkWe remove the conflict by forcing LeaveOvernight to occur before Remove(Spare,Trunk)Conflicts with effects of Remove(Spare,Trunk)The only way to resolve the conflict is to undo LeaveOvernightuse the action Remove(Flat,Axle)

44 This time, we choose Remove(Flat,Axle)Pick up At(Spare,Trunk) and choose Start to achieve itPick up At(Flat,Axle) and choose Start to achieve it.We now have a complete consistent partially ordered plan

45 POP Algorithm (1)Backtrack when fails to resolve a threat or find an operatorCausal linksRecognize when to abandon a doomed plan without wasting time expanding irrelevant part of the planallow early pruning of inconsistent combination of actionsWhen actions include variables, we need to find appropriate substitutionsTypically we try to delay commitments to instantiating a variable until we have no other choice (least commitment)POP is sound, complete, and systematic (no repetition)

46 POP Algorithm (2) Decomposes the problem (advantage)But does not represent states explicitly: it is hard to design heuristic to estimate distance from goalExample: Number of open preconditions – those that match the effects of the start node. Not perfect (same problems as before)A heuristic can be used to choose which plan to refine (which precondition to pick-up):Choose the most-constrained precondition, the one satisfied by the least number of actions. Like in CSPs!When no action satisfies a precondition, backtrack!When only one action satisfies a precondition, pick up the precondiction.

48 Planning Graph  Is special data structure used forDeriving better heuristic estimatesExtract a solution to the planning problem: GraphPlan algorithmIs a sequence S0,A0,S1,A1,…,Si of levelsAlternating state levels & action levelsLevels correspond to time stampsStarting at initial stateState level is a set of (propositional) literalsAll those literals that could be true at that levelAction level is a set of (propositionalized) actionsAll those actions whose preconditions appear in the state level (ignoring all negative interactions, etc.)Propositionalization may yield combinatorial explosition in the presence of a large number of objects

49 Focus Building the Planning Graph Using it for Heuristic EstimationUsing it for generating the plan

51 Example of a Planning Graph (2)At each state level, list all literals that may hold at that levelAt each action level, list all noops & all actions whose preconditions may hold at previous levelsRepeat until plan ‘levels off,’ no new literals appears (Si=Si+1)Building the Planning Graph is a polynomial processAdd (binary) mutual exclusion (mutex) links between conflicting actions and between conflicting literalsMutual exclusion linksS1 represents multiple states

52 Mutex Links between ActionsInconsistent effects: one action negates an effect of anotherEat(Cake) & noop of Have(Cake) disagree on effect Have(Cake)Interference: An action effect negates the precondition of anotherEat(Cake) negates precondition of the noop of Have(Cake):Competing needs: A precondition on an action is mutex with the precondition of anotherBake(Cake) & Eat(Cake): compete on Have(Cake) precondition

53 Mutex Links between LiteralsTwo literals are negation of each otherInconsistent support: Each pair of actions that can achieve the two literals is mutex. Examples:In S1, Have(Cake) & Eaten(Cake) are mutexIn S2, they are not because Bake(Cake) & the noop of Eaten(Cake) are not mutex

54 Focus Building the Planning Graph Using it for Heuristic EstimationPlanning graph as a relaxation of original problemEasy to build (compute)Using it for generating the plan

55 Planning Graph for Heuristic EstimationA literal that does not appear in the final level cannot be achieved by any planState-space search: Any state containing an unachievable literal has cost h(n)=POP: Any plan with an unachievable open condition has cost h(n)=The estimate cost of any goal literal is the first level at which it appearsEstimate is admissible for individual literalsEstimate can be improved by serializing the graph (serial planning graph: one action per level) by adding mutex between all actions in a given levelThe estimate of a conjunction of goal literalsThree heuristics: max level, level sum, set level

56 Estimate of Conjunction of Goal LiteralsMax-levelThe largest level of a literal in the conjunctionAdmissible, not very accurateLevel sumUnder subgoal independence assumption, sums the level costs of the literalsInadmissible, works well for largely decomposable problemsSet levelFinds the level at which all literals appear w/o any pair of them being mutexDominates max-level, works extremely well on problems where there is a great deal of interaction among subplans

57 Focus Building the Planning Graph Using it for Heuristic EstimationUsing it for generating the planGraphPlan algorithm [Blum & Furst, 95]

63 Backtrack Search for Solution ExtractionStarting at the highest fact levelEach goal is put in a goal list for the current fact layerSearch iterates thru each fact in the goal list trying to find an action to support it which is not mutex with any other chosen actionWhen an action is chosen, its preconditions are added to the goal list of the lower levelWhen all facts in the goal list of the current level have a consistent assignment of actions, the search moves to the next levelSearch backtracks to the previous level when it fails to assign an action to each fact in the goal list at a given levelSearch succeeds when the first level is reached.

65 Optimality of GraphPlanThe plans generated by GraphPlanAre optimal in the number of steps needed to execute the planNot necessarily optimal in the number of actions in the plan (GraphPlan produces partially ordered plans)