This is a short course that aims to provide an introduction to the techniques currently used for the decision making of non-player characters (NPCs) in commercial video games, and show how a simple deliberation technique from academic artificial intelligence research can be employed to advance the state-of-the art. For more information and downloading the supplementary material please use the following links: http://stavros.lostre.org/2012/05/19/video-games-sapienza-roma-2012/ http://tinyurl.com/AI-NPC-LaSapienza

1. Stavros Vassos, University of Athens, Greece stavrosv@di.uoa.gr May 2012
INTRODUCTION TO AI
STRIPS PLANNING
.. and Applications to Video-games!

2. Course overview
2
 Lecture 1: Game-inspired competitions for AI research,
AI decision making for non-player characters in games
 Lecture 2: STRIPS planning, state-space search
 Lecture 3: Planning Domain Definition Language (PDDL),
using an award winning planner to solve Sokoban
 Lecture 4: Planning graphs, domain independent
heuristics for STRIPS planning
 Lecture 5: Employing STRIPS planning in games:
SimpleFPS, iThinkUnity3D, SmartWorkersRTS
 Lecture 6: Planning beyond STRIPS

3. STRIPS planning
3
 STRIPS! So why do we like this formalism?

4. STRIPS planning
4
 STRIPS! So why do we like this formalism?
 Simple formalism for representing planning problems
 Easy to compute applicable actions
 Check whether the list of preconditions is a subset of the
state description: PRECONDITIONS  S
 Easy to compute the successor states
 Add the list of positive effects to the state description and
remove the list of negative effects:
S’ = (S / NEGATIVE-EFFECTS)  POSITIVE-EFFECTS
 Easy to check if the goal is satisfied
 Check whether the goal is a subset of the state description:
GS

5. STRIPS planning
5
 STRIPS! So why do we like this formalism?
 It can already describe difficult and complex problems
(in more challenging domains than the example we saw)
 …let’s see how we can solve this kind of problems

6. STRIPS planning: state-based search
6
 Finding a solution to the planning problem following
a state-based search
 Init( On(Α,Table)  On(Β,Table)  … )
 Goal( On(Α,Β)  …)
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  …
EFFECTS: On(b,y)  … )
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  …
EFFECTS: On(b,Table)  …)

7. STRIPS planning: state-based search
7
 Finding a solution to the planning problem following
a state-based search
 Init( where to start from )
 Goal( when to stop searching )
 Action( how to generate
the “graph” )
 Progression planning: forward state-based search
 Regression planning: backward state-based search

8. Progression planning
8
 Start from the initial state
 Check if the current state satisfies the
goal
 Compute applicable actions to the
current state
 Compute the successor states
 Pick one of the successor states as the
current state
 Repeat until a solution is found or the
state space is exhausted

9. Progression planning
9
 Start from the initial state
On(Α,Table)
On(Β,Table)
Α Β C On(C,Table)
Clear(Α)
Clear(Β)
Clear(C)

10. Progression planning
10
 Check if the current state satisfies the goal
 No
On(Α,Table)
On(Β,Table) On(Α,Β)
Α Β C On(C,Table) On(Β,C)
Clear(Α)
Clear(Β)
Clear(C)

11. Progression planning
11
 Compute applicable actions to the current state
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y))
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b))
On(Α,Table)
On(Β,Table)
Α Β C On(C,Table)
Clear(Α)
Clear(Β)
Clear(C)

12. Progression planning
12
 Compute applicable actions to the current state
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y))
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b))
 Move(Β,Table,C)
On(Α,Table)
Preconditions: On(Β,Table)
 On(Β,Table) Α Β C On(C,Table)
 Clear(Β) Clear(Α)
 Clear(C) Clear(Β)
Clear(C)
 Applicable action!

13. Progression planning
13
 Compute applicable actions to the current state
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y))
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b))
 Move(Α,Table,Β)
On(Α,Table)
 Move(Α,Table,C) On(Β,Table)
 Move(Β,Table,Α) Α Β C On(C,Table)
 Move(Β,Table,C) Clear(Α)
Clear(Β)
 Move(C,Table,Α) Clear(C)
 Move(C,Table,Β)
 All these are applicable actions!

14. Progression planning
14
 Compute applicable actions to the current state
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y))
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b))
 Move(Β,Table,Β)
On(Α,Table)
Preconditions: On(Β,Table)
 On(Β,Table) Α Β C On(C,Table)
 Clear(Β) Clear(Α)
Clear(Β)
Clear(C)
 This is also an applicable action!

15. Progression planning
15
 Compute applicable actions to the current state
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y))
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b))
 MoveTT(Β,Table)
On(Α,Table)
Preconditions: On(Β,Table)
 On(Β,Table) Α Β C On(C,Table)
 Clear(Β) Clear(Α)
Clear(Β)
Clear(C)
 This is also an applicable action!

16. Progression planning
16
 Compute applicable actions to the current state
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y))
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b))
 Move(Α,Table,Α)
On(Α,Table)
 Move(Β,Table,Β) On(Β,Table)
 Move(C,Table,C) Α Β C On(C,Table)
 MoveTT(Α,Table) Clear(Α)
Clear(Β)
 MoveTT(Β,Table) Clear(C)
 MoveTT(C,Table)
 All these are applicable also actions!

17. Progression planning
17
 Compute the successor states
 Action( Move(b,x,y),
EFFECTS: On(b,y) 
Clear(x)  Β
On(b,x)  Α C
Clear(y) )
 Move(Β,Table,C) On(Α,Table)
Effects: On(Β,Table)
Α Β C On(C,Table)
 On(Β,C)
Clear(Α)
 Clear(Table)
Clear(Β)
  On(Β,Table) Clear(C)
 Clear(C)

18. Progression planning
18
 Compute the successor states
Move(Α,Table,C) Α Β
Move(Β,Table,C)
Β C Α C
Move(Α,Table,Β) Α Β Move(Β,Table,Α)
C Β Α Β C C Α
C C Move(C,Table,Α)
Move(C,Table,Β)
Α Β Β Α

19. Progression planning
19
 Pick one of the successor states as current..
Α Β
Β C Α C
Α Β
C Β Α Β C C Α
C C
Α Β Β Α

20. Progression planning
20
 ..and repeat!
Α
Α Β Β
Β C Α C C
Α Β
C Β Α Β C C Α
C C
Α Β Β Α

21. Progression planning
21
 Pick one of the successor states as current
Α
Α Β Β
Β C Α C C
Α Β
C Β Α Β C C Α
C C
Α Β Β Α

22. Progression planning
22
 Is it guaranteed that progressive planning will find
a solution if one exists?

23. Progression planning
23
 Is it guaranteed that progressive planning will find
a solution if one exists?
 Given that the state-space is finite (ground atoms, no
function symbols, finite number of constants)..
 ..Yes! As long as we visit each state only once..

24. Progression planning
24
 Pick one of the not-visited successor states as current
Α
Α Β Β
Β C Α C C
Α Β
C Β Α Β C C Α
C C
Α Β Β Α

25. Progression planning
25
 Is it guaranteed that progressive planning will find
a solution if one exists?
 Given that the state-space is finite (ground atoms, no
function symbols, finite number of constants)..
 ..Yes! As long as we visit each state only once..
 But it may have to explore the whole state-space

26. Progression planning
26
Β Α
Α Α Β Β
C Β C Α C C
C C
Α Α Β Β
Β C Β Α Β C C Α Α
Α Β
C C C C
Β Α Β Β Α Α

27. Progression planning
27
 Unlike this simple example, in many problems the state
space is actually huge.
 Even this simple example becomes challenging if we
consider 100 boxes and 1000s of applicable actions
of the form Move(b,x,y) in each state.
 Similar to search problems we can make use of
heuristics that help progression planning pick the most
promising states to investigate first.

28. Heuristics for progression planning
28
 A* search
 Evaluation function f(s) = g(s) + h(s)
 g(s): the number of actions needed to reach state s from
the initial state (accurate)
 h(s): the number of actions needed to reach a goal state
from state s (estimated)
 Use f(s) to order the successor states and pick the
most promising one.

29. Heuristics for progression planning
29
 Consider a heuristic h(s) with the following behavior
Move(Α,Table,C) Move(Β,Table,C)
g(s)=1 Α Β g(s)=1
Β C Α C h(s)=1
h(s)=2
Move(Α,Table,Β) Move(Β,Table,Α)
g(s)=1 Α Β g(s)=1
C Β Α Β C C Α h(s)=2
h(s)=1
Move(C,Table,Β) Move(C,Table,Α)
g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

30. Heuristics for progression planning
30
 So why is it different than the usual search problems?
 E.g.,in grid-based problems we could define h(s) using a
“relaxed” distance metric such as Manhattan distance.

31. Heuristics for progression planning
31
 So why is it different than the usual search problems?
 E.g.,in grid-based problems we could define h(s) using a
“relaxed” distance metric such as Manhattan distance.
 The action schemas provide valuable information that
can be used to specify domain independent heuristic
functions!
 Moreover they provide a logical specification of
the problem that allows approaches for finding a
solution that are different than search

32. Heuristics for progression planning
32
 Empty list of preconditions
 h(s)
= number of actions needed to achieve the goal if
we assume that all actions are always applicable
 Empty list of negative effects
 h(s)
= number of actions needed to achieve the goal if
we disregard the negative effects of actions
 Planning graphs
 Simple example: h(s) = number of literals in the
goal that are missing from s

33. Heuristics for progression planning
33
 Start from the initial state
 Check if the current state satisfies the
goal
 Compute applicable actions to the
current state
 Compute the successor states
 Pick one the most promising of the
successor states as the current state
 Repeat until a solution is found or the
state space is exhausted

34. Heuristics for progression planning
34
 Compute the successor states
Move(Α,Table,C) Move(Β,Table,C)
g(s)=1 Α Β g(s)=1
Β C Α C h(s)=1
h(s)=2
Move(Α,Table,Β) Move(Β,Table,Α)
g(s)=1 Α Β g(s)=1
C Β Α Β C C Α h(s)=2
h(s)=1
Move(C,Table,Β) Move(C,Table,Α)
g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

35. Heuristics for progression planning
35
 Compute the successor states
Move(Α,Table,C) Move(Β,Table,C)
g(s)=1 Α Β g(s)=1
Β C Α C h(s)=1
h(s)=2
Move(Α,Table,Β) Move(Β,Table,Α)
g(s)=1 Α Β g(s)=1
C Β Α Β C C Α h(s)=2
h(s)=1
Move(C,Table,Β) Move(C,Table,Α)
g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

36. Heuristics for progression planning
36
 Pick the most promising successor state wrt f(s)
Move(Α,Table,C) Move(Β,Table,C)
g(s)=1 Α Β g(s)=1
Β C Α C h(s)=1
h(s)=2
Move(Α,Table,Β) Move(Β,Table,Α)
g(s)=1 Α Β g(s)=1
C Β Α Β C C Α h(s)=2
h(s)=1
Move(C,Table,Β) Move(C,Table,Α)
g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

37. Heuristics for progression planning
37
 Compute the successor states
Move(Α,Table,C) Move(Β,Table,C)
g(s)=1 Α Β g(s)=1
Β C Α C h(s)=1
h(s)=2
C Move(Β,Table,Α)
Α Α Β g(s)=1
Β C Β Α Β C C Α h(s)=2
Move(C,Table,Α)
g(s)=2 Move(C,Table,Β) Move(C,Table,Α)
h(s)=1 g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

38. Heuristics for progression planning
38
 Compute the successor states
Move(Α,Table,C) Move(Β,Table,C)
g(s)=1 Α Β g(s)=1
Β C Α C h(s)=1
h(s)=2
C Move(Β,Table,Α)
Α Α Β g(s)=1
Β C Β Α Β C C Α h(s)=2
Move(C,Table,Α)
g(s)=2 Move(C,Table,Β) Move(C,Table,Α)
h(s)=1 g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

39. Heuristics for progression planning
39
 Pick the most promising successor state wrt f(s)
Move(Α,Table,C) Move(Β,Table,C)
g(s)=1 Α Β g(s)=1
Β C Α C h(s)=1
h(s)=2
C Move(Β,Table,Α)
Α Α Β g(s)=1
Β C Β Α Β C C Α h(s)=2
Move(C,Table,Α)
g(s)=2 Move(C,Table,Β) Move(C,Table,Α)
h(s)=1 g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

40. Heuristics for progression planning
40
Move(Α,Table,Β)
 Compute the successor states g(s)=2
h(s)=0
Move(Α,Table,C) Α
g(s)=1 Α Β Β
Β C Α C C
h(s)=2
C Move(Β,Table,Α)
Α Α Β g(s)=1
Β C Β Α Β C C Α h(s)=2
Move(C,Table,Α)
g(s)=2 Move(C,Table,Β) Move(C,Table,Α)
h(s)=1 g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

41. Heuristics for progression planning
41
Move(Α,Table,Β)
 Compute the successor states g(s)=2
h(s)=0
Move(Α,Table,C) Α
g(s)=1 Α Β Β
Β C Α C C
h(s)=2
C Move(Β,Table,Α)
Α Α Β g(s)=1
Β C Β Α Β C C Α h(s)=2
Move(C,Table,Α)
g(s)=2 Move(C,Table,Β) Move(C,Table,Α)
h(s)=1 g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

42. Heuristics for progression planning
42
 Pick the most promising successor state wrt f(s)
Move(Α,Table,C) Α
g(s)=1 Α Β Β
Β C Α C C
h(s)=2
C Move(Β,Table,Α)
Α Α Β g(s)=1
Β C Β Α Β C C Α h(s)=2
Move(C,Table,Α)
g(s)=2 Move(C,Table,Β) Move(C,Table,Α)
h(s)=1 g(s)=1 C C g(s)=1
Α Β Β Α h(s)=2
h(s)=2

43. Regression planning
43
 Start from the goal as current goal
 Check if the initial state satisfies the
current goal
 Compute the relevant and consistent
actions for the current goal
 Compute the predecessor states
 Pick one of the predecessor states as
the current goal
 Repeat until a solution is found or the
state space is exhausted

44. Research in STRIPS planning
44
 Planning Domain Definition Language (PDDL)
 Formal language for specifying planning problems
 Formal syntax similar to a programming language
 Includes STRIPS and ADL, and many more features
 Provides the ground for performing a direct comparison
between planning techniques and evaluating against
classes of problems

45. Research in STRIPS planning
45
 Planning Domain Definition Language (PDDL)
 International Planning Competition 1998 – today
 SAT Plan
 TL Plan Planning problems in
PDDL, e.g., Blocks
 FF world, Storage, Trucks,
…
 BlackBox
 SHOP2
 TALPlanner
Direct comparison between
… planning techniques! E.g.,
evaluation of heuristics, …

46. Research in STRIPS planning
46
 Planning Domain Definition Language (PDDL)
 We will see more of PDDL and off-the-shelve planners
in Lecture 3

47. Next lecture
47
 Lecture 1: Game-inspired competitions for AI research,
AI decision making for non-player characters in games
 Lecture 2: STRIPS planning, state-space search
 Lecture 3: Planning Domain Definition Language (PDDL),
using an award winning planner to solve Sokoban
 Lecture 4: Planning graphs, domain independent
heuristics for STRIPS planning
 Lecture 5: Employing STRIPS planning in games:
SimpleFPS, iThinkUnity3D, SmartWorkersRTS
 Lecture 6: Planning beyond STRIPS

48. Bibliography
48
 Material
 Artificial
Intelligence: A Modern Approach 2nd Ed. Stuart Russell,
Peter Norvig. Prentice Hall, 2003 Section 11.2
 References
 PDDL - The Planning Domain Definition Language. Drew
McDermott, Malik Ghallab, Adele Howe, Craig Knoblock, Ashwin
Ram, Manuela Veloso, Daniel Weld, David Wilkins. Technical
report, Yale Center for Computational Vision and Control, TR-98-
003, 1998.