This is a short course that aims to provide an introduction to the techniques currently used for the decision making of non-player characters (NPCs) in commercial video games, and show how a simple deliberation technique from academic artificial intelligence research can be employed to advance the state-of-the art. For more information and downloading the supplementary material please use the following links: http://stavros.lostre.org/2012/05/19/video-games-sapienza-roma-2012/ http://tinyurl.com/AI-NPC-LaSapienza

1. Stavros Vassos, University of Athens, Greece stavrosv@di.uoa.gr May 2012
INTRODUCTION TO AI
STRIPS PLANNING
.. and Applications to Video-games!

2. Course overview
2
 Lecture 1: Game-inspired competitions for AI research,
AI decision making for non-player characters in games
 Lecture 2: STRIPS planning, state-space search
 Lecture 3: Planning Domain Definition Language (PDDL),
using an award winning planner to solve Sokoban
 Lecture 4: Planning graphs, domain independent
heuristics for STRIPS planning
 Lecture 5: Employing STRIPS planning in games:
SimpleFPS, iThinkUnity3D, SmartWorkersRTS
 Lecture 6: Planning beyond STRIPS

3. Goal Oriented Action Planning (GOAP)
3
 Video Games:
 Finite State Machines
 Decision Diagrams
 Behavior Trees
 Goal Oriented Action Planning
 Academic AI on agents:
 Knowledge representation, First-order logic,
Classical planning, Planning with preferences, …
 Belief-Desire-Intention architecture, Agent-based
programming, …
 Probabilistic reasoning, Bayesian networks,
Utility theory, Markov Decision Processes, …

4. Goal Oriented Action Planning (GOAP)
4
 Search for a plan that achieves the goal in the current
state
Move Preconditions: Door open In room
into room Effects: In room
Move Preconditions: -
to door Effects: At door
Unlock Preconditions: Hold key
door Effects: Door open
Kick Preconditions: -
door Effects: Door open

5. Goal Oriented Action Planning (GOAP)
5
 Search for a plan that achieves the goal in the current
state
 There is a standard formalism for representing such
problems in AI literature and a large body of work!
 Video-games: The best strategy is to use the existing
formalism and take advantage of the work in the
literature

6. Course overview
6
 Lecture 1: Game-inspired competitions for AI research,
AI decision making for non-player characters in games
 Lecture 2: STRIPS planning, state-space search
 Lecture 3: Planning Domain Definition Language (PDDL),
using an award winning planner to solve Sokoban
 Lecture 4: Planning graphs, domain independent
heuristics for STRIPS planning
 Lecture 5: Employing STRIPS planning in games:
SimpleFPS, iThinkUnity3D, SmartWorkersRTS
 Lecture 6: Planning beyond STRIPS

7. STRIPS planning
7
 Typical description of a planning problem:
 Initial state
 Goal
 Available actions
 Typical solution of a planning problem:
A sequence of actions that when executed in the initial
state results in a state that satisfies the goal condition.
 Planning: The automatic discovery of such solutions

8. What is a planning problem?
8
 Let’s start with a simple example..

9. What is a planning problem?
9
 Given:
 Initial state

10. What is a planning problem?
10
 Given:
 Initial state
 Goal

11. What is a planning problem?
11
 Given:
 Initial state
 Goal
 Available actions

12. What is a planning problem?
12
 Given:
 Initial state
 Goal
 Available actions
 Find:
A sequence of actions that achieves the goal
 E.g.: [Left, Down, Left, Up, …]

13. What is a planning problem?
13
 Given:
 Initial state
 Goal
 Available actions
 Find:
A sequence of actions that achieves the goal
 A method that is capable of finding a solution for every
initial state and goal

14. What is a planning problem?
14
 Given:
 Initial state
 Goal
 Available actions
 Find:
A sequence of actions that achieves the goal
 A method that is capable of finding a solution for every
application domain

15. Use of planning in real applications
15
 Scheduling
 Hubble Space Telescope

16. Use of planning in real applications
16
 KIVA Robots: robots make inventory isles move
instead of workers
youtube link

17. Use of planning in real applications
17
 Autonomous agents with planning capabilities for
specifying the behavior (Intelligent Agents/
Cognitive Robots)
 UAV DARPA Grand/Urban Challenge, Honda
ASIMO

18. STRIPS planning
18
 Typical description of a planning problem:
 Initial state
 Goal
 Available actions

19. STRIPS planning
19
 Blocks world domain
 Initial state: s0 Α
Β
Α Β C C
 Goal: g
s0 g
 Available actions: moving a block
 from the table to the top of another block
 from the top of another block to the table
 from the top of one block to the top of another block

20. STRIPS planning
20
 Initial state
 Representation of the properties of the Α Β C
domain using first order logic literals
On(Α,Table)
 Blocks world On(Β,Table)
On(C,Table)
 On(b,x): Clear(Α)
Clear(Β)
 block b is on top of x, where x is another
Clear(C)
block or the table
 Clear(x):
a block can be placed on top of x s0

21. STRIPS planning
21
 Initial state
 Representation of the properties of the Α Β C
domain using first order logic literals
On(Α,Table)
 Ground and function-free On(Β,Table)
On(C,Table)
Clear(Α)
 Completely specified Clear(Β)
Clear(C)
based on a closed-world assumption
s0

22. STRIPS planning
22
 Initial state On(Α,Α)
On(Α,Β)
On(Α,C)
 Closed-world On(Β,Α) Α Β C
assumption On(Β,Β)
On(Β,C)
On(C,Α) On(Α,Table)
On(C,Β)
 Any literal not On(Β,Table)
On(C,C) On(C,Table)
mentioned in the On(Α,Α) Clear(Α)
description of the On(Α,Β) Clear(Β)
state are assumed On(Α,C) Clear(C)
On(Table,Α)
to be false! On(Table,Β)
On(Table,C)
On(Table,Table) s0
Clear(Table)

23. STRIPS planning
23
 Goal Α
Β
 Representation of the properties of the C
domain using first order logic literals
On(Α,Β)
 Ground and function-free On(Β,C)
 Partiallyspecified
(no closed-world assumption)
A state s satisfies goal g if it contains all g
literals in g (more literals may be in s)

24. STRIPS planning
24
 The initial state and goal Α
condition are described using Β
first-order logic literals: Α Β C C
 ground
 function-free
On(Α,Table)
On(Α,Β)
 positive On(Β,Table)
On(Β,C)
On(C,Table)
Clear(Α)
Clear(Β)
the list of literals typically Clear(C)
forms a logical conjunction
s0 g

25. STRIPS planning
25
Available actions
Move:
Α Β C
 from the table to the
top of another block
 from the top of On(Α,Table)
another block to the On(Β,Table)
table On(C,Table)
Clear(Α)
 from the top of one Clear(Β)
block to the top of Clear(C)
another block
s0

26. STRIPS planning
26
Available actions
Move: Β
Α Β C Α C
 from the table to the
top of another block
 from the top of On(Α,Table)
another block to the On(Β,Table)
table On(C,Table)
Clear(Α) ???
 from the top of one Clear(Β)
block to the top of Clear(C)
another block
Move(Β,Table,C)
s0 s1

27. STRIPS planning
27
 Available actions
Β
Α Β C Α C
 Preconditions: literals
denoting what needs to
be in the state for the On(Α,Table)
action to be applicable On(Β,Table)
On(C,Table)
Clear(Α) ???
 Effects: literals denoting Clear(Β)
Clear(C)
how the state is
transformed when the Move(Β,Table,C)
action is applied.
s0 s1

28. STRIPS planning
28
 Available actions
Β
Α Β C Α C
 Preconditions:
 On(b,x)
 Clear(b) On(Α,Table)
 Clear(y) On(Β,Table)
On(C,Table)
Clear(Α) ???
 Effects: Clear(Β)
Clear(C)
 On(b,y)
 Clear(x)
Move(Β,Table,C)
  On(b,x)
s0 s1
 Clear(y)

29. STRIPS planning
29
 Available actions
Β
Α Β C Α C
 Preconditions:
 On(Β,Table)
 Clear(Β) On(Α,Table)
 Clear(C) On(Β,Table)
On(C,Table)
Clear(Α) ???
 Effects: Clear(Β)
Clear(C)
 On(Β,C)
 Clear(Table)
Move(Β,Table,C)
  On(Β,Table)
s0 s1
 Clear(C)

30. STRIPS planning
30
 Available actions
Β
Α Β C Α C
 Preconditions:
 On(Β,Table)
 Clear(Β) On(Α,Table) On(Α,Table)
 Clear(C) On(Β,Table) On(Β,C)
On(C,Table) On(C,Table)
Clear(Α) Clear(Α)
 Effects: Clear(Β) Clear(Β)
Clear(C) Clear(Table)
 On(Β,C)
 Clear(Table)
Move(Β,Table,C)
  On(Β,Table)
s0 s1
 Clear(C)

31. STRIPS planning
31
 Available actions
Β
???
Α C
 Preconditions:
 On(b,x)
 Clear(b) On(Α,Table)
 Clear(y) On(Β,C)
On(C,Table)
Clear(Α) ???
 Effects: Clear(Β)
Clear(Table)
 On(b,y)
 Clear(x)
Move(C,Table,Α)
  On(b,x)
s1 s2
 Clear(y)

32. STRIPS planning
32
 Available actions
Β
???
Α C
 Preconditions:
 On(C,Table)
 Clear(C) On(Α,Table)
 Clear(Α) On(Β,C)
On(C,Table)
Clear(Α) ???
 Effects: Clear(Β)
Clear(Table)
 On(C,Α)
 Clear(Table)
Move(C,Table,Α)
  On(C,Table)
s1 s2
 Clear(Α)

33. STRIPS planning
33
 Available actions Α
Β Β
Α C C
 Preconditions:
 On(Α,Table)
 Clear(Α) On(Α,Table)
 Clear(Β) On(Β,C)
On(C,Table)
Clear(Α) ???
 Effects: Clear(Β)
Clear(Table)
 On(Α,Β)
 Clear(Table)
Move(Α,Table,Β)
  On(Α,Table)
s1 s2
 Clear(Β)

34. STRIPS planning
34
 Available actions Α
Β Β
Α C C
 Preconditions:
 On(Α,Table)
 Clear(Α) On(Α,Table) On(Α,Β)
 Clear(Β) On(Β,C) On(Β,C)
On(C,Table) On(C,Table)
Clear(Α) Clear(Α)
 Effects: Clear(Β) Clear(Table)
Clear(Table)
 On(Α,Β)
 Clear(Table)
Move(Α,Table,Β)
  On(Α,Table)
s1 s2
 Clear(Β)

35. STRIPS planning
35
Α
Β Β
Α Β C Α C C
On(Α,Table) On(Α,Table) On(Α,Β)
On(Β,Table) On(Β,C) On(Β,C)
On(C,Table) On(C,Table) On(C,Table)
Clear(Α) Clear(Α) Clear(Α)
Clear(Β) Clear(Β) Clear(Table)
Clear(C) Clear(Table)
Move(Β,Table,C) Move(Α,Table,Β)
s0 s1 s2

36. STRIPS planning
36
Α Α
Β Β Β
Α Β C Α C C C
On(Α,Table) On(Α,Table) On(Α,Β)
On(Α,Β)
On(Β,Table) On(Β,C) On(Β,C)
On(Β,C)
On(C,Table) On(C,Table) On(C,Table)
Clear(Α) Clear(Α) Clear(Α)
Clear(Β) Clear(Β) Clear(Table)
Clear(C) Clear(Table)
Move(Β,Table,C) Move(Α,Table,Β)
s0 s1 s2 g

37. STRIPS planning
37
 Is our STRIPS representation of the Α
blocks world correct? Β
C
 Consider moving back blocks A
and B back to the table
On(Α,Β)
On(Β,C)
On(C,Table)
Clear(Α)
Clear(Table)

38. STRIPS planning
38
Α
Β Β
C Α C Α Β C
On(Α,Β) On(Α,Table)
On(Β,C) On(Β,C)
On(C,Table) On(C,Table)
Clear(Α) Clear(Α) ???
Clear(Table) Clear(Β)
Move(A,B,Table) Move(B,C,Table)
s2 s3 s4

39. STRIPS planning
39
 Is our STRIPS representation of the Α
blocks world correct? Β
C
 Consider moving back blocks A
and B back to the table
On(Α,Β)
On(Β,C)
On(C,Table)
 Clear(Table) needs to be treated Clear(Α)
differently Clear(Table)
 Move(b,x,y)
 MoveToTable(b,x)

40. STRIPS planning
40
 Init( On(Α,Table)  On(Β,Table)  On(C,Table)
 Clear(Α)  Clear(Β)  Clear(C) )
 Goal( On(Α,Β)  On(Β,C) )
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y)
EFFECTS: On(b,y)  Clear(x)  On(b,x)
 Clear(y) )
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b)
EFFECTS: On(b,Table)  Clear(x)  On(b,x) )

41. STRIPS planning
41
 Init( On(Α,Table)  On(Β,Table)  On(C,Table)
! Variables thatappear in  Clear(C) ) and effects
 Clear(Α) Clear(Β)
preconditions
need to be parameters of the action schema
 Goal( On(Α,Β)  On(Β,C) )
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y)
EFFECTS: On(b,y)  Clear(x)  On(b,x)
 Clear(y) )
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b)
EFFECTS: On(b,Table)  Clear(x)  On(b,x) )

42. STRIPS planning
42
 Init( On(Α,Table)  On(Β,Table)  On(C,Table)
! Variables thatappear in  Clear(C) ) and effects
 Clear(Α) Clear(Β)
preconditions
need to be parameters of the action schema
 Goal( On(Α,Β)  On(Β,C) )
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y)
EFFECTS: On(b,y)  Clear(x)  On(b,x)
 Clear(y) )
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b)
EFFECTS: On(b,Table)  Clear(x)  On(b,x) )

43. STRIPS planning
43
 Init( On(Α,Table)  On(Β,Table)  On(C,Table)
! Variables thatappear in  Clear(C) ) and effects
 Clear(Α) Clear(Β)
preconditions
need to be parameters of the action schema
 Goal( On(Α,Β)  On(Β,C) )
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y)
EFFECTS: On(b,y)  Clear(x)  On(b,x)
 Clear(y) )
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b)
EFFECTS: On(b,Table)  Clear(x)  On(b,x) )

44. STRIPS planning
44
 Init( On(Α,Table)  On(Β,Table)  On(C,Table)
 Clear(Α)  Clear(Β)  Clear(C) )
 Goal( On(Α,Β)  On(Β,C) )
 Action( Move(b,x,y),
PRECONDITIONS: On(b,x)  Clear(b)  Clear(y)
EFFECTS: On(b,y)  Clear(x)  On(b,x)
 Clear(y) )
 Action( MoveToTable(b,x),
PRECONDITIONS: On(b,x)  Clear(b)
EFFECTS: On(b,Table)  Clear(x)  On(b,x) )
! Simplified STRIPS version of Figure 11.4 in AIMA 2nd Ed.

45. STRIPS planning
45
 Gripper domain
 Init( On-Table(Α)  On-Table(Β)  On-table(C)
 Clear(Α)  Clear(Β)  Clear(C)  Grip-Empty )
 Goal( On(Α,Β)  On(Β,C) )
 Action( Pick-up(x), … )
 Action( Put-down(x), … )
 Action( Stack(top,below), … )
 Action( Unstack(top,below), … )

46. STRIPS planning
46
 Reasoning about action and change
 Since 1971 that STRIPS planning was proposed by R. E.
Fikes and N. J. Nilsson, more expressive planning
formalisms have been developed based on STRIPS!
 ADL allows open worlds, conditional effects, quantifiers
[Pednault 1988]
 Situation calculus [McCarthy, Hayes 1969] [Reiter 2001]
is based on first-order logic allowing rich
representations
 Action languages A, Fluent calculus, Event calculus, …

47. Bibliography
47
 Material
 Artificial Intelligence: A Modern Approach 2nd Ed. Stuart Russell, Peter
Norvig. Prentice Hall, 2003 Section 11.1
 References
 STRIPS: A new approach to the application of theorem proving to
problem solving. Richard E. Fikes, Nils J. Nilsson. Artificial Intelligence,
Vol. 2, No. 3-4, 1971.
 Synthesizing plans that contain actions with context-dependent effects.
E. P. D. Pednault. Computational Intelligence, Vol. 4, No. 3. 1988.
 Some Philosophical Problems from the Standpoint of Artificial
Intelligence. John McCarthy, Patrick J. Hayes. Machine Intelligence, Vol.
4, 1969.
 Knowledge in Action: Logical Foundations for Specifying and
Implementing Dynamical Systems. Raymond Reiter. MIT Press, 2001.