Presentation of the work "Evolutionary FSM-Based Agents for Playing Super Mario Game" at LION 2013 (LION 7). by Rosa Hidalgo, María Sandra Rodríguez, Antonio M. Mora, Pablo García, Juan Julián Merelo y Antonio J. Fernández Conference http://www.intelligent-optimization.org/LION7/

1. Evolutionary FSM-Based
Agents for Playing
Super Mario Game
LION 2013
R.M. Hidalgo-Bermúdez, M.S. Rodríguez-Domingo, A.M. Mora
P. García-Sánchez, J.J. Merelo, Antonio J. Fernández-Leiva

2. INDEX
1. INTRODUCTION
2. PRELIMINARY CONCEPTS AND BACKGROUND
3. MARIO AI: COMPETITION AND ENVIRONMENT
4. EVOLUTIONARY FSM-BASED AGENT
5. EXPERIMENTS AND RESULTS
6. CONCLUSIONS AND FUTURE WORK

3. INTRODUCTION
Why did we choose Mario Bros?
This game is worldwide known and remains very
popular nowadays.
There is an open source clone of the game named
Infinite Mario Bros.

4. INTRODUCTION
The main goal of the game is to go across lateral
platforming levels, trying to avoid different
enemies and obstacles by using some useful
items.
We have developed an autonomous agent using
different approaches based on Finite State
Machines (FSMs) combined with Genetic
Algorithms (GAs) techniques.

5. PRELIMINARY CONCEPTS AND
BACKGROUND
GAs are based on the model of darwinistic evolution. A
standard GA’s procedure is:

6. PRELIMINARY CONCEPTS AND
BACKGROUND
A Finite State Machine (FSM) is a computational model which
represents a set of states and connections between them.
It is represented as a directed graph, where each node
corresponds to a state.

7. MARIO AI: COMPETITION AND
ENVIRONMENT
Infinite Mario Bros is an open source modification of
Super Mario Bros. It includes wide support for
implementing autonomous agents to control Mario
character using AI techniques.
The Agent follows the rules of Mario AI
Championship. The tracks are:
 Gameplay  Turing Test
 Learning  Level Generation

8. MARIO AI: COMPETITION AND
ENVIRONMENT
The possible actions of the Agent are:
 Go Right  Go Left
 Jump  Down
 Shoot/Run
Mario can be in three different modes:
 Fire  Big  Small

9. MARIO AI: COMPETITION AND
ENVIRONMENT
Game data is presented in the form of two 19x19
matrices. Each cell of each matrix holds a value,
which represents a type of obstacle or enemy ,
respectively

10. MARIO AI: COMPETITION AND
ENVIRONMENT
Obstacles and useful items:

11. MARIO AI: COMPETITION AND
ENVIRONMENT
Enemies:

12. EVOLUTIONARY FSM-BASED AGENT
Possible states

13. EVOLUTIONARY FSM-BASED AGENT
Chromosome (modelling a FSM)

14. EVOLUTIONARY FSM-BASED AGENT
Two approaches:
Mono-seed
All the individuals are tested in the same level
The length of the level and the time limit to complete it grow with
each generation
Multi-seed
Every individual is tested in 30 random levels
The fitness is computed considering the results of all the plays

15. EVOLUTIONARY FSM-BASED AGENT
There are two types of fitness:
 Generic fitness
 Hierarchical fitness
The selection mechanism considers the best individual and a
percentage of the best ones, selected by tournament according to
their fitness.
Crossover is performed considering the best individual of the
present generation as one of the parents, and one of the
individuals with positive fitness as the other parent.
In the mutation operator, various genes in each of these
individuals are randomly selected to be mutated

16. EXPERIMENTS AND RESULTS
Mono-seed approach

17. EXPERIMENTS AND RESULTS
Mono-seed approach

18. EXPERIMENTS AND RESULTS

19. EXPERIMENTS AND RESULTS

20. EXPERIMENTS AND RESULTS
Multi-seed approach

21. SOME OF OUR APPROACHES
Difficulty level 1 (completed):
http://www.youtube.com/watch?v=6Pj6dZCE070
Difficulty level 2 (completed):
http://www.youtube.com/watch?v=gtfuY-L0WDA
Difficulty level 3 (completed):
http://www.youtube.com/watch?v=qQVQ43sWwYY
Difficulty level 12 (stacked):
http://www.youtube.com/watch?v=zNGfBApX7sk

22. CONCLUSIONS AND FUTURE WORK
An agent has been implemented using Finite State Machine (FSM)
models, improved by means of Genetic Algorithms (GA) and
considering different evaluation schemes: mono-seed and a multi-
seed evaluation approaches.
Mono-seed can yield excellent agents for the level where they were
‘trained’ (evolved), having a quite bad behaviour in a different level.
Multi-seed takes much more computational time and has higher
resource requirements, but the agents it yields are very good
playing in any level of the considered difficulty (in the evolution).
All these agents play much better than an expert human player and
can complete the levels in a time impossible to get for the human.

23. Thanks!