This was a triumph: Evolving
intelligent bots for videogames.
And for Science.
Pablo García Sánchez @fergunet
@geneura
University of Granada

Index
• Introduction
• Evolutionary
Algorithms
• FPS
• RTS
• Content
Generation
commons.wikipedia.org

Evolutionary
Algorithms
hideyourarms.com

What are the Evolutionary
Algorithms?
• Metaheuristics
• Solve Optimization problem
• Based on a fitness function
• Population based
• Natural selection
• Stochastic

Individuals
No Yes Yes No Yes No F=3
BlueVenom
Fire Fly
Fangs
Horns

Mating
No Yes Yes No Yes No
No Yes No No Yes Yes

Offspring
No Yes Yes
No Yes NoNo Yes No
No Yes Yes
F=2
F=4

Mutation
No Yes Yes No Yes Yes F=4Yes F=5

EA stages

Example: Super Mario Bot
• The actions of
Mario are
Evolved
• VIDEO TIME!

FPS
commons.wikipedia.org

Unreal Tournament 2004
• Edited with UnrealScript
• Java engine to access the game: POGAMUT

Unreal Bot
• Parameter optimization of a FSM to play Unreal
Tournament 2004
• Each chromosome is a vector of parameters

UnrealBot Team
• The same, but optimizing the parameters of a team.

Unreal ExpertBot
• Two-level FSM based in the experience of a expert player (Andalusia
champion) in 1vs1 matches.
• For example, it models which weapon to use. It wins “normal” human
players.

Unreal InteractiveBot
• From ExpertBot
• Experts perform actions
during the algorithm
• Expert player: answer
questions to block some
parts of the chromosome
• Expert researcher
• Second position
2KBotPrice 2014

RTS
commons.wikipedia.org

Real Time Strategy (RTS)
• Road traffic, finances or weahter forecasting:
• Complex and dynamic environments
• No completed nor perfect information
• RTS->simplification
• Uncertainty decisión taking
• Real time plannification

RTS features
• Simultaneous movement
• Durative actions (require time)
• Real time (quick decisions)
• Partially observable (fog of war)
• Undeterministic
• Complex

Challenges in RTS
• Planning
• Learning
• Uncertainty
• Spacial and temporal
reasoning
• Domain knowledge
exploitation
• Task decomposition
andreita42 en deviantart

Challenges: Planification
• Different abstract levels
• Long term strategy (match)
• Short term strategy (battles)
• Techniques: sampling or
hierarchical decomposition
pixabay.com

Challenges: Learning
• Existent learning (ex. matches
DB)
• In-game learning (ex.
reinforcement learning)
• Inter-game learning (ej. learn
from previous game to win
the next)
soft9000.com

Challenges: Uncertainty
• Spatial uncertainty (fog of
war)
• Enemy action
commons.wikipedia.org

Challenges: Reasoning
• Spatial reasoning: where to
build the base
• Temporal reasoning: when to
create technology
fc02.deviant.org

Challenges: task
decomposition
• Strategy
• Tactics
• Reactive control
• Terrain analisys
• Intelligent gathering
• Micro and macro tasks
insightly.com

Planet Wars
gaskin1982.deviantart.com

Planet Wars

Problem description
• Restrictions:
• 1 second to decide
• No memory can be used
• Current state: planets and fleets
• Actions: a text file read by the game
Current
state
Actions to
perform
Bot

AresBot
• Select base
• Select objective
• For each colony:
• Reinforce base
• Reinforce attack
(este es Antares)

FSM

Genebot Operation
• AresBot parameter
optimization
• Chromosome: parameter
array
• 2-Tournament
• BLX-alpha crossover
• Re-evaluation
• 40 seconds to evaluate an
individual
Propiedad de EA (no me demandéis!)

Exp-Genebot
• Detect the type of map to chose a previously
optimized set of parameters

GP-bot
• New approach: Genetic Programming
• The genome is a TREE
• Decisions: e.g. if distance to the base is lower than
0.25
• Actions: e.g. attack the base with the 20% of ships
• Java code compiled in each evaluation using CAssist

Example of a chromosome
(tree)
actualMyShipsRatio>0.711
attackWeakestNeutralPlanet(0.3)
attackNearestEnemyPlanet(0.2)
attackEnemyBase(0.91)
myShipsLandedFlyingRatio>0.2

Results (validation)
3G 7G UG 3E 7E UE
100200300400500
Configuration
Turnstobedefeated
3G 7G UG 3E 7E UE
10203040506070
Configuration
Percentageofvictories

Stop criteria?
• Number of generations
• Outlier age
• Replacement rate
• Fitness threshold
• Fitness improvement respect to average

When to stop?

What we learnt?
• Compare with previous version to IMPROVE
• Use hierarchical fitness
• Individuals reevaluation
• Validate always aginst bots/maps not used from
learning
• Replacement based stop criteria
• Use free software

StarCraft
Propiedad de Blizzard

Why StarCraft?
• Competitions: CIG, AIIDE, SCSC...
• Available frameworks and libraries
• Replays database (ej: http://bwreplays.com/)
• Unified Test-bed in research on RTS

BWAPI
• Create AIs (in .dll)
• Information about:
• Game: minerals, map, game type, latency...
• Units: distance, speed, detection, status...
• Player: race, status, resources, deaths...

ChaosLauncher

Tournament Manager

StarCraftGP
• Framework to automatically generate bots to play
Starcraft using Genetic Programming
• Basad on uGP3
• We use Linear-GP to generate strategies for bots using
the uGP3 framework (using OpprimoBot as base)
• In collaboration with INRA Politecnico di Torino

External Evaluator
Evolutionary Core
Constraints
.txt
.txt
TournamentManager
External Evaluator
Individual.DLL
results.txtParser
Fitness values
VM Server
VM Client
VM Client
VM Client
…
VStudio Compiler
Individual.cpp

Bot example

Results

Content
Generation
Story generation in massively
distributed environments: MADE
Propiedad de Bethesda

The problem

MADE
• Massive Artificial Drama Engine
• A GA optimizes a multi-agent system
• Archetype based
• Available in made.velonuboso.com

Archetypes
Laffent in deviantart.org

Methodology

Agent

Libraries

References (Planet Wars)
• A. Fernández-Ares, A. Miguel Mora, J. J. Merelo Guervós, P. García-Sánchez, C. M. Fernandes:
Optimizing player behavior in a real-time strategy game using evolutionary algorithms. IEEE
Congress on Evolutionary Computation 2011: 2017-2024 (2012)
• A. Miguel Mora, A. Fernández-Ares, J. J. Merelo Guervós, P. García-Sánchez, C. M. Fernandes:
Effect of Noisy Fitness in Real-Time Strategy Games Player Behaviour Optimisation Using
Evolutionary Algorithms. J. Comput. Sci. Technol. 27(5): 1007-1023 (2012)
• Antonio Fernández-Ares, Pablo García-Sánchez, Antonio Miguel Mora, Juan J. Merelo Guervós:
Adaptive bots for real-time strategy games via map characterization. CIG 2012: 417-721 (2012)
• Antonio Fernández-Ares, Pablo García-Sánchez, Antonio M. Mora, Pedro A. Castillo Valdivieso,
Juan Julián Merelo Guervós, Maria I. García Arenas, Gustavo Romero: It's Time to Stop: A
Comparison of Termination Conditions in the Evolution of Game Bots. EvoApplications 2015:
355-368
• Pablo García-Sánchez, Antonio Fernández-Ares, Antonio Miguel Mora, Pedro A. Castillo
Valdivieso, Jesús González, Juan Julián Merelo Guervós: Tree Depth Influence in Genetic
Programming for Generation of Competitive Agents for RTS Games. EvoApplications 2014:
411-421

References (Unreal)
• Antonio Miguel Mora, Ramón Montoya, Juan Julián Merelo Guervós,
Pablo García-Sánchez, Pedro A. Castillo, Juan Luís Jiménez Laredo,
Ana Isabel Martínez García, Anna Esparcia-Alcázar: Evolving Bot AI
in Unreal. EvoApplications (1) 2010: 171-180
• Antonio Miguel Mora, Francisco Aisa, Ricardo Caballero, Pablo
García-Sánchez, Juan Julián Merelo Guervós, Pedro A. Castillo,
Raúl Lara-Cabrera: Designing and Evolving an Unreal
TournamentTM 2004 Expert Bot. IWANN (2) 2013: 312-323
• Antonio Miguel Mora, M. A. Moreno, Juan Julián Merelo Guervós,
Pedro A. Castillo, Maribel García Arenas, Juan Luís Jiménez Laredo:
Evolving the cooperative behaviour in Unreal™ bots. CIG 2010:
241-248

References (MADE)
• Rubén Héctor García-Ortega, Pablo García-Sánchez, Juan Julián Merelo
Guervós, María Isabel García Arenas, Pedro A. Castillo Valdivieso, Antonio
M. Mora: How the World Was MADE: Parametrization of Evolved Agent-
Based Models for Backstory Generation. EvoApplications 2015: 443-454
• Garcia-Ortega, Ruben H., et al. My life as a sim: evolving unique and
engaging life stories using virtual worlds. ALIFE 14: The Fourteenth
Conference on the Synthesis and Simulation of Living Systems. Vol. 14.
Pages 580-587, 2014
• http://made.velonuboso.com
• @made_project
• @rubenhek

Free Software and Open
Science
• GitHub: fergunet, deantares, amorag, jjmerelo,
geneura, geneura-papers
• http://geneura.ugr.es
• Text in this presentation is CC-SA-BY (free license
images used, non commercial, except images from
Blizzard, Bethesda, EA and clippart.co )

Thanks!
• @fergunet
• @amoragar
• @geneura
• @antaress