Contenu connexe

Similaire à This was a triumph: Evolving intelligent bots for videogames. And for Science. (20)


This was a triumph: Evolving intelligent bots for videogames. And for Science.

  1. This was a triumph: Evolving intelligent bots for videogames. And for Science. Pablo García Sánchez @fergunet @geneura University of Granada
  2. Index • Introduction • Evolutionary Algorithms • FPS • RTS • Content Generation
  3. Evolutionary Algorithms
  4. What are the Evolutionary Algorithms? • Metaheuristics • Solve Optimization problem • Based on a fitness function • Population based • Natural selection • Stochastic
  5. Individuals No Yes Yes No Yes No F=3 BlueVenom Fire Fly Fangs Horns
  6. Mating No Yes Yes No Yes No No Yes No No Yes Yes
  7. Offspring No Yes Yes No Yes NoNo Yes No No Yes Yes F=2 F=4
  8. Mutation No Yes Yes No Yes Yes F=4Yes F=5
  9. EA stages
  10. Example: Super Mario Bot • The actions of Mario are Evolved • VIDEO TIME!
  11. FPS
  12. Unreal Tournament 2004 • Edited with UnrealScript • Java engine to access the game: POGAMUT
  13. Unreal Bot • Parameter optimization of a FSM to play Unreal Tournament 2004 • Each chromosome is a vector of parameters
  14. UnrealBot Team • The same, but optimizing the parameters of a team.
  15. Unreal ExpertBot • Two-level FSM based in the experience of a expert player (Andalusia champion) in 1vs1 matches. • For example, it models which weapon to use. It wins “normal” human players.
  16. Unreal InteractiveBot • From ExpertBot • Experts perform actions during the algorithm • Expert player: answer questions to block some parts of the chromosome • Expert researcher • Second position 2KBotPrice 2014
  17. RTS
  18. Real Time Strategy (RTS) • Road traffic, finances or weahter forecasting: • Complex and dynamic environments • No completed nor perfect information • RTS->simplification • Uncertainty decisión taking • Real time plannification
  19. RTS features • Simultaneous movement • Durative actions (require time) • Real time (quick decisions) • Partially observable (fog of war) • Undeterministic • Complex
  20. Challenges in RTS • Planning • Learning • Uncertainty • Spacial and temporal reasoning • Domain knowledge exploitation • Task decomposition andreita42 en deviantart
  21. Challenges: Planification • Different abstract levels • Long term strategy (match) • Short term strategy (battles) • Techniques: sampling or hierarchical decomposition
  22. Challenges: Learning • Existent learning (ex. matches DB) • In-game learning (ex. reinforcement learning) • Inter-game learning (ej. learn from previous game to win the next)
  23. Challenges: Uncertainty • Spatial uncertainty (fog of war) • Enemy action
  24. Challenges: Reasoning • Spatial reasoning: where to build the base • Temporal reasoning: when to create technology
  25. Challenges: task decomposition • Strategy • Tactics • Reactive control • Terrain analisys • Intelligent gathering • Micro and macro tasks
  26. Planet Wars
  27. Planet Wars
  28. Problem description • Restrictions: • 1 second to decide • No memory can be used • Current state: planets and fleets • Actions: a text file read by the game Current state Actions to perform Bot
  29. AresBot • Select base • Select objective • For each colony: • Reinforce base • Reinforce attack (este es Antares)
  30. FSM
  31. Genebot Operation • AresBot parameter optimization • Chromosome: parameter array • 2-Tournament • BLX-alpha crossover • Re-evaluation • 40 seconds to evaluate an individual Propiedad de EA (no me demandéis!)
  32. Exp-Genebot • Detect the type of map to chose a previously optimized set of parameters
  33. GP-bot • New approach: Genetic Programming • The genome is a TREE • Decisions: e.g. if distance to the base is lower than 0.25 • Actions: e.g. attack the base with the 20% of ships • Java code compiled in each evaluation using CAssist
  34. Example of a chromosome (tree) actualMyShipsRatio>0.711 attackWeakestNeutralPlanet(0.3) attackNearestEnemyPlanet(0.2) attackEnemyBase(0.91) myShipsLandedFlyingRatio>0.2
  35. Results (validation) 3G 7G UG 3E 7E UE 100200300400500 Configuration Turnstobedefeated 3G 7G UG 3E 7E UE 10203040506070 Configuration Percentageofvictories
  36. Stop criteria? • Number of generations • Outlier age • Replacement rate • Fitness threshold • Fitness improvement respect to average
  37. When to stop?
  38. What we learnt? • Compare with previous version to IMPROVE • Use hierarchical fitness • Individuals reevaluation • Validate always aginst bots/maps not used from learning • Replacement based stop criteria • Use free software
  39. StarCraft Propiedad de Blizzard
  40. Why StarCraft? • Competitions: CIG, AIIDE, SCSC... • Available frameworks and libraries • Replays database (ej: • Unified Test-bed in research on RTS
  41. BWAPI • Create AIs (in .dll) • Information about: • Game: minerals, map, game type, latency... • Units: distance, speed, detection, status... • Player: race, status, resources, deaths...
  42. ChaosLauncher
  43. Tournament Manager
  44. StarCraftGP • Framework to automatically generate bots to play Starcraft using Genetic Programming • Basad on uGP3 • We use Linear-GP to generate strategies for bots using the uGP3 framework (using OpprimoBot as base) • In collaboration with INRA Politecnico di Torino
  45. External Evaluator Evolutionary Core Constraints .txt .txt TournamentManager External Evaluator Individual.DLL results.txtParser Fitness values VM Server VM Client VM Client VM Client … VStudio Compiler Individual.cpp
  46. Bot example
  47. Results
  48. Content Generation Story generation in massively distributed environments: MADE Propiedad de Bethesda
  49. The problem
  50. MADE • Massive Artificial Drama Engine • A GA optimizes a multi-agent system • Archetype based • Available in
  51. Archetypes Laffent in
  52. Methodology
  53. Agent
  54. Libraries
  55. References (Planet Wars) • A. Fernández-Ares, A. Miguel Mora, J. J. Merelo Guervós, P. García-Sánchez, C. M. Fernandes: Optimizing player behavior in a real-time strategy game using evolutionary algorithms. IEEE Congress on Evolutionary Computation 2011: 2017-2024 (2012) • A. Miguel Mora, A. Fernández-Ares, J. J. Merelo Guervós, P. García-Sánchez, C. M. Fernandes: Effect of Noisy Fitness in Real-Time Strategy Games Player Behaviour Optimisation Using Evolutionary Algorithms. J. Comput. Sci. Technol. 27(5): 1007-1023 (2012) • Antonio Fernández-Ares, Pablo García-Sánchez, Antonio Miguel Mora, Juan J. Merelo Guervós: Adaptive bots for real-time strategy games via map characterization. CIG 2012: 417-721 (2012) • Antonio Fernández-Ares, Pablo García-Sánchez, Antonio M. Mora, Pedro A. Castillo Valdivieso, Juan Julián Merelo Guervós, Maria I. García Arenas, Gustavo Romero: It's Time to Stop: A Comparison of Termination Conditions in the Evolution of Game Bots. EvoApplications 2015: 355-368 • Pablo García-Sánchez, Antonio Fernández-Ares, Antonio Miguel Mora, Pedro A. Castillo Valdivieso, Jesús González, Juan Julián Merelo Guervós: Tree Depth Influence in Genetic Programming for Generation of Competitive Agents for RTS Games. EvoApplications 2014: 411-421
  56. References (Unreal) • Antonio Miguel Mora, Ramón Montoya, Juan Julián Merelo Guervós, Pablo García-Sánchez, Pedro A. Castillo, Juan Luís Jiménez Laredo, Ana Isabel Martínez García, Anna Esparcia-Alcázar: Evolving Bot AI in Unreal. EvoApplications (1) 2010: 171-180 • Antonio Miguel Mora, Francisco Aisa, Ricardo Caballero, Pablo García-Sánchez, Juan Julián Merelo Guervós, Pedro A. Castillo, Raúl Lara-Cabrera: Designing and Evolving an Unreal TournamentTM 2004 Expert Bot. IWANN (2) 2013: 312-323 • Antonio Miguel Mora, M. A. Moreno, Juan Julián Merelo Guervós, Pedro A. Castillo, Maribel García Arenas, Juan Luís Jiménez Laredo: Evolving the cooperative behaviour in Unreal™ bots. CIG 2010: 241-248
  57. References (MADE) • Rubén Héctor García-Ortega, Pablo García-Sánchez, Juan Julián Merelo Guervós, María Isabel García Arenas, Pedro A. Castillo Valdivieso, Antonio M. Mora: How the World Was MADE: Parametrization of Evolved Agent- Based Models for Backstory Generation. EvoApplications 2015: 443-454 • Garcia-Ortega, Ruben H., et al. My life as a sim: evolving unique and engaging life stories using virtual worlds. ALIFE 14: The Fourteenth Conference on the Synthesis and Simulation of Living Systems. Vol. 14. Pages 580-587, 2014 • • @made_project • @rubenhek
  58. Free Software and Open Science • GitHub: fergunet, deantares, amorag, jjmerelo, geneura, geneura-papers • • Text in this presentation is CC-SA-BY (free license images used, non commercial, except images from Blizzard, Bethesda, EA and )
  59. Thanks! • @fergunet • @amoragar • @geneura • @antaress