SlideShare a Scribd company logo
1 of 11
Download to read offline
Car racing competition(s):
   lessons learned and
     future directions
  Julian Togelius, Daniele Loiacono, Pier Luca Lanzi
Car Racing Competition:
  2007, 2008, 2009...?
• We want to make this a recurring event,
  increasing both the quality of submitted
  controllers and of the competition
  organization
• Last year: used the simplerace game
  (lightweight Java implementation)
• 5 entries for CIG, 12 for CEC
Comparing TORCS
     to simplerace
• More advanced/realistic (e.g. better
  dynamics and collision handling, gear shifting)
• Harder (in a sense)
• Completely deterministic (no noise)
• Slower. Much slower...
• Not completely cross-platform
• Not designed for learning algorithms
Not designed for
  learning algorithms...
• Overhead from restarting
• Memory leak
• Not simple for client to select track
• Instant shutdown from excessive car damage
• Exploits (degenerate strategies possible)
 • crossing the start line backwards!
However...

• All of the problems (except memory leak)
  have been solved with client- or server-side
  patches
• Taken together, TORCS is the best
  alternative we’ve found
The future of the car
  racing competition
• We want to make this a recurring event,
  continuously improving the quality of both
  competition and entries
• Next iteration confirmed for CIG 2008
• Several questions regarding in which
  direction to evolve the competition...
 • we want your input!
The future of the car
  racing competition
• What can we improve?
 • Measuring learning rather than design
 • Accessibility and participation
 • Validity and generality of results
 • Dissemination
Measuring learning
rather than design skills
• How do we measure the power of learning
  algorithms and representations rather than
  the competitors’ programming skills?
  • Varying the task (e.g. tracks, cars)
    between training and scoring
  • Automatic, track-specific learning phase
    after submission
• Is this important?
Accessibility and
       participation
• Last year we had much higher participation
  second time around (same software)
• How can we make it easier to participate?
 • Interfaces in more languages? (which?)
 • More example trainers / controllers?
• Should we reach out to other communities?
  (classical RL people, game developers etc.)
Validity and generality
(what can we learn?)
• That a controller based on algorithm X
  wins, does not prove that algorithm X is
  better than others for car racing...
  • How do we improve the validity of the
    competition results?
• ...it also does not prove that algorithm X is
  good for any other (car control) tasks
  • How do we ensure generality?
Dissemination

• More people will submit better controllers
  if they can get a publication out of it
• Last year’s competitions became a 37-page
  GPEM paper...
• Is there a better publication format? Special
  issues? Workshop proceedings?

More Related Content

Similar to Car Racing Competition at WCCI2008 - Summary

Complexity...and the illusion of simplicity: Part One
Complexity...and the illusion of simplicity: Part OneComplexity...and the illusion of simplicity: Part One
Complexity...and the illusion of simplicity: Part One
Graham Beale
 
Machine_Learning_and_drive_grading_HEPIX_October_2015
Machine_Learning_and_drive_grading_HEPIX_October_2015Machine_Learning_and_drive_grading_HEPIX_October_2015
Machine_Learning_and_drive_grading_HEPIX_October_2015
Amit Chattopadhyay
 

Similar to Car Racing Competition at WCCI2008 - Summary (20)

Car Care.pptx
Car Care.pptxCar Care.pptx
Car Care.pptx
 
Phil Koopman's ISSRE 2016 Keynote
Phil Koopman's ISSRE 2016 KeynotePhil Koopman's ISSRE 2016 Keynote
Phil Koopman's ISSRE 2016 Keynote
 
Investing in a good software factory and automating the build process
Investing in a good software factory and automating the build processInvesting in a good software factory and automating the build process
Investing in a good software factory and automating the build process
 
Building Beautiful High Performance Connected Car Applications
Building Beautiful High Performance Connected Car ApplicationsBuilding Beautiful High Performance Connected Car Applications
Building Beautiful High Performance Connected Car Applications
 
Vehicle Detection using Camera
Vehicle Detection using CameraVehicle Detection using Camera
Vehicle Detection using Camera
 
AWS DeepRacer desde cero - Meetup de awsvalencia (2021/08/12)
AWS DeepRacer desde cero - Meetup de awsvalencia (2021/08/12)AWS DeepRacer desde cero - Meetup de awsvalencia (2021/08/12)
AWS DeepRacer desde cero - Meetup de awsvalencia (2021/08/12)
 
Total cost of ownership
Total cost of ownershipTotal cost of ownership
Total cost of ownership
 
Download Magazine.pdf
Download Magazine.pdfDownload Magazine.pdf
Download Magazine.pdf
 
Interactive Image Processing Demos for the Web
Interactive Image Processing Demos for the WebInteractive Image Processing Demos for the Web
Interactive Image Processing Demos for the Web
 
Road traffic rules synthesis using ge
Road traffic rules synthesis using geRoad traffic rules synthesis using ge
Road traffic rules synthesis using ge
 
AWS Finland Meetup June 2019 - DeepRacer story
AWS Finland Meetup June 2019 - DeepRacer storyAWS Finland Meetup June 2019 - DeepRacer story
AWS Finland Meetup June 2019 - DeepRacer story
 
Complexity...and the illusion of simplicity: Part One
Complexity...and the illusion of simplicity: Part OneComplexity...and the illusion of simplicity: Part One
Complexity...and the illusion of simplicity: Part One
 
Machine_Learning_and_drive_grading_HEPIX_October_2015
Machine_Learning_and_drive_grading_HEPIX_October_2015Machine_Learning_and_drive_grading_HEPIX_October_2015
Machine_Learning_and_drive_grading_HEPIX_October_2015
 
Open Source CMS 2009
Open Source CMS 2009Open Source CMS 2009
Open Source CMS 2009
 
Open Source Cms 2009
Open Source Cms 2009Open Source Cms 2009
Open Source Cms 2009
 
Influence of Tester in DevOps World
Influence of Tester in DevOps WorldInfluence of Tester in DevOps World
Influence of Tester in DevOps World
 
"How to Test and Validate an Automated Driving System," a Presentation from M...
"How to Test and Validate an Automated Driving System," a Presentation from M..."How to Test and Validate an Automated Driving System," a Presentation from M...
"How to Test and Validate an Automated Driving System," a Presentation from M...
 
2016 ScalaMatsuri LT / セプテーニ・オリジナルの文化と歴史
2016 ScalaMatsuri LT / セプテーニ・オリジナルの文化と歴史2016 ScalaMatsuri LT / セプテーニ・オリジナルの文化と歴史
2016 ScalaMatsuri LT / セプテーニ・オリジナルの文化と歴史
 
How Euroflorist is preparing for Artificial Intelligence
How Euroflorist is preparing for Artificial IntelligenceHow Euroflorist is preparing for Artificial Intelligence
How Euroflorist is preparing for Artificial Intelligence
 
Intro to software development
Intro to software developmentIntro to software development
Intro to software development
 

More from Daniele Loiacono

2010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-20102010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-2010
Daniele Loiacono
 
2010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-20102010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-2010
Daniele Loiacono
 
CIG-2008 Car Racing Competition
CIG-2008 Car Racing CompetitionCIG-2008 Car Racing Competition
CIG-2008 Car Racing Competition
Daniele Loiacono
 

More from Daniele Loiacono (20)

EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013
 
2013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-20132013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-2013
 
2012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-20122012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-2012
 
2012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-20122012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-2012
 
2012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-20122012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-2012
 
Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012
 
XCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental ForgettingXCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental Forgetting
 
Testing learning classifier systems
Testing learning classifier systemsTesting learning classifier systems
Testing learning classifier systems
 
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
 
One Step Fits All
One Step Fits AllOne Step Fits All
One Step Fits All
 
Introducing LCS to Digital Design Verification
Introducing LCS to Digital Design VerificationIntroducing LCS to Digital Design Verification
Introducing LCS to Digital Design Verification
 
A temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networksA temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networks
 
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
 
Automatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier SystemsAutomatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier Systems
 
Voting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label ClassificationVoting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label Classification
 
2011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-20112011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-2011
 
2010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-20102010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-2010
 
2010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-20102010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-2010
 
CIG-2008 Car Racing Competition
CIG-2008 Car Racing CompetitionCIG-2008 Car Racing Competition
CIG-2008 Car Racing Competition
 
Car Racing Competition at WCCI2008 - Diego Perez
Car Racing Competition at WCCI2008 - Diego PerezCar Racing Competition at WCCI2008 - Diego Perez
Car Racing Competition at WCCI2008 - Diego Perez
 

Recently uploaded

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Recently uploaded (20)

Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 

Car Racing Competition at WCCI2008 - Summary

  • 1. Car racing competition(s): lessons learned and future directions Julian Togelius, Daniele Loiacono, Pier Luca Lanzi
  • 2. Car Racing Competition: 2007, 2008, 2009...? • We want to make this a recurring event, increasing both the quality of submitted controllers and of the competition organization • Last year: used the simplerace game (lightweight Java implementation) • 5 entries for CIG, 12 for CEC
  • 3. Comparing TORCS to simplerace • More advanced/realistic (e.g. better dynamics and collision handling, gear shifting) • Harder (in a sense) • Completely deterministic (no noise) • Slower. Much slower... • Not completely cross-platform • Not designed for learning algorithms
  • 4. Not designed for learning algorithms... • Overhead from restarting • Memory leak • Not simple for client to select track • Instant shutdown from excessive car damage • Exploits (degenerate strategies possible) • crossing the start line backwards!
  • 5. However... • All of the problems (except memory leak) have been solved with client- or server-side patches • Taken together, TORCS is the best alternative we’ve found
  • 6. The future of the car racing competition • We want to make this a recurring event, continuously improving the quality of both competition and entries • Next iteration confirmed for CIG 2008 • Several questions regarding in which direction to evolve the competition... • we want your input!
  • 7. The future of the car racing competition • What can we improve? • Measuring learning rather than design • Accessibility and participation • Validity and generality of results • Dissemination
  • 8. Measuring learning rather than design skills • How do we measure the power of learning algorithms and representations rather than the competitors’ programming skills? • Varying the task (e.g. tracks, cars) between training and scoring • Automatic, track-specific learning phase after submission • Is this important?
  • 9. Accessibility and participation • Last year we had much higher participation second time around (same software) • How can we make it easier to participate? • Interfaces in more languages? (which?) • More example trainers / controllers? • Should we reach out to other communities? (classical RL people, game developers etc.)
  • 10. Validity and generality (what can we learn?) • That a controller based on algorithm X wins, does not prove that algorithm X is better than others for car racing... • How do we improve the validity of the competition results? • ...it also does not prove that algorithm X is good for any other (car control) tasks • How do we ensure generality?
  • 11. Dissemination • More people will submit better controllers if they can get a publication out of it • Last year’s competitions became a 37-page GPEM paper... • Is there a better publication format? Special issues? Workshop proceedings?