1. Car Racing
Competition
Daniele Loiacono, Julian Togelius
and Pier Luca Lanzi
CIG, 17 December 2008, Perth, Australia

2. Goal
 Learn or design a controller for TORCS that
races as fast as possible alone or in the
presence of others drivers
 “Spiritual successor” of CEC 2007
Competition, direct successor of WCCI 2008

3. Car Racing Competition meets
TORCS
 More representative of real game AI
 Better interaction with human players
 Many good programmed controllers available
 Challenges
 How to make it easy accessible?
 Not designed for Machine Learning!
 More similar to a real-world problem

4. 4

5. The Open Racing Car
Simulator

6. The Open Racing Car
Simulator
 TORCS is a state of the art open source simulator
written in C++
 Main features
 Sophisticated dynamics
 Provided with several
cars, tracks, and
controllers
 Active community of
users and developers
 Easy to develop your
own controller
 OS Support
 Linux: binaries and building from sources suppo
 Windows: binaries and “limited” bulding from sources support
 OSX: legacy binaries and no building from sources support 

7. Competition API
 To make TORCS more easy to
use we developed an API based
TORCS
on socket (UDP)
Patch
 Values of sensors and effectors
are sent through UDP Server BOT
 3 components
 Torcs Patch
UDP
 Server Bot (C++)
 Client API (C++ and Java)
Client
Controlller

8. Sensors
 Rangefinders to sense
 Opponents
 Edges of the track
 Speed
 Positionon track
 Rotation speed of wheels Opponent Sensors
 RPM
 Angle with track
 Distance raced
 Fuel and damage
 ...
Track Sensors

9. Effectors
 Basically 4 effectors
 Steeringwheel [-1,+1]
 Gas pedal [0, +1]
 Brake pedal [0,+1]
 Gearbox {-1,0,1,2,3,4,5,6}

10. Effectors
 Basically 4 effectors
 Steeringwheel [-1,+1]
 Gas pedal [0, +1]
 Brake pedal [0,+1]
 Gearbox {-1,0,1,2,3,4,5,6}

11. The Competitors

12. Submissions
5 entries have been submitted to the competition:
 Matt Simmerson – Independent Researcher
 Luigi Cardamone – Politecnico di Milano
 Chung-Cheng Chiu “Redjava” - Academia Sinica
 Diego Perez - University Carlos III, Madrid
 Aravind Gowrisankar - UT Austin, USA
3 more controllers have been considered
 Daniele’s heuristic C++ controller
 Julian’s heuristic Java controller
 WCCI 2008 champion (entered by Matt Simmerson)

13. Matt Simmerson:
NEAT Controller
• Controller implemented as an arbitrary-
topology neural network, evolved with the
NEAT4j implementation of NEAT
• Inputs: speed, angle to track axis, wall
rangefinders, track position, gear, wheel
spin, RPM
• Output: driving and gear selection

14. Politecnico di Milano
Evolving a
Fast Controller
for TORCS
Using NEAT
Luigi Cardamone
Politecnico di Milano
Neuroevolution in TORCS Cardamone
Luigi

15. Overview
 We applied NEAT to evolve a fast controller for the
Car Racing Simulator
 To evaluate the candidate solution we measured
their performance on a complete lap
 The evolved neural controller was then coupled with
a gear shifting policy and with a basic obstacle
avoidance behavior
Neuroevolution in TORCS Cardamone
Luigi

16. Sensors and effectors
Neuroevolution in TORCS Cardamone
Luigi

17. Sensors and effectors
Neuroevolution in TORCS Cardamone
Luigi

18. Sensors and effectors
Neuroevolution in TORCS Cardamone
Luigi

19. Sensors and effectors
Neuroevolution in TORCS Cardamone
Luigi

20. Sensors and effectors
Neuroevolution in TORCS Cardamone
Luigi

21. Sensors and effectors
Neuroevolution in TORCS Cardamone
Luigi

22. Sensors and effectors
 The following sensors were used:
– Track sensors at -90°,-60°,-30°,+30°,+60°,+90°
– Frontal sensor: max reading among the frontal track
sensors at -10°,0°,10°
– Car speed
 The network controls
– Steering wheel
– Gas/Brake pedals
Neuroevolution in TORCS Cardamone
Luigi

23. Controller design
 To avoid wasting time with fast but slower controller,
we set gas pedal to 1 (the max value) when the car
is on a straight (i.e., when the frontal sensor return
100m as reading)
 In addition, the neural controller does not deal with
gear shifting and the scripted policy provided is
used instead
 We also used a very simple scripted policy to avoid
at least the opponents that are “close and in front”
of the bot
Neuroevolution in TORCS Cardamone
Luigi

24. Evaluation of Controllers
 The fitness is computed on an entire lap as
– Where Tout is the number of game tics the bot was
outside the track
– Savg is the average speed
– D is the distance raced
 As soon as a Tout becomes greater than 500 game
tics, the evaluation is immediately stopped
Neuroevolution in TORCS Cardamone
Luigi

25. An example of behaviors evolved
Neuroevolution in TORCS Cardamone
Luigi

26. Aravind Gowrisankar
 Based on NEAT
 Preliminary work
20

27. “Redjava” (Chung-Cheng Chiu)
 The driving policy is based on determining which sensor detects the
most distant value, and turn the steer according to its angle between
driving direction.
 When the car is in the track and the gear is above 3, then it act in the
similar policy as SimpleDriver (but without speed limit). Otherwise,
accelerate. (brake is opposite)
 The policy except steering adopt similar code as SimpleDriver (also the
unstuck detection).
 The code is naive since my original attempt wai to apply a machine
learning method but I got bad results, thus I removed that part.
21

28. The Results

29. Submissions
5 entries have been submitted to the competition:
 Matt Simmerson – Independent Researcher
 Luigi Cardamone – Politecnico di Milano
 Redjava - ???
 Diego Perez - University Carlos III, Madrid
 Aravind - ???
3 more controllers have been considered
 Daniele’s heuristic C++ controller
 Julian’s heuristic Java controller
 WCCI 2008 champion (entered by Matt Simmerson)

30. Scoring setup
A server with 2 Quad-core Xeon 2.66 GHz,
8GB RAM, running Fedora Core 6

31. Scoring process: first stage
 Scoring
process involves three tracks
(unknown to the competitors):
 C-Speedway
 E-Track6
 Wheel 2
 Onlya controller at once is tested and
performance is defined as the distance
covered within 10000 game tics

32. First Stage: C-Speedway
C-Speedway
Redjava
Luigi
Diego
Competitors
Matt
Aravind
WCCI Champ
Daniele
Julian
0 5000 10000 15000 20000
Distance Raced

33. First Stage: E-Track 6
E-Track 6
Luigi
Redjava
Aravind
Competitors
Diego
Matt
WCCI Champ
Daniele
Julian
0 2000 4000 6000 8000
Distance Raced

34. First Stage: Wheel 2
Wheel 2
Redjava
Luigi
Matt
Competitors
Diego
Aravind
WCCI Champ
Daniele
Julian
0 2250 4500 6750 9000
Distance Raced

35. Comments on firs stage
 Two clear winners:
 Redjava
 Luigi
 WCCI Champ still competitive in all but
Wheel 2 track
 Matt’s new entry is outperformed by the
WCCI champ in all tracks but Wheel 2 (the
most difficult track)

36. The final stage
 Who is the best controller in presence of
opponents?
 In the final stage we compared the best
entries with the previous champion
 For each track we run 5 races with random
starting grids
 The final score is computed as the median of
the 5 races using the GP2 point system (10
to first, 8 to second, 6 to third AND 1 bonus
point for the fastest lap)

38. Final results
 After 25 races…
Luigi Cardamone is the winner of the
CIG 2008 Simulated Car Racing

39. Summary of results
 Why did Luigi win this competition?
 It is not the fastest but the most reliable one:
 Redjava’s controller is extremely fast but often
looses the control of the car
 It often reaches the max amount of damage or
just crashes into other opponents
 Better shifting policy? (though both hard-coded)

40. Summary of results
 Good things:
 Significant improvements with respect to the previous
competition!
 One new learned controller performed significantly better
than programmed controllers!
 Hand-coding still very competitive
 An important step to the actual level of the best
programmed controller in TORCS… but the distance is still
big
 Bad things:
 Still poor performance in avoiding and overtaking
opponents!
 Many crazy and “dirty” behaviors to go as fast as possible…
is really what we are looking for