Slides from the Paris Game/AI Conference 2011 talk by Neil Henning - covering the

1. Preparing AI for Parallelism Lessons from NASCAR The Game 2011 Neil Henning – Technology Lead Neil Henning neil@codeplay.com Paris Game AI Conference 2011

2. Introduction Neil Henning neil@codeplay.com Paris Game AI Conference 2011

3. Introduction I am sure some of you are wondering... Why a guy from is doing a talk about which was developed by Neil Henning neil@codeplay.com Paris Game AI Conference 2011

4. Introduction Team from Codeplay worked for 15 months on game Neil Henning neil@codeplay.com Paris Game AI Conference 2011

6. 43 cars on screen at the same time

7. Cars race in tight packs on the circuit

8. Overtaking is all about navigating through these packs Cannot simply make the AI use LODs, nearly always in view Neil Henning neil@codeplay.com Paris Game AI Conference 2011

9. Agenda Neil Henning neil@codeplay.com Paris Game AI Conference 2011

12. Why prepare your AI for parallelism?

13. What changes should be made?

14. What common issues are there?

15. How did these changes help when optimizing NASCAR?

16. How did we make use of the PS3's unique hardware? What performance improvement was achieved? Neil Henning neil@codeplay.com Paris Game AI Conference 2011

17. Why prepare your AI for parallelism? Neil Henning neil@codeplay.com Paris Game AI Conference 2011

19. Say we have four bots

20. In serial – can easily fit in a frameframe length Neil Henning neil@codeplay.com Paris Game AI Conference 2011

22. Have to either optimize or parallelize (or both)frame length Neil Henning neil@codeplay.com Paris Game AI Conference 2011

24. Only possible with parallelismframe length Neil Henning neil@codeplay.com Paris Game AI Conference 2011

26. This generation of consoles are multicore

27. Sony's new PS Vita is quad core

28. Even iPad usesdual core processors now!

29. Being able to split work amongst cores is key

30. Might not be required yet, but could be essential laterNeil Henning neil@codeplay.com Paris Game AI Conference 2011

32. Optimization being sought throughout engine

33. Either optimize engine or cut features

34. Have AI prepared to become parallel

35. Optimization folks will love you!Neil Henning neil@codeplay.com Paris Game AI Conference 2011

36. What changes should be made? Neil Henning neil@codeplay.com Paris Game AI Conference 2011

38. In NASCAR, had 18 components for each carDriving Controllers Stay Behind Obstacle Detection Stay Beside Neil Henning neil@codeplay.com Paris Game AI Conference 2011

40. All components in a group can be run in parallelNeil Henning neil@codeplay.com Paris Game AI Conference 2011

42. Each car’s groups can be run in parallel too0 1 2 … … 42 Neil Henning neil@codeplay.com Paris Game AI Conference 2011

44. Two phases for your AI

45. Read phase can read world/other car state

46. Write phase can modify own car stateNeil Henning neil@codeplay.com Paris Game AI Conference 2011

48. In read phase, store needed reads into temporary data

49. In write phase, read from the temporary data

50. AI is one frame behind world events

51. Effect on AI is minimalNeil Henning neil@codeplay.com Paris Game AI Conference 2011

54. Only components that required world/other car stateNeil Henning neil@codeplay.com Paris Game AI Conference 2011

59. Documenting the relationship saves time and effort laterNeil Henning neil@codeplay.com Paris Game AI Conference 2011

60. What common issues are there? Neil Henning neil@codeplay.com Paris Game AI Conference 2011

62. ~500-1200 cycles on PowerPC if virtual lookup misses cache

63. Can equate to a large amount of time doing no workNeil Henning neil@codeplay.com Paris Game AI Conference 2011

65. Based on previous game (Supercar Challenge)

66. 16 cars in previous, now 43 cars

67. 5 component types in previous, now 18 component types

68. Now read/write phase too

69. 80 virtual calls to update became 1333 virtual calls!Neil Henning neil@codeplay.com Paris Game AI Conference 2011

71. In real terms, 3ms of virtual function lookup per frame

72. First optimization was to have typed buckets of components

73. 1333 virtual calls went to 31 virtual calls

74. Platform agnostic (PS3, 360 and Wii all sped up)Neil Henning neil@codeplay.com Paris Game AI Conference 2011

76. Not knowing the size of data kills SPU/Compute developmentNeil Henning neil@codeplay.com Paris Game AI Conference 2011

78. Avoid/reduce/remove locks if possible

81. AI required knowledge of obstacles

82. Physics system used, raycast to find problematic obstacles

83. Each call to raycast used a mutex, every thread would halt!

84. Had to refactor code to remove need for lockingNeil Henning neil@codeplay.com Paris Game AI Conference 2011

86. How did these changes help when optimizing NASCAR? Neil Henning neil@codeplay.com Paris Game AI Conference 2011

88. Allowed work to be split across all 6 threads

89. Each thread was given 1/6th of the cars to process

90. Takes 2ms of all CPU resources on 360 in a frameNeil Henning neil@codeplay.com Paris Game AI Conference 2011

92. Only 2 threads on PS3, but have 6 sub processors (the SPUs)

93. Both threads on PS3 were completely full

95. How did these changes help when optimizing NASCAR? On PS3 most costly components were targeted Neil Henning neil@codeplay.com Paris Game AI Conference 2011

97. And all components in a group being able to be run in parallel

98. Costly groups were made to use the SPUs

99. Knowing relationship between data was key

100. Well documented code made life so much easier!Neil Henning neil@codeplay.com Paris Game AI Conference 2011

101. How did we make use of the PS3's unique hardware? Neil Henning neil@codeplay.com Paris Game AI Conference 2011

103. Very tight deadlines, 1 month to reduce time taken in AI

104. No main thread time left – have to use the SPUs

105. Our Offload compiler technology crucialNeil Henning neil@codeplay.com Paris Game AI Conference 2011

107. They are amazingly fast, if you code correctly for them

108. Normally requires total rewrite of existing codebase

109. Painful to access global variables

110. Virtual functions are a complete write offNeil Henning neil@codeplay.com Paris Game AI Conference 2011

112. Common to have 4-5 SPU programmers for ~10 months

113. Not feasible for late-in-cycle development

114. Offload aims to mitigate the issues with getting code onto SPU

115. Can offload code to SPU much quicker (typically a few man days)

116. Much easier to move existing code bases to SPUNeil Henning neil@codeplay.com Paris Game AI Conference 2011

118. Any work within an offload block is performed on the SPU

119. All PPU code is duplicated for the SPUNeil Henning neil@codeplay.com Paris Game AI Conference 2011

121. Just use them as normal!Neil Henning neil@codeplay.com Paris Game AI Conference 2011

123. Just have to specify which virtual functions may be calledNeil Henning neil@codeplay.com Paris Game AI Conference 2011

125. Four components taking most of the frame timeNeil Henning neil@codeplay.com Paris Game AI Conference 2011

128. How did we make use of the PS3's unique hardware? Stay Behind Other Car Stay Beside Other Car Looked at Stay Behind/Beside Other Car together In the same group, can be run in parallel Neil Henning neil@codeplay.com Paris Game AI Conference 2011

129. How did we make use of the PS3's unique hardware? Stay Behind Other Car Stay Beside Other Car Moved Stay Behind component to SPU Stay Beside component would continue to be run on PPU Neil Henning neil@codeplay.com Paris Game AI Conference 2011

130. How did we make use of the PS3's unique hardware? Stay Behind Other Car Stay Beside Other Car As long as SPU work was less time than the PPU work, no cost! Effectively ‘hid’ the cost of calculating Stay Behind component Neil Henning neil@codeplay.com Paris Game AI Conference 2011

133. Changes were purely optimizationAIObstacle** obstacles; unsigned intnumObstacles; offloadThread_thandle = __offload(obstacles, numObstacles) { for(unsigned int i = 0; i < numObstacles; i++) { AIObstacle* obstacle = obstacles[i]; // use obstacle for calculations } }; Neil Henning neil@codeplay.com Paris Game AI Conference 2011

135. Changes were purely optimization// array of AIObstacle* ’s on main memory AIObstacle** obstacles; unsigned intnumObstacles; offloadThread_thandle = __offload(obstacles, numObstacles) { for(unsigned int i = 0; i < numObstacles; i++) { // AIObstacle* points to main memory AIObstacle* obstacle = obstacles[i]; // use obstacle for calculations } }; Neil Henning neil@codeplay.com Paris Game AI Conference 2011

137. Changes were purely optimization// array of AIObstacle* ’s on main memory AIObstacle** obstacles; unsigned intnumObstacles; offloadThread_thandle = __offload(obstacles, numObstacles) { CachedPointer<AIObstacle*> innerObstacles(obstacles, numObstacles); for(unsigned int i = 0; i < numObstacles; i++) { // AIObstacle* points to main memory CachedPointer<AIObstacle> obstacle(innerObstacles[i]); // use obstacle for calculations } }; Neil Henning neil@codeplay.com Paris Game AI Conference 2011

138. What performance improvement was achieved? Neil Henning neil@codeplay.com Paris Game AI Conference 2011

140. ~100 lines of source code changed

141. 2½ weeks development timeNeil Henning neil@codeplay.com Paris Game AI Conference 2011

143. ~100 lines of source code changed

144. 2½ weeks development timeNeil Henning neil@codeplay.com Paris Game AI Conference 2011

146. ~50 lines of source code changed

147. 1 week development timeNeil Henning neil@codeplay.com Paris Game AI Conference 2011

149. ~50 lines of source code changed

150. 1 week development timeNeil Henning neil@codeplay.com Paris Game AI Conference 2011

152. ~20 lines of source code changed

153. 8 hours development timeNeil Henning neil@codeplay.com Paris Game AI Conference 2011

155. ~20 lines of source code changed

156. 8 hours development timeNeil Henning neil@codeplay.com Paris Game AI Conference 2011

158. 50% speed improvement on PS3Neil Henning neil@codeplay.com Paris Game AI Conference 2011

159. Takeaway Neil Henning neil@codeplay.com Paris Game AI Conference 2011

161. But need code ready to be parallelised

162. Small changes in coding style lead to hugely better results

163. Better to plan systems from beginning with multicore in mindNeil Henning neil@codeplay.com Paris Game AI Conference 2011

164. Questions? Can also catch me on twitter @sheredom Neil Henning neil@codeplay.com Paris Game AI Conference 2011