HTML 5 game engine and performance optimization techniques for mobile devices

1. Web Engine-Aware
Performace Optimization of
HTML5 Games for Mobile Devices
Sang Seok Lim (sangseok.lim@sk.com),
Director, SK planet

2. Agenda
● Web Engine Internal
● Web Engine Performance Analysis
● HTML5 Game Basics
● Performace Optimization Techniques
● Case study

3. Commercial Use Cases of HTML5 Games
● In-app HTML5 games
○ OK Cashbag Arcade (June, 2014)
○ Syrup Plant Pop (October, 2014)
● In-app means a hybrid app
○ embedded through WebView
○ not for a browser which is incapable of enabling BM
● Goal from the business viewpoint
○ user rentention for keeping/increasing UV of native
apps

4. HTML5 Game Architecture Overview
● from the viewpoint of performance,
DOM/JavaScript APIs of HTML5 are tightly
coupled to underlying SW/HW layers
Game Logic (HTML/CSS/JavaScript)
Game Engine (JavaScript, WebWorker)
DOM/JavaScript API
Webview Widget
Webkit/Blink Engine
Android/iOS
HW(ARM CPU, OpenGL ES GPU)

5. Chrome Browser Architecture
similar to Chrome Webview

6. HTML5 Game RunTime: WebKit
● Layout/Rendering
○ 2D, 3D graphics by GPU, text rendering
● Sound
○ <audio>, WebAudio
● Physics
○ Javascript engine, JIT, multi-core by WebWorker
DOM <canvas> Physics Engine
Parser Layout Rendering
2D
graphics
3D
graphics
Text
Rendering
JavaScript Engine
(JIT)
multimedia
(sound)
Game
Web
engine
Platform

7. Parsing before Rendering

8. WebKit Internal: Layout
● Calculate left top,
width, height of all
DOM nodes
contained in a
HTML document
● Group a set of
DOM nodes then,
stack the resultant
groups accordingly
left, top, width, height
ordering layers
https://developer.mozilla.org/en-US/docs/Web/Guide/CSS/Understanding_z_index/The_stacking_context

9. Internal Data Structure for Rendering

10. WebKit Internal: Rendering
● DOM Rendering
○ Image + text
○ CSS 2D/3D rendering
■ GPU Acceleration
● 2D graphics
○ canvas/SW
○ canvas/OpenGL ES
● 3D graphics
○ WebGL/OpenGL ES

11. DOM Rendering: Image + Text
● It’s all about handling a image
○ <img> tag represent a image
○ text is broken into a serises of images
Penguin
P e
n
g
u
i
n

12. DOM Rendering: CSS2D/3D
● 2D/3D transform matrix
rotate
scale translate
skew

13. DOM Rendering: CSS2D/3D
● RenerLayers are selectively mapped into
GPU textures for being rendered by GPU

14. DOM Rendering: CSS2D/3D by HW acceleration
● 2D/3D transform of DOM nodes by GPU
Penguin
Penguin
Penguin
Texture generation by CPU Texture based animation by GPU

15. Texture Visualization by Inspector

16. <canvas> Rendering
● 2D primitive
○ path, curve, polygon
○ fill, stroke
○ image manipulation
○ pixel manipulation
● <canvas> primitive HW acceleraton
○ image manipulation
● <canvas> buffer itself is a GPU texture

17. WebGL Rendering
● JavaScript wrapper of OpenGL

18. Compatibility across Mobile Devices
● Not all of the rendering models can be used
for mobile devices right now
● Compatibility summary
○ CSS 2D/3D (HW)
■ ICS 4.0 >, iOS 4.0 >
○ canvas (SW)
○ Android 2.3 <=
● canvas (HW)
○ Android 4.0 >, iOS 5.0 >
● WebGL
○ iOS 8.0 >
○ Android L >

19. OS Distribution of Customers
OK Cashbag App (June, 2014)
the critical
defect by
Google

20. General Rendering Back-end
for Mobile Devices
● <canvas> 2D for Game object rendering
○ HW acceleration
● DOM for Game UI
○ 2D/3D by GPU

21. App-embedded Branded Casual Game
● In-app play for user retention
● No app update via app store

22. <canvas> based Game Development
Browser limitations
● Browser engine as a game platform
○ rendering
■ <canvas>, WebGL, DOM
○ sound
■ <audio>
■ WebAudio
○ resource loading
■ XHR/<img>
○ input
■ touchstart, touchmove, touchend
○ DPI management
■ image resource scale up/down

23. <canvas> based Game Development
Into Reality
● The limitations of open source game engine
○ only for PC chrome/Firefox
○ does not meet the hard requirements for mobile OS
● In-house game engine development
○ to achieve stable 20 fps on our customer’s device
■ higher is better even on android 2.3 and iOS 5.0
○ reasonable touch response delay
○ no garbage collection whiling playing
● Further information
○ http://www.slideshare.net/senxation/html5-ok-33344593

24. <canvas> Game Main Loop
● As easy as animation by flipping pages

25. Representaion of Game Objects
● Two approaches
○ static image generation: offline image based
○ dynamic image generation: drawing primitive
combination based
● Each approach has its own pros and cons
○ performance
○ design-constraint

26. Image based Object Representation
● A series of sprite image cropping/scaling
○ ctx.drawImage(src, dest)
○ the performance of game is decided by drawImage()
● Static image generation (offline)
○ one for each low, mid, high resolution devices

27. Drawing Primitive Combination
● Dynamic image generation
○ draw an object onto a buffer with the prior
knowledge of device DPI, target pixel dimension
○ minimize image scaling during rendering
http://simeonvisser.hubpages.com/hub/HTML5-Tutorial-Drawing-Circles-and-Arcs

28. Summary of Pros and Cons
● Image based (offline)
○ overall performance limited by image scaling
primitive
○ designer-friendly
● Drawing primitive based (online)
○ higher performance
○ be an enemy to a developer with lower IQ
○ be an enemy to a designer

29. Performance of <canvas> in Detail
in WebView for Hybrid App
● The killing bottleneck is “PAINTING”
● Performance challenges
○ no <canvas> HW acceleration: android 2.3, iOS 4.0
○ Kitkat Chrome Webview’s critical error by Google
■ no HW acceleration on 4.4, 4.4.2 Chrome
Webview
■ fixed at 4.4.3
■ but market share of 4.4.2 in Korea → 65%

30. The Problem Definition
when it comes to developing <canvas> game for mobile devices
● Android 2.3 without canvas HW acceleration
○ together with canvas acceleration
● Android /4.4[0-2]/ without canvas HW
acceleration

31. Performance Analysis of Kitkat Webview
● To find the performance limiting factors
○ painting size
○ the number of drawing primitive calls
○ DOM tree complexity
● System under test
○ Optimus G Pro (4.4.2)
○ Nexus 5 (4.4)

32. Performan Analysis I
● Partial rendering
○ reuse the content of a previous frame buffer
● Full rendering
○ reset a frame buffer every rendering
painting area size
fps
The Larger, The Slower

33. Performance Analysis II
● Increase the number of painting areas
○ a painting area size is fixed to 25x25
fps
the number of painting area

34. Performance Analysis III
● Impact analysis with respect to DOM complexity

35. Performance Analysis V
● Impact analysis with respect to DOM complexity
● Severe fps fluctuation

36. Summary of Performance Analysis
● In proportion to the size of painting area,
painting performance (fps) is decreased
● The more complex DOM tree containing
<cavas> is, the more severely fps fluctuates

37. Performance Improvement
● 5 practical techniques
○ DOM/<canvas> hybrid rendering for game objects
○ Static object pool for keeping garbage collector
asleep
○ Smart repaint by tracing invaldated areas
○ Image resource offline manipulation
○ Minimize a DOM tree complexity
● not all of them are applicable at the same
time
○ depending on game scenario, game performance
requirement, smart repaint/hybrid rendering can be
selectively applicable

38. DOM/<canvas> Hybrid Rendering
painting size minimization
● Selectively separate game objects contained in
<canvas>, and then paint them through DOM
canvas only canvas/DOM Hybrid

39. DOM/<canvas> Hybrid Rendering
implementation detail: DOM node pool
● Manage/reuse a set of DOM nodes with a
background image of a game object

40. Smart Repaint
Repaint Region Trace
● Reuse the content of the frame buffer of n-1
frame for rendering n frame
● Trace the invalidated area, paint only
invalidated area

41. Source Image Size Matters A LOT!
● A sprite image is generally used to minimize # of
network req
● Performance anomaly
○ the larger the source image is, the slower fps is, even when the
destination size is fixed
separate

42. Garbage Collector Stops The World
● All objects are allocated from a object pool
● Implementation of a object pool matters
○ single Array, double Array, linked list
○ http://jsperf.com/objectpool-performance-comparison

43. Static Object Pool Performance Comparision
● Single array by Array.push/pop is the best
○ poor performance of Array.spice()
● Complete suppression of GC is impossible

44. Dive into Resolution Issue
● A designer-generated image is scaled
up/down 4 times at most implicitly
○ canvas.width, canvas.height
○ canvas.style.width, canvas.style.height
○ ctx.drawImage(source, destination)
○ window.devicePixelRatio
● Try not to scale canvas.width/height, canvas.
style.width/height
○ how to adapt device resolution in performant way
○ no fitting/add padding is good for performance
○ sub optimal display for a user

45. <canvas> Isolation
● fps swing around 30% up and down, when
<canvas> is contained within a document
with complex DOM tree
● Seprate <canvas> from DOM based game UI
○ not easy in the real field where designers/publisher
and developers from different teams are co-working

46. Case Study: Flappy Ball
● DOM/<canvas> hybrid
● Smart repaint, sprite image separation

47. 2048
● DOM/GPU only
● Keeping GPU textures as long as possible

48. Space 2014
● Radically reuse a previous frame without
clearance
○ Android 2.3
● clearRect/fillRect

49. Closing Remark
● For commercialization on mobile devices,
performance matters a lot
○ understading Web-engine internals is essential for
developing commercial level game successfully
● open sourced
○ http://skpla.net/bPqc