This presentation presents the strengths and challenges of using HTML5 audio. It then presents a working game audio system architecture which leverages HTML5 audio. Finally, it looks at the future of web audio.

1. EFFECTIVE HTML5 GAME AUDIO
Chris Khoo (chris.khoo@wappworks.com)

2. TOPICS
1. HTML5 Audio
 Definition
 The good, the bad, and the ugly
2. HTML5 audio game layer architecture

3. INTRODUCTION TO HTML5
AUDIO

4. HTML5 AUDIO
 HTML5 audio is part of the HTML5 specifications
managed by the World Wide Web Consortium (W3C)
 The <audio> element
 Represents a sound or audio stream
 Not designed for real-time audio mixing
 W3C is working on a Web Audio API as an alternative
audio solution
 Advanced sound manipulation functionality
 Not purely stream related

6. THE GOOD
 HTML5 audio API support is standard across all
modern browsers
 EXCEPTION: Firefox versions prior to Jan 1, 2012 do
not support the loop flag
It is possible to implement a game sound engine
using HTML5 audio

7. THE BAD
 Limited audio control
 Coarse control of audio playback position and volume
 No audio panning support
 No sound filters
 Non-contiguous looping
 Expect a slight pause whenever a sound sample loops

8. THE BAD
 Non-standard audio formats across browsers
 No one audio format works for all browsers
 A list of supported audio formats on a per browser basis
is available on Mozilla‟s website
 To ensure the highest browser compatibility, be sure to
support multiple audio formats (ogg, mp3, wav)

9. THE BAD
 Spotty HTML5 audio support on default mobile
device web browsers
 Android
 Android 2.x: Audio does not play
 Android 3.0+: Only one audio instance allowed to produce
sound at a time
 iOS
 Audio must be triggered through user interaction
 Apple is actively working to prevent workarounds
 Support in third party mobile web browsers is much
better
 Firefox mobile implements HTML5 audio perfectly

10. THE UGLY
 No guarantee of timely activation
 Some browsers have a noticeable delay between calling
the play() function and sound playback starting
 Safari - always true
 IE 9 - occurs when modifying an audio element‟s src
parameter prior to play back
 Beware of audio stream stalls
 HTML5 audio streaming can sometimes stall without
reason
 It seems to happen more frequently with WebKit browsers
 A “stalled” event is fired whenever this occurs
 Call the load() function to manually restart streaming

11. THE UGLY
 Server functionality can impact HTML5 audio behavior
 On WebKit browsers (Chrome and Safari), audio/video seek
functionality is quietly disabled if the audio web server does
not support partial requests
 Disabling seek causes
 play() call completes without error but playback does not occur
 „ended‟ event does not fire on playback complete
 „Sound file format corrupted‟ exception is thrown when trying to
replay sounds
 Generally happens the first time an audio is referenced
 Doesn‟t happen on subsequent accesses if the audio is cached
 Other known quirks
 Firefox 3.6: short sounds don‟t play
 No longer a problem since Mozilla is phasing out this version
 Safari: Terrible audio playback performance

12. DEMO TIME

13. SHOULD WE USE HTML5 AUDIO?
Pros
 Browser audio support is steadily improving
 It works across all platforms including mobile web
 It‟s possible to work around most of the issues through careful
sound design and good engineering but audio experience, in
general, is compromised
 Popular HTML5 game acceleration platforms support
audio through the <audio> element
 CocoonJs and AppMobi take this approach
Cons
 Limited audio manipulation functionality
 Inherently more complicated to implement
 Audio experience still varies from browser to browser

14. QUESTIONS?

15. BUILDING AN HTML5 GAME
AUDIO SYSTEM

16. SYSTEM REQUIREMENTS
 Graceful degradation
 What if there‟s no HTML5 audio support?
 Sound loading/caching
 Sound playback
 Sound looping support
 Mute/Unmute
 Volume control
 Globally
 Per active sound
 Multiple sound channels
 Per channel volume control
 Stop all active sounds in a channel

17. SYSTEM REQUIREMENTS
 Polyphony limits
 Limit maximum number of simultaneous sounds per
channel
 Why?
 Performance reasons: sound playback has a performance
impact
 Less is more: too many sounds playing simultaneously can be
chaotic
 Option to drop the oldest sound or ignore the latest
sound request when polyphony limit reached

18. GAME AUDIO STACK

19. GAME AUDIO SYSTEM

20. GAME AUDIO SYSTEM
AudioSys
 Game-specific audio interface
 Isolates game audio implementation from the rest
of the game systems
 Helps in implementing graceful degradation
 Game must continue to function when audio
functionality is not available
 Audio samples are loaded and managed by an
AudioCache instance
 Music and SFX audio are played back using
separate AudioChannel instances

21. GAME AUDIO SYSTEM

22. GAME AUDIO SYSTEM
AudioCache
 Handles audio asset loading
 selects the best audio format
 Maps sound IDs to actual audio samples

23. GAME AUDIO SYSTEM

24. GAME AUDIO SYSTEM
AudioChannel
 Handles sound playback
 Supports looping
 Provides channel playback controls
 Stop
 Volume control
 Mute
 Monitors and enforces polyphony limits
 Plays sounds from an AudioCache and returns an
AudioChannel::Handle for managing an actively
playing sound
 Reusable component

25. GAME AUDIO SYSTEM

26. GAME AUDIO SYSTEM
AudioChannel::Handle
 Façade class for managing an AudioChannel‟s
active audio instance
 Provides playback functionality
 Stop
 Set volume
 Provide functionality to check playback status

27. GAME AUDIO SYSTEM

28. GAME AUDIO SYSTEM
AudioFactory
 Instantiates augmented HTML5 audio instances
 Why?
 Adds a custom loop API
 Addresses missing loop parameter support in earlier Firefox
versions
 Replaces the existing volume control with a two tier
volume control system
 Augmentation is applied on an audio instance
instead of through class extension
 It is not recommended to extend an HTML5 audio class
directly because HTML5 audio class name is not
standard

29. GAME AUDIO SYSTEM

30. GAME AUDIO SYSTEM

31. DEMO TIME

32. AUDIO SYSTEM IMPROVEMENTS
 Decompose AudioSys into multiple classes based
on use case
 Improves readability and maintainability
 Hyper Grav‟s audio system decomposition
 AudioMan: Game audio system
 AudioListener: In-game specific audio interface
 Modify AudioCache to manage sound pools on a
per sound asset basis
 AudioChannel requests an audio instance from
AudioCache and it returns an instance from the sound
asset‟s instance pool
 Fixes sound playback delay in IE 9

33. AUDIO SYSTEM IMPROVEMENTS
 Support asset filtering in AudioCache
 Current implementation selects format based on the
order of source URLs and the browser‟s audio format
support
 Add filtering to control audio format selection on a per
platform basis
 Why?
 On some platforms, audio format selection has to be made for
performance reasons e.g. audio compression has a significant
performance impact on mobile

34. ALTERNATIVE WEB AUDIO APIS
 W3C Web Audio API
 Official Web Audio API
 Designed to handle more complex audio applications
(e.g. games)
 API is still in development
 Currently, only Chrome implements a version of the
Web Audio API
 Mozilla Audio API extension
 Extends existing HTML5 audio and video elements to
support more complex audio application
 Mozilla has officially declared the API deprecated

35. THE END
Questions?

36. REFERENCES
 W3C Audio Working Group
http://www.w3.org/2011/audio/
 Latest W3C HTML5 audio specification
http://www.w3.org/TR/html5/the-audio-element.html
 Mozilla‟s audio data browser compatibility list
https://developer.mozilla.org/en-
US/docs/Media_formats_supported_by_the_audio_and_
video_elements#Browser_compatibility
 Proposed W3C Web Audio API
http://www.w3.org/TR/webaudio/
 Mozilla Audio API extension
https://developer.mozilla.org/en-
US/docs/Introducing_the_Audio_API_Extension