Update on the Web Application Video Ecosystem (WAVE) Project by John Simmons of Microsoft.

1. WAVE
The Web Application
Video Ecosystem project
John Simmons
Media Platform Architect
Microsoft
johnsim@microsoft.com

2. Web Application
Video Ecosystem
WAVE
November 2018
WAVE Background
• WAVE history and origin
• WAVE relationship to global media standards
WAVE Today
• The what and who of WAVE
• WAVE and content
• WAVE and applications
• WAVE and devices
WAVE Future
• WAVE and progressive web apps
• The why of WAVE
Discussion
2

3. WAVE History and Origin: 2008-2015
October 2008 Microsoft Athens Project to standardize DRM-interoperable Streaming
September 2009 Publication of the PIFF container format and Smooth Streaming Transport
March 2010 MPEG Call for Proposal to standardize HTTP Streaming
February 2011 Microsoft and Netflix Proposals to W3C to support DRM and Adaptive Streaming
February 2012 Dynamic Adaptive Streaming over HTTP (DASH) and Common Encryption published
June 2012 HTML5 Encrypted Media Extensions (EME) and Media Source Extensions (MSE) Start
April 2014 Microsoft “Interoperability, Digital Rights Management and the Web” paper delivered at NAB
January 2015 Microsoft and Apple disclose CMAF project under NDA at CES. And the CTA proposes WAVE.
December 2015 The Consumer Technology Association (CTA) launches WAVE project based on Microsoft NAB paper
November 2018 3

4. WAVE History and Origin: 2016-present
January 2016 First CTA Web Application Video Ecosystem (WAVE) meeting held at CES in Las Vegas
February 2016 CMAF proposal submitted to MPEG supported by Adobe, Akamai, Apple, BBC, Cisco, Comcast, DTG,
Ericsson, Fraunhofer, iStreamPlanet, LG, Microsoft, BAMTech, Stars, Telecom Italia, Turner and
Verimatrix.
April 2016 First WAVE Technical Working Group meeting held at NAB
November 2016 HTML5 Media Source Extensions (MSE) a formal W3C Recommendation
September 2017 HTML5 Encrypted Media Extensions (EME) a formal W3C Recommendation
December 2017 CTA WAVE publishes “Web Application Video Ecosystem - Web Media API Snapshot 2017”
January 2018 Common Media Application Format (CMAF) becomes a published ISO Specification
April 2018 CTA WAVE publishes “Web Application Video Ecosystem – Content Specification”
November 2018 4

5. WAVE AND GLOBAL WEB MEDIA STANDARDS
ISO MPEG CENC
ISO MPEG DASH
Apple published in IETF
November 2018 5

6. DRM 3 Proprietary
Container
DRM 3
Proprietary
Encoding and
Encryption
BEFORE COMMON ENCRYPTION
Common Container
DRM1 Header
DRM2 Header
DRM3 Header
Common Encoding and
Encryption
ISO MPEG CENC
DRM 2 Proprietary
Container
DRM 2
Proprietary
Encoding and
Encryption
DRM 1 Proprietary
Container
DRM 1
Proprietary
Encoding and
Encryption
WITH COMMON ENCRYPTION
November 2018 6

7. ▪ An 18-month Apple-Microsoft co-development
activity. January 2015, Microsoft and Apple shared
their idea of a common media format between HLS
and DASH
▪ Proposed at MPEG’s 114th meeting in San Diego in
Feb 2016.
▪ Requirements submitted by: Adobe, Akamai, Apple,
BBC, Cisco, Comcast, DTG, Ericsson, Fraunhofer,
iStreamPlanet, LG Electronics, Microsoft, MLBAM,
Qualcomm, Samsung, Starz, Telecom Italia, Turner,
Verimatrix, WWE.
▪ Draft specification submitted by: Apple, Microsoft,
MLBAM, Cisco, Akamai and Comcast.
▪ Became a published ISO/IEC spec January 2018.
CMAF was designed with live linear streaming, content protection and ad signaling in mind.
November 2018 7

8. CDN
delivery
Cloud
encoding
mpd
m3u8
CMAF
mpd
m3u8
CMAF
mpd
m3u8
CMAF
mpd
m3u8
CMAF
mpd
m3u8
CMAF
mpd
m3u8
CMAF
• June 15, 2016 Apple announced HLS would support CMAF.
• Now the top two adaptive delivery manifest formats - DASH and HLS -
support the same content packaging
1) DASH, HLS with CMAF reduce storage and packaging costs,
ISO MPEG DASH
Apple published in IETF
2) improve edge caching efficiency - very
important for live linear streaming,
Origin
Storage
Source
3) and enable device interoperability.
Mobile or PC App
Smart TV App
Game Machine App
Set top box appNovember 2018 8

9. Player Framework
HTML5 Media Application
Web Application APIs
EME MSE MCE
Device Platform
DRM Media Decoder
Mobile or PC Game ConsoleSmart TV Set top box
▪ Media players use existing player frameworks – e.g. open
source Shaka player, dash.js, hls.js and video.js, proprietary
jwplayer, THEOplayer and the Bitmovin player.
▪ Encrypted Media Extensions (EME) interface to embedded
DRM, Media Source Extensions (MSE) interface to the Media
Decoder for adaptive playback.
▪ A Media Capabilities Extension (MCE) is being incubated
which will enable device capability discovery.
▪ CTA WAVE and the W3C are co-publishing recommendations
on embedded device browser API support.November 2018 9

10. Web Media Interop
For Content, Application Platform and Devices – through
specifications and industry engagement
November 2018 10

11. WAVE
Specification
Work
▪ WAVE addresses global Web media interop issues by
defining interop points based on global standards,
targeting desktop and embedded browsers – laptops,
phones, tables, smart TVs, media sticks and set-top
boxes.
▪ Encoding issues are being worked on by the Content
Specification Task Force, published in the WAVE content
specification, based on profiles of the new ISO IEC CMAF
specification.
▪ Platform issues are being addressed by the HTML5 API
Task Force, in the published Web Media API Snapshot
spec, as well as the upcoming Web Media Application
Developers Guidelines and the Web Media Porting
specification.
▪ Playback issues are being worked on by the Device
Playback Capabilities Task Force, in the upcoming Device
Playback Capabilities specification.
▪ WAVE Specs are free - https://cta.tech/WAVE
November 2018 11

12. WAVE
engagement
with Industry
▪ WAVE addresses global media interop issues by working
with Industry Fora (e.g. DASH IF, W3C) and SDOs (e.g.
ISO MPEG) to drive improvements in baseline standards.
▪ Encoding improvements are being address by getting
CMAF Media Profiles published in organizations like ETSI
and the Alliance for Open Media, and by driving
enhancements to the base CMAF specification, e.g. to
address multi-stream audio formats and splice
conditioning.
▪ Platform improvements are being addressed by
promoting changes to HTML5 EME, MSE, Media
Capabilities and Timed Data APIs (e.g. emsg box).
▪ Playback improvements are being addressed by
developing test WAVE compliance testing for WAVE
media profiles and the WAVE Web Media API Snapshot
spec.
▪ Join WAVE by emailing standards@CTA.tech
November 2018 12

13. WAVE Organization
Steering
Committee
Technical
Working Group
Content
Specification
Task Force
Device Playback
Capabilities Task
Force
HTML5 API Task
Force
Will Law
Akamai
Mark Vickers
Comcast
Thomas Stockhammer
Qualcomm
John Simmons
Microsoft
Microsoft, Qualcomm,
Samsung, Sky, Sony,
Starz, TP Vision, WWE
Adobe, Akamai,
Apple, Comcast,
Disney Streaming,
Google, LG,
November 2018 13

14. Adobe Systems, Inc
AGP, LLC
Akamai
Amazon.com
Apple
AT&T Inc
AwoX
BBC Research & Development
BitRouter
Brightcove Inc.
Cable Television Labs
castLabs
Comcast Cable
Consumer Technology
Association
Cox Communications, Inc.
Digital Entertainment Content
Ecosystem (DECE)
Discovery Communications
Disney Streaming
Disney/ABC/ESPN
Dolby Laboratories Inc.
DTS, Inc.
Ericsson
Eurofins Digital Testing
Facebook
Fraunhofer
Google Inc.
Home Box Office (HBO)
Intel Corporation
JW Player
LG Electronics
Microsoft Corporation
Motion Picture Association of
America
Motion Picture Laboratories
Nagravision
National Association of
Broadcasters
Netflix, Inc.
Nevelex
P Thomsen Consulting
Qualcomm Incorporated
RK Entertainment Technology
Consulting LLC
Samsung Electronics
SERAPHIC Information
Technology
SET - Brazilian Society of
Television Engineering
Sharp Electronics Corporation
Sky
Society of Cable
Telecommunications Engineers
Solekai Systems
Sony Electronics
SpireSpark International
Starz
Streamroot
TBT, Inc.
Toshiba
TP Vision Holding B.V.
Turner
UStudio
Verance Corporation
Verimatrix
Verizon
Vewd Software
Viacom
Vizio
WJR Consulting
World Wide Web Consortium
(W3C)
WWE
WAVE Membership
November 2018 14

15. CMAF Codec bindings or “Media
Profiles” with market support
will be published annually in the
CTA WAVE Content Specification.
... ...
Some media profiles are identified in the ISO MPEG CMAF specification.Additional media profiles will be published elsewhere.
CMAF Codec bindings to Global
ISO MPEG and non-ISO MPEG
Codecs, including new royalty
free codecs like AV1 from the
Alliance for Open Media.
November 2018 15

16. WAVE Content Specification 2018 - Video Profiles
Media Profile
Name
INFORMATIVE
Codec
INFORMATIVE
Profile
INFORMATIVE
Level
INFORMATIVE
Color
primaries &
matrix
coefficients
INFORMATIVE
Transfer
Characteristics
INFORMATIVE
‘codecs’
MIME subparameters
NORMATIVE
CMAF Brand
NORMATIVE
Normative
Reference
HD AVC High 4.0 1 (BT.709) 1 (BT.709 OETF) avc1.640028
avc3.640028
‘cfhd’ [CMAF]
Table A.1
HHD10 HEVC Main10
MainTier
4.1 1 (BT.709) 1 (BT.709) hev1.2.4.L123.B0
hvc1.2.4.L123.B0
‘chh1’ [CMAF]
Table B.1
UHD10 HEVC Main10
MainTier
10-bit
5.1 1 (BT.709)
9 (BT.2020)
1 (BT.709 OETF)
14 (BT.2020 OETF)
hev1.2.4.L153.B0
hvc1.2.4.L153.B0
‘cud1’ [CMAF]
Table B.1
HLG10 HEVC Main10
MainTier
10-bit
5.1 9 (BT-2020) 18 (BT.2100 Table 5
HLG OETF)
14 (BT.2020 OETF)
hev1.2.4.L153.B0
hvc1.2.4.L153.B0
‘clg1’ [CMAF]
Table B.1
HDR10 HEVC Main10
MainTier
10-bit
5.1 9 (BT.2020) 16 (BT.2100 Table 4 PQ
EOTF)
hev1.2.4.L153.B0
hvc1.2.4.L153.B0
‘chd1’ [CMAF]
Table B.1
The 2018 Edition of the WAVE Content Specification includes the following video Media Profiles. Additional media
profiles are likely to be added in an amendment prior to the 2019 edition of the WAVE Content Specification –
including a CMAF binding for AV-1.November 2018 16

17. WAVE Content Spec 2018 - Audio Profiles
Media Profile
Name
INFORMATIVE
Codec
Family
INFORMATIVE
Allowed
Codecs or
Profiles
INFORMATIVE
Level
INFORMATIVE
‘codecs’
MIME
subparameter
NORMATIVE
CMAF Brand
NORMATIVE
Normative
Reference
AAC Core AAC AAC-LC, HE-
AAC or HE-
AAC v2
2 mp4a.40.2
mp4a.40.5
mp4a.40.29
‘caac’ [CMAF]
Table A.2
Adaptive AAC
Core
AAC AAC-LC, HE-
AAC or HE-
AAC v2
2 mp4a.40.2
mp4a.40.5
mp4a.40.29
‘caaa’ [CMAF]
Table A.2
AAC
Multichannel
AAC AAC-LC, HE-
AAC
6 mp4a.40.2
mp4a.40.5
mp4a.40.29
‘camc’ [CMAF A1]
Table i.2
Enhanced AC-
3, including
AC-3
AC-3 EAC-3 AC-3
EAC-3
n.a. ec-3 ‘ceac’ [EAC3]
AC-4, Single
Stream
AC-4 AC-4 3 ac-4.02.01.03 ‘ca4s’ [AC4]
MPEG-H,
Single Stream
MPEG-H Low
Complexity
(LC)
3 mhm1.0x0B
mhm1.0x0C
mhm1.0x0D
‘cmhs’ [CMAF A1]
Table j.2
▪ Some organizations outside
MPEG have begun publishing
bindings specifications for
CMAF.
▪ The first of these organizations
is ETSI, which is publishing
CMAF bindings specifications
for both Dolby and DTS audio
codecs.
▪ Other organization have
suggested they will publish
CMAF bindings in 2018.
▪ The WAVE content specification
also includes both IMSC1 Text
and Image CMAF bindings.
▪ IMSC1 is being standardized
because of its compatibility
with EBU_TT_D used in Europe.November 2018 17

18. WAVE Programs and Live Linear Content
WAVE Program
CMAF
Presentation 1
Audio Switching
Set
Video Switching
Set
Subtitle
Switching Set
CMAF
Presentation 2
Audio Switching
Set
Video Switching
Set
CMAF
Presentation 3
Audio Switching
Set
Video Switching
Set
Subtitle
Switching Set
Continuous Rendering for a
continuous user experience
▪ Live linear content with ad insertions may require delivering
not one but a sequence of CMAF Presentations.
▪ To address this likelihood, the WAVE content spec defines
WAVE programs, which are a sequence of one or more
CMAF Presentations.
▪ When a WAVE Program includes more than one CMAF
presentations, it can optionally conform to a WAVE Splice
Constraint Profile.
▪ The 2018 Edition of the WAVE Content Spec defines a
Baseline Splice Constraint Profile, encoding constraints
intended to allow continuous rendering of Sequential
Switching Sets in WAVE Programs on most existing adaptive
streaming Players in the market today.
▪ We anticipate that as new devices enter the market, the
more advanced Splice Constraint Profiles will be published
by WAVE.
November 2018 18

19. HTML5 APIs: Reference Platform
19
WAVE/CMAF
One Content Format... Reference Platform
HTML5
tests
Write reference
tests in HTML5…
… then port tests to
device platforms.
(HTML5 platforms run
tests directly.)
...but multiple devices
mobile
apps
PC
apps
TV
apps
game
apps
set-top
apps

20. HTM5 API Task Force: Work Plan
• Web Media API Community Group:
• w3.org/community/webmediaapi/
1. Annual Web Media API spec
• define baseline web APIs to support media
web apps.
2. Guidelines for media web app
developers
3. Identify gaps in current web APIs
• work with W3C Working Groups to update
web standards.
20

21. Web Media API Snapshot 2017
• First annual API Snapshot published
20 December 2017:
https://www.w3.org/2017/12/webmediaapi.html
▪ Lists key APIs supported in 2017 in all major
HTML code bases.
▪ CTA-W3C agreement to co-publish this spec.
▪ Plan to propose Community Group spec as a
W3C standards track spec
▪ CTA WAVE released a test suite for all listed APIs
based on W3C API tests
(https://webapitests2017.ctawave.org).
▪ Test suite will enable manufacturers to test that
their HTML support is up-to-date!
21

22. OTT Device Performance Challenges
• Ad splicing problems
• Regional profiles (50/60Hz)
• Request protocol deficiencies
• Unknown codec capabilities
• Unknown rendering capabilities
• Partial profile support
• Codec incompatibility
• Long-term playback instability
• Late Binding Synchronization
• Audio discontinuities
• Glitches when switching bitrate
• Memory problems
• Limited processing power
• Long start-up delay
• Performance monitoring
• DRM support
• Variable HDR support
• Scaling display issues
22

23. Device Types
• WAVE includes
different types of
OTT clients, both
HTML and Native
• Requirements apply
universally, tests
focused on
(referenced to)
HTML
23
App-driven
players:
Native App
Devices
Native
Platform
Player
HTML5 App
Player
JavaScript & MSE
HTML5
Platform
Player
video object
Native App
Player
Embedded
players:
HTML5 App
Devices

24. Video Source Buffer
Audio Source Buffer
Abstracted Device Playback Model
24
Device Playback
Model
Video Source Buffer
Audio Source Buffer
Subtitle Source Buffer
Visual Rendering
Audio Rendering
Device Capabilities
Stimulus/Input Observations
Requirements: If you input WAVE content, this shall be the observation

25. DPCTF Specification Objectives
• Provide testable requirements for device performance
challenges
• Provide capability code points for WAVE content
• Enable the qualification of existing platforms for their WAVE
content playback capabilities
• Generate a forward-looking specification for advanced media
playback requirements, including new codecs and experiences
• Prioritize challenges and address the highest priority items first
25

26. November 2018 26

27. Mobile or PC Game ConsoleSmart TV Set top box
EdgeHTML Gecko WebKitBlink
APP-LIKE Feels like an app, works like a website - Progressive
Web Apps (PWAs) are powered by Web technology
and delivered with Web Infrastructure.
PROGRESSIVE PWAs will be supported by most embedded and
desktop web browser engines.
RESPONSIVE Easily built targeting multiple form factor devices,
using the new W3C Manifest for Web Apps spec.
BUT NETWORK
INDEPENDENT
Works with or without network connection for offline
playback, thanks to "service workers"
BUT ALWAYS
FRESH
PWAs are always up to date, updates do not need to
be resubmitted to store.
DISCOVERABLE
AND RE-
ENGAGEABLE
Search engines can find the app, because of scope of
W3C manifest for Web Apps and Web Worker
registration – and supports push notifications.
INSTALLABLE
AND LINKABLE
Installs like a native application and can be shared
with a URL – e.g. from the base website.
November 2018 27

28. November 2018 28
WHY WAVE?

29. Internet
Inflection
Point in
1989
November 2018 29

30. Web
Inflection
Point in
1993
November 2018 30

31. Hypertext
Transport
Protocol
(HTTP)
Hypertext
Markup
Language
(HTML)
Portable
Browser
(Mosaic)
Mobility of
Internet
Experience
(Web)
Common
Media
Application
Format
(CMAF)
Dynamic
Adaptive
Streaming
over HTTP
(DASH)
HTML
Media
Extensions
(EME, MSE)
Progressive
Web Apps
(PWAs)
Mobility of
Web Media
Experience
2018
1993
Global Web Media Standards Inflection Point
November 2018 31

32. Three Tiers of Commercial Web Media
Tier-1 mainstream: linear content from cable/satellite/telco, most
popular and limited in variety because linear is bandwidth constrained
Tier-2 direct-to-consumer: Tier-1 channels or channel
programs going direct to consumer. Inherit interop challenge.
Tier-3 Long tail: vertical niche content made practical
by global Internet standards for commercial media
which are cloud addressable objects. “Farm league”
for linear content.
MassMarket
Niche Market
November 2018 32

33. Devices
Mobile, Game
Console, Smart TV
Mobile, Game
Console, Smart TV
Impact of Global Industry Adoption
Set top box
Desktop
Set top box
Content Industry
Tier-1
Mainstream
Tier-2
Direct to Consumer
Tier-3
Long Tail
AuthZ Providers
Cable/Sat/Telco
Online Video
Distributors
Specialty
providers
Desktop
Legacy
Transition
Future
Set top box
November 2018 33

34. ISO MPEG CENC
ISO MPEG DASH
Apple published in IETF
November 2018 34

35. ISO MPEG CENC
ISO MPEG DASH
Apple published in IETF
Web App Manifest, W3C Working Draft 08 May 2018,
www.w3.org/TR/appmanifest/
Media Source Extensions, W3C Recommendation 17 November 2016,
http://www.w3.org/TR/media-source/
Encrypted Media Extensions, W3C Recommendation 18 September 2017,
http://www.w3.org/TR/encrypted-media/
HTTP Live Streaming (HLS), RFC8216, https://tools.ietf.org/html/rfc8216
ISO/IEC 23009-1:2014, Information technology – Dynamic adaptive streaming over
HTTP (DASH) – Part 1: Media presentation description and segment formats,
https://www.iso.org/standard/65274.html
ISO/IEC 23000-19, Information technology — Coding of audio-visual objects — Part
19: Common media application format (CMAF) for segmented media.
https://www.iso.org/standard/71975.html
ISO/IEC 23001-7:2016, Information technology – MPEG systems technologies – Part
7: Common encryption in ISO base media file format files,
https://www.iso.org/standard/68042.html
November 2018 35

36. Discussion
November 2018 36