Tutorial Session on QoE and Standardization at QoMEX'16: http://www.qomex2016.org/

1. Adaptive Media Streaming: The Role of Standards
Dynamic Adaptive Streaming over HTTP
Priv.-Doz. Dr. Christian Timmerer
[Ack: Ali C. Begen, MediaMelon, Inc., OzyeginUniversity]
Alpen-Adria-Universität Klagenfurt (AAU) w Faculty of Technical Sciences (TEWI) w Department of Information
Technology (ITEC) w Multimedia Communication (MMC) w Sensory Experience Lab (SELab)
http://blog.timmerer.com w http://selab.itec.aau.at/ w http://dash.itec.aau.at w christian.timmerer@itec.aau.at
Chief Innovation Officer (CIO) at bitmovin GmbH
http://www.bitmovin.comw christian.timmerer@bitmovin.com
Tutorial @ QoMEX 2016, June 2016
http://www.slideshare.net/christian.timmerer

2. Importance of Multimedia Delivery
• Multimedia is predominant on
the Internet
• Real-time entertainment
– Streaming video and audio
– More than 70% of Internet traffic
at peak periods
• Popular services
– YouTube (17.85%), Netflix
(37.05%), Amazon Video (3.11%),
Hulu (2.58%)
– All delivered over-the-top (OTT)
June 2016 QoMEX 2016 Tutorial 2
Global Internet Phenomena Report: Dec 2015

3. Open Digital Media Value Chain
June 2016 QoMEX 2016 Tutorial 3
Create
Content
Aggregate
Monetize
Distribute
Content
Consume
Content
Any Content Any Storefront Any Network Any Device
CDNsMedia
Protocols
Internet
Transport
DRM
Encoding
Encapsulation
Dynamic
Ads
Clients
Happy User

4. What is Streaming?
June 2016 QoMEX 2016 Tutorial 4
Streaming is transmission of a continuous content from a server
to a client and its simultaneous consumption by the client
Two Main Characteristics
1. Client consumption rate may be limited by real-time constraints as opposed to just
bandwidth availability
2. Server transmission rate (loosely or tightly) matches to client consumption rate

5. Common Annoyances in Streaming
• Wrong format
• Wrong protocol
• Plugin requirements
• DRM issues
• Long start-up delay
• Poor quality
• Frequent stalls
• Quality oscillations
• No seeking features
June 2016 QoMEX 2016 Tutorial 5

6. Over-The-Top – Adaptive Media Streaming
• In a nutshell…
June 2016 QoMEX 2016 Tutorial 6
Adaptation logic is within the
client, not normatively
specified by the standard,
subject to research and
development

7. Multi-Bitrate Encoding and Representation Switching
June 2016 QoMEX 2016 Tutorial 7
Contents on the Web Server
Movie A – 200 Kbps
Movie A – 400 Kbps
Movie A – 1.2 Mbps
Movie A – 2.2 Mbps
. . .
. . .
Movie K – 200 Kbps
Movie K – 500 Kbps
Movie K – 1.1 Mbps
Movie K – 1.8 Mbps
. . .
. . .
Time (s)
Start quickly
Keep requesting
Improve quality
Loss/congestion detection
Revamp quality
. . .
. . .
Segments

8. Adaptive Streaming over HTTP
June 2016 QoMEX 2016 Tutorial 8
…
…
…
…
HTTPGETs
Client
Buffer
Media
Player
HTTP
Server

9. Formats and Standards
• Adobe
– HTTP Dynamic Streaming (HDS)
– Almost dead
• Apple
– HTTP Live Streaming (HLS)
– Required for iOS
• Microsoft
– Smooth Streaming
– Almost dead, too! Or not?
• MPEG Dynamic Adaptive Streaming over HTTP (DASH)
– Supported by Netflix, YouTube, Bitmovin, etc.
June 2016 QoMEX 2016 Tutorial 9
Source: http://xkcd.com/927/

10. Scope of DASH: what is specified?
June 2016 QoMEX 2016 Tutorial 10
Media Presentation on
HTTP Server
DASH-enabled ClientMedia Presentation
Description
.
.
.
Segment
…
.
.
.Segment
…
.
.
.
Segment
…
.
.
.Segment
…
…
Segments located
by HTTP-URLs
DASH Control Engine
HTTP/1.1
HTTP
Client
MPD
Parser
Media
Engine
On-time HTTP
requests to
segments

11. Scope of DASH: what is specified?
June 2016 QoMEX 2016 Tutorial 11
Media Presentation on
HTTP Server
DASH-enabled ClientMedia Presentation
Description
.
.
.
Segment
…
.
.
.Segment
…
.
.
.
Segment
…
.
.
.Segment
…
…
Segments located
by HTTP-URLs
DASH Control Engine
HTTP/1.1
HTTP
Client
MPD
Parser
Media
Engine
On-time HTTP
requests to
segments

12. DASH Data Model
June 2016 QoMEX 2016 Tutorial 12
MPD
Period id = 1
start = 0 s
Period id = 3
start = 300 s
Period id = 4
start = 850 s
Period id = 2
start = 100 s
Adaptation Set 0
subtitle turkish
Adaptation Set 2
audio english
Adaptation Set 1
BaseURL=http://abr.rocks.com/
Representation 2
Rate = 1 Mbps
Representation 4
Rate = 3 Mbps
Representation 1
Rate = 500 Kbps
Representation 3
Rate = 2 Mbps
Resolution = 720p
Segment Info
Duration = 10 s
Template:
3/$Number$.mp4
Segment Access
Initialization Segment
http://abr.rocks.com/3/0.mp4
Media Segment 1
start = 0 s
http://abr.rocks.com/3/1.mp4
Media Segment 2
start = 10 s
http://abr.rocks.com/3/2.mp4
Adaptation Set 3
audio german
Adaptation Set 1
video
Period id = 2
start = 100 s
Representation 3
Rate = 2 Mbps
Selection of
components/tracks
Well-defined
media format
Selection of
representations
Splicing of arbitrary
content like ads
Chunks with addresses
and timing

13. June 2016 QoMEX 2016 Tutorial 13
type=static typically,
for on demand content
Base URL of the
segments
Subtitles
Audio adaptation set
with different
representations (bw)
Video adaptation set
with different
representations (bw)
Different codecs
(profiles)
Segment URL constructed
with template and base
URL

14. http://www.dash-player.com/demo/
June 2016 QoMEX 2016 Tutorial 14

15. O Standard, Where Art Thou?
What is a Standard?
http://en.wikipedia.org/wiki/Standards
A technical standard is an established
norm or requirement.
It is usually a formal document that
establishes uniform engineering or
technical criteria, methods, processes
and practices.
è I N T E R O P E R A B I L I T Y
Types of Standard
• Open standards
– ITU-T, W3C, IETF, MPEG, Linux, …
• Closed standards
– MS MediaPlayer, …
• De jure standards
– ANSI, ISO, DIN, …
• De facto standards
– MS Windows, OS X, Unix, Linux, …
• Mandatory standards
– Health, radiation, …
• Voluntary standards
– JPEG, …
June 2016 QoMEX 2016 Tutorial 15

16. Adaptive Streaming Content Workflow
June 2016 QoMEX 2016 Tutorial 16
Source Transcoding Encapsulation Encryption
Origin
Server
HelperDistribution
Client
Linear: Multicast
VoD: FTP, RTMP, HTTP, etc.
Unicast HTTP (PUSH), FTP, etc.
HTTP GET small objects
Single highest-bitrate
stream
Multiple streams at
target bitrates
Multiple streams at
target encapsulation formats
Large video/virtual
files and manifests

17. Adaptive Streaming Content Workflow Simplified
June 2016 QoMEX 2016 Tutorial 17
Standard Delivery
Infrastructure (CDN)
Source Transcoding Encapsulation Encryption
Multiple streams: video [bitrate (32000-20000000), profile (baseline,
main, high), preset (standard, professional, premium), height (128-
7680), width (96-4320), frame rate (1-120), codec (h264, hevc)],
audio: [bitrate (8000-256000), sample rate (0, 8000, 11025, 12000,
16000, 22050, 24000, 32000, 44100, 48000, 64000, 88200, 96000)]
Single highest-bitrate stream:
HTTP, FTP, RTMP; mp4, ts; AVC,
AAC, Subtitles
Multiple streams at target
encapsulation formats: DASH
(MPD + mp4), HLS (m3u8, ts)
Multiple streams with multiple DRM
formats: MPEG-CENC, Widewine,
PlayReady, PrimeTime, Fairplay
Player
Heterogeneous Clients e.g.
Bitmovin HTML5 Adaptive Player
DASH, HLS, HTML5, MSE, EME

18. QoE for DASH Services
• Different application domains have different QoE
requirements
– Need to provide specializations of the general QoE
definition
– Take into account requirements formulated by means of
influence factors and features of QoE
• QoE influence factors for DASH
– Initial/start-up delay (low)
– Buffer underruns, stalls, freezes (zero)
– Quality switches (low)
– Media throughput (high)
– …
June 2016 QoMEX 2016 Tutorial 18

19. !
http://www.bitcodin.com/
http://www.dash-player.com/
QoE Evaluation for DASH-based Services
• Test sequence
– Many datasets available
– Adopted Big Buck Bunny & DASHed it with bitcodin
• Players
– bitdash
– Proprietary solutions (smooth, HLS, HDS)
– YouTube, dash.js, DASH-JS
– …and compare it with ten different adaptation algorithms
• Objective evaluation
– Common test setup using network emulation & bandwidth shaping
– Predefined bandwidth trajectory (or real network traces)
• Subjective evaluation
– Lab [ITU-T B.500 / P.910] vs.
crowdsourcing with special
platforms or social networks
June 2016 QoMEX 2016 Tutorial 19

20. Crowdsourced QoE Evaluation
• Quality of Experience …
– Mean Opinion Score [0..100]
– [other objective metrics:
start-up time, throughput, stalls]
• … Web-based Adaptive HTTP Streaming Clients …
– HTML5+MSE: DASH-JS (dash.itec.aau.at), dash.js (DASH-IF, v1.1.2), YouTube
• … Real-World Environments …
– DASH-JS, dash.js hosted at ITEC/AAU (~ 10Gbit/s)
– YouTube hosted at Google data centers
– Content: Tears of Steel @ 144p (250 kbit/s), 240p (380 kbit/s), 360p (740 kbit/s), 480p (1308 kbit/s), and
720p (2300 kbit/s); segment size: 2s
– Users access content over the open Internet (i.e., real-world environment)
• … Crowdsourcing
– Campaign at Microworker platform (others also possible: Mechanical Turk, social networks) limited to
Europe, USA/Canada, India
– Screening Techniques: Browser fingerprinting, stimulus presentation time, QoE ratings and pre-
questionnaire
June 2016 QoMEX 2016 Tutorial 20
B. Rainer, C. Timmerer, “Quality ofExperience ofWeb-based Adaptive HTTP Streaming Clients in Real-World Environments using Crowdsourcing”,
Proceedings of International Workshop on VideoNext: Design,Quality andDeployment of Adaptive Video Streaming, Sydney, Australia,Dec. 2014.

21. MOS and Average Bitrate
• 288 microworkers, 33 screened (Fingerprinting: 20, presentation time: 6, QoE
ratings and pre- questionnaire: 7), 175 male and 80 female, majority (80%) is aged
between 18 and 37
June 2016 QoMEX 2016 Tutorial 21

22. Startup Time and Number of Stalls
June 2016 QoMEX 2016 Tutorial 22

23. Results Summary
• DASH-JS
– High start-up time
– Low number of stalls
– Good throughput, QoE
• dash.js
– Low start-up time
– High # stalls
– Low throughput
– Low QoE
• YouTube
– Low start-up time
– Low number of stalls
– Best throughput, QoE
June 2016 QoMEX 2016 Tutorial 23

24. Now, 10 different
adaptation logics …
June 2016 QoMEX 2016 Tutorial 24
Adaptation logics well-known in
research literature
Predefined bandwidth trajectory and
test setup
Different segment sizes, RTTs, HTTP/2,
etc.
C. Timmerer, M. Maeiro, B. Rainer, “Which Adaption
Logic? An Objective and Subjective Performance
Evaluation of HTTP-basedAdaptiveMedia Streaming
Systems”, arXiv cs.MM, June 2016,
http://arxiv.org/abs/1606.00341.

25. June 2016 QoMEX 2016 Tutorial 25

26. June 2016 QoMEX 2016 Tutorial 26

27. DASH-JS vs. bitdash
June 2016 QoMEX 2016 Tutorial 27
C. Timmerer, D. Weinberger, C. Mueller, and S. Lederer, “Ultra-High-Definition-Quality of Experience with MPEG-DASH”,
Proceedings of the Broadcast EngineeringConference (BEC), NAB2015, Las Vegas, NV, USA, April 2015.

28. Objective Evaluations
June 2016 QoMEX 2016 Tutorial 28
Stalls (lower is better)Average Bitrate (higher is better)

29. Stalls are really bad…
June 2016 QoMEX 2016 Tutorial 29
Conviva: Viewer Experience Report. 2014

30. Conclusions (1)
• MPEG-DASH defines formats only
– Media Presentation Description (MPD)
– Segment format: mp4, ts
• MPEG-DASH is not
– System, protocol, presentation, codec, interactivity, DRM, client specification
– Other standards required for a complete ecosystem: e.g., DASH-IF, WAVE,
HMTL5, MSE, EME
• Do we need MPEG-DASH? (for adaptive media streaming)
– Not necessarily: e.g., WebM + VPx + manifest & control end-to-end
– Required to address heterogeneous environments
• Role of standards sometimes overrated but often underestimated
June 2016 QoMEX 2016 Tutorial 30

31. Conclusions (2)
• QoE for DASH-based services (a rule of thumb)
– Startup delay (low [but live vs. on-demand & short vs. long-tail
content])
– Buffer underrun / stalls (zero)
– Quality switches (low) and media throughput (high)
– Energy- and cost-awareness (data plan)
• No general applicable QoE model for DASH
– (Too) many factors influencing / features of QoE for DASH-based
services
– Methodology for reproducible research is in place and well established
– Ample research opportunities
June 2016 QoMEX 2016 Tutorial 31
Main QoE
factors for DASH
➡ Come up with our own QoE factor and design, conduct, analyze a small-scale experiment