As Flash continues to decline, HTML5 video technologies increasingly bring the promise of heightened performance and better QOE. This workshop provides an in-depth look at HTML5 players, their features and strengths, as well as the open-source media engine frameworks available on the market today. We begin by examining the main components in a video player, then discuss how to choose a player adapted to one’s use case, examining how several open-source solutions compare. Finally, we use an interactive example to build features and demonstrate several optimizations, offering tips and best practices and pointing out potential production issues as we go along.
Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...
2016 Streaming Media West: Transitioning from Flash to HTML5
1. Choosing an HTML5 player
An overview of how media engines work & benchmark
of open-source frameworks
Streaming Media West – Track D
Tuesday, May 10, 2016
1:45 to 2:30 pm
2. ● Pioneers in hybrid video delivery systems to accompany exponential growth in OTT
traffic
● Experts in HTML5 and consultants in the transition from Flash
Infinite scale, limitless delivery.
Streamroot: Who are we?
3. And who are you?
Infinite scale, limitless delivery.
4. I. HTML5 basics
A. Why make the transition
B. Formats, APIs, standardization
II. Delving in to the migration
A. Formats & Encoding
B. Players
C. DRMs, ads and business features
D. HTTPs
III. Choosing a player
A. Modern player architecture
B. Considerations & what to look for
C. Benchmark of open-source and proprietary options
What we’ll be talking about today.
Infinite scale, limitless delivery.
5. IV. Examples: hls.js & dash.js
Debugging, subtitles, encryption, ads, analytics, Streamroot, ABR
V. Going from POC to production
A. What could go wrong
B. What happens after I do a POC? Workflow, fallbacks, etc.
C. Smoothly transitioning into production: QoS metrics, AB testing
D. Useful tools & links
What we’ll be talking about today.
Infinite scale, limitless delivery.
7. Infinite scale, limitless delivery.
I. HTML5 Basics
APIs
1. Media Source Extensions
• Makes HTTP adaptive streaming
possible in HMTL5
• Use JavaScript to build streams and
inject data into the video tag’s buffer
Source: W3C specs
8. Infinite scale, limitless delivery.
I. HTML5 Basics
APIs
2. Encrypted Media Extensions
• Allows for DRMs in HMTL5
• Provides a way to interact with
content protection systems
• Plugin-free
• Common Encryption (CENC):
standardized key and encryption
methods - multiple DRMs for same
file
Source: W3C specs
14. Infinite scale, limitless delivery.
II. Delving into the migration
Formats & Encoding
Significant changes to the encoding side that can take some time
Encode to MP4 and then repackage as necessary for target platforms
Lots of packaging solutions out there: Wowza, Unified Streaming Platform, etc.
15. Infinite scale, limitless delivery.
II. Delving into the migration
Players
More flexible architecture, with separation between UX and media engine logic
When choosing: look at use case and feature requirements
Section III.
16. Infinite scale, limitless delivery.
II. Delving into the migration
Business features
DRMs: switch from token authorization to HMTL5 DRM
Ads: rewrite the ad logic in HTML5
Business logic: make sure 3rd party plugins are also compatible with HTML5
17. Infinite scale, limitless delivery.
II. Delving into the migration
HTTPS
Beware!
Playing HTTP streams even in an HTTPS environment is not an option in HTML5.
Changing to HTTPs can be expensive and long, depending on your CDN.
18. Infinite scale, limitless delivery.
It’s like...
III. Player architecture
Modern player architecture
19. … your favorite burger.
Infinite scale, limitless delivery.
It’s like...
III. Player architecture
Modern player architecture
23. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
UI
Social sharing
III. Player architecture
24. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
UI
Media Engine(s)
Social sharing
III. Player architecture
25. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
Media Engine(s)
UI
Media Engine(s)
Social sharing
III. Player architecture
26. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
Social sharing
Media Engine(s)
UI
Media Engine(s)
Playback
& DRM
III. Player architecture
27. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
Content decryption
module
Media Engine(s)
DRM
Manager
UI
Media Engine(s)
Playback
& DRM
Social sharing
III. Player architecture
28. Infinite scale, limitless delivery.
Skin
Ads
Playlists
DRM
Manager
Decoder /
Renderer
Authentication
Content decryption
module
Media Engine(s)
UI
Media Engine(s)
Playback
& DRM
Social sharing
III. Player architecture
29. Infinite scale, limitless delivery.
1. Skin - the graphic design
of your player
III. Player architecture
User Interface
30. Infinite scale, limitless delivery.
1. Skin - the graphic design
of your player
2. UI logic - features defining
all interaction with the user
on top of video playback
Lots of plugins available, video.js for
example
III. Player architecture
User Interface
31. Infinite scale, limitless delivery.
3. Business logic
Authentication
Payment
Ads
*Configuration / device detection logic!
III. Player architecture
User Interface
32. Infinite scale, limitless delivery.
Sample UI workflow with authentication, channels and pre-roll advertisement
III. Player architecture
User Interface
33. Infinite scale, limitless delivery.
Easily customizable, with many plugins available for use or inspiration.
Add features as plugins/modules to the core UI base.
Create a unified user experience across browsers, even if the media engine behind it
may vary from device to device.
Check out tools such as React native, Haxe
III. Player architecture
User Interface
40. Infinite scale, limitless delivery.
Extremely important yet too often neglected.
Most often necessary to have several media engines to reach your audience.
III. Player architecture
Media Engine
42. 1. General criteria
- device, format and codec dependencies
Infinite scale, limitless delivery.
III. What to look for
2. Custom features
- DRMs
- DVR subtitles
- QoS, etc.
3. Perfs & Quality
- startup time
- ABR
- error strategies
4. Street cred, robustness, ease of use
43. Light-touch dev:
- Simplicity and stablity of the media engine
- Assess extensibility & ease of incorporating special features
- Media engine customization
- Events exposed
- Debugging
Infinite scale, limitless delivery.
Digging deeper:
- Robust yet flexible core design
- Tests and testing coverage
- community & support
III. What to look for
44. Infinite scale, limitless delivery.
III. What’s out there?
Disclaimer!
- What follows are all GOOD options - probably the best out there.
- We’ve tried to be solely objective based on our research and experience.
- Features, support and upkeep are always changing.
45. Infinite scale, limitless delivery.
PROs
Supported by DASH-IF
Pushed & maintained by Akamai + tier-1s (BBC)
Highest visibility, big community
Lots of features and use-cases handled
Widely used in production
Huge test suite + online test page
CONs
A little bit of technical debt and complexity
Non-trivial API and customization config
H264/AAC support only
III. What’s out there?
46. Infinite scale, limitless delivery.
PROs
Built by an entire at Google: solid & smart
Performances and robustness, quick to improve
Simple to get started, good tutorials
Only one supporting WebM, VP8, VP9, open audio codecs
Good support on github and Google groups
CONs
Google-centric
Today lacks some features for large broadcasters
Stricter PR & features policy
No ES6 support
Fewer OVP and open-source all-in-one integrations
III. What’s out there?
47. Infinite scale, limitless delivery.
PROs
Lead by Dailymotion, built from scratch by author of Flashls
Enormous traction and visibility
Clear architecture design, easily extendable
Good robustness and debug demo, responsive support
Widely used in prod by all-in-one players + tier-1s
CONs
Not all HLS features supported yet
Some restrictions from the transmuxing & HLS: no DRMs today,
only AES128
III. What’s out there?
48. Infinite scale, limitless delivery.
PROs
Seamless Flash fallback with a MediaSource polyfill
De facto solution for HLS with VideoJS
Large community of users (Brightcove + Videojs)
In production with Brightcove with a wide range of customers
CONs
Videojs plugin: not usable without videojs
Lack of public debug tools or pages
Learning curve on providers & tech behind it
III. What’s out there?
49. But there are also a lot of off-the-shelf options.
Infinite scale, limitless delivery.
III. What’s out there?
50. All available online on github:
https://github.com/streamroot/benchmarking-player
Contains:
- Media engines code
- Examples
Infinite scale, limitless delivery.
IV. Examples
58. STRONG POINTS COULD BE IMPROVED
Very simple and understandable
Startup time constraint could be improved to get
the lowest level first
Handles CPU & player size constraints
Conservative BW adjustment to avoid oscillation,
with EWMA smoothing
Sound emergency abort mechanism
Infinite scale, limitless delivery.
IV. Examples: hls.js sumup
59. 1. Tweak the parameters
https://github.com/dailymotion/hls.js/blob/master/API.md#fine-tuning
Dropped FPS:
capLevelOnFPSDrop: false,
fpsDroppedMonitoringPeriod: 5000,
fpsDroppedMonitoringThreshold: 0.2
PlayerSize:
capLevelToPlayerSize: false,
2. Write your own rules!
AbrController: AbrController
capLevelController: CapLevelController,
fpsController: fpsController
Bandwidth estimation:
abrEwmaFastLive: 5.0,
abrEwmaSlowLive: 9.0,
abrEwmaFastVoD: 4.0,
abrEwmaSlowVoD: 15.0,
abrEwmaDefaultEstimate: 500000,
abrBandWidthFactor: 0.8,
abrBandWidthUpFactor: 0.7,
Infinite scale, limitless delivery.
IV. Examples: hls.js how to improve
62. STRONG POINTS COULD BE IMPROVED
Smoothes bandwidth No quantization of bitrates
Segment abort mechanism to avoid buffering during
network drops
Doesn’t handle CPU & Player size constraints
Rich buffer threshold to avoid BW oscillations
Infinite scale, limitless delivery.
IV. Examples: Dash.js sumup
63. 1. Tweak the Parameters
ThroughputRule:
AVERAGE_THROUGHPUT_SAMPLE_AMOUNT_LIVE = 2;
AVERAGE_THROUGHPUT_SAMPLE_AMOUNT_VOD = 3;
AbandonRequestRule:
GRACE_TIME_THRESHOLD = 500;
ABANDON_MULTIPLIER = 1.5;
2. Write your own rules
https://github.com/Dash-Industry-Forum/dash.js/wiki/Migration-2.0#extending-dashjs
https://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/rules/abr/ABRRulesCollection.js
BufferOccupancyRule:
RICH_BUFFER_THRESHOLD = 20
Infinite scale, limitless delivery.
IV. Examples: Dash.js how to improve
65. What could go wrong?
- - Encoding
- - DRMs
- - HTTPS
Infinite scale, limitless delivery.
V. The Path to Production!
66. I did a POC. Now what?
- - Do you need a Flash Fallback?
- - Moving the UX to HTML5
- - Build the media engine extendable architecture (providers)
- - Fallback alternatives
https://github.com/streamroot/videojs5-hlsjs-source-handler/blob/master/lib/videojs5-hlsjs-source-handler.js
https://github.com/videojs/video.js/blob/66922a818e588b8954989c439e1be790777966ae/docs/guides/tech.md
https://github.com/faisalman/ua-parser-js
Infinite scale, limitless delivery.
V. The Path to Production!
67. Making a smooth transition.
- - HTML5 player rollout
- - Playback & QoS Metrics
- - AB Testing!
Infinite scale, limitless delivery.
V. The Path to Production!
68. Infinite scale, limitless delivery.
Streamroot
workflow
V. The Path to Production!
AB testing pipeline: General workflow
69. Many options based on your requirements: open-source, proprietary, Cloud...
Infinite scale, limitless delivery.
V. The Path to Production!
AB testing pipeline: What to chose from
Message broker /
stream processor
Storage + possible
query/visualization
BI/viz tools Advanced analytics
Kafka (Kinesis) Druid Caravel R
RabbitMQ InfluxDB Grafana Matlab
ActiveMQ Cassandra (DynamoDB) Tableau
PostGresSQL QlikView
ElasticSearch Kibana
71. Client AB testing methods:
1) Dynamic Config Injection
2) Different builds with a reverse proxy
Infinite scale, limitless delivery.
V. The Path to Production!
AB testing pipeline: General setup
72. Infinite scale, limitless delivery.
V. The Path to Production!
AB testing pipeline: Config injection
73. Infinite scale, limitless delivery.
V. The Path to Production!
AB testing pipeline: Config injection
74. Infinite scale, limitless delivery.
V. The Path to Production!
AB testing pipeline: Dynamic config injection
PROs
Simple & easy splits
Separate - fast to deploy & rollback
Scalable
CONs
Limited scope
Complications in code
Need to handle scaling & high availability
WARNING
Async implementation!
75. Infinite scale, limitless delivery.
V. The Path to Production!
AB testing pipeline: Different builds with reverse proxy
76. Infinite scale, limitless delivery.
PROs
Serve very different (major) versions of player
Perfect for rolling out new features smoothly
(10% then 30% then...)
CONs
Good CI/CD necessary to roll out new
versions quickly
Heavy stack to maintain and scale
WARNING
Make sure the file request is not slower!
Single point of failure!
V. The Path to Production!
AB testing pipeline: Different builds with reverse proxy
77. How much time does it take?
The build vs. buy tradeoff.
- Do I have the internal resources to develop,
integrate, customize and maintain?
- Are the features I need already available in
off-the-shelf and/or open-source solutions?
Consider time to market.
Infinite scale, limitless delivery.
V. The Path to Production!
79. Nikolay Rodionov, Co-Founder and CPO, nikolay@streamroot.io
Erica Beavers, Head of Partnerships, erica@streamroot.io
Infinite scale, limitless delivery.
Get in touch!
80. We’d like to send out a huge thanks to Ludovic Bostral and Benjamin Pott from
Afrostream for letting us borrow their burger analogy. Without them this talk would not
have been half as juicy.
(Also their platform rocks!)
Infinite scale, limitless delivery.
Credits
Notes de l'éditeur
@charlo - Eurosport video player example that we used in blog
@charlo - Eurosport video player example that we used in blog
@charlo - Eurosport video player example that we used in blog
Checks the max CapLevel corresponding to current player size
Every 1000ms.
You can also add up manual level caps on initialization.
If the cap level is bigger that the last one (which means the player size has grown, like in Fullscreen for exemple), then you flush the current buffer and ask for a new quality right away (force the buffer)
Calculates the dropped frames per second ratio.
If it is > 0.2, bans the level for ever => goes into restricated levels
Not activated in production!
fpsDroppedMonitoringThreshold
fpsDroppedMonitoringPeriod
First segment always from the lowest quality, then it continues with normal rule (very simple simple rule in practice!)
Another optimization is just to load this level (and playlist), and don’t wait for the other levels to have been loaded
Simple algorithm,
One of the most important ones here
What happens if you started a request and then BW drops ? Especially important when you ahve long fragments, this can very easily lead to a buffer underrun!
After Half of the needed time, compare the estimate time of arrival to time of buffer underrun. And then see if there is another level that could solve the issue?
Pros:
Simple implementation, taking into account a lot of different params
Works as good as the other implementation at Dailymotion! (alshls, android, iPhone… etc)
Cons:
Still Naive bandwidth estimation => possible overestimation, and possible oscillation around bitrates?
We can do a lot of improvements on bandwidth estimation! difficult to correlate a unique segment download time to the real device’s available bandwidth, for several reasons:
You can have very quick bandwidth changes, especially on a mobile network, as well as unexpected bandwidth drops
The requests can be living in parallel with other TCP request (HTTP or any other on the user’s device)
This can lead to frequent estimation oscillations!
The different static constants more for you use-case?
You can play with them
You can also easily build your own rule!
Here is an example on Github?
First explain how to do that?
DASH.Js has 4 different Rules
ThroughputRule calculates bandwidth with some smoothing!
No real quantizing (have a real estimate and no other values)
AbandonRequestsRule cancels if takes more than 1.5x of donwload
BufferOccupancyRule to now go down if buffer large enough (RICH BUFFER TRESHOLD)
InsufficientBufferRule au tas
You can easily take the best out of hls.js here! Write a player size rule, a FPS drop rule… change the Abandonrate rule!
It’s all very easy to do!