Making Virtual Reality Real: 4K VR/AR Encoding and Global Delivery

Making Virtual Reality Real: 4K VR/AR Encoding and Global Delivery
Apurva Agrawal

COMPRESSED
BIT RATE
(Kbps)
50,000
40,000
30,000
20,000
10,000
2001
QVGA
MPEG1
550 Kbps
VGA/SDTV
MPEG2-4
1,800 Kbps
2004 2016
4K-UHD
HEVC/H.265
16,000-30,000 Kbps
HD-1080p
AVC/H.264
7,500 Kbps
201
1
HD-720p
AVC/H.264
3,500 Kbps
2007
Quality Requires Bits

©2017 AKAMAI | FASTER FORWARD
TM

TM
Agenda
● Understanding all the new terms in the industry
● Challenges of 4K and VR workflows
● Real world use cases
● Recommendations
● Key Takeaways

TM
UNDERSTANDING THE TERMS

TM
Core Principles - Picture Quality
Resolution
Refers to the number of pixels that compose the picture on the screen
1080p VS 4K*
A screen with 1080 rows and 1920 columns of pixels VS a screen 2160 rows and 3840 columns of pixels.
4K simply means a clearer picture. It's more pixels (8,294,400 to be exact) on the screen at once that creates images that
are crisper and capable of showing more details than standard HD.
*when we talk about 4K we are mostly talking about UHD..more on that in the next slide

TM
4K ≠ UHD
UHD is a consumer display and broadcast standard.
UHD quadruples that resolution to 3,840 by 2,160 from Std HD (1,920 by 1,080) but also improves picture quality in 7
different areas.
When we talk about 4K OTT streaming we are mostly talking about the UHD standard in terms of resolution

©2015 AKAMAI | FASTER FORWARDTM
Ultra High Definition video represents an improvement
in quality in 7 different areas
Size Color HDR FPS COD ART Sound UHD
Spatial
resolution
Color
Range
Dynamic
range
Temporal
resolution
Component
coding
Compression
artifacts
Sound
Systems

TM
UHD Spec - More than just resolution

TM
What is HDR or High Dynamic Range?
While 4K promises more pixels on the screen, HDR is promising better, richer pixels. HDR screens are capable of displaying
millions more colors than SDR televisions, and the contrast between the darkest part of the image and the brightest part can
be expanded even further.
4K = more pixels <> HDR = better pixels
HDR10 vs. HDR HLG vs. Dolby Vision?
HLG (Hybrid Log Gama) and HDR10 are open source formats. While Dolby Vision is a proprietary standard for HDR
made by Dolby. HLG maintains some backward compatibility with older SDR TVs. HDR10 is not backward compatible but
offers richer colors and brightness levels. Dolby Vision is a proprietary standard that has an edge in terms of colors and
contrast, but also provides a way for content creators to include a metadata channel that offers a smooth way to downgrade
the signal to HDR10 or SDR without the loss of quality that comes from an automatic conversion.
Devices that support HLG, HDR10 or DoVi?
While support for HDR10 is broad, including popular brands of TVs, Game Consoles and OTT Boxes. The support for Dolby
Vision took longer to get going, but now major brands across the device spectrum are starting to support DoVi. TVs: Sony,
TCL/Roku, Philips, Vizio and LG; OTT Boxes: Chromecast Ultra, Apple TV 4K; and Mobile Devices: such as the iPhone X.

Media Innovation Zone Demo: HDR & 3D Audio with Dolby
Vision & Atmos
Why:
• Give people a personal experience with HDR
• The highest quality experiences are now
starting to come to OTT first
Discussion Point:
• HDR displays are starting to take off and
content is becoming available. What are your
you doing about it?
Standard
Dynamic Range
High
Dynamic Range
What:
• Side by Side experience of Dolby Vision &
Atmos (audio format) vs. Standard Definition &
Stereo Sound

TM
Core Principles - Encoding for 4K or VR 360
What is a Codec?
A codec is a device or program that allows you to shrink large video files (encoding) and make them available for later
viewing or editing (decoding) – the name comes from both “(co)mpressor/(dec)ompressor” and “(co)de/(dec)ode”.
H.265 or HEVC
H.265, also known as HEVC, is the successor to the current generation of H.264 codec (also known as AVC) and is fully
capable of encoding/decoding 4K resolutions. While there are no costs associated with HEVC/H.265 use for end users,
hardware manufacturers, proprietary software vendors and paid streaming services are required to pay royalties for using
HEVC technology in their products and/or services. For example, H.265 codecs are included with the latest 4K monitors and
4K smart TVs
VP9
VP9 is an open and royalty free video coding format developed by Google. VP9 is a successor to VP8 and competes mainly
with MPEG's High Efficiency Video Coding (HEVC/H.265).
AV1
While VP9 was not completely successful in that task, it has laid the foundation for Google’s next generation codec,
AOMedia Video 1 (AV1), which is supported by companies like AMD, ARM, Intel, NVIDIA, Google, Microsoft, Mozilla, Netflix
and Amazon. AV1 is royalty free.

TM
Core Principles - Formats 4K

TM
What is VR/AR/MR?
VR - Virtual Reality
360° video is an immersive experience using pre-filmed real-world content as the central media. 360° video is a version of
VR created with only real-world content
AR- Augmented Reality
Augmented reality is an overlay of content on the real world, but that content is not anchored to or part of it.
MR - Mixed Reality
Mixed reality is an overlay of synthetic content on the real world that is anchored to and interacts with the real world.
These technologies are all in their infancy. We will focus on 360 degree videos for VR.

TM
ALL OF THESE FEATURES
REQUIRE BANDWIDTH

Typical bitrates for high quality content

TM
Use Case: Delivering a 360° VR Video

TM
360° VR Video Challenges
● Requires low latency - Only have 20-40ms to get new picture
when user turns their head
● High bandwidth - Atleast 20mbps for 4K and even more for 8K
● Streaming the entire sphere takes up bandwidth
● Requires high quality

TM
Typical 360° VR Video Workflow
Courtesy: harmonic

TM
360° VR Video Workflow
● VR Video Capturing
○ Content can be produced in 2D/3D
○ Nokia OZO, GoPro Odyssey, Orah & Lots of cheaper options on the market
● VR Video Stitching
○ Different camera angles have to be stitched together. For live you will stitch the video before encoding
○ For VOD stitching can be done in the post production phase
○ Lots of cameras will support live stitching or a stitching software can take SDI/HDMI as input
● VR Encoding
○ Codec: Mostly H.265 (HEVC) Main at 3840X2160 with ABR
○ Ingest : SDI/IP/MPEG-4/depends on the encoder
○ Packaging can be DASH or HLS.
○ Tiled Encoding
● VR Viewing Devices
○ Head Mounted Displays - Samsung Gear VR, Oculus Rift, HTC Vive

TM
HEVC Tiled Approach - Recommendations
● Akamai released a paper at IBC 2017 with Tiledmedia describing one approach
○ https://show.ibc.org/__media/Technical-Papers-2017/R-VanBrandenburg----CDN-OPTIMIZATION-FOR-VR-STREAMING.pdf
● During encoding, split video up into multiple independently decodable ‘tiles’
● Required bitrate about 1/8th of the full resolution (12k VR ~ 4K UHD)
● Streaming tiles that are only in the FOV (Field of View) - 90 degrees by 90 degrees.Player then only requests the tiles that are in the FOv
● LOW RES fallback layer always available if the network can’t keep up
● CDN with a lot of points of presence - low RTT
● High chance of cache miss - Tilemedia’s client gives hints to Akamai on what tiles will most likely be requested
● Prefetching tiles with byte range requests to keep tiles in cache warm
● Using QUIC protocol with Akamai’s Media Acceleration feature

2
Poor
Network
Conditions
Cellular
Network
Congestion
WiFi
Interference
Akamai Intelligent Platform Media Acceleration SDKs
Media Acceleration 2.0
Introduces new highly secure distribution sources and ensures viewers
receive content via the most efficient available source
Sustainable Economics | Scale | Quality Improvements

TM
Use Case: Delivering a Live 4K
Experience

TM
Challenges
● High bandwidth requirements
● Resilient ingest
● Choice of encoding formats, delivery formats
● Monitoring and failover

TM
Media Services Live 4.x = Purpose Built New Architecture for 4K

The Solution – Media Services Live with liveOriginTM Capabilities
Low-Latency Live content 1-2 seconds behind broadcast
Self Healing
Brings the reliability and availability required for live
24/7 streaming content & large events
Allows for improved ingestion performance over the
open internet to match broadcast quality
Media Ingest Acceleration
Enhanced Monitoring & Alerting
Allowing customers to quickly identify and mitigate
first mile issues
DVR and Archive
Provide end users ways to match the
TV experience online

TM
Recommendations when delivering 4K Live
● Encoding
○ Use HEVC or VP9 as the video codec. AV1 will become an option in Q1-2018 to
Chrome and Firefox clients.
● Ingest
○ If live, use IAS for ingest and MAE (QUIC) for delivery.
○ Use MSL 4.x
● Segment Size
○ If live, reduce segment size to <= 4s, to keep segment file size manageable.
○ Ingest and deliver on the same continent. Try to avoid trans-atlantic and trans-pacific
traffic.
● Manage bitrates
○ When the player starts up, have it estimate the throughput and if it’s too low, warn
the user that they may not have sufficient bandwidth for great 4K experience.

TM
Key Takeaways
● New streaming technologies are coming to OTT first instead of
broadcast!
● Akamai has products and services that can help you architect
the best in class solution for your company

Making Virtual Reality Real: 4K VR/AR Encoding and Global Delivery

Making Virtual Reality Real: 4K VR/AR Encoding and Global Delivery

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (19)

Similaire à Making Virtual Reality Real: 4K VR/AR Encoding and Global Delivery

Similaire à Making Virtual Reality Real: 4K VR/AR Encoding and Global Delivery (20)

Plus de Akamai Developers & Admins

Plus de Akamai Developers & Admins (20)

Dernier

Dernier (20)

Making Virtual Reality Real: 4K VR/AR Encoding and Global Delivery