3. Function of WebRTC media servers
http://www.kurento.org
3
Transcoding media server
VP8 H.264
MCU/SFU media server
Recording media server
4. Evolution of media servers
http://www.kurento.org
4
What common WebRTC
Media Servers do:
• Transcoding
• MCU
• Recording
What future Media
Servers will do:
• Flexible processing
• Augmented reality
• Blending
• Mixing
• Analyzing
• Etc.
Media is
here
Media goes
there
Transcoding
MCU/SFU
Recording
Current media servers
Media is
here
Rich Media
goes there
Future media servers
Media
Events
Transcoding,
MCU/SFU,
Recording,
Enrich,
Augment,
Analyze,
Combine,
Transform,
Adapt, …
Context
Content
Commands
5. Why is this important
WebRTC
Making
calls
Content and
entertainment
IoT
M2M
P2M
Multimedia
Making calls
WARNING! Overcrowded
Advertising
Broadcasting
Gaming
eLearning
Blue Ocean
Smart cities
Emergencies
Security
Blue Ocean
6. Kurento Media Server
http://www.kurento.org
6
SinkSRC
Sink
SRC
SRCSink
Sink
Media Element
• Provides a specific media
functionality
› Send/receive media
› Process media
› Transform media
• Exchange media through
› Sources
› Sinks
Media pipeline
• Chain of media elements
implementing the desired media
logic.
• The Media API provides the
capability of creating media
pipelines by joining media
elements of the toolbox
Media Element
Sink
SRC
7. Creating applications basing on
Kurento Media Server
http://www.kurento.org
7
Protocols
& Codecs
Computer
Vision
Augmented
Reality
Mul sensory
Mul media
Media
Repository
IPTV
Integra on
Group
Communica ons
Sink%
H pEndpoint
Sink%SRC%
RtpEndpoint
SRC$
DataChannelEndpointSink%SRC%
WebRtcEndpoint
Sink%
SRC%
BarCodeReader
Sink%
SRC%
PointerTracker
Sink%
SRC%
FaceDetector
Sink%
SRC%
FaceOverlay
Sink%
SRC%
ChromaFilter
Sink%
SRC%
HeartRateBlender
Sink%
RecorderEndpoint
SRC$
PlayerEndpoint
Sink%
CdnUpload
Sink%
IPTVConnector Mixer
SinkSRC
Sink
SRC
Sink
SRCSink
Sink
Application 2
Sink
SRC
Sink
SRCSink
SinkSRC
Sink
Application 3
SinkSRC
SRCSink
SRC Sink
SinkSRC
Application 1
Toolbox of media elements
8. Beyond media servers: WebRTC
clouds and the problem of scalability
http://www.kurento.org
8
WebRTC Application based on media infrastructure
WebRTC Cloud
9. Cloud models for WebRTC
infrastructures
http://www.kurento.org
9
PaaS
IaaS SaaS
Provider
- Computing resources
Developer
- Installation
- Administration
- Security
- Application logic
Provider
- Development API
Developer
- Application logic
Provider
- Service
Developer
- Nothing to do
10. WebRTC PaaS APIs: Requirements
• Requirements of WebRTC PaaS APIs
– Functional requirements
• Media transport
– Media endpoint
– Media replication
– Media routing
• Media persistence
– Media storage
– Media recovery
• Media processing
– Transcoding
– Analysis
– Augmentation
– Non-functional requirements
• Security
• Dependability
• Scalability
http://www.kurento.org
10
11. WebRTC multimedia session
• RFC 5117
– A multimedia session is an association among a group of participants
engaged in the communication via one or more RTP Sessions.
• Characterized by
– Communication topology
• Graph of multimedia flows
– Multimedia processing
• Function of each edge of the graph of media flows
http://www.kurento.org
11
Transport
Transcode
Record
Analyze
Etc.
12. Scalability of RTC multimedia services
http://www.kurento.org
12
Number of concurrent sessions
Numberofuserspersession
WebRTC PaaS Phone system
TV
Broadcasting
CDNs
Media Server
based Services
“RTC calls”
“Home-made
RTC”
“Broadcasting
Non-RTC”
“Beyond RTC calls”
13. Scalability of RTC multimedia services
http://www.kurento.org
13
Number of concurrent sessions
Numberofuserspersession
WebRTC PaaS Phone system
TV
Broadcasting
CDNs
Media Server
based Services
“RTC calls”
“Home-made
RTC”
“Broadcasting
Non-RTC”
“Beyond RTC calls”
14. The scalability problem in “call”
clouds
CallCalll Call CallCall
CallCallCall Call CallCall
CallCall Call CallCall
15. Anatomy of WebRTC PaaS for call models:
Flat Architecture
http://www.kurento.org
15
LoadBalancer
Function
IaaS Cloud Manager
ApplicationServer
Function
BrokerFunction
MediaServer
Function
Cloud Orchestrator
16. Cloud functions: IaaS
manager
http://www.kurento.org
16
IaaS Cloud Manager
Cloud
Orchestrator
• Function
– Provides APIs for IaaS
management
• Images
• Instances
• Storage
• Metrics
• Security
• Etc.
• Requires
– Physical infrastructure
IaaS Cloud Manager
Cloud
Orchestrator
Image
Management
Computing
Management
Storage
Management
Networking
Management
Security
Management
Metrics and
KPIs
Infrastructure as a Service APIs
Load%Balancer%
Func.on%
IaaS%Cloud%Manager%
Applica.on%Server%
%Func.on%
Broker%Func.on%
Media%Server%
Func.on%
Cloud%Orchestrator%
17. Cloud functions: Cloud
Orchestrator
http://www.kurento.org
17
IaaS Cloud Manager
Cloud
Orchestrator
• Function
– Lifecycle management of the
platform
• It acquires virtual resources and
allocate them to the specific
services
– Runtime management with
autoscaling
• It scales out new service instances
in situations of peak load
• It scales in service instances
whenever they are not required
any longer
• Requires
– Autoscaling rules
• Ex. If average load is over 60% add
two new instances
Image
Management
Computing
Management
Storage
Management
Networking
Management
Security
Management
Metrics and
KPIs
Infrastructure as a Service APIs
Load%Balancer%
Func.on%
IaaS%Cloud%Manager%
Applica.on%Server%
%Func.on%
Broker%Func.on%
Media%Server%
Func.on%
Cloud%Orchestrator%
20. Application Server
http://www.kurento.org
20
Application Server Instance
Signaling frontend
Security logic
API logic
Client request
• Function
– Signaling
• Send/receive of signaling messages
– Security logic
• Authentication, Authorization,
Accounting
– PaaS API logic
• Control of media server functions
for providing API semantics
• Requires
– Signaling protocol
implementation
• SIP, JSON, etc.
– Security rules
• ACLs, CAP, etc.
– Specific logic
• Media server dependent
Load%Balancer%
Func.on%
IaaS%Cloud%Manager%
Applica.on%Server%
%Func.on%
Broker%Func.on%
Media%Server%
Func.on%
Cloud%Orchestrator%
21. Load balancer
• Function
– Distributes client requests
among available AS
instances
– Usually stateful
• Requires
– Balancing policy
• Round robin
• Random
• Less load
• Etc.
• Scaling needs
– Low
http://www.kurento.org
21
Load Balancer Function
Client App.
Application
Server
Instance
Client App. Client App.
Application
Server
Instance
Application
Server
Instance
Load%Balancer%
Func.on%
IaaS%Cloud%Manager%
Applica.on%Server%
%Func.on%
Broker%Func.on%
Media%Server%
Func.on%
Cloud%Orchestrator%
22. The flat Nubomedia implementation
http://www.kurento.org
22
LoadBalancer
Function
IaaS Cloud Manager
ApplicationServer
Function
BrokerFunction
MediaServer
Function
Cloud Orchestrator
23. Instance Lifecycle
http://www.kurento.org 23
OpenStack Swift
OpenStack Glance
Media Server
Image
Application
Server Image
Broker
Image
Load Balancer
Image
Heat Orchestration
Template
OpenStack Nova
Scalability
Group
Open Stack HeatPacker
Autoscaling
Rules
Launch
configurations
Scalability
Group
Scalability
Group
Scalability
Group
Load
Balancer
Instance
Application
Server
Instance
Application
Server
Instance
Broker
Instance
Media
Server
Instance
Media
Server
Instance
Media
Server
Instance
24. Scalability of RTC multimedia services
http://www.kurento.org
24
Number of concurrent sessions
Numberofuserspersession
WebRTC PaaS Phone system
TV
Broadcasting
CDNs
Media Server
based Services
“RTC calls”
“Home-made
RTC”
“Broadcasting
Non-RTC”
“Beyond RTC calls”
25. Beyond calls: convergence of
broadcasting and phone-like services
http://www.kurento.org
25
Users in call Monitoring users
Users can dynamically
change their role
26. The scalability problem in “beyond
call” clouds
http://www.kurento.org
26
Media stream
OnetoMANY
27. Anatomy of WebRTC PaaS for call
models: Hierarchical Architecture
http://www.kurento.org
27
LoadBalancer
Function
IaaS Cloud Manager
ApplicationServer
Function
BrokerFunction
MediaServer
Function
Cloud Orchestrator
29. The elastic media server
http://www.kurento.org
29
• Elasticity
– On the number of media
pipelines
• Number of concurrent
sessions
– On the number of elements
per media pipeline
• Number of concurrent users
per session
– Media Pipeline
• Distributed media pipeline
• Rigidity
– The media element is a
monolithic (non distributed)
entity
SinkSRC
SRCSink
SinkSRC
Sink
SRC
SRCSinkSRCSink
SRCSink
Load%Balancer%
Func.on%
IaaS%Cloud%Manager%
Applica.on%Server%
%Func.on%
Broker%Func.on%
Media%Server%
Func.on%
Cloud%Orchestrator%
30. Broker Function
http://www.kurento.org
30
Broker Function
Application
Server
Instance • Function
– Assigns “call legs” to specific
media server instances
• Give me a media server instance to
take care of this call
– “call” are split among different
media server instances
• Requires
– Scheduling policy
• Topology aware
• Network aware
• SLA aware
• Etc.
– Very complex problem
• Leg adding may require additional
media server instances
• Churn is very complex to manage
Media
Server
Instance
Media
Server
Instance
Media
Server
Instance
Media
Server
Instance
Media
Server
Instance
Load%Balancer%
Func.on%
IaaS%Cloud%Manager%
Applica.on%Server%
%Func.on%
Broker%Func.on%
Media%Server%
Func.on%
Cloud%Orchestrator%
31. Hierarchical Nubomedia implementation:
Work in progress
http://www.kurento.org
3131
Load%Balancer%
Func.on%
IaaS%Cloud%Manager%
Applica.on%Server%
%Func.on%
Broker%Func.on%
Media%Server%
Func.on%
Cloud%Orchestrator%
Own
implementation
32. Lessons learnt
• CAP theorem
– You cannot be available and consistent at the same time
• Always chose availability
• Managing failures is tricky
– Beware of poison messages
– Logs need to be centralized
• Video streams are not like water streams
– FIRs & PLIs will break bandwidth
• …. and the battery
• Scaling out is simpler than scaling in
– Move or wait?
• Managing video quality with limited BW is THE PROBLEM
– Simulcast Vs Transcoding
http://www.kurento.org
32
33. Nubomedia: the elastic RTC
multimedia infrastructure
http://www.kurento.org
33
Thanks
Partners
Luis Lopez
lulop@kurento.org