Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Recording and media manipulation of WebRTC streams

This presentation introduces Kurento technologies to developers at the WebRTC Conference & Expo 2014 in San Jose. It focuses on Kurento Client APIs and on its capabilities for recording and manipulating the audio and video streams in WebRTC sessions.

  • Soyez le premier à commenter

Recording and media manipulation of WebRTC streams

  1. 1. Developing WebRTC applications with advanced media processing Luis Lopez lulop@kurento.org www.kurento.org
  2. 2. WebRTC infrastructures: at the heart of the long tail Peer-to-Peer WebRTC Application (without media infrastructure) WebRTC video stream WebRTC Application based on media infrastructure media infrastructure
  3. 3. WebRTC media servers: the simplest WebRTC infrastructure • Media server – Monolithic software – Based on shared memory – Scales well vertically – Can’t scale horizontally • Media server vendors – Software vendors – Hardware vendors
  4. 4. WebRTC media server vendors
  5. 5. Common functions of WebRTC media servers Transcoding media server VP8 H.264 MCU/SFU media server Recording media server
  6. 6. Evolution of media servers: functionality What common WebRTC Media Servers do: • Transcoding • MCU • Recording What future Media Servers will do: • Flexible processing • Augmented reality • Blending • Mixing • Analyzing • Etc. Media is here Media goes there Current media servers Transcoding MCU/SFU Recording Media is here Rich Media goes there Future media servers Media Events Transcoding, MCU/SFU, Recording, Enrich, Augment, Analyze, Combine, Transform, Adapt, … Context Content Commands
  7. 7. Why is this important? Explicit augmentation with target information Implicit augmentation with external knowledge Why do you communicate? To see other’s face? WebRTC convergence with Telco Why don’t converge with Content?
  8. 8. Kurento Media Server: an open source software WebRTC media server Applications define the processing of streams getting through KMS Send Receive Analyze Augment Enrich Transform Transcode Record Process Replicate Media Source Media Sink KMS Java API JavaScript API Other languages
  9. 9. What does Kurento mean?
  10. 10. Kurento History 2012 •Kurento goes open source LGPL v2.1. Professional services provided by Naevatec (startup) June’13 •1M€ of investment for providing multimedia capabilities to FI-WARE Dec’13 •Kurento Featured at WebRTC Expo (Paris) Feb’14 •3.3M€ of investment for making Kurento an elastic cloud platform (NUBOMEDIA) April’14 •Kurento featured at IMS World Conference (Barcelone) July’14 •Kurento featured at DevCon5 (New York) Sep’14 •500K€ of investment for maintenance of Kurento FI-WARE APIs
  11. 11. Kurento Client API Sink SRC Sink SRC SRC Sink Sink  Media Element • Provides a specific media functionality › Send/receive media › Process media › Transform media • Exchange media through › Sources › Sinks  Media pipeline • Chain of media elements implementing the desired media logic. • The Media API provides the capability of creating media pipelines by joining media elements of the toolbox Media Element Sink SRC
  12. 12. Creating applications: the Lego game H pEndpoint WebRtcEndpoint Protocols & Codecs Application 2 IPTVConnector Mixer FaceDetector PointerTracker BarCodeReader Computer Vision HeartRateBlender ChromaFilter FaceOverlay Augmented Reality Mul sensory Mul media DataChannelEndpoint PlayerEndpoint RecorderEndpoint Media Repository IPTV Integra on Group Communica ons Sink% Sink% SRC% RtpEndpoint SRC$ Sink% SRC% Sink% SRC% Sink% Sink% SRC% SRC% Sink% SRC% Sink% Sink% SRC% SRC% Sink% Sink% SRC$ CdnUpload Sink% Sink SRC Sink Sink SRC SRC Sink Sink Application 3 Sink SRC Sink Sink SRC Sink Sink SRC Application 1 Sink SRC Sink SRC Sink SRC Sink SRC Toolbox of media elements
  13. 13. Architecting a Kurento application
  14. 14. Ready?
  15. 15. Installing Kurento • The doc – http://www.kurento.org/docs/current/installation_guide.html • The video – https://www.youtube.com/watch?v=RGdDpyZAPus • In short sudo add-apt-repository ppa:kurento/kurento wget -O - http://ubuntu.kurento.org/kurento.gpg.key | sudo apt-key add - sudo apt-get update sudo apt-get install kurento-media-server
  16. 16. Get a Kurento cloud instance for free • FI-WARE (http://www.fi-ware.org) – Funded through European Commission research programmes – PaaS cloud infrastructure offering a bunch of capabilities and enablers for the “Future Internet” – You can get money developing WebRTC applications with Kurento and FI-WARE • http://www.fi-ware.org/accelerators/ • How to get a Kurento instance in FIWARE in 5 minutes – https://www.youtube.com/watch?v=lFM12pXeWu8
  17. 17. Kurento “Hello World”
  18. 18. Source & demo https://github.com/Kurento/kurento-tutorial-js/ tree/master/kurento-hello-world
  19. 19. Recording and playing Media Pipeline WebRTC Streaming Sink SRC Sink Media to file or URI Media Pipeline WebRTC Streaming Sink SRC Media from file or URI SRC WebRtcEndpoint WebRtcEndpoint RecorderEndpoint PlayerEndpoint
  20. 20. Source code kurentoClient.create("MediaPipeline", function(error, p) { if(error) return onError(error); pipeline = p; pipeline.create("WebRtcEndpoint", function(error, webRtc){ if(error) return onError(error); pipeline.create(”RecorderEndpoint”, function(error, recorder){ if(error) return onError(error); webRtc.connect(recorder, function(error){ if(error) return onError(error); webRtc.connect(webRtc, function(error){ if(error) return onError(error); webRtc.processOffer(sdpOffer, … if(error) return onError(error); webRtcPeer.processSdpAnswer(sdpAnswer); }); }); }); }); });
  21. 21. I don’t like callbacks From that code structure to this code structure
  22. 22. Second try … generators • https://github.com/Kurento/kurento-tutorial-js/ tree/master/kurento-hello-world-recorder-generator • Transactions very soon … – By end Q4’14
  23. 23. Advanced media processing
  24. 24. Source & demo • https://github.com/Kurento/kurento-tutorial-js/ tree/master/kurento-magic-mirror
  25. 25. Create your own filter: working with Kurento external modules WebRTC Streaming Sink SRC WebRtcEndpoint Sink ArMarkerDetector SRC
  26. 26. Source & demo • https://github.com/Kurento/kurento-tutorial-js/ tree/master/kurento-alvar
  27. 27. Putting it altogether: calls with recording and filtering
  28. 28. Source & demo: let’s switch to Java for fun • https://github.com/Kurento/kurento-tutorial-java/ tree/master/kurento-one2one-call-advanced
  29. 29. Bonus: going beyond the media server vision WebRTC Application based on media infrastructure WebRTC Cloud
  30. 30. WebRTC cloud infrastructures IaaS SaaS PaaS Provider - Computing resources Provider Developer - Development API - Installation - Administration Developer - Security - Application logic - Application logic Provider - Service Developer - Nothing to do
  31. 31. Multimedia PaaS: scalability types “Broadcasting Non-RTC” Number of concurrent sessions Number of users per session WebRTC PaaS Phone system TV Broadcasting CDNs Media Server based Services “RTC calls” “Media server RTC” “Beyond RTC calls”
  32. 32. The scalability problem in call clouds Call Call Call Call Call Call Call Call Call Call Call Call Call Call Call Call Call Call
  33. 33. The scalability problem beyond call clouds Media stream One to MANY
  34. 34. WebRTC services beyond the call model Users in call Monitoring users Users can dynamically change their role
  35. 35. Nubomedia: Kurento as an elastic media server Load%Balancer%Func. on% Applica. on%Server%Func. on% Broker%Func. on% Media%Server%Func. on% IaaS%Cloud%Manager% Cloud% Orchestrator%
  36. 36. Thank you Partners Luis Lopez lulop@kurento.org www.kurento.org