SlideShare a Scribd company logo

TADS Developer Summit WebRTC Dan Burnett

•
•
Alan Quayle
Alan Quayle

Dan Burnett, editor of the WebRTC specification and author of The WebRTC Book provides an excellent tutorial on WenRTC at TADS, 21-22 Nov 2013 in Bangkok

Technology
1 of 93
Download to read offline
WebRTC Introduc)on  to  WebRTC   Dan  Burne4   Chief  Scien)st,  Tropo   Director  of  Standards,  Voxeo     Alan  Johnston   Dis)nguished  Engineer   Avaya    
WebRTC  Tutorial  Topics   •  •  •  •  •  •  •  What  is  WebRTC?   How  to  Use  WebRTC   WebRTC  Peer-­‐to-­‐Peer  Media   WebRTC  Protocols  and  IETF  Standards   WebRTC  W3C  API  Overview   Pseudo  Code  Walkthrough   PracFcal  bits   TAD  Summit  Bangkok  2013   2  
What  is  WebRTC?  
WebRTC  is  “Voice  &  Video  in  the  browser”   •  Access  to  camera  and  microphone  without  a   plugin   –  No  proprietary  plugin  required!     •  Audio/video  direct  from  browser  to  browser   •  Why  does  it  maWer?   –  Media  can  stay  local   –  Mobile  devices  eventually  dropping  voice  channel   anyway   –  Games   TAD  Summit  Bangkok  2013   4  
The  Browser  RTC  FuncFon   Web   Server   Signaling   Server   HTTP  or  WebSockets     JavaScript/HTML/CSS   Other  APIs   Web   Browser   •  WebRTC  adds  new  Real-­‐ Time  CommunicaFon  (RTC)   FuncFon  built-­‐in  to   browsers   –  No  download   HTTP  or  WebSockets  –  No  Flash  or  other  plugins     (Signaling)   •  Contains   –  Audio  and  video  codecs   –  Ability  to  negoFate  peer-­‐to-­‐ peer  connecFons   On-­‐the-­‐wire  protocols   –  Echo  cancellaFon,  packet  loss   (Media  or  Data)   concealement   RTC  APIs   Browser   RTC   FuncFon   NaFve  OS  Services   •  In  Chrome  &  Firefox  today,   Internet  Explorer  someFme   and  Safari  eventually   TAD  Summit  Bangkok  2013   5  
Benefits  of  WebRTC   For  Developer     For  User   •  Streamlined  development  –   one  placorm   •  Simple  APIs  –  detailed   knowledge  of  RTC  protocols   not  needed   •  NAT  traversal  only  uses   expensive  relays  when  no   other  choice   •  Advanced  voice  and  video   codecs  without  licensing     •  No  download  or  install  –   easy  to  use   •  All  communicaton   encrypted  –  private   •  Reliable  session   establishment     –  “just  works”   •  Excellent  voice  and  video   quality     •  Many  more  choices  for  real-­‐ Fme  communicaFon   TAD  Summit  Bangkok  2013   6  
WebRTC  Support  of  MulFple  Media   Microphone  Audio   ApplicaFon  Sharing  Video   Front  Camera  Video   Rear  Camera  Video   WebCam  Video   Stereo  Audio   Browser  L         on  Laptop   Browser  M   on  Mobile   •  MulFple  sources  of  audio  and  video  are  assumed   and  supported   •  All  media,  voice  and  video,  and  feedback  messages   are  mulFplexed  over  the  same  transport  address   TAD  Summit  Bangkok  2013   7  
WebRTC  Triangle   Web  Server   (ApplicaFon)   Peer  ConnecFon  (Audio,  Video,  and/or  Data)   Browser  L   Browser  M   (Running  HTML5  ApplicaFon     from  Web  Server)   (Running  HTML5  ApplicaFon     from  Web  Server)   •  Both  browsers  running  the  same  web  applicaFon  from  web   server   •  Peer  ConnecFon  established  between  them  with  the  help  of   the  web  server   TAD  Summit  Bangkok  2013   8  
WebRTC  Trapezoid   Web  Server  A   (ApplicaFon  A)   Browser  M   SIP     or  Jingle   Web  Server  B   (ApplicaFon  B)   Peer  ConnecFon  (Audio  and/or  Video)   Browser  T   (Running  HTML5  ApplicaFon     from  Web  Server  B)   (Running  HTML5  ApplicaFon     from  Web  Server  A)     •  Similar  to  SIP  Trapezoid     •  Web  Servers  communicate  using  SIP  or  Jingle  or  proprietary   •  Could  become  important  in  the  future.   TAD  Summit  Bangkok  2013   9  
WebRTC  and  SIP   Web  Server     SIP   SIP  Server   SIP   Browser  M   Peer  ConnecFon  (Audio  and/or  Video)   SIP  Client     •  SIP  (Session  IniFaFon  Protocol)  is  a  signaling  protocol  used  by  service   providers  and  enterprises  for  real-­‐Fme  communcaFon   •  Peer  ConnecFon  appears  as  a  standard  RTP  session,  described  by  SDP   •  SIP  Endpoint  must  support  RTCWEB  media  extensions       TAD  Summit  Bangkok  2013   10  
WebRTC  and  Jingle   Web  Server   Jingle   XMPP  Server   Jingle   Peer  ConnecFon  (Audio  and/or  Video)   Browser  M   Jingle  Client   •  Jingle  is  a  signaling  extension  to  XMPP  (Extensible  Messaging  and   Presence  Protocol,  aka  Jabber)   •  Peer  ConnecFon  SDP  can  be  mapped  to  Jingle   •  Jingle  Endpoint  must  support  RTCWEB  Media  extensions   TAD  Summit  Bangkok  2013   11  
WebRTC  and  PSTN   Web  Server   Peer  ConnecFon  (Audio)   PSTN  Gateway   Browser  M   Phone   •  Peer  ConnecFon  terminates  on  a  PSTN  Gateway   •  Audio  Only   •  EncrypFon  ends  at  Gateway   TAD  Summit  Bangkok  2013   12  
WebRTC  with  SIP   Web  Server     SIP  Proxy/Registrar  Server     WebSocket  (SIP)   HTTP     (HTML5/CSS/ JavaScript)   Browser  M   (running  JavaScript  SIP  UA)   HTTP     WebSocket   (HTML5/CSS/ (SIP)   JavaScript)   SRTP  Media   Browser  T   (running  JavaScript  SIP  UA)     •  Browser  runs  a  SIP  User  Agent  by  running  JavaScript  from  Web  Server     •  SRTP  media  connecFon  uses  WebRTC  APIs   •  Details  in  [dram-­‐iec-­‐sipcore-­‐websocket]  that  defines  SIP  transport  over   TAD  Summit  Bangkok  2013   13   WebSockets  
WebRTC  Signaling  Approaches   •  Signaling  is  required  for  exchange  of  candidate  transport   addresses,  codec  informaFon,  media  keying  informaFon   •  Many  opFons  –  choice  is  up  to  web  developer   TAD  Summit  Bangkok  2013   14  
How  to  Use  WebRTC  
WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   All  media  added   Set  Up  Peer   ConnecFon   Peer  ConnecFon  established   AWach  Media   or  Data   AWach  more  media  or  data   Ready  for  call   Exchange   Offer/Answer   2013   TAD  Summit  Bangkok   16  
WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   •  getUserMedia() –  Audio  and/or  video   –  Constraints   –  User  permissions   All  media  added   Set  Up  Peer   ConnecFon   •  Browser  must  ask  before   allowing  a  page  to  access   microphone  or  camera   Peer  ConnecFon  established   AWach  Media   or  Data   AWach  more  media  or  data   •  MediaStream •  MediaStreamTrack –  CapabiliFes   –  States  (sepngs)   Ready  for  call   Exchange   Offer/Answer   TAD  Summit  Bangkok  2013   17  
WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   •  RTCPeerConnection –  All  media  added   –  –  Set  Up  Peer   –  ConnecFon   –  Peer  ConnecFon  established   –  AWach  Media   AWach  more  media  or  data   –  or  Data   –  Ready  for  call   Direct  media   Between  two  peers   ICE  processing   SDP  processing   DTMF  support   Data  channels   IdenFty  verificaFon   StaFsFcs  reporFng   Exchange   Offer/Answer   TAD  Summit  Bangkok  2013   18  
WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   •  addStream() –  Doesn't  change  media  state!   •  removeStream() All  media  added   –  DiWo!   Set  Up  Peer   ConnecFon   •  createDataChannel() Peer  ConnecFon  established   AWach  Media   or  Data   –  Depends  on  transport   AWach  more  media  or  data   Ready  for  call   Exchange   Offer/Answer   TAD  Summit  Bangkok  2013   19  
WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   All  media  added   Set  Up  Peer   ConnecFon   Peer  ConnecFon  established   AWach  Media   or  Data   •  createOffer(), createAnswer() •  setLocalDescription(), setRemoteDescription() •  Applying  SDP  answer  makes   the  magic  happen AWach  more  media  or  data   Ready  for  call   Exchange   Session   DescripFons   TAD  Summit  Bangkok  2013   20  
WebRTC  usage  –  a  bit  more  detail   Set  Up  Signaling   Channel   Obtain  Local   Media   Get  more  media   Set  Up  Peer   ConnecFon   AWach  Media  or   Data   Exchange   Session   TAD  Summit  Bangkok  2013   DescripFons   AWach  more  media  or  data   21  
SDP  offer/answer   •  Session  DescripFons   –  Session  DescripFon  Protocol  created  for  use  by   SIP  in  sepng  up  voice  (and  video)  calls   –  Describes  real-­‐Fme  media  at  low  level  of  detail   •  Which  IP  addresses  and  ports  to  use   •  Which  codecs  to  use   •  Offer/answer  model  (JSEP)   –  One  side  sends  an  SDP  offer  lisFng  what  it  wants   to  send  and  what  it  can  receive   –  Other  side  replies  with  an  SDP  answer  lisFng  what   it  will  receive  and  send   TAD  Summit  Bangkok  2013   22  
WebRTC  Peer-­‐to-­‐Peer  Media  
Media  Flows  in  WebRTC   Web  Server       Internet   Home  WiFi   Router   Router           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  Router       Browser  L     TAD  Summit  Bangkok  2013   24  
Media  without  WebRTC   Web  Server       Internet   Home  WiFi   Router   Router           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  Router       Browser  L     TAD  Summit  Bangkok  2013   25  
Peer-­‐to-­‐Peer  Media  with  WebRTC   Web  Server       Internet   Home  WiFi   Router   Router           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  Router       Browser  L     TAD  Summit  Bangkok  2013   26  
NAT  Complicates  Peer-­‐to-­‐Peer  Media   Web  Server   Most  browsers  are  behind  NATs   on  the  Internet,  which   complicates  the  establishment   of  peer-­‐to-­‐peer  media  sessions.         Internet   Router  with   NAT   Home  WiFi     with  NAT           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   27  
What  is  a  NAT?   •  Network  Address  Translator  (NAT)   •  Used  to  map  an  inside  address  (usually  a   private  IP  address)  to  outside  address   (usually  a  public  IP  address)  at  Layer  3   •  Network  Address  and  Port  TranslaFon   (NAPT)  also  changes  the  transport  port   number  (Layer  4)   – These  are  omen  just  called  NATs  as  well   •  One  reason  for  NAT  is  the  IP  address   shortage   TAD  Summit  Bangkok  2013   28  
NAT  Example   Internet   “Outside”      Public  IP  Address       203.0.113.4 “Inside”      Private  IP  Addresses       192.168.x.x Home  WiFi     with  NAT     Browser  M   192.168.0.5     Browser  T   192.168.0.6       TAD  Summit  Bangkok  2013   29  
NATs  and  ApplicaFons   •  NATs  are  compaFble  with  client/server  protocols   such  as  web,  email,  etc.   •  However,  NATs  generally  block  peer-­‐to-­‐peer   communicaFon   •  Typical  NAT  traversal  for  VoIP  and  video   services  today  use  a  media  relay  whenever  the   client  is  behind  a  NAT   –  Omen  done  with  an  SBC  –  Session  Border   Controller   –  This  is  a  major  expense  and  complicaFon  in   exisFng  VoIP  and  video  systems   •  WebRTC  has  a  built-­‐in  NAT  traversal  strategy:   InteracFve  ConnecFvity  Establishment  (ICE)   TAD  Summit  Bangkok  2013   30  
Peer-­‐to-­‐Peer  Media  Through  NAT   Web  Server   ICE  connecFvity  checks  can   omen  establish  a  direct  peer-­‐ to-­‐peer  session  between   browsers  behind  different   NATs         Internet   Router  with   NAT   Home  WiFi     with  NAT           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   31  
ICE  ConnecFvity  Checks   •  ConnecFvity  through  NAT  can  be  achieved  using  ICE   connecFvity  checks   •  Browsers  exchange  a  list  of  candidates   –  Local:  read  from  network  interfaces     –  Reflexive:  obtained  using  a  STUN  Server   –  Relayed:  obtained  from  a  TURN  Server  (media  relay)   •  Browsers  aWempt  to  send  STUN  packets  to  the   candidate  list  received  from  other  browser   •  Checks  performed  by  both  sides  at  same  Fme   •  If  one  STUN  packet  gets  through,  a  response  is  sent   and  this  connecFon  used  for  communicaFon   –  TURN  relay  will  be  last  resort  (lowest  priority)   TAD  Summit  Bangkok  2013   32  
P2P  Media  Can  Stay  Local  to  NAT   If  both  browsers  are   behind  the  same  NAT,   connecFvity  checks  can   omen  establish  a   connecFon  that  never   leaves  the  NAT.     Web  Server       Internet   Router  with   NAT   Home  WiFi     with  NAT           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   33  
ICE  Servers   Web  Server       STUN  Server   TURN  Server   198.51.100.9   198.51.100.2       ICE  uses  STUN  and  TURN   servers  in  the  public   Internet  to  help  with  NAT   traversal.     Internet   Home  WiFi     with  NAT   203.0.113.4 Router  with   NAT         Browser  M   192.168.0.5         Browser  D     Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   34  
Browser  Queries  STUN  Server     Web  Server       STUN  Server   TURN  Server   198.51.100.9   198.51.100.2       Browser  sends  STUN  test   packet  to  STUN  server  to   learn  its  public  IP  address   (address  of  the  NAT).     Internet   Home  WiFi     with  NAT   203.0.113.4 Router  with   NAT         Browser  M   192.168.0.5         Browser  D     Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   35  
TURN  Server  Can  Relay  Media   Web  Server       STUN   TURN  Server  as  a   Server   Media  Relay       In  some  cases,  connecFvity   checks  fail,  and  a  TURN   Media  Relay  on  the  public   Internet  must  be  used.       Internet   Router  with   NAT   Home  WiFi     with  NAT           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   36  
WebRTC  Protocols  and  IETF   Standards  
WebRTC:  A  Joint  Standards  Effort   •  Internet  Engineering  Task  Force  (IETF)  and  World   Wide  Web  ConsorFum  (W3C)  are  working  together  on   WebRTC   •  IETF   –  Protocols  –  “bits  on  wire”   –  Main  protocols  are  already  RFCs,  but  many  extensions  in   progress   –  RTCWEB  (Real-­‐Time  CommunicaFons  on  the  Web)  Working   Group  is  the  main  focus,  but  other  WGs  involved  as  well   –  hWp://www.iec.org     •  W3C   –  APIs  –  used  by  JavaScript  code  in  HTML5   –  hWp://www.w3c.org   TAD  Summit  Bangkok  2013   38  
WebRTC  Protocols   ApplicaFon  Layer   HTTP   ICE   WebSocket   SRTP   SDP   STUN   TURN   Transport  Layer   TLS   TCP   Network  Layer   DTLS   UDP   SCTP   IP   SIP  is  not  shown  as  it  is  opFonal   TAD  Summit  Bangkok  2013   39  
IETF  RTCWEB  Documents   Document) Ref) Overview' “Overview:'Real'Time'Protocols'for' Browser6based'Applications”' draft6ietf6rtcweb6 overview' Use'Cases'and'Requirements' “Web'Real6Time'Communication' Use6cases'and'Requirements”' draft6ietf6rtcweb6 use6cases6and6 requirements' RTP'Usage' “Web'Real6Time'Communication' (WebRTC):'Media'Transport'and' Use'of'RTP”' draft6ietf6rtcweb6 rtp6usage' Security'Architecture' “RTCWEB'Security'Architecture”' draft6ietf6rtcweb6 security6arch' Threat'Model' “Security'Considerations'for'RTC6 Web”' draft6ietf6rtcweb6 security' Data'Channel' “RTCWeb'Data'Channels”' draft6ietf6rtcweb6 data6channel' JSEP' “JavaScript'Session'Establishment' Protocol”' draft6ietf6rtcweb6 jsep' Audio' “WebRTC'Audio'Codec'and' Processing'Requirements”' draft6ietf6rtcweb6 audio' Quality'of'Service' ' Title) “DSCP'and'other'packet'markings' for'RTCWeb'QoS”' draft6ietf6rtcweb6 qos' TAD  Summit  Bangkok  2013   40  
Codecs                      RFC  6716                  .                 •  Mandatory  to  Implement  (MTI)  audio  codecs  are   seWled  on  Opus  and  G.711  (finally!)   •  Video  is  not  yet  decided!   TAD  Summit  Bangkok  2013   41  
WebRTC  W3C  API  Overview  
Two  primary  API  secFons   •  Handling  local  media   –  Media  Capture  and  Streams  (getUserMedia)   specificaFon   •  Transmipng  media   –  WebRTC  (Peer  ConnecFon)  specificaFon   TAD  Summit  Bangkok  2013   43  
Local  Media  Handling   Audio   PresentaFon   Video   “Audio”  Track   Presenter  Stream   “PresentaFon”    Track   “Audio”  Track   Microphone  Audio   Presenter  Video   ApplicaFon  Sharing  Video   Front  Camera  Video   Rear  Camera  Video   “Presenter”     Track   DemonstraFon  Stream   “Audio”  Track   DemonstraFon   Video   Browser  M   Sources   PresentaFon  Stream   Captured   MediaStreams   •  In  this  example   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  Captured  4  local  media  streams   –  Created  3  media  streams  from  them   –  Sent  streams  over  Peer  ConnecFon   TAD  Summit  Bangkok  2013   44  
Local  Media  Handling   Audio   PresentaFon   Video   Microphone  Audio   ApplicaFon  Sharing  Video   Browser  M   Sources   •  Sources   “Audio”  Track   Presenter  Stream   “PresentaFon”    Track   “Audio”  Track   Presenter  Video   Front  Camera  Video   Rear  Camera  Video   PresentaFon  Stream   “Presenter”     Track   DemonstraFon  Stream   “Audio”  Track   DemonstraFon   Video   Captured   MediaStreams   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  Encoded  together   –  Can't  manipulate  individually   TAD  Summit  Bangkok  2013   45  
Local  Media  Handling   Audio   PresentaFon   Video   Microphone  Audio   ApplicaFon  Sharing  Video   Browser  M   “Audio”  Track   Presenter  Stream   “PresentaFon”    Track   “Audio”  Track   Presenter  Video   Front  Camera  Video   Rear  Camera  Video   PresentaFon  Stream   “Presenter”     Track   DemonstraFon  Stream   “Audio”  Track   DemonstraFon   Video   Sources   Captured   MediaStreams   •  Tracks  (MediaStreamTrack)   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  Tied  to  a  source   –  Exist  primarily  as  part  of  Streams;  single  media  type   –  Globally  unique  ids;  opFonally  browser-­‐labeled   TAD  Summit  Bangkok  2013   46  
Local  Media  Handling   PresentaFon  Stream   Audio   “Audio”  Track   PresentaFon   Video   Microphone  Audio   ApplicaFon  Sharing  Video   Presenter  Stream   “Audio”  Track   Presenter  Video   “Presenter”     Track   DemonstraFon  Stream   Front  Camera  Video   Rear  Camera  Video   Browser  M   Sources   “PresentaFon”    Track   “Audio”  Track   DemonstraFon   Video   Captured   MediaStreams   •  Captured  MediaStream   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  Returned  from  getUserMedia() –  Permission  check  required  to  obtain   TAD  Summit  Bangkok  2013   47  
Local  Media  Handling   Audio   PresentaFon   Video   Microphone  Audio   ApplicaFon  Sharing  Video   Browser  M   Sources   “Audio”  Track   Presenter  Stream   “PresentaFon”    Track   “Audio”  Track   Presenter  Video   Front  Camera  Video   Rear  Camera  Video   PresentaFon  Stream   “Presenter”     Track   DemonstraFon  Stream   “Audio”  Track   DemonstraFon   Video   Captured   MediaStreams   •  MediaStream   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  All  contained  tracks  are  synchronized   –  Can  be  created,  transmiWed,  etc.   TAD  Summit  Bangkok  2013   48  
Local  Media  Handling   •  Sepngs   –  Current  values  of  source  properFes  (height,  width,   etc.)   –  Exposed  on  MediaStreamTrack •  CapabiliFes   –  Allowed  values  for  source  properFes   –  Exposed  on  MediaStreamTrack •  Constraints   –  Requested  ranges  for  track  properFes   –  Used  in  getUserMedia(),  applyConstraints() TAD  Summit  Bangkok  2013   49  
Transmipng  media   •  Signaling  channel   –  Non-­‐standard   –  Must  exist  to  set  up  Peer  ConnecFon   •  Peer  ConnecFon   –  Links  together  two  peers   –  Add/Remove  Media  Streams   •  addStream(),  removeStream() –  Handlers  for  ICE  or  media  change   –  Data  Channel  support   TAD  Summit  Bangkok  2013   50  
Peer  ConnecFon   •  "Links"  together  two  peers   –  Via  new RTCPeerConnection() –  Generates  Session  DescripFon  offers/answers   •  createOffer(),  createAnswer() –  From  SDP  answers,  iniFates  media   •  setLocalDescription(),  setRemoteDescription() –  Offers/answers  MUST  be  relayed  by  applicaFon   code!   –  ICE  candidates  can  also  be  relayed  and  added  by  app   •  addIceCandidate()   TAD  Summit  Bangkok  2013   51  
Peer  ConnecFon   •  Handlers  for  signaling,  ICE  or  media  change   –  onsignalingstatechange –  onicecandidate,   oniceconnectionstatechange –  onaddstream,  onremovestream –  onnegotiationneeded –  A  few  others   TAD  Summit  Bangkok  2013   52  
Peer  ConnecFon   •  “Extra”  APIs   –  Data   –  DTMF   –  StaFsFcs   –  IdenFty   •  Grouped  separately  in  WebRTC  spec   –  but  really  part  of  RTCPeerConnection   definiFon   –  all  are  mandatory  to  implement   TAD  Summit  Bangkok  2013   53  
Data  Channel  API   •  RTCDataChannel createDataChannel() •  Configurable  with   –  –  –  –  ordered maxRetransmits,  maxRetransmitTime negotiated id •  Provides  RTCDataChannel  with   –  send() –  onopen,  onerror,  onclose,  onmessage*   TAD  Summit  Bangkok  2013   54  
DTMF  API   •  RTCDTMFSender createDTMFSender() –  Associates  track  input  parameter  with  this   RTCPeerConnection •  RTCDTMFSender  provides   –  boolean canInsertDTMF() –  insertDTMF() –  ontonechange –  (other  stuff)   TAD  Summit  Bangkok  2013   55  
StaFsFcs  API   •  getStats() –  Callback  returns  staFsFcs  for  given  track   •  StaFsFcs  available  (local/remote)  are:   –  Bytes/packets  xmiWed   –  Bytes/packets  received   •  May  be  useful  for  congesFon-­‐based   adjustments   TAD  Summit  Bangkok  2013   56  
IdenFty  API   •  setIdentityProvider(),   getIdentityAssertion() •  Used  to  verify  idenFty  via  third  party,  e.g.,   Facebook  Connect   •  Both  methods  are  opFonal   •  onidentity  handler  called  amer  any   verificaFon  aWempt   •  RTCPeerConnection.peerIdentity  holds   any  verified  idenFty  asserFon   TAD  Summit  Bangkok  2013   57  
Pseudo  Code  Walkthrough  
Pseudo  Code   •  Close  to  real  code,  but  .  .  .   •  No  HTML,  no  signaling  channel,  not   asynchronous,  and  API  is  sFll  in  flux   •  Don't  expect  this  to  work  anywhere   TAD  Summit  Bangkok  2013   59  
Back  to  first  diagram   Microphone  Audio   ApplicaFon  Sharing  Video   Front  Camera  Video   Rear  Camera  Video   WebCam  Video   Stereo  Audio   Browser  L         on  Laptop   Browser  M   on  Mobile   •  Mobile  browser  "calls"  laptop  browser   •  Each  sends  media  to  the  other   TAD  Summit  Bangkok  2013   60  
Mobile  browser  code  outline   var signalingChannel = createSignalingChannel(); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( •  We  will  look  next  at  each  of  these   •  .  .  .  except  for  creaFng  the  signaling   channel   [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   61  
Mobile  browser  produces  .  .  .   Audio   PresentaFon  Stream   “Audio”  Track   “PresentaFon”  Track   PresentaFon  Video   Microphone  Audio   ApplicaFon  Sharing  Video   “Audio”  Track   Presenter  Video   “Presenter”  Track   DemonstraFon  Stream   Front  Camera  Video   Rear  Camera  Video   Presenter  Stream   “Audio”  Track   DemonstraFon  Video   “DemonstraFon”  Track   Browser  M   Sources   Captured  MediaStreams   Created  MediaStreams   •  At  least  3  calls  to  getUserMedia() •  Three  calls  to  new MediaStream() •  App  sends  stream  ids,  then  streams   TAD  Summit  Bangkok  2013   Tracks   62  
funcFon  getMedia()  [1]   navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } . . . function attachMedia() { presentation = new MediaStream( •  Get  audio   •  (Get  window  video  –  out  of  scope)   [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   63  
funcFon  getMedia()  [2]   . . . constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); •  Get  front-­‐facing  camera   •  Get  rear-­‐facing  camera   constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   64  
funcFon  createPC()   var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; pc = new RTCPeerConnection(configuration); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; •  Create  RTCPeerConnection •  Set  handlers   function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   65  
Mobile  browser  consumes  .  .  .   Audio  &  Video  Stream   Display   “Video”  Track   Right  Headphone   “Right”  Track   Lem  Headphone   Browser  M   “Lem”  Track   (Audio  &  Video  Stream  selected)   Stereo  Stream   “Right”  Track   “Lem”  Track   “Mono”  track   Mono  Stream   Sinks   MediaStreams   •  Receives  three  media  streams   •  Chooses  one •  Sends  tracks  to  output  channels   TAD  Summit  Bangkok  2013   Tracks   66  
FuncFon  handleIncomingStream()   if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); •  If  incoming  stream  has  video  track,  set  to   av_stream  and  display  it   •  If  it  has  two  audio  tracks,  must  be  stereo   •  Otherwise,  must  be  the  mono  stream   pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   67  
FuncFon  show_av(st)   display.srcObject = new MediaStream(st.getVideoTracks()[0]); left.srcObject = new MediaStream(st.getAudioTracks()[0]); right.srcObject = new MediaStream(st.getAudioTracks()[1]); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); •  Turn  streams  into  URLs   •  Set  as  source  for  media  elements   } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   68  
Mobile  browser  code  outline   var signalingChannel = createSignalingChannel(); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( •  We  will  look  next  at  each  of  these   •  .  .  .  except  for  creaFng  the  signaling   channel   [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   69  
funcFon  aWachMedia()  [1]   presentation = new MediaStream( [microphone.getAudioTracks()[0], application.getVideoTracks()[0]]); presenter = new MediaStream( [microphone.getAudioTracks()[0], front.getVideoTracks()[0]]); demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); . . . // Audio // Presentation "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // Audio // Presenter // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( // Audio // Demonstration •  Create  3  new  streams,  all  with  same   audio  but  different  video   TAD  Summit  Bangkok  2013   var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; 70  
funcFon  aWachMedia()  [2]   pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); •  AWach  all  3  streams  to  Peer  ConnecFon   •  Send  stream  ids  to  peer  (before  streams!)   } function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   71  
Mobile  browser  code  outline   var signalingChannel = createSignalingChannel(); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( •  We  will  look  next  at  each  of  these   •  .  .  .  except  for  creaFng  the  signaling   channel   [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   72  
funcFon  call()   pc.createOffer(gotDescription, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function gotDescription(desc) { pc.setLocalDescription(desc, s, e); function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); } pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter •  Ask  browser  to  create  SDP  offer   •  Set  offer  as  local  descripFon   •  Send  offer  to  peer   demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   73  
How  do  we  get  the  SDP  answer?   signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter }; demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); •  Signaling  channel  provides  message   •  If  SDP,  set  as  remote  descripFon   •  If  ICE  candidate,  tell  the  browser   TAD  Summit  Bangkok  2013   } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; 74  
And  now  the  laptop  browser  .  .  .   •  Watch  for  the  following   –  We  set  up  media  *amer*  receiving  the  offer   –  but  the  signaling  channel  sFll  must  exist  first!   –  Also,  need  to  save  incoming  stream  ids   TAD  Summit  Bangkok  2013   75  
Signaling  channel  message  is  trigger   signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; . . . pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; }; // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { •  Set  up  PC  and  media  if  not  already  done   if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; TAD  Summit  Bangkok  2013   76  
Signaling  channel  message  is  trigger   signalingChannel.onmessage = function (msg) { . . . if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); }; •  If  SDP,  *also*  answer   •  But  if  neither  SDP  nor  ICE  candidate,  must   be  set  of  incoming  stream  ids,  so  save   TAD  Summit  Bangkok  2013   signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; 77  
FuncFon  prepareForIncomingCall()   createPC(); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; getMedia(); var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); attachMedia(); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; •  No  suprises  here   •  Media  obtained  is  a  liWle  different   •  But  aWached  the  same  way   TAD  Summit  Bangkok  2013   // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; 78  
FuncFon  answer()   pc.createAnswer(gotDescription, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", function gotDescription(desc) { pc.setLocalDescription(desc, s, e); "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = signalingChannel.send(JSON.stringify({ "sdp": desc })); function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); } constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; •  createAnswer()  automaFcally  uses   value  of  remoteDescription  when   generaFng  new  SDP   // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; TAD  Summit  Bangkok  2013   79  
Laptop  browser  consumes  .  .  .   PresentaFon  Stream   “Audio”  Track   “PresentaFon”  Track   Presenter  Stream     “Audio”  Track   Speaker   “Presenter”  Track   Display   DemonstraFon  Stream   Display   “Audio”  Track   Display   “DemonstraFon”  Track   Browser  L   (All  video  streams  selected)   Tracks   MediaStreams   Sinks   •  Three  input  streams •  All  have  same  #  of  audio  and  video  tracks •  Need  stream  ids  to  disFnguish   TAD  Summit  Bangkok  2013   80  
FuncFon  handleIncomingStream()   if (st.id === incoming.presentation) { speaker.srcObject = new MediaStream(st.getAudioTracks()[0]); win1.srcObject = new MediaStream(st.getVideoTracks()[0]); } else if (st.id === incoming.presenter) { win2.srcObject = new MediaStream(st.getVideoTracks()[0]); } else { win3.srcObject = new MediaStream(st.getVideoTracks()[0]); } •  Use  ids  to  disFnguish  streams   •  Extract  one  audio  and  all  video  tracks   •  Assign  to  element  sources   TAD  Summit  Bangkok  2013   var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; 81  
Laptop  browser  produces  .  .  .   Audio  &  Video  Stream   Video   “Video”  Track   WebCam   Lem  Microphone   “Right”  Track   “Lem”  Track   Lem  Audio   Right  Microphone   Stereo  Stream   Browser  L   “Right”  Track   Right  Audio   “Lem”  Track   “Mono”  Track   Mono  Stream   Tracks   Created  MediaStreams   Captured   MediaStreams   Sources   •  Three  calls  to  getUserMedia() •  Three  calls  to  new MediaStream() •  No  stream  ids  needed   TAD  Summit  Bangkok  2013   82  
FuncFon  getMedia()  [1]   navigator.getUserMedia({"video": true}, function (stream) { webcam = stream; }, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); . . . pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; •  Request  webcam  video   // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; TAD  Summit  Bangkok  2013   83  
FuncFon  getMedia()  [2]   . . . constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) {left = stream;}, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) {right = stream;}, e); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); •  Request  lem  and  right  audio  streams   •  Save  them  as  left  and  right  variables   function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; TAD  Summit  Bangkok  2013   84  
TADS Developer Summit WebRTC Dan Burnett
TADS Developer Summit WebRTC Dan Burnett
TADS Developer Summit WebRTC Dan Burnett
TADS Developer Summit WebRTC Dan Burnett
TADS Developer Summit WebRTC Dan Burnett
TADS Developer Summit WebRTC Dan Burnett
TADS Developer Summit WebRTC Dan Burnett
TADS Developer Summit WebRTC Dan Burnett
TADS Developer Summit WebRTC Dan Burnett

Recommended

TADS Developer Summit Apidaze Philippe Sultan
TADS Developer Summit Apidaze Philippe SultanTADS Developer Summit Apidaze Philippe Sultan
TADS Developer Summit Apidaze Philippe SultanAlan Quayle
 
TADS Developer Summit Tropo Jose de Castro
TADS Developer Summit Tropo Jose de CastroTADS Developer Summit Tropo Jose de Castro
TADS Developer Summit Tropo Jose de CastroAlan Quayle
 
Audio codes presentation
Audio codes presentationAudio codes presentation
Audio codes presentationTed Wentzel
 
WebRTC introduction
WebRTC introductionWebRTC introduction
WebRTC introductiongotouch
 
PLNOG 4: Ido Miran - Service Provider Class 5 VoIP - An Integrated Approach
PLNOG 4: Ido Miran - Service Provider Class 5 VoIP - An Integrated ApproachPLNOG 4: Ido Miran - Service Provider Class 5 VoIP - An Integrated Approach
PLNOG 4: Ido Miran - Service Provider Class 5 VoIP - An Integrated ApproachPROIDEA
 
Developing rich SIP applications with SIPSIMPLE SDK
Developing rich SIP applications with SIPSIMPLE SDKDeveloping rich SIP applications with SIPSIMPLE SDK
Developing rich SIP applications with SIPSIMPLE SDKSaúl Ibarra Corretgé
 
Building Tropo Apps with Grails
Building Tropo Apps with GrailsBuilding Tropo Apps with Grails
Building Tropo Apps with GrailsChristopher Bartling
 
Video Interoperability with Skype for Business using Pexip Infinity
Video Interoperability with Skype for Business using Pexip InfinityVideo Interoperability with Skype for Business using Pexip Infinity
Video Interoperability with Skype for Business using Pexip Infinitymscug
 

Recommended

TADS Developer Summit Apidaze Philippe Sultan
TADS Developer Summit Apidaze Philippe SultanTADS Developer Summit Apidaze Philippe Sultan
TADS Developer Summit Apidaze Philippe SultanAlan Quayle
 
TADS Developer Summit Tropo Jose de Castro
TADS Developer Summit Tropo Jose de CastroTADS Developer Summit Tropo Jose de Castro
TADS Developer Summit Tropo Jose de CastroAlan Quayle
 
Audio codes presentation
Audio codes presentationAudio codes presentation
Audio codes presentationTed Wentzel
 
WebRTC introduction
WebRTC introductionWebRTC introduction
WebRTC introductiongotouch
 
PLNOG 4: Ido Miran - Service Provider Class 5 VoIP - An Integrated Approach
PLNOG 4: Ido Miran - Service Provider Class 5 VoIP - An Integrated ApproachPLNOG 4: Ido Miran - Service Provider Class 5 VoIP - An Integrated Approach
PLNOG 4: Ido Miran - Service Provider Class 5 VoIP - An Integrated ApproachPROIDEA
 
Developing rich SIP applications with SIPSIMPLE SDK
Developing rich SIP applications with SIPSIMPLE SDKDeveloping rich SIP applications with SIPSIMPLE SDK
Developing rich SIP applications with SIPSIMPLE SDKSaúl Ibarra Corretgé
 
Building Tropo Apps with Grails
Building Tropo Apps with GrailsBuilding Tropo Apps with Grails
Building Tropo Apps with GrailsChristopher Bartling
 
Video Interoperability with Skype for Business using Pexip Infinity
Video Interoperability with Skype for Business using Pexip InfinityVideo Interoperability with Skype for Business using Pexip Infinity
Video Interoperability with Skype for Business using Pexip Infinitymscug
 
Rebaca's Video Delivery Expertise Overview
Rebaca's Video Delivery Expertise OverviewRebaca's Video Delivery Expertise Overview
Rebaca's Video Delivery Expertise OverviewArshad Mahmood
 
Pexip Infinity Fusion for Skype for Business
Pexip Infinity Fusion for Skype for BusinessPexip Infinity Fusion for Skype for Business
Pexip Infinity Fusion for Skype for BusinessGraham Walsh
 
Pexip and Microsoft - Certified for Skype for Business Server
Pexip and Microsoft - Certified for Skype for Business ServerPexip and Microsoft - Certified for Skype for Business Server
Pexip and Microsoft - Certified for Skype for Business ServerGraham Walsh
 
Балансировщики KEMP для Microsoft Lync, Skype for Business
Балансировщики KEMP для Microsoft Lync, Skype for BusinessБалансировщики KEMP для Microsoft Lync, Skype for Business
Балансировщики KEMP для Microsoft Lync, Skype for BusinessAlexander Zhuravlev, Ph.D.
 
Flash media gateway
Flash media gatewayFlash media gateway
Flash media gatewayChithiraiselvan Deivasigamani
 
Pexip Infinity 13 adds more than 20 new features and enhancements
Pexip Infinity 13 adds more than 20 new features and enhancementsPexip Infinity 13 adds more than 20 new features and enhancements
Pexip Infinity 13 adds more than 20 new features and enhancementsAnders Løkke
 
Pexip presentation
Pexip presentationPexip presentation
Pexip presentationTed Wentzel
 
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCC
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCCMicrosoft Surface Hub - How to integrate with legacy videoconferencing and UCC
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCCGraham Walsh
 
Pexip architecture
Pexip architecturePexip architecture
Pexip architectureMatthieu LIENART
 
Basic VoIP Drupal Intro - for BioRAFT
Basic VoIP Drupal Intro - for BioRAFTBasic VoIP Drupal Intro - for BioRAFT
Basic VoIP Drupal Intro - for BioRAFTMicky Metts
 
Simple hybrid voice deployments with Sonus
Simple hybrid voice deployments with SonusSimple hybrid voice deployments with Sonus
Simple hybrid voice deployments with SonusAdam Hand
 
Avaya lync integration 6 16
Avaya lync integration 6 16Avaya lync integration 6 16
Avaya lync integration 6 16Chris Stegh
 
Microsoft Lync 2010 Call Flows Explained
Microsoft Lync 2010 Call Flows ExplainedMicrosoft Lync 2010 Call Flows Explained
Microsoft Lync 2010 Call Flows ExplainedMUCUGL
 
Byt ut din gamla växel – en djupdykning i Lync telefoni
Byt ut din gamla växel – en djupdykning i Lync telefoniByt ut din gamla växel – en djupdykning i Lync telefoni
Byt ut din gamla växel – en djupdykning i Lync telefoniTommy Clarke
 
Lync 2010 Conferencing Deep Dive
Lync 2010 Conferencing Deep DiveLync 2010 Conferencing Deep Dive
Lync 2010 Conferencing Deep DiveHarold Wong
 
Ken Rehor's presentation at eComm 2008
Ken Rehor's presentation at eComm 2008Ken Rehor's presentation at eComm 2008
Ken Rehor's presentation at eComm 2008eComm2008
 
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...Louis Göhl
 
Tc 2008 11 19
Tc 2008 11 19Tc 2008 11 19
Tc 2008 11 19jeffiel
 
Integrating Microsoft Surface Hub with legacy videoconferencing and UCC
Integrating Microsoft Surface Hub with legacy videoconferencing and UCCIntegrating Microsoft Surface Hub with legacy videoconferencing and UCC
Integrating Microsoft Surface Hub with legacy videoconferencing and UCCAnders Løkke
 
Pexip and Microsoft Interoperability
Pexip and Microsoft InteroperabilityPexip and Microsoft Interoperability
Pexip and Microsoft InteroperabilityGraham Walsh
 
WebRTC Overview by Dan Burnett
WebRTC Overview by Dan BurnettWebRTC Overview by Dan Burnett
WebRTC Overview by Dan BurnettMojo Lingo
 
DevCon 5 (December 2013) - WebRTC & WebSockets
DevCon 5 (December 2013) - WebRTC & WebSocketsDevCon 5 (December 2013) - WebRTC & WebSockets
DevCon 5 (December 2013) - WebRTC & WebSocketsCrocodile WebRTC SDK and Cloud Signalling Network
 

More Related Content

What's hot

Rebaca's Video Delivery Expertise Overview
Rebaca's Video Delivery Expertise OverviewRebaca's Video Delivery Expertise Overview
Rebaca's Video Delivery Expertise OverviewArshad Mahmood
 
Pexip Infinity Fusion for Skype for Business
Pexip Infinity Fusion for Skype for BusinessPexip Infinity Fusion for Skype for Business
Pexip Infinity Fusion for Skype for BusinessGraham Walsh
 
Pexip and Microsoft - Certified for Skype for Business Server
Pexip and Microsoft - Certified for Skype for Business ServerPexip and Microsoft - Certified for Skype for Business Server
Pexip and Microsoft - Certified for Skype for Business ServerGraham Walsh
 
Балансировщики KEMP для Microsoft Lync, Skype for Business
Балансировщики KEMP для Microsoft Lync, Skype for BusinessБалансировщики KEMP для Microsoft Lync, Skype for Business
Балансировщики KEMP для Microsoft Lync, Skype for BusinessAlexander Zhuravlev, Ph.D.
 
Pexip Infinity 13 adds more than 20 new features and enhancements
Pexip Infinity 13 adds more than 20 new features and enhancementsPexip Infinity 13 adds more than 20 new features and enhancements
Pexip Infinity 13 adds more than 20 new features and enhancementsAnders Løkke
 
Pexip presentation
Pexip presentationPexip presentation
Pexip presentationTed Wentzel
 
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCC
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCCMicrosoft Surface Hub - How to integrate with legacy videoconferencing and UCC
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCCGraham Walsh
 
Basic VoIP Drupal Intro - for BioRAFT
Basic VoIP Drupal Intro - for BioRAFTBasic VoIP Drupal Intro - for BioRAFT
Basic VoIP Drupal Intro - for BioRAFTMicky Metts
 
Simple hybrid voice deployments with Sonus
Simple hybrid voice deployments with SonusSimple hybrid voice deployments with Sonus
Simple hybrid voice deployments with SonusAdam Hand
 
Avaya lync integration 6 16
Avaya lync integration 6 16Avaya lync integration 6 16
Avaya lync integration 6 16Chris Stegh
 
Microsoft Lync 2010 Call Flows Explained
Microsoft Lync 2010 Call Flows ExplainedMicrosoft Lync 2010 Call Flows Explained
Microsoft Lync 2010 Call Flows ExplainedMUCUGL
 
Byt ut din gamla växel – en djupdykning i Lync telefoni
Byt ut din gamla växel – en djupdykning i Lync telefoniByt ut din gamla växel – en djupdykning i Lync telefoni
Byt ut din gamla växel – en djupdykning i Lync telefoniTommy Clarke
 
Lync 2010 Conferencing Deep Dive
Lync 2010 Conferencing Deep DiveLync 2010 Conferencing Deep Dive
Lync 2010 Conferencing Deep DiveHarold Wong
 
Ken Rehor's presentation at eComm 2008
Ken Rehor's presentation at eComm 2008Ken Rehor's presentation at eComm 2008
Ken Rehor's presentation at eComm 2008eComm2008
 
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...Louis Göhl
 
Tc 2008 11 19
Tc 2008 11 19Tc 2008 11 19
Tc 2008 11 19jeffiel
 
Integrating Microsoft Surface Hub with legacy videoconferencing and UCC
Integrating Microsoft Surface Hub with legacy videoconferencing and UCCIntegrating Microsoft Surface Hub with legacy videoconferencing and UCC
Integrating Microsoft Surface Hub with legacy videoconferencing and UCCAnders Løkke
 
Pexip and Microsoft Interoperability
Pexip and Microsoft InteroperabilityPexip and Microsoft Interoperability
Pexip and Microsoft InteroperabilityGraham Walsh
 

What's hot (20)

Rebaca's Video Delivery Expertise Overview
Rebaca's Video Delivery Expertise OverviewRebaca's Video Delivery Expertise Overview
Rebaca's Video Delivery Expertise Overview
 
Pexip Infinity Fusion for Skype for Business
Pexip Infinity Fusion for Skype for BusinessPexip Infinity Fusion for Skype for Business
Pexip Infinity Fusion for Skype for Business
 
Pexip and Microsoft - Certified for Skype for Business Server
Pexip and Microsoft - Certified for Skype for Business ServerPexip and Microsoft - Certified for Skype for Business Server
Pexip and Microsoft - Certified for Skype for Business Server
 
Балансировщики KEMP для Microsoft Lync, Skype for Business
Балансировщики KEMP для Microsoft Lync, Skype for BusinessБалансировщики KEMP для Microsoft Lync, Skype for Business
Балансировщики KEMP для Microsoft Lync, Skype for Business
 
Flash media gateway
Flash media gatewayFlash media gateway
Flash media gateway
 
Pexip Infinity 13 adds more than 20 new features and enhancements
Pexip Infinity 13 adds more than 20 new features and enhancementsPexip Infinity 13 adds more than 20 new features and enhancements
Pexip Infinity 13 adds more than 20 new features and enhancements
 
Pexip presentation
Pexip presentationPexip presentation
Pexip presentation
 
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCC
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCCMicrosoft Surface Hub - How to integrate with legacy videoconferencing and UCC
Microsoft Surface Hub - How to integrate with legacy videoconferencing and UCC
 
Pexip architecture
Pexip architecturePexip architecture
Pexip architecture
 
Basic VoIP Drupal Intro - for BioRAFT
Basic VoIP Drupal Intro - for BioRAFTBasic VoIP Drupal Intro - for BioRAFT
Basic VoIP Drupal Intro - for BioRAFT
 
Simple hybrid voice deployments with Sonus
Simple hybrid voice deployments with SonusSimple hybrid voice deployments with Sonus
Simple hybrid voice deployments with Sonus
 
Avaya lync integration 6 16
Avaya lync integration 6 16Avaya lync integration 6 16
Avaya lync integration 6 16
 
Microsoft Lync 2010 Call Flows Explained
Microsoft Lync 2010 Call Flows ExplainedMicrosoft Lync 2010 Call Flows Explained
Microsoft Lync 2010 Call Flows Explained
 
Byt ut din gamla växel – en djupdykning i Lync telefoni
Byt ut din gamla växel – en djupdykning i Lync telefoniByt ut din gamla växel – en djupdykning i Lync telefoni
Byt ut din gamla växel – en djupdykning i Lync telefoni
 
Lync 2010 Conferencing Deep Dive
Lync 2010 Conferencing Deep DiveLync 2010 Conferencing Deep Dive
Lync 2010 Conferencing Deep Dive
 
Ken Rehor's presentation at eComm 2008
Ken Rehor's presentation at eComm 2008Ken Rehor's presentation at eComm 2008
Ken Rehor's presentation at eComm 2008
 
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...
CLI319 Microsoft Desktop Optimization Pack: Planning the Deployment of Micros...
 
Tc 2008 11 19
Tc 2008 11 19Tc 2008 11 19
Tc 2008 11 19
 
Integrating Microsoft Surface Hub with legacy videoconferencing and UCC
Integrating Microsoft Surface Hub with legacy videoconferencing and UCCIntegrating Microsoft Surface Hub with legacy videoconferencing and UCC
Integrating Microsoft Surface Hub with legacy videoconferencing and UCC
 
Pexip and Microsoft Interoperability
Pexip and Microsoft InteroperabilityPexip and Microsoft Interoperability
Pexip and Microsoft Interoperability
 

Similar to TADS Developer Summit WebRTC Dan Burnett

WebRTC Overview by Dan Burnett
WebRTC Overview by Dan BurnettWebRTC Overview by Dan Burnett
WebRTC Overview by Dan BurnettMojo Lingo
 
Short introduction to WebRTC at the Amsterdam WebRTC Meetup, March 26, 2014
Short introduction to WebRTC at the Amsterdam WebRTC Meetup, March 26, 2014Short introduction to WebRTC at the Amsterdam WebRTC Meetup, March 26, 2014
Short introduction to WebRTC at the Amsterdam WebRTC Meetup, March 26, 2014Bart Uelen
 
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptxBridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptxAlberto González Trastoy
 
WebRTC overview
WebRTC overviewWebRTC overview
WebRTC overviewRouyun Pan
 
WebRTC Standards & Implementation Q&A - WebRTC Standards Feature Complete 
No...
WebRTC Standards & Implementation Q&A - WebRTC Standards Feature Complete 
No...WebRTC Standards & Implementation Q&A - WebRTC Standards Feature Complete 
No...
WebRTC Standards & Implementation Q&A - WebRTC Standards Feature Complete 
No...Amir Zmora
 
Getting Started with WebRTC
Getting Started with WebRTCGetting Started with WebRTC
Getting Started with WebRTCChad Hart
 
Workshop web rtc customers and use cases
Workshop web rtc customers and use casesWorkshop web rtc customers and use cases
Workshop web rtc customers and use casesDouglas Tait
 
What is WebRTC and How does it work?
What is WebRTC and How does it work?What is WebRTC and How does it work?
What is WebRTC and How does it work?SandipPatel533958
 
Boost JBoss AS7 with HTML5 WebRTC for Real Time Communications
Boost JBoss AS7 with HTML5 WebRTC for Real Time CommunicationsBoost JBoss AS7 with HTML5 WebRTC for Real Time Communications
Boost JBoss AS7 with HTML5 WebRTC for Real Time Communicationstelestax
 

Similar to TADS Developer Summit WebRTC Dan Burnett (20)

WebRTC Overview by Dan Burnett
WebRTC Overview by Dan BurnettWebRTC Overview by Dan Burnett
WebRTC Overview by Dan Burnett
 
DevCon 5 (December 2013) - WebRTC & WebSockets
DevCon 5 (December 2013) - WebRTC & WebSocketsDevCon 5 (December 2013) - WebRTC & WebSockets
DevCon 5 (December 2013) - WebRTC & WebSockets
 
WebRTC
WebRTCWebRTC
WebRTC
 
Short introduction to WebRTC at the Amsterdam WebRTC Meetup, March 26, 2014
Short introduction to WebRTC at the Amsterdam WebRTC Meetup, March 26, 2014Short introduction to WebRTC at the Amsterdam WebRTC Meetup, March 26, 2014
Short introduction to WebRTC at the Amsterdam WebRTC Meetup, March 26, 2014
 
WebRTC Seminar Report
WebRTC Seminar ReportWebRTC Seminar Report
WebRTC Seminar Report
 
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptxBridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
 
WebRTC overview
WebRTC overviewWebRTC overview
WebRTC overview
 
Asterisk World (January 2014) - Taking Enterprise Telephony into the Web World
Asterisk World (January 2014) - Taking Enterprise Telephony into the Web WorldAsterisk World (January 2014) - Taking Enterprise Telephony into the Web World
Asterisk World (January 2014) - Taking Enterprise Telephony into the Web World
 
Html5 RTC - 1
Html5 RTC - 1Html5 RTC - 1
Html5 RTC - 1
 
WebRTC Standards & Implementation Q&A - WebRTC Standards Feature Complete 
No...
WebRTC Standards & Implementation Q&A - WebRTC Standards Feature Complete 
No...WebRTC Standards & Implementation Q&A - WebRTC Standards Feature Complete 
No...
WebRTC Standards & Implementation Q&A - WebRTC Standards Feature Complete 
No...
 
Webrtc and tokbox
Webrtc and tokboxWebrtc and tokbox
Webrtc and tokbox
 
Getting Started with WebRTC
Getting Started with WebRTCGetting Started with WebRTC
Getting Started with WebRTC
 
WebRTC presentation
WebRTC presentationWebRTC presentation
WebRTC presentation
 
WebRTC standards update (Jul 2014)
WebRTC standards update (Jul 2014)WebRTC standards update (Jul 2014)
WebRTC standards update (Jul 2014)
 
Web rtc 入門
Web rtc 入門Web rtc 入門
Web rtc 入門
 
WebRTC on Mobile
WebRTC on MobileWebRTC on Mobile
WebRTC on Mobile
 
Workshop web rtc customers and use cases
Workshop web rtc customers and use casesWorkshop web rtc customers and use cases
Workshop web rtc customers and use cases
 
What is WebRTC and How does it work?
What is WebRTC and How does it work?What is WebRTC and How does it work?
What is WebRTC and How does it work?
 
Boost JBoss AS7 with HTML5 WebRTC for Real Time Communications
Boost JBoss AS7 with HTML5 WebRTC for Real Time CommunicationsBoost JBoss AS7 with HTML5 WebRTC for Real Time Communications
Boost JBoss AS7 with HTML5 WebRTC for Real Time Communications
 
DevCon5 (July 2014) - Intro to WebRTC
DevCon5 (July 2014) - Intro to WebRTCDevCon5 (July 2014) - Intro to WebRTC
DevCon5 (July 2014) - Intro to WebRTC
 

More from Alan Quayle

Supercharging CPaaS Growth & Margins with Identity and Authentication, Aditya...
Supercharging CPaaS Growth & Margins with Identity and Authentication, Aditya...Supercharging CPaaS Growth & Margins with Identity and Authentication, Aditya...
Supercharging CPaaS Growth & Margins with Identity and Authentication, Aditya...Alan Quayle
 
Building a sub-second virtual ThunderDome: Considerations for mass scale sub-...
Building a sub-second virtual ThunderDome: Considerations for mass scale sub-...Building a sub-second virtual ThunderDome: Considerations for mass scale sub-...
Building a sub-second virtual ThunderDome: Considerations for mass scale sub-...Alan Quayle
 
What makes a cellular IoT API great? Tobias Goebel
What makes a cellular IoT API great? Tobias GoebelWhat makes a cellular IoT API great? Tobias Goebel
What makes a cellular IoT API great? Tobias GoebelAlan Quayle
 
eSIM as Root of Trust for IoT security, João Casal
eSIM as Root of Trust for IoT security, João CasaleSIM as Root of Trust for IoT security, João Casal
eSIM as Root of Trust for IoT security, João CasalAlan Quayle
 
Architecting your WebRTC application for scalability, Arin Sime
Architecting your WebRTC application for scalability, Arin SimeArchitecting your WebRTC application for scalability, Arin Sime
Architecting your WebRTC application for scalability, Arin SimeAlan Quayle
 
CPaaS Conversational Platforms and Conversational Customer Service – The Expe...
CPaaS Conversational Platforms and Conversational Customer Service – The Expe...CPaaS Conversational Platforms and Conversational Customer Service – The Expe...
CPaaS Conversational Platforms and Conversational Customer Service – The Expe...Alan Quayle
 
Programmable Testing for Programmable Telcos, Andreas Granig
Programmable Testing for Programmable Telcos, Andreas GranigProgrammable Testing for Programmable Telcos, Andreas Granig
Programmable Testing for Programmable Telcos, Andreas GranigAlan Quayle
 
How to best maximize the conversation data stream for your business? Surbhi R...
How to best maximize the conversation data stream for your business? Surbhi R...How to best maximize the conversation data stream for your business? Surbhi R...
How to best maximize the conversation data stream for your business? Surbhi R...Alan Quayle
 
Latest Updates and Experiences in Launching Local Language Tools, Karel Bourgois
Latest Updates and Experiences in Launching Local Language Tools, Karel BourgoisLatest Updates and Experiences in Launching Local Language Tools, Karel Bourgois
Latest Updates and Experiences in Launching Local Language Tools, Karel BourgoisAlan Quayle
 
What Everyone Needs to Know about Protecting the CPaaS Ecosystem from Unlawfu...
What Everyone Needs to Know about Protecting the CPaaS Ecosystem from Unlawfu...What Everyone Needs to Know about Protecting the CPaaS Ecosystem from Unlawfu...
What Everyone Needs to Know about Protecting the CPaaS Ecosystem from Unlawfu...Alan Quayle
 
Master the Audience Experience Multiverse: AX Best Practices and Success Stor...
Master the Audience Experience Multiverse: AX Best Practices and Success Stor...Master the Audience Experience Multiverse: AX Best Practices and Success Stor...
Master the Audience Experience Multiverse: AX Best Practices and Success Stor...Alan Quayle
 
Open Source Telecom Software Survey 2022, Alan Quayle
Open Source Telecom Software Survey 2022, Alan QuayleOpen Source Telecom Software Survey 2022, Alan Quayle
Open Source Telecom Software Survey 2022, Alan QuayleAlan Quayle
 
OpenSIPS 3.3 – Messaging in the IMS and UC ecosystems. Bogdan-Andrei Iancu
OpenSIPS 3.3 – Messaging in the IMS and UC ecosystems. Bogdan-Andrei IancuOpenSIPS 3.3 – Messaging in the IMS and UC ecosystems. Bogdan-Andrei Iancu
OpenSIPS 3.3 – Messaging in the IMS and UC ecosystems. Bogdan-Andrei IancuAlan Quayle
 
TADS 2022 - Shifting from Voice to Workflow Management, Filipe Leitao
TADS 2022 - Shifting from Voice to Workflow Management, Filipe LeitaoTADS 2022 - Shifting from Voice to Workflow Management, Filipe Leitao
TADS 2022 - Shifting from Voice to Workflow Management, Filipe LeitaoAlan Quayle
 
What happened since we last met TADSummit 2022, Alan Quayle
What happened since we last met TADSummit 2022, Alan QuayleWhat happened since we last met TADSummit 2022, Alan Quayle
What happened since we last met TADSummit 2022, Alan QuayleAlan Quayle
 
Stacuity - TAD Summit 2022 - Time to ditch the dumb-pipe, Mike Bromwich
Stacuity - TAD Summit 2022 - Time to ditch the dumb-pipe, Mike BromwichStacuity - TAD Summit 2022 - Time to ditch the dumb-pipe, Mike Bromwich
Stacuity - TAD Summit 2022 - Time to ditch the dumb-pipe, Mike BromwichAlan Quayle
 
AWA – a Telco bootstrapping product development: Challenges with dynamic mark...
AWA – a Telco bootstrapping product development: Challenges with dynamic mark...AWA – a Telco bootstrapping product development: Challenges with dynamic mark...
AWA – a Telco bootstrapping product development: Challenges with dynamic mark...Alan Quayle
 
Founding a Startup in Telecoms. The good, the bad and the ugly. João Camarate
Founding a Startup in Telecoms. The good, the bad and the ugly. João CamarateFounding a Startup in Telecoms. The good, the bad and the ugly. João Camarate
Founding a Startup in Telecoms. The good, the bad and the ugly. João CamarateAlan Quayle
 
How to bring down your own RTC platform. Sandro Gauci
How to bring down your own RTC platform. Sandro GauciHow to bring down your own RTC platform. Sandro Gauci
How to bring down your own RTC platform. Sandro GauciAlan Quayle
 

More from Alan Quayle (20)

What is a vCon?
What is a vCon?What is a vCon?
What is a vCon?
 
Supercharging CPaaS Growth & Margins with Identity and Authentication, Aditya...
Supercharging CPaaS Growth & Margins with Identity and Authentication, Aditya...Supercharging CPaaS Growth & Margins with Identity and Authentication, Aditya...
Supercharging CPaaS Growth & Margins with Identity and Authentication, Aditya...
 
Building a sub-second virtual ThunderDome: Considerations for mass scale sub-...
Building a sub-second virtual ThunderDome: Considerations for mass scale sub-...Building a sub-second virtual ThunderDome: Considerations for mass scale sub-...
Building a sub-second virtual ThunderDome: Considerations for mass scale sub-...
 
What makes a cellular IoT API great? Tobias Goebel
What makes a cellular IoT API great? Tobias GoebelWhat makes a cellular IoT API great? Tobias Goebel
What makes a cellular IoT API great? Tobias Goebel
 
eSIM as Root of Trust for IoT security, João Casal
eSIM as Root of Trust for IoT security, João CasaleSIM as Root of Trust for IoT security, João Casal
eSIM as Root of Trust for IoT security, João Casal
 
Architecting your WebRTC application for scalability, Arin Sime
Architecting your WebRTC application for scalability, Arin SimeArchitecting your WebRTC application for scalability, Arin Sime
Architecting your WebRTC application for scalability, Arin Sime
 
CPaaS Conversational Platforms and Conversational Customer Service – The Expe...
CPaaS Conversational Platforms and Conversational Customer Service – The Expe...CPaaS Conversational Platforms and Conversational Customer Service – The Expe...
CPaaS Conversational Platforms and Conversational Customer Service – The Expe...
 
Programmable Testing for Programmable Telcos, Andreas Granig
Programmable Testing for Programmable Telcos, Andreas GranigProgrammable Testing for Programmable Telcos, Andreas Granig
Programmable Testing for Programmable Telcos, Andreas Granig
 
How to best maximize the conversation data stream for your business? Surbhi R...
How to best maximize the conversation data stream for your business? Surbhi R...How to best maximize the conversation data stream for your business? Surbhi R...
How to best maximize the conversation data stream for your business? Surbhi R...
 
Latest Updates and Experiences in Launching Local Language Tools, Karel Bourgois
Latest Updates and Experiences in Launching Local Language Tools, Karel BourgoisLatest Updates and Experiences in Launching Local Language Tools, Karel Bourgois
Latest Updates and Experiences in Launching Local Language Tools, Karel Bourgois
 
What Everyone Needs to Know about Protecting the CPaaS Ecosystem from Unlawfu...
What Everyone Needs to Know about Protecting the CPaaS Ecosystem from Unlawfu...What Everyone Needs to Know about Protecting the CPaaS Ecosystem from Unlawfu...
What Everyone Needs to Know about Protecting the CPaaS Ecosystem from Unlawfu...
 
Master the Audience Experience Multiverse: AX Best Practices and Success Stor...
Master the Audience Experience Multiverse: AX Best Practices and Success Stor...Master the Audience Experience Multiverse: AX Best Practices and Success Stor...
Master the Audience Experience Multiverse: AX Best Practices and Success Stor...
 
Open Source Telecom Software Survey 2022, Alan Quayle
Open Source Telecom Software Survey 2022, Alan QuayleOpen Source Telecom Software Survey 2022, Alan Quayle
Open Source Telecom Software Survey 2022, Alan Quayle
 
OpenSIPS 3.3 – Messaging in the IMS and UC ecosystems. Bogdan-Andrei Iancu
OpenSIPS 3.3 – Messaging in the IMS and UC ecosystems. Bogdan-Andrei IancuOpenSIPS 3.3 – Messaging in the IMS and UC ecosystems. Bogdan-Andrei Iancu
OpenSIPS 3.3 – Messaging in the IMS and UC ecosystems. Bogdan-Andrei Iancu
 
TADS 2022 - Shifting from Voice to Workflow Management, Filipe Leitao
TADS 2022 - Shifting from Voice to Workflow Management, Filipe LeitaoTADS 2022 - Shifting from Voice to Workflow Management, Filipe Leitao
TADS 2022 - Shifting from Voice to Workflow Management, Filipe Leitao
 
What happened since we last met TADSummit 2022, Alan Quayle
What happened since we last met TADSummit 2022, Alan QuayleWhat happened since we last met TADSummit 2022, Alan Quayle
What happened since we last met TADSummit 2022, Alan Quayle
 
Stacuity - TAD Summit 2022 - Time to ditch the dumb-pipe, Mike Bromwich
Stacuity - TAD Summit 2022 - Time to ditch the dumb-pipe, Mike BromwichStacuity - TAD Summit 2022 - Time to ditch the dumb-pipe, Mike Bromwich
Stacuity - TAD Summit 2022 - Time to ditch the dumb-pipe, Mike Bromwich
 
AWA – a Telco bootstrapping product development: Challenges with dynamic mark...
AWA – a Telco bootstrapping product development: Challenges with dynamic mark...AWA – a Telco bootstrapping product development: Challenges with dynamic mark...
AWA – a Telco bootstrapping product development: Challenges with dynamic mark...
 
Founding a Startup in Telecoms. The good, the bad and the ugly. João Camarate
Founding a Startup in Telecoms. The good, the bad and the ugly. João CamarateFounding a Startup in Telecoms. The good, the bad and the ugly. João Camarate
Founding a Startup in Telecoms. The good, the bad and the ugly. João Camarate
 
How to bring down your own RTC platform. Sandro Gauci
How to bring down your own RTC platform. Sandro GauciHow to bring down your own RTC platform. Sandro Gauci
How to bring down your own RTC platform. Sandro Gauci
 

Recently uploaded

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilV3cube
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 

Recently uploaded (20)

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 

TADS Developer Summit WebRTC Dan Burnett

  • 1. WebRTC Introduc)on  to  WebRTC   Dan  Burne4   Chief  Scien)st,  Tropo   Director  of  Standards,  Voxeo     Alan  Johnston   Dis)nguished  Engineer   Avaya    
  • 2. WebRTC  Tutorial  Topics   •  •  •  •  •  •  •  What  is  WebRTC?   How  to  Use  WebRTC   WebRTC  Peer-­‐to-­‐Peer  Media   WebRTC  Protocols  and  IETF  Standards   WebRTC  W3C  API  Overview   Pseudo  Code  Walkthrough   PracFcal  bits   TAD  Summit  Bangkok  2013   2  
  • 4. WebRTC  is  “Voice  &  Video  in  the  browser”   •  Access  to  camera  and  microphone  without  a   plugin   –  No  proprietary  plugin  required!     •  Audio/video  direct  from  browser  to  browser   •  Why  does  it  maWer?   –  Media  can  stay  local   –  Mobile  devices  eventually  dropping  voice  channel   anyway   –  Games   TAD  Summit  Bangkok  2013   4  
  • 5. The  Browser  RTC  FuncFon   Web   Server   Signaling   Server   HTTP  or  WebSockets     JavaScript/HTML/CSS   Other  APIs   Web   Browser   •  WebRTC  adds  new  Real-­‐ Time  CommunicaFon  (RTC)   FuncFon  built-­‐in  to   browsers   –  No  download   HTTP  or  WebSockets  –  No  Flash  or  other  plugins     (Signaling)   •  Contains   –  Audio  and  video  codecs   –  Ability  to  negoFate  peer-­‐to-­‐ peer  connecFons   On-­‐the-­‐wire  protocols   –  Echo  cancellaFon,  packet  loss   (Media  or  Data)   concealement   RTC  APIs   Browser   RTC   FuncFon   NaFve  OS  Services   •  In  Chrome  &  Firefox  today,   Internet  Explorer  someFme   and  Safari  eventually   TAD  Summit  Bangkok  2013   5  
  • 6. Benefits  of  WebRTC   For  Developer     For  User   •  Streamlined  development  –   one  placorm   •  Simple  APIs  –  detailed   knowledge  of  RTC  protocols   not  needed   •  NAT  traversal  only  uses   expensive  relays  when  no   other  choice   •  Advanced  voice  and  video   codecs  without  licensing     •  No  download  or  install  –   easy  to  use   •  All  communicaton   encrypted  –  private   •  Reliable  session   establishment     –  “just  works”   •  Excellent  voice  and  video   quality     •  Many  more  choices  for  real-­‐ Fme  communicaFon   TAD  Summit  Bangkok  2013   6  
  • 7. WebRTC  Support  of  MulFple  Media   Microphone  Audio   ApplicaFon  Sharing  Video   Front  Camera  Video   Rear  Camera  Video   WebCam  Video   Stereo  Audio   Browser  L         on  Laptop   Browser  M   on  Mobile   •  MulFple  sources  of  audio  and  video  are  assumed   and  supported   •  All  media,  voice  and  video,  and  feedback  messages   are  mulFplexed  over  the  same  transport  address   TAD  Summit  Bangkok  2013   7  
  • 8. WebRTC  Triangle   Web  Server   (ApplicaFon)   Peer  ConnecFon  (Audio,  Video,  and/or  Data)   Browser  L   Browser  M   (Running  HTML5  ApplicaFon     from  Web  Server)   (Running  HTML5  ApplicaFon     from  Web  Server)   •  Both  browsers  running  the  same  web  applicaFon  from  web   server   •  Peer  ConnecFon  established  between  them  with  the  help  of   the  web  server   TAD  Summit  Bangkok  2013   8  
  • 9. WebRTC  Trapezoid   Web  Server  A   (ApplicaFon  A)   Browser  M   SIP     or  Jingle   Web  Server  B   (ApplicaFon  B)   Peer  ConnecFon  (Audio  and/or  Video)   Browser  T   (Running  HTML5  ApplicaFon     from  Web  Server  B)   (Running  HTML5  ApplicaFon     from  Web  Server  A)     •  Similar  to  SIP  Trapezoid     •  Web  Servers  communicate  using  SIP  or  Jingle  or  proprietary   •  Could  become  important  in  the  future.   TAD  Summit  Bangkok  2013   9  
  • 10. WebRTC  and  SIP   Web  Server     SIP   SIP  Server   SIP   Browser  M   Peer  ConnecFon  (Audio  and/or  Video)   SIP  Client     •  SIP  (Session  IniFaFon  Protocol)  is  a  signaling  protocol  used  by  service   providers  and  enterprises  for  real-­‐Fme  communcaFon   •  Peer  ConnecFon  appears  as  a  standard  RTP  session,  described  by  SDP   •  SIP  Endpoint  must  support  RTCWEB  media  extensions       TAD  Summit  Bangkok  2013   10  
  • 11. WebRTC  and  Jingle   Web  Server   Jingle   XMPP  Server   Jingle   Peer  ConnecFon  (Audio  and/or  Video)   Browser  M   Jingle  Client   •  Jingle  is  a  signaling  extension  to  XMPP  (Extensible  Messaging  and   Presence  Protocol,  aka  Jabber)   •  Peer  ConnecFon  SDP  can  be  mapped  to  Jingle   •  Jingle  Endpoint  must  support  RTCWEB  Media  extensions   TAD  Summit  Bangkok  2013   11  
  • 12. WebRTC  and  PSTN   Web  Server   Peer  ConnecFon  (Audio)   PSTN  Gateway   Browser  M   Phone   •  Peer  ConnecFon  terminates  on  a  PSTN  Gateway   •  Audio  Only   •  EncrypFon  ends  at  Gateway   TAD  Summit  Bangkok  2013   12  
  • 13. WebRTC  with  SIP   Web  Server     SIP  Proxy/Registrar  Server     WebSocket  (SIP)   HTTP     (HTML5/CSS/ JavaScript)   Browser  M   (running  JavaScript  SIP  UA)   HTTP     WebSocket   (HTML5/CSS/ (SIP)   JavaScript)   SRTP  Media   Browser  T   (running  JavaScript  SIP  UA)     •  Browser  runs  a  SIP  User  Agent  by  running  JavaScript  from  Web  Server     •  SRTP  media  connecFon  uses  WebRTC  APIs   •  Details  in  [dram-­‐iec-­‐sipcore-­‐websocket]  that  defines  SIP  transport  over   TAD  Summit  Bangkok  2013   13   WebSockets  
  • 14. WebRTC  Signaling  Approaches   •  Signaling  is  required  for  exchange  of  candidate  transport   addresses,  codec  informaFon,  media  keying  informaFon   •  Many  opFons  –  choice  is  up  to  web  developer   TAD  Summit  Bangkok  2013   14  
  • 15. How  to  Use  WebRTC  
  • 16. WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   All  media  added   Set  Up  Peer   ConnecFon   Peer  ConnecFon  established   AWach  Media   or  Data   AWach  more  media  or  data   Ready  for  call   Exchange   Offer/Answer   2013   TAD  Summit  Bangkok   16  
  • 17. WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   •  getUserMedia() –  Audio  and/or  video   –  Constraints   –  User  permissions   All  media  added   Set  Up  Peer   ConnecFon   •  Browser  must  ask  before   allowing  a  page  to  access   microphone  or  camera   Peer  ConnecFon  established   AWach  Media   or  Data   AWach  more  media  or  data   •  MediaStream •  MediaStreamTrack –  CapabiliFes   –  States  (sepngs)   Ready  for  call   Exchange   Offer/Answer   TAD  Summit  Bangkok  2013   17  
  • 18. WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   •  RTCPeerConnection –  All  media  added   –  –  Set  Up  Peer   –  ConnecFon   –  Peer  ConnecFon  established   –  AWach  Media   AWach  more  media  or  data   –  or  Data   –  Ready  for  call   Direct  media   Between  two  peers   ICE  processing   SDP  processing   DTMF  support   Data  channels   IdenFty  verificaFon   StaFsFcs  reporFng   Exchange   Offer/Answer   TAD  Summit  Bangkok  2013   18  
  • 19. WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   •  addStream() –  Doesn't  change  media  state!   •  removeStream() All  media  added   –  DiWo!   Set  Up  Peer   ConnecFon   •  createDataChannel() Peer  ConnecFon  established   AWach  Media   or  Data   –  Depends  on  transport   AWach  more  media  or  data   Ready  for  call   Exchange   Offer/Answer   TAD  Summit  Bangkok  2013   19  
  • 20. WebRTC  usage  in  brief   Obtain  Local   Media   Get  more  media   All  media  added   Set  Up  Peer   ConnecFon   Peer  ConnecFon  established   AWach  Media   or  Data   •  createOffer(), createAnswer() •  setLocalDescription(), setRemoteDescription() •  Applying  SDP  answer  makes   the  magic  happen AWach  more  media  or  data   Ready  for  call   Exchange   Session   DescripFons   TAD  Summit  Bangkok  2013   20  
  • 21. WebRTC  usage  –  a  bit  more  detail   Set  Up  Signaling   Channel   Obtain  Local   Media   Get  more  media   Set  Up  Peer   ConnecFon   AWach  Media  or   Data   Exchange   Session   TAD  Summit  Bangkok  2013   DescripFons   AWach  more  media  or  data   21  
  • 22. SDP  offer/answer   •  Session  DescripFons   –  Session  DescripFon  Protocol  created  for  use  by   SIP  in  sepng  up  voice  (and  video)  calls   –  Describes  real-­‐Fme  media  at  low  level  of  detail   •  Which  IP  addresses  and  ports  to  use   •  Which  codecs  to  use   •  Offer/answer  model  (JSEP)   –  One  side  sends  an  SDP  offer  lisFng  what  it  wants   to  send  and  what  it  can  receive   –  Other  side  replies  with  an  SDP  answer  lisFng  what   it  will  receive  and  send   TAD  Summit  Bangkok  2013   22  
  • 24. Media  Flows  in  WebRTC   Web  Server       Internet   Home  WiFi   Router   Router           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  Router       Browser  L     TAD  Summit  Bangkok  2013   24  
  • 25. Media  without  WebRTC   Web  Server       Internet   Home  WiFi   Router   Router           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  Router       Browser  L     TAD  Summit  Bangkok  2013   25  
  • 26. Peer-­‐to-­‐Peer  Media  with  WebRTC   Web  Server       Internet   Home  WiFi   Router   Router           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  Router       Browser  L     TAD  Summit  Bangkok  2013   26  
  • 27. NAT  Complicates  Peer-­‐to-­‐Peer  Media   Web  Server   Most  browsers  are  behind  NATs   on  the  Internet,  which   complicates  the  establishment   of  peer-­‐to-­‐peer  media  sessions.         Internet   Router  with   NAT   Home  WiFi     with  NAT           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   27  
  • 28. What  is  a  NAT?   •  Network  Address  Translator  (NAT)   •  Used  to  map  an  inside  address  (usually  a   private  IP  address)  to  outside  address   (usually  a  public  IP  address)  at  Layer  3   •  Network  Address  and  Port  TranslaFon   (NAPT)  also  changes  the  transport  port   number  (Layer  4)   – These  are  omen  just  called  NATs  as  well   •  One  reason  for  NAT  is  the  IP  address   shortage   TAD  Summit  Bangkok  2013   28  
  • 29. NAT  Example   Internet   “Outside”      Public  IP  Address       203.0.113.4 “Inside”      Private  IP  Addresses       192.168.x.x Home  WiFi     with  NAT     Browser  M   192.168.0.5     Browser  T   192.168.0.6       TAD  Summit  Bangkok  2013   29  
  • 30. NATs  and  ApplicaFons   •  NATs  are  compaFble  with  client/server  protocols   such  as  web,  email,  etc.   •  However,  NATs  generally  block  peer-­‐to-­‐peer   communicaFon   •  Typical  NAT  traversal  for  VoIP  and  video   services  today  use  a  media  relay  whenever  the   client  is  behind  a  NAT   –  Omen  done  with  an  SBC  –  Session  Border   Controller   –  This  is  a  major  expense  and  complicaFon  in   exisFng  VoIP  and  video  systems   •  WebRTC  has  a  built-­‐in  NAT  traversal  strategy:   InteracFve  ConnecFvity  Establishment  (ICE)   TAD  Summit  Bangkok  2013   30  
  • 31. Peer-­‐to-­‐Peer  Media  Through  NAT   Web  Server   ICE  connecFvity  checks  can   omen  establish  a  direct  peer-­‐ to-­‐peer  session  between   browsers  behind  different   NATs         Internet   Router  with   NAT   Home  WiFi     with  NAT           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   31  
  • 32. ICE  ConnecFvity  Checks   •  ConnecFvity  through  NAT  can  be  achieved  using  ICE   connecFvity  checks   •  Browsers  exchange  a  list  of  candidates   –  Local:  read  from  network  interfaces     –  Reflexive:  obtained  using  a  STUN  Server   –  Relayed:  obtained  from  a  TURN  Server  (media  relay)   •  Browsers  aWempt  to  send  STUN  packets  to  the   candidate  list  received  from  other  browser   •  Checks  performed  by  both  sides  at  same  Fme   •  If  one  STUN  packet  gets  through,  a  response  is  sent   and  this  connecFon  used  for  communicaFon   –  TURN  relay  will  be  last  resort  (lowest  priority)   TAD  Summit  Bangkok  2013   32  
  • 33. P2P  Media  Can  Stay  Local  to  NAT   If  both  browsers  are   behind  the  same  NAT,   connecFvity  checks  can   omen  establish  a   connecFon  that  never   leaves  the  NAT.     Web  Server       Internet   Router  with   NAT   Home  WiFi     with  NAT           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   33  
  • 34. ICE  Servers   Web  Server       STUN  Server   TURN  Server   198.51.100.9   198.51.100.2       ICE  uses  STUN  and  TURN   servers  in  the  public   Internet  to  help  with  NAT   traversal.     Internet   Home  WiFi     with  NAT   203.0.113.4 Router  with   NAT         Browser  M   192.168.0.5         Browser  D     Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   34  
  • 35. Browser  Queries  STUN  Server     Web  Server       STUN  Server   TURN  Server   198.51.100.9   198.51.100.2       Browser  sends  STUN  test   packet  to  STUN  server  to   learn  its  public  IP  address   (address  of  the  NAT).     Internet   Home  WiFi     with  NAT   203.0.113.4 Router  with   NAT         Browser  M   192.168.0.5         Browser  D     Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   35  
  • 36. TURN  Server  Can  Relay  Media   Web  Server       STUN   TURN  Server  as  a   Server   Media  Relay       In  some  cases,  connecFvity   checks  fail,  and  a  TURN   Media  Relay  on  the  public   Internet  must  be  used.       Internet   Router  with   NAT   Home  WiFi     with  NAT           Browser  M     Browser  D   Browser  T     Coffee  Shop   WiFi  with   NAT       Browser  L     TAD  Summit  Bangkok  2013   36  
  • 37. WebRTC  Protocols  and  IETF   Standards  
  • 38. WebRTC:  A  Joint  Standards  Effort   •  Internet  Engineering  Task  Force  (IETF)  and  World   Wide  Web  ConsorFum  (W3C)  are  working  together  on   WebRTC   •  IETF   –  Protocols  –  “bits  on  wire”   –  Main  protocols  are  already  RFCs,  but  many  extensions  in   progress   –  RTCWEB  (Real-­‐Time  CommunicaFons  on  the  Web)  Working   Group  is  the  main  focus,  but  other  WGs  involved  as  well   –  hWp://www.iec.org     •  W3C   –  APIs  –  used  by  JavaScript  code  in  HTML5   –  hWp://www.w3c.org   TAD  Summit  Bangkok  2013   38  
  • 39. WebRTC  Protocols   ApplicaFon  Layer   HTTP   ICE   WebSocket   SRTP   SDP   STUN   TURN   Transport  Layer   TLS   TCP   Network  Layer   DTLS   UDP   SCTP   IP   SIP  is  not  shown  as  it  is  opFonal   TAD  Summit  Bangkok  2013   39  
  • 40. IETF  RTCWEB  Documents   Document) Ref) Overview' “Overview:'Real'Time'Protocols'for' Browser6based'Applications”' draft6ietf6rtcweb6 overview' Use'Cases'and'Requirements' “Web'Real6Time'Communication' Use6cases'and'Requirements”' draft6ietf6rtcweb6 use6cases6and6 requirements' RTP'Usage' “Web'Real6Time'Communication' (WebRTC):'Media'Transport'and' Use'of'RTP”' draft6ietf6rtcweb6 rtp6usage' Security'Architecture' “RTCWEB'Security'Architecture”' draft6ietf6rtcweb6 security6arch' Threat'Model' “Security'Considerations'for'RTC6 Web”' draft6ietf6rtcweb6 security' Data'Channel' “RTCWeb'Data'Channels”' draft6ietf6rtcweb6 data6channel' JSEP' “JavaScript'Session'Establishment' Protocol”' draft6ietf6rtcweb6 jsep' Audio' “WebRTC'Audio'Codec'and' Processing'Requirements”' draft6ietf6rtcweb6 audio' Quality'of'Service' ' Title) “DSCP'and'other'packet'markings' for'RTCWeb'QoS”' draft6ietf6rtcweb6 qos' TAD  Summit  Bangkok  2013   40  
  • 41. Codecs                      RFC  6716                  .                 •  Mandatory  to  Implement  (MTI)  audio  codecs  are   seWled  on  Opus  and  G.711  (finally!)   •  Video  is  not  yet  decided!   TAD  Summit  Bangkok  2013   41  
  • 42. WebRTC  W3C  API  Overview  
  • 43. Two  primary  API  secFons   •  Handling  local  media   –  Media  Capture  and  Streams  (getUserMedia)   specificaFon   •  Transmipng  media   –  WebRTC  (Peer  ConnecFon)  specificaFon   TAD  Summit  Bangkok  2013   43  
  • 44. Local  Media  Handling   Audio   PresentaFon   Video   “Audio”  Track   Presenter  Stream   “PresentaFon”    Track   “Audio”  Track   Microphone  Audio   Presenter  Video   ApplicaFon  Sharing  Video   Front  Camera  Video   Rear  Camera  Video   “Presenter”     Track   DemonstraFon  Stream   “Audio”  Track   DemonstraFon   Video   Browser  M   Sources   PresentaFon  Stream   Captured   MediaStreams   •  In  this  example   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  Captured  4  local  media  streams   –  Created  3  media  streams  from  them   –  Sent  streams  over  Peer  ConnecFon   TAD  Summit  Bangkok  2013   44  
  • 45. Local  Media  Handling   Audio   PresentaFon   Video   Microphone  Audio   ApplicaFon  Sharing  Video   Browser  M   Sources   •  Sources   “Audio”  Track   Presenter  Stream   “PresentaFon”    Track   “Audio”  Track   Presenter  Video   Front  Camera  Video   Rear  Camera  Video   PresentaFon  Stream   “Presenter”     Track   DemonstraFon  Stream   “Audio”  Track   DemonstraFon   Video   Captured   MediaStreams   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  Encoded  together   –  Can't  manipulate  individually   TAD  Summit  Bangkok  2013   45  
  • 46. Local  Media  Handling   Audio   PresentaFon   Video   Microphone  Audio   ApplicaFon  Sharing  Video   Browser  M   “Audio”  Track   Presenter  Stream   “PresentaFon”    Track   “Audio”  Track   Presenter  Video   Front  Camera  Video   Rear  Camera  Video   PresentaFon  Stream   “Presenter”     Track   DemonstraFon  Stream   “Audio”  Track   DemonstraFon   Video   Sources   Captured   MediaStreams   •  Tracks  (MediaStreamTrack)   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  Tied  to  a  source   –  Exist  primarily  as  part  of  Streams;  single  media  type   –  Globally  unique  ids;  opFonally  browser-­‐labeled   TAD  Summit  Bangkok  2013   46  
  • 47. Local  Media  Handling   PresentaFon  Stream   Audio   “Audio”  Track   PresentaFon   Video   Microphone  Audio   ApplicaFon  Sharing  Video   Presenter  Stream   “Audio”  Track   Presenter  Video   “Presenter”     Track   DemonstraFon  Stream   Front  Camera  Video   Rear  Camera  Video   Browser  M   Sources   “PresentaFon”    Track   “Audio”  Track   DemonstraFon   Video   Captured   MediaStreams   •  Captured  MediaStream   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  Returned  from  getUserMedia() –  Permission  check  required  to  obtain   TAD  Summit  Bangkok  2013   47  
  • 48. Local  Media  Handling   Audio   PresentaFon   Video   Microphone  Audio   ApplicaFon  Sharing  Video   Browser  M   Sources   “Audio”  Track   Presenter  Stream   “PresentaFon”    Track   “Audio”  Track   Presenter  Video   Front  Camera  Video   Rear  Camera  Video   PresentaFon  Stream   “Presenter”     Track   DemonstraFon  Stream   “Audio”  Track   DemonstraFon   Video   Captured   MediaStreams   •  MediaStream   “DemonstraFon”    Track   Created   MediaStreams   Tracks   –  All  contained  tracks  are  synchronized   –  Can  be  created,  transmiWed,  etc.   TAD  Summit  Bangkok  2013   48  
  • 49. Local  Media  Handling   •  Sepngs   –  Current  values  of  source  properFes  (height,  width,   etc.)   –  Exposed  on  MediaStreamTrack •  CapabiliFes   –  Allowed  values  for  source  properFes   –  Exposed  on  MediaStreamTrack •  Constraints   –  Requested  ranges  for  track  properFes   –  Used  in  getUserMedia(),  applyConstraints() TAD  Summit  Bangkok  2013   49  
  • 50. Transmipng  media   •  Signaling  channel   –  Non-­‐standard   –  Must  exist  to  set  up  Peer  ConnecFon   •  Peer  ConnecFon   –  Links  together  two  peers   –  Add/Remove  Media  Streams   •  addStream(),  removeStream() –  Handlers  for  ICE  or  media  change   –  Data  Channel  support   TAD  Summit  Bangkok  2013   50  
  • 51. Peer  ConnecFon   •  "Links"  together  two  peers   –  Via  new RTCPeerConnection() –  Generates  Session  DescripFon  offers/answers   •  createOffer(),  createAnswer() –  From  SDP  answers,  iniFates  media   •  setLocalDescription(),  setRemoteDescription() –  Offers/answers  MUST  be  relayed  by  applicaFon   code!   –  ICE  candidates  can  also  be  relayed  and  added  by  app   •  addIceCandidate()   TAD  Summit  Bangkok  2013   51  
  • 52. Peer  ConnecFon   •  Handlers  for  signaling,  ICE  or  media  change   –  onsignalingstatechange –  onicecandidate,   oniceconnectionstatechange –  onaddstream,  onremovestream –  onnegotiationneeded –  A  few  others   TAD  Summit  Bangkok  2013   52  
  • 53. Peer  ConnecFon   •  “Extra”  APIs   –  Data   –  DTMF   –  StaFsFcs   –  IdenFty   •  Grouped  separately  in  WebRTC  spec   –  but  really  part  of  RTCPeerConnection   definiFon   –  all  are  mandatory  to  implement   TAD  Summit  Bangkok  2013   53  
  • 54. Data  Channel  API   •  RTCDataChannel createDataChannel() •  Configurable  with   –  –  –  –  ordered maxRetransmits,  maxRetransmitTime negotiated id •  Provides  RTCDataChannel  with   –  send() –  onopen,  onerror,  onclose,  onmessage*   TAD  Summit  Bangkok  2013   54  
  • 55. DTMF  API   •  RTCDTMFSender createDTMFSender() –  Associates  track  input  parameter  with  this   RTCPeerConnection •  RTCDTMFSender  provides   –  boolean canInsertDTMF() –  insertDTMF() –  ontonechange –  (other  stuff)   TAD  Summit  Bangkok  2013   55  
  • 56. StaFsFcs  API   •  getStats() –  Callback  returns  staFsFcs  for  given  track   •  StaFsFcs  available  (local/remote)  are:   –  Bytes/packets  xmiWed   –  Bytes/packets  received   •  May  be  useful  for  congesFon-­‐based   adjustments   TAD  Summit  Bangkok  2013   56  
  • 57. IdenFty  API   •  setIdentityProvider(),   getIdentityAssertion() •  Used  to  verify  idenFty  via  third  party,  e.g.,   Facebook  Connect   •  Both  methods  are  opFonal   •  onidentity  handler  called  amer  any   verificaFon  aWempt   •  RTCPeerConnection.peerIdentity  holds   any  verified  idenFty  asserFon   TAD  Summit  Bangkok  2013   57  
  • 59. Pseudo  Code   •  Close  to  real  code,  but  .  .  .   •  No  HTML,  no  signaling  channel,  not   asynchronous,  and  API  is  sFll  in  flux   •  Don't  expect  this  to  work  anywhere   TAD  Summit  Bangkok  2013   59  
  • 60. Back  to  first  diagram   Microphone  Audio   ApplicaFon  Sharing  Video   Front  Camera  Video   Rear  Camera  Video   WebCam  Video   Stereo  Audio   Browser  L         on  Laptop   Browser  M   on  Mobile   •  Mobile  browser  "calls"  laptop  browser   •  Each  sends  media  to  the  other   TAD  Summit  Bangkok  2013   60  
  • 61. Mobile  browser  code  outline   var signalingChannel = createSignalingChannel(); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( •  We  will  look  next  at  each  of  these   •  .  .  .  except  for  creaFng  the  signaling   channel   [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   61  
  • 62. Mobile  browser  produces  .  .  .   Audio   PresentaFon  Stream   “Audio”  Track   “PresentaFon”  Track   PresentaFon  Video   Microphone  Audio   ApplicaFon  Sharing  Video   “Audio”  Track   Presenter  Video   “Presenter”  Track   DemonstraFon  Stream   Front  Camera  Video   Rear  Camera  Video   Presenter  Stream   “Audio”  Track   DemonstraFon  Video   “DemonstraFon”  Track   Browser  M   Sources   Captured  MediaStreams   Created  MediaStreams   •  At  least  3  calls  to  getUserMedia() •  Three  calls  to  new MediaStream() •  App  sends  stream  ids,  then  streams   TAD  Summit  Bangkok  2013   Tracks   62  
  • 63. funcFon  getMedia()  [1]   navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } . . . function attachMedia() { presentation = new MediaStream( •  Get  audio   •  (Get  window  video  –  out  of  scope)   [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   63  
  • 64. funcFon  getMedia()  [2]   . . . constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); •  Get  front-­‐facing  camera   •  Get  rear-­‐facing  camera   constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   64  
  • 65. funcFon  createPC()   var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; pc = new RTCPeerConnection(configuration); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; •  Create  RTCPeerConnection •  Set  handlers   function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   65  
  • 66. Mobile  browser  consumes  .  .  .   Audio  &  Video  Stream   Display   “Video”  Track   Right  Headphone   “Right”  Track   Lem  Headphone   Browser  M   “Lem”  Track   (Audio  &  Video  Stream  selected)   Stereo  Stream   “Right”  Track   “Lem”  Track   “Mono”  track   Mono  Stream   Sinks   MediaStreams   •  Receives  three  media  streams   •  Chooses  one •  Sends  tracks  to  output  channels   TAD  Summit  Bangkok  2013   Tracks   66  
  • 67. FuncFon  handleIncomingStream()   if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); •  If  incoming  stream  has  video  track,  set  to   av_stream  and  display  it   •  If  it  has  two  audio  tracks,  must  be  stereo   •  Otherwise,  must  be  the  mono  stream   pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   67  
  • 68. FuncFon  show_av(st)   display.srcObject = new MediaStream(st.getVideoTracks()[0]); left.srcObject = new MediaStream(st.getAudioTracks()[0]); right.srcObject = new MediaStream(st.getAudioTracks()[1]); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); •  Turn  streams  into  URLs   •  Set  as  source  for  media  elements   } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   68  
  • 69. Mobile  browser  code  outline   var signalingChannel = createSignalingChannel(); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( •  We  will  look  next  at  each  of  these   •  .  .  .  except  for  creaFng  the  signaling   channel   [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   69  
  • 70. funcFon  aWachMedia()  [1]   presentation = new MediaStream( [microphone.getAudioTracks()[0], application.getVideoTracks()[0]]); presenter = new MediaStream( [microphone.getAudioTracks()[0], front.getVideoTracks()[0]]); demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); . . . // Audio // Presentation "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // Audio // Presenter // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( // Audio // Demonstration •  Create  3  new  streams,  all  with  same   audio  but  different  video   TAD  Summit  Bangkok  2013   var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; 70  
  • 71. funcFon  aWachMedia()  [2]   pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); •  AWach  all  3  streams  to  Peer  ConnecFon   •  Send  stream  ids  to  peer  (before  streams!)   } function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   71  
  • 72. Mobile  browser  code  outline   var signalingChannel = createSignalingChannel(); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( •  We  will  look  next  at  each  of  these   •  .  .  .  except  for  creaFng  the  signaling   channel   [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   72  
  • 73. funcFon  call()   pc.createOffer(gotDescription, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function gotDescription(desc) { pc.setLocalDescription(desc, s, e); function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); } pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter •  Ask  browser  to  create  SDP  offer   •  Set  offer  as  local  descripFon   •  Send  offer  to  peer   demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; TAD  Summit  Bangkok  2013   73  
  • 74. How  do  we  get  the  SDP  answer?   signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var microphone, application, front, rear; var presentation, presenter, demonstration; var remote_av, stereo, mono; var display, left, right; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); getMedia(); createPC(); attachMedia(); call(); function getMedia() { // get local audio (microphone) navigator.getUserMedia({"audio": true }, function (stream) { microphone = stream; }, e); // get local video (application sharing) ///// This is outside the scope of this specification. ///// Assume that 'application' has been set to this stream. // constraint = {"video": {"mandatory": {"videoFacingModeEnum": "front"}}}; navigator.getUserMedia(constraint, function (stream) { front = stream; }, e); constraint = {"video": {"mandatory": {"videoFacingModeEnum": "rear"}}}; navigator.getUserMedia(constraint, function (stream) { rear = stream; }, e); } function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function attachMedia() { presentation = new MediaStream( [microphone.getAudioTracks()[0], // Audio application.getVideoTracks()[0]]); // Presentation presenter = new MediaStream( [microphone.getAudioTracks()[0], // Audio front.getVideoTracks()[0]]); // Presenter }; demonstration = new MediaStream( [microphone.getAudioTracks()[0], rear.getVideoTracks()[0]]); // Audio // Demonstration pc.addStream(presentation); pc.addStream(presenter); pc.addStream(demonstration); •  Signaling  channel  provides  message   •  If  SDP,  set  as  remote  descripFon   •  If  ICE  candidate,  tell  the  browser   TAD  Summit  Bangkok  2013   } signalingChannel.send( JSON.stringify({ "presentation": presentation.id, "presenter": presenter.id, "demonstration": demonstration.id })); function call() { pc.createOffer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.getVideoTracks().length == 1) { av_stream = st; show_av(av_stream); } else if (st.getAudioTracks().length == 2) { stereo = st; } else { mono = st; } } function show_av(st) { display.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); left.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); right.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[1])); } signalingChannel.onmessage = function (msg) { var signal = JSON.parse(msg.data); if (signal.sdp) { pc.setRemoteDescription( new RTCSessionDescription(signal.sdp), s, e); } else { pc.addIceCandidate( new RTCIceCandidate(signal.candidate)); } }; 74  
  • 75. And  now  the  laptop  browser  .  .  .   •  Watch  for  the  following   –  We  set  up  media  *amer*  receiving  the  offer   –  but  the  signaling  channel  sFll  must  exist  first!   –  Also,  need  to  save  incoming  stream  ids   TAD  Summit  Bangkok  2013   75  
  • 76. Signaling  channel  message  is  trigger   signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; . . . pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; }; // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { •  Set  up  PC  and  media  if  not  already  done   if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; TAD  Summit  Bangkok  2013   76  
  • 77. Signaling  channel  message  is  trigger   signalingChannel.onmessage = function (msg) { . . . if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); }; •  If  SDP,  *also*  answer   •  But  if  neither  SDP  nor  ICE  candidate,  must   be  set  of  incoming  stream  ids,  so  save   TAD  Summit  Bangkok  2013   signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; 77  
  • 78. FuncFon  prepareForIncomingCall()   createPC(); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; getMedia(); var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); attachMedia(); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; •  No  suprises  here   •  Media  obtained  is  a  liWle  different   •  But  aWached  the  same  way   TAD  Summit  Bangkok  2013   // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; 78  
  • 79. FuncFon  answer()   pc.createAnswer(gotDescription, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", function gotDescription(desc) { pc.setLocalDescription(desc, s, e); "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = signalingChannel.send(JSON.stringify({ "sdp": desc })); function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); } constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; •  createAnswer()  automaFcally  uses   value  of  remoteDescription  when   generaFng  new  SDP   // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; TAD  Summit  Bangkok  2013   79  
  • 80. Laptop  browser  consumes  .  .  .   PresentaFon  Stream   “Audio”  Track   “PresentaFon”  Track   Presenter  Stream     “Audio”  Track   Speaker   “Presenter”  Track   Display   DemonstraFon  Stream   Display   “Audio”  Track   Display   “DemonstraFon”  Track   Browser  L   (All  video  streams  selected)   Tracks   MediaStreams   Sinks   •  Three  input  streams •  All  have  same  #  of  audio  and  video  tracks •  Need  stream  ids  to  disFnguish   TAD  Summit  Bangkok  2013   80  
  • 81. FuncFon  handleIncomingStream()   if (st.id === incoming.presentation) { speaker.srcObject = new MediaStream(st.getAudioTracks()[0]); win1.srcObject = new MediaStream(st.getVideoTracks()[0]); } else if (st.id === incoming.presenter) { win2.srcObject = new MediaStream(st.getVideoTracks()[0]); } else { win3.srcObject = new MediaStream(st.getVideoTracks()[0]); } •  Use  ids  to  disFnguish  streams   •  Extract  one  audio  and  all  video  tracks   •  Assign  to  element  sources   TAD  Summit  Bangkok  2013   var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; 81  
  • 82. Laptop  browser  produces  .  .  .   Audio  &  Video  Stream   Video   “Video”  Track   WebCam   Lem  Microphone   “Right”  Track   “Lem”  Track   Lem  Audio   Right  Microphone   Stereo  Stream   Browser  L   “Right”  Track   Right  Audio   “Lem”  Track   “Mono”  Track   Mono  Stream   Tracks   Created  MediaStreams   Captured   MediaStreams   Sources   •  Three  calls  to  getUserMedia() •  Three  calls  to  new MediaStream() •  No  stream  ids  needed   TAD  Summit  Bangkok  2013   82  
  • 83. FuncFon  getMedia()  [1]   navigator.getUserMedia({"video": true}, function (stream) { webcam = stream; }, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); . . . pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; •  Request  webcam  video   // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; TAD  Summit  Bangkok  2013   83  
  • 84. FuncFon  getMedia()  [2]   . . . constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) {left = stream;}, e); var pc; var configuration = {"iceServers":[{"url":"stun:198.51.100.9"}, {"url":"turn:198.51.100.2", "credential":"myPassword"}]}; var webcam, left, right; var av, stereo, mono; var incoming; var speaker, win1, win2, win3; function s(sdp) {} // stub success callback function e(error) {} // stub error callback var signalingChannel = createSignalingChannel(); function prepareForIncomingCall() { createPC(); getMedia(); } attachMedia(); function createPC() { pc = new RTCPeerConnection(configuration); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) {right = stream;}, e); pc.onicecandidate = function (evt) { signalingChannel.send( JSON.stringify({ "candidate": evt.candidate })); }; pc.onaddstream = function (evt) {handleIncomingStream(evt.stream);}; } function getMedia() { navigator.getUserMedia({"video": true }, function (stream) { webcam = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "left"}}}; navigator.getUserMedia(constraint, function (stream) { left = stream; }, e); constraint = {"audio": {"mandatory": {"audioDirectionEnum": "right"}}}; navigator.getUserMedia(constraint, function (stream) { right = stream; }, e); } function attachMedia() { av = new MediaStream( [webcam.getVideoTracks()[0], left.getAudioTracks()[0], right.getAudioTracks()[0]]); stereo = new MediaStream( [left.getAudioTracks()[0], right.getAudioTracks()[0]]); mono = left; // Video // Left audio // Right audio // Left audio // Right audio // Treat the left audio as the mono stream pc.addStream(av); pc.addStream(stereo); pc.addStream(mono); } function answer() { pc.createAnswer(gotDescription, e); •  Request  lem  and  right  audio  streams   •  Save  them  as  left  and  right  variables   function gotDescription(desc) { pc.setLocalDescription(desc, s, e); signalingChannel.send(JSON.stringify({ "sdp": desc })); } } function handleIncomingStream(st) { if (st.id === incoming.presentation) { speaker.src = URL.createObjectURL( new MediaStream(st.getAudioTracks()[0])); win1.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else if (st.id === incoming.presenter) { win2.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } else { win3.src = URL.createObjectURL( new MediaStream(st.getVideoTracks()[0])); } } signalingChannel.onmessage = function (msg) { if (!pc) { prepareForIncomingCall(); } var sgnl = JSON.parse(msg.data); if (sgnl.sdp) { pc.setRemoteDescription( new RTCSessionDescription(sgnl.sdp), s, e); answer(); } else if (sgnl.candidate) { pc.addIceCandidate(new RTCIceCandidate(sgnl.candidate)); } else { incoming = sgnl; } }; TAD  Summit  Bangkok  2013   84