Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Quality Assurance for WebRTC Services

1 217 vues

Publié le

This session, given at Kranky Geek India, offers suggestions on how to go about building test automation that is suitable for WebRTC services.

Publié dans : Technologie
  • Login to see the comments

Quality Assurance for WebRTC Services

  1. 1. Quality Assurance for WebRTC Services March 2016, Kranky Geek India Tsahi Levent-Levi
  2. 2. People don’t usually test their WebRTC implementations Seriously 2
  3. 3. And when they do, they test things manually 3
  4. 4. Testing and automation are mandatory and widely used Meanwhile, on other software fields… 4
  5. 5. How do you go about automating your WebRTC testbed? 7 suggestions for you 5 W. Heath Robinson
  6. 6. • WebRTC changes… monthly • You have more than a voice call to test Rely on the browser 6 40 41 42 43 44 45 46 47 48 49 35 36 37 38 39 40 41 42 43 44 45 Dec-14 Apr-15 Jul-15 Oct-15 Jan-16 May-16 Chrome Firefox Selenium is your friend 1
  7. 7. Rely on the browser… and peer into the future 7 1 Stable What your customers are using Beta What you should expect in the next release Dev Where you can complain about breakage – and expect fixes in time Canary A nightly build of whatever is available at that moment in time Testing only against the Stable version of browsers means you find out issues only after the service breaks for your customers.
  8. 8. > ffmpeg -i MYFILE.mp4 -pix_fmt yuv420p > sed -i '0,/C420mpeg2/s//C420/' *.y4m > google-chrome --use-fake-device-for-media-stream --use-file-for-fake-video-capture=MYFILE.y4m See http://testrtc.com/y4m-video-chrome/ for more details Handle your media feeds 8 and think about the type of content you need 2
  9. 9. For some arcane reason (called AGC or something), browsers will loop your fake video but won’t look your fake audio Loop an input 6 times (=expand 5 minutes to 30) > ffmpeg -f concat -i <(for i in {1..6}; do echo "file '$PWD/INPUT.wav'"; done) –c copy OUTPUT.wav > google-chrome --use-fake-device-for-media-stream --use-file-for-fake-audio-capture=OUTPUT.wav 9 2 Handle your media feeds and make sure your audio recording is suitably long
  10. 10. Take care of the network 10 Use netfilter.org iptables Network Impairments • Bitrate limitation • Packet loss • Jitter & latency Firewall Configuration • None • Force TURN relay • HTTP only • HTTPS only Multiple Locations • Virtualization sucks • Stability & Throughput is what you need • Explore barebone options 3
  11. 11. • Caller / Callee • Caller / Agent • Caller / Expert • Meeting point • Conference • Webinar Handle synchronization 11 4
  12. 12. Can you see the problem? Visualize the results 12 5
  13. 13. Visualize the results 13 Chrome 48 with VP8 Chrome 49 with VP9 5
  14. 14. Are these real network issues with packet losses or bugs in these products and how they report packet loss? Visualize the results (2) 14 5
  15. 15. • How much time does it take to connect the session? • Are the media channels you need really there? • Are you sure there aren’t any “additional” media channels? • Is the bitrate following your restrictions? Don’t forget to set your expectations 15 6
  16. 16. You never know when things will become handy… Here are a few thoughts: • chrome://webrtc-internals dump • Browser console logs • Media recordings • Machine performance Collect EVERYTHING 16 7
  17. 17. Don’t take WebRTC testing lightly 17
  18. 18. Thank You Tsahi Levent-Levi http://bloggeek.me tsahil@bloggeek.me 18

×