How OpenWebRTC was used in Ericsson’s and Volvo CE’s Remote Control over Mobile Networks prototype – demoed at MWC 2015. Also a quick review of OpenWebRTC and a few words on the addition of Data Channel support. Presentation by Ericsson Research.
1. OpenWebRTC – Not just
for Video Chat
Kristofer Dovstam
Master Researcher
Multimedia Applications and Protocols, Ericsson Research
2. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 2
› Mobile first WebRTC client framework
› Permissive Open Source BSD 2-
clause license
› Independent, interoperable
implementation
› Being integrated into WebKit
› Native applications
› GitHub
What is
OpenWebRTC?
3. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 3
› What is it?
– Not just audio and video, but data too
– Can control reliability and ordering properties depending on needs
– Encrypted real-time data
› OpenWebRTC now with Data Channel support!
– Using the reference SCTP implementation - libusrsctp
– Contributed by Collabora Ltd.
– Has been tested in real use cases…
WebRTC Data Channel
4. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 4
› Remote Control over Mobile Networks
– Remote control an excavator from a long
way away
– Use WebRTC technology
– Optimize solution for high-performance and
low-latency
› Demo at MWC 2014
– ‘Toy’ excavator
– Earlier internal version of OpenWebRTC
– iPhone
– Oculus Rift DK1
IDEA and History
5. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 5
…2015
6. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 6
Excavator
› Research collaboration
with Volvo CE
› Not a product!
7. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 7
Excavator
› Excavator
– Excavators from Volvo CE
– Modified to handle electrical control
signals
– Powerful Linux PC
– Python application with custom
signaling built on top of
OpenWebRTC
8. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 8
Audio / Video
Capture
› 2048x2048 24fps
camera
› 360 degree lens
› 52.5 degrees up, 62.5
degrees down
› Avoids stitching artifacts
› Microphone array of four
microphones
9. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 9
Control Rig
› Simulator from Oryx and
Volvo CE
› Mac OS X computer
› OS X Cocoa application
with custom signaling
built on top of
OpenWebRTC
10. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 10
› Oculus Rift DK2 (1920x1080 75Hz –
960x1080 for each eye)
– Still much too low resolution!
› Doughnut to panorama texture
mapped on a sphere
› Local rendering minimizes motion-to-
light latency of head movements
› Four channel audio rendered as four
independent sound sources in a 3D
space
Audio / Video
Rendering
11. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 11
Controls
› Simulator rig with analogue controls
› Computer measures analogue signals
and converts to digital measurements
› Control signals sent via WebRTC Data
Channel to excavator
› Excavator computer converts digital
signals into analogue electrical signals
› It’s alive!
12. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 12
Media Processing
Architecture
Camera
Microphone Array
Excavator
OpenWebRTC
Audio / Video
Data Channel
OpenWebRTC
Audio / Video
Data Channel
Oculus Rift DK2
Headphones
Control Rig
Network
Mobile, fixed, …
Network
Mobile, fixed, …
Audio
Video
Data
Audio
Video
Data
Audio
Video
Data
Audio
Video
Data
13. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 13
› Data Channel
– 1 stream
– Unordered
– Unreliable (full control signal
state in every message so
only the newest message is
relevant)
– libusrsctp
› Video
– 2048 x 2048
– ~24fps
– H.264
– OpenH264
– 10Mbps
› Audio
– Single 4-channel stream
– Signed 16-bit
– 48kHz
– Opus
– libopus
– 256 kbps total bitrate
Media Stream Specifications
14. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 14
Mobile world congress - 2015
15. ER Day Internal template | Ericsson Internal | 2014-11-24 | Page 15
Notes de l'éditeur
1
It is a mobile first WebRTC client framework. We have put quite a lot of effort to try to make it work well on mobile platforms
It’s what we call permissive open source under the BSD 2-clause license
It is an independent, interoperable implementation of WebRTC
It’s currently being integrated into WebKit
We’ve also had native applications in mind when developing OpenWebRTC – especially for mobile platforms
WebKitGTK+
2015-03-04 – data channel support was contributed to OpenWebRTC
With the data channel we can send and receive basically any kind of data, not just audio and video
You can control reliability and ordering properties depending on needs (basically SCTP functionality)
It gives us the possibility to use encrypted real-time data
The OpenWebRTC implementation of the data channel is based on the reference SCTP implementation
It was contributed by Collabora
And, it has actually already been tested in real use cases… (the Excavator prototype)
Reliability - retransmission
A little over a year ago we came up with the idea of remote controlling over mobile networks, to show case the new kind of applications that the next generation mobile networks can make real
This year, we took this project to a whole different level…
We used real machines…
This was made possible together with Volvo CE
In the excavator
One reason to use a 360 lens is to avoid stitching artifacts
On the controlling side…
The audio is surprisingly important in order to control the excavator well…
The remote control over mobile network proof point at the Mobile World Congress 2015 with the Excavator prototype was a huge success with a constant flow of customers lining up to test the system!