Submit Search
Upload
Camera Architecture from Failure to Success
•
2 likes
•
775 views
Ryo Jin
Follow
Camera Architecture from Failure to Success - Symphony Teleca
Read less
Read more
Technology
Art & Photos
Report
Share
Report
Share
1 of 21
Download now
Download to read offline
Recommended
Utf 8'en'ibm sametime 9 - voice and video deployment
Utf 8'en'ibm sametime 9 - voice and video deployment
a8us
OpenEye DVR Administer Training
OpenEye DVR Administer Training
openeyevideo
OpenEye Client Software Training
OpenEye Client Software Training
openeyevideo
OpenEye Digital Video Recorder Overview
OpenEye Digital Video Recorder Overview
openeyevideo
OpenEye IP Video Basics
OpenEye IP Video Basics
openeyevideo
Live Streaming from A-Z
Live Streaming from A-Z
Brightcove
XS Japan 2008 BitVisor English
XS Japan 2008 BitVisor English
The Linux Foundation
OpenEye Optix Network Cameras
OpenEye Optix Network Cameras
openeyevideo
Recommended
Utf 8'en'ibm sametime 9 - voice and video deployment
Utf 8'en'ibm sametime 9 - voice and video deployment
a8us
OpenEye DVR Administer Training
OpenEye DVR Administer Training
openeyevideo
OpenEye Client Software Training
OpenEye Client Software Training
openeyevideo
OpenEye Digital Video Recorder Overview
OpenEye Digital Video Recorder Overview
openeyevideo
OpenEye IP Video Basics
OpenEye IP Video Basics
openeyevideo
Live Streaming from A-Z
Live Streaming from A-Z
Brightcove
XS Japan 2008 BitVisor English
XS Japan 2008 BitVisor English
The Linux Foundation
OpenEye Optix Network Cameras
OpenEye Optix Network Cameras
openeyevideo
XS Boston 2008 OVF
XS Boston 2008 OVF
The Linux Foundation
MIPI DevCon Bangalore 2017: Emulation of DUT Using MIPI RMMI (M-PHY Module In...
MIPI DevCon Bangalore 2017: Emulation of DUT Using MIPI RMMI (M-PHY Module In...
MIPI Alliance
Unified communications with IBM
Unified communications with IBM
Vincent Perrin
Keynote Speech: Xen ARM Virtualization
Keynote Speech: Xen ARM Virtualization
The Linux Foundation
Video Conferencing over BGAN
Video Conferencing over BGAN
Videoguy
Sametime 8.5 Audio Video
Sametime 8.5 Audio Video
Vincent Perrin
Last videoppt
Last videoppt
Manni Singh
Sony PXW-X500 Camcorder
Sony PXW-X500 Camcorder
AV ProfShop
Streaming Video into Second Life
Streaming Video into Second Life
Videoguy
BP502 IBM Connect 2014
BP502 IBM Connect 2014
Peter Lurie
Movi presentation Singapore video tech meetup
Movi presentation Singapore video tech meetup
Lars-Erik M Ravn
Linux Foundation Collaboration Summit 13 :10 years of Xen and Beyond
Linux Foundation Collaboration Summit 13 :10 years of Xen and Beyond
The Linux Foundation
Vimantra Cloud Video Platform
Vimantra Cloud Video Platform
Thomson Reuters
XS Boston 2008 Malware & Training
XS Boston 2008 Malware & Training
The Linux Foundation
The Grid Collateral Galicki
The Grid Collateral Galicki
Bee Gee
JMP206 : Calling Home: Enabling the IBM Sametime Softphone in ST9
JMP206 : Calling Home: Enabling the IBM Sametime Softphone in ST9
Keith Brooks
Sony PMW-F3K
Sony PMW-F3K
AV ProfShop
Virtual STB / Cloud UI Streaming revisited
Virtual STB / Cloud UI Streaming revisited
Dr. Randolph Nikutta
WebRTC for Mobile
WebRTC for Mobile
Shubh Agarwal
IVI Essential Requirements:BSP and Hardware.
IVI Essential Requirements:BSP and Hardware.
Yuichi Kusakabe
Mips track a
Mips track a
Alona Gradman
Video Conferencing Update
Video Conferencing Update
Videoguy
More Related Content
What's hot
XS Boston 2008 OVF
XS Boston 2008 OVF
The Linux Foundation
MIPI DevCon Bangalore 2017: Emulation of DUT Using MIPI RMMI (M-PHY Module In...
MIPI DevCon Bangalore 2017: Emulation of DUT Using MIPI RMMI (M-PHY Module In...
MIPI Alliance
Unified communications with IBM
Unified communications with IBM
Vincent Perrin
Keynote Speech: Xen ARM Virtualization
Keynote Speech: Xen ARM Virtualization
The Linux Foundation
Video Conferencing over BGAN
Video Conferencing over BGAN
Videoguy
Sametime 8.5 Audio Video
Sametime 8.5 Audio Video
Vincent Perrin
Last videoppt
Last videoppt
Manni Singh
Sony PXW-X500 Camcorder
Sony PXW-X500 Camcorder
AV ProfShop
Streaming Video into Second Life
Streaming Video into Second Life
Videoguy
BP502 IBM Connect 2014
BP502 IBM Connect 2014
Peter Lurie
Movi presentation Singapore video tech meetup
Movi presentation Singapore video tech meetup
Lars-Erik M Ravn
Linux Foundation Collaboration Summit 13 :10 years of Xen and Beyond
Linux Foundation Collaboration Summit 13 :10 years of Xen and Beyond
The Linux Foundation
Vimantra Cloud Video Platform
Vimantra Cloud Video Platform
Thomson Reuters
XS Boston 2008 Malware & Training
XS Boston 2008 Malware & Training
The Linux Foundation
The Grid Collateral Galicki
The Grid Collateral Galicki
Bee Gee
JMP206 : Calling Home: Enabling the IBM Sametime Softphone in ST9
JMP206 : Calling Home: Enabling the IBM Sametime Softphone in ST9
Keith Brooks
Sony PMW-F3K
Sony PMW-F3K
AV ProfShop
What's hot
(17)
XS Boston 2008 OVF
XS Boston 2008 OVF
MIPI DevCon Bangalore 2017: Emulation of DUT Using MIPI RMMI (M-PHY Module In...
MIPI DevCon Bangalore 2017: Emulation of DUT Using MIPI RMMI (M-PHY Module In...
Unified communications with IBM
Unified communications with IBM
Keynote Speech: Xen ARM Virtualization
Keynote Speech: Xen ARM Virtualization
Video Conferencing over BGAN
Video Conferencing over BGAN
Sametime 8.5 Audio Video
Sametime 8.5 Audio Video
Last videoppt
Last videoppt
Sony PXW-X500 Camcorder
Sony PXW-X500 Camcorder
Streaming Video into Second Life
Streaming Video into Second Life
BP502 IBM Connect 2014
BP502 IBM Connect 2014
Movi presentation Singapore video tech meetup
Movi presentation Singapore video tech meetup
Linux Foundation Collaboration Summit 13 :10 years of Xen and Beyond
Linux Foundation Collaboration Summit 13 :10 years of Xen and Beyond
Vimantra Cloud Video Platform
Vimantra Cloud Video Platform
XS Boston 2008 Malware & Training
XS Boston 2008 Malware & Training
The Grid Collateral Galicki
The Grid Collateral Galicki
JMP206 : Calling Home: Enabling the IBM Sametime Softphone in ST9
JMP206 : Calling Home: Enabling the IBM Sametime Softphone in ST9
Sony PMW-F3K
Sony PMW-F3K
Similar to Camera Architecture from Failure to Success
Virtual STB / Cloud UI Streaming revisited
Virtual STB / Cloud UI Streaming revisited
Dr. Randolph Nikutta
WebRTC for Mobile
WebRTC for Mobile
Shubh Agarwal
IVI Essential Requirements:BSP and Hardware.
IVI Essential Requirements:BSP and Hardware.
Yuichi Kusakabe
Mips track a
Mips track a
Alona Gradman
Video Conferencing Update
Video Conferencing Update
Videoguy
Ip Cam
Ip Cam
Robert Bosch
ELCE 2010 - State Of Multimedia In 2010 Embedded Linux Devices
ELCE 2010 - State Of Multimedia In 2010 Embedded Linux Devices
Benjamin Zores
option-2_all IP cameras_Wavestore budget and Design.pptx
option-2_all IP cameras_Wavestore budget and Design.pptx
PawachMetharattanara
hcm381-481_oi.pdf
hcm381-481_oi.pdf
PawachMetharattanara
Монетизация сетевой инфраструктуры
Монетизация сетевой инфраструктуры
BAKOTECH
Video Server
Video Server
nnmaurya
Bitmovin LIVE Tech Talks: Data Driven Video Workflows
Bitmovin LIVE Tech Talks: Data Driven Video Workflows
Bitmovin Inc
BRKEVT-2311_joeking_pbr.pptx
BRKEVT-2311_joeking_pbr.pptx
Joel W. King
Zencoder Guide to Closed Captions
Zencoder Guide to Closed Captions
Zencoder
MIKE BY DHI SAAS - New features as of October 2014
MIKE BY DHI SAAS - New features as of October 2014
Stephen Flood
Trends in Visualization for Armoured Vehicles
Trends in Visualization for Armoured Vehicles
jder
Considerations for Creating Streamed Video Content over 3G ...
Considerations for Creating Streamed Video Content over 3G ...
Videoguy
We4IT LCTY 2013 - Infra-Man - Deep Dive into IBM Sametime - Advanced Video Te...
We4IT LCTY 2013 - Infra-Man - Deep Dive into IBM Sametime - Advanced Video Te...
We4IT Group
My Profile
My Profile
Shibasis Ganguly
Ipcam 090727060856-phpapp02
Ipcam 090727060856-phpapp02
azad12915
Similar to Camera Architecture from Failure to Success
(20)
Virtual STB / Cloud UI Streaming revisited
Virtual STB / Cloud UI Streaming revisited
WebRTC for Mobile
WebRTC for Mobile
IVI Essential Requirements:BSP and Hardware.
IVI Essential Requirements:BSP and Hardware.
Mips track a
Mips track a
Video Conferencing Update
Video Conferencing Update
Ip Cam
Ip Cam
ELCE 2010 - State Of Multimedia In 2010 Embedded Linux Devices
ELCE 2010 - State Of Multimedia In 2010 Embedded Linux Devices
option-2_all IP cameras_Wavestore budget and Design.pptx
option-2_all IP cameras_Wavestore budget and Design.pptx
hcm381-481_oi.pdf
hcm381-481_oi.pdf
Монетизация сетевой инфраструктуры
Монетизация сетевой инфраструктуры
Video Server
Video Server
Bitmovin LIVE Tech Talks: Data Driven Video Workflows
Bitmovin LIVE Tech Talks: Data Driven Video Workflows
BRKEVT-2311_joeking_pbr.pptx
BRKEVT-2311_joeking_pbr.pptx
Zencoder Guide to Closed Captions
Zencoder Guide to Closed Captions
MIKE BY DHI SAAS - New features as of October 2014
MIKE BY DHI SAAS - New features as of October 2014
Trends in Visualization for Armoured Vehicles
Trends in Visualization for Armoured Vehicles
Considerations for Creating Streamed Video Content over 3G ...
Considerations for Creating Streamed Video Content over 3G ...
We4IT LCTY 2013 - Infra-Man - Deep Dive into IBM Sametime - Advanced Video Te...
We4IT LCTY 2013 - Infra-Man - Deep Dive into IBM Sametime - Advanced Video Te...
My Profile
My Profile
Ipcam 090727060856-phpapp02
Ipcam 090727060856-phpapp02
More from Ryo Jin
Why is EFL used on Tizen?
Why is EFL used on Tizen?
Ryo Jin
Samsung Z4 User Manual
Samsung Z4 User Manual
Ryo Jin
Samsung ARTIK 050 (ARTIK ZERO) Modules Data Sheet
Samsung ARTIK 050 (ARTIK ZERO) Modules Data Sheet
Ryo Jin
Introduction to Watch Face Development with Tizen Studio
Introduction to Watch Face Development with Tizen Studio
Ryo Jin
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
Ryo Jin
Panduan Penggunaan Perangkat Wearable Tizen
Panduan Penggunaan Perangkat Wearable Tizen
Ryo Jin
Cara Menggunakan Smartphone Tizen
Cara Menggunakan Smartphone Tizen
Ryo Jin
Gear Fit2 Watchface Design Guide
Gear Fit2 Watchface Design Guide
Ryo Jin
Samsung Indonesia: Tizen Store
Samsung Indonesia: Tizen Store
Ryo Jin
Samsung Indonesia: Tizen Wearables
Samsung Indonesia: Tizen Wearables
Ryo Jin
Samsung Indonesia: Tizen Web Apps
Samsung Indonesia: Tizen Web Apps
Ryo Jin
Samsung Indonesia: Tizen Native App
Samsung Indonesia: Tizen Native App
Ryo Jin
Samsung Indonesia: Tizen Platform Overview and IoT
Samsung Indonesia: Tizen Platform Overview and IoT
Ryo Jin
Russian Tizen Project
Russian Tizen Project
Ryo Jin
Samsung SM-R360 Tizen User Manual
Samsung SM-R360 Tizen User Manual
Ryo Jin
Tizen Micro Profile for IoT device
Tizen Micro Profile for IoT device
Ryo Jin
Panduan Dasar Pemrograman Tizen
Panduan Dasar Pemrograman Tizen
Ryo Jin
The Story of Enlightenment, EFL, Tizen and Wayland
The Story of Enlightenment, EFL, Tizen and Wayland
Ryo Jin
Tizen PASS
Tizen PASS
Ryo Jin
Tizen PASS
Tizen PASS
Ryo Jin
More from Ryo Jin
(20)
Why is EFL used on Tizen?
Why is EFL used on Tizen?
Samsung Z4 User Manual
Samsung Z4 User Manual
Samsung ARTIK 050 (ARTIK ZERO) Modules Data Sheet
Samsung ARTIK 050 (ARTIK ZERO) Modules Data Sheet
Introduction to Watch Face Development with Tizen Studio
Introduction to Watch Face Development with Tizen Studio
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
Tizen 3.0's Window System Integration Layer of OpenGLES/EGL & Vulkan Driver
Panduan Penggunaan Perangkat Wearable Tizen
Panduan Penggunaan Perangkat Wearable Tizen
Cara Menggunakan Smartphone Tizen
Cara Menggunakan Smartphone Tizen
Gear Fit2 Watchface Design Guide
Gear Fit2 Watchface Design Guide
Samsung Indonesia: Tizen Store
Samsung Indonesia: Tizen Store
Samsung Indonesia: Tizen Wearables
Samsung Indonesia: Tizen Wearables
Samsung Indonesia: Tizen Web Apps
Samsung Indonesia: Tizen Web Apps
Samsung Indonesia: Tizen Native App
Samsung Indonesia: Tizen Native App
Samsung Indonesia: Tizen Platform Overview and IoT
Samsung Indonesia: Tizen Platform Overview and IoT
Russian Tizen Project
Russian Tizen Project
Samsung SM-R360 Tizen User Manual
Samsung SM-R360 Tizen User Manual
Tizen Micro Profile for IoT device
Tizen Micro Profile for IoT device
Panduan Dasar Pemrograman Tizen
Panduan Dasar Pemrograman Tizen
The Story of Enlightenment, EFL, Tizen and Wayland
The Story of Enlightenment, EFL, Tizen and Wayland
Tizen PASS
Tizen PASS
Tizen PASS
Tizen PASS
Recently uploaded
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Anna Loughnan Colquhoun
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
Martijn de Jong
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Igalia
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Delhi Call girls
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
wesley chun
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
naman860154
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
Principled Technologies
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Delhi Call girls
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Maria Levchenko
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
The Digital Insurer
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
V3cube
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Puma Security, LLC
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Rafal Los
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
debabhi2
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
Recently uploaded
(20)
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
Camera Architecture from Failure to Success
1.
Camera Architecture –
Options for Success
2.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 2 Tizen-IVI offers the whole world of HTML5/CSS/JS solutions from a UI point of view, however many of the features require developers to step out of the WRT for native support. The Camera related use case is a good example where JS alone may not be enough to build whole SW stack. Considering available technologies, and the specifics of the use cases, there might be several ways to implement Camera support. Within STC we did some exploration of the options, and would like to present details of this investigation. We hope this knowledge sharing will be helpful to other developers. We will briefly go through possible HW configurations and related use cases before really stepping into the SW side. Then we will introduce SW stack details and the possible solutions as they relate to the configurations and use cases. Introduction
3.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 3 As with any camera related project it is important to pay attention to the following key requirements: • Desired FPS • Video Resolution • Video Lag • Distance of camera(s) from the processing unit • Number of cameras to connect • Number of cameras to display simultaneously • Time to switch camera view if user allowed to select camera from the list of inactive devices • Performance of CPU or assisting en(de)coding modules Considering the level of abstraction of Tizen-IVI applications with respect to Native solutions, it is extremely important to fix these requirements before the HW selection step. We will review some of the implications in the next slides. Typical requirements to consider
4.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 4 It is important to introduce possible types of HW first, as use cases may split in its requirements to particular HW combinations. We could recommend first to focus on these three groups of HW in selecting particular combinations of HW for the setup. • Camera HW which could be USB, Analog and IP ones. • Host system main board offering USB root hub (one or many, USB2.0 or USB3.0), a means of connecting an Analog video capture board, and one or more Network interfaces. • Analog video capture board offering single or multiple channel support, together with a video driver which may or may not perform channel multiplexing ( mapping of multiple channels onto a single Linux video device ). Types of HW
5.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 5 At a low level, some of available options will limit possible use cases. Typical points to consider: • IP cameras to support required quality of the video (frame size/fps) with MJPEG over TCP/IP and/or H.264 over RTSP/RTP depending on particular needs. There is not too much HW related specific in this option if you have TCP connection to the camera working along with UDP packets going well and you satisfied with the quality versus performance of the network and host system. Why it is important to care about particular HW set up?
6.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 6 • Analog video capture board which supports non-multiplexed Linux video devices. Many cheap and small miniPCI Express cards support many input channels, but their drivers expose only one Linux video device, and only one channel sends frames to the interface at one time. Hence multichannel support requires rapid switching between channels. To minimize problems on the SW side, it is preferable to use a capture board whose driver supports as many Linux video devices as required, ideally one per camera. Another option would be several video capture boards, if allowed by the number of MiniPCI Express ports on the mainboard. Cards with higher total FPS should be preferred. If channel multiplexing is required, then higher FPS would help to minimize time lost ( about 3-4 frames ) due to tuning phase after each channel switch. It’s important to distinguish between per-channel FPS, and the total FPS supported by the video chip/card. Card manufacturers have a habit of not keeping their drivers up to date. Check that you can build your card’s driver using the particular kernel used on the target system. Why it is important to care about particular HW set up? ( cont. )
7.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 7 • USB cameras to support required quality of the video with acceptable USB bandwidth considering other devices connected to same USB Root Hub at the host system mainboard. Feasibility of multiple USB camera solutions is determined by single camera USB bandwidth allocation for acceptable quality video, the number of cameras needed, and the number of USB Root Hubs. The main issue here is the difficulty in knowing how much USB bandwidth will be allocated by a Hub for a given camera – it is not a simple function of video bandwidth, but depends on options presented to the USB driver by the device. Typical USB Bandwidth allocations to expect for different camera profiles are from 20% up to 60%+. 20% of bus bandwidth is reserved for non-isochronous devices. Hence this might be a limiting factor in multiple camera projects. Another complication is that you would never find this USB bandwidth usage written on the box. Even more, it might depend to particular Firmware version. So, you would need to be in touch with manufacturer or perform hands-on experiments with sample cameras. Why it is important to care about particular HW set up? ( cont. )
8.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 8 • Mitigations for USB bandwidth issues Consider requiring USB 3.0 support in Root Hub and Camera as this might relax some of the limits around USB bandwidth. Consider H.264 support in the camera and availability of the SW options to deal with such interfaces. Why it is important to care about particular HW set up? ( cont. )
9.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 9 In context of Tizen-IVI, let’s assume we are always talking about Web Applications presenting a video stream in WebKit or Crosswalk browser. This is the ‘Tizen IVI way’. Typical use cases and the most important requirements to consider might be: • Slow motion camera - Intermediate quality of the video, no requirements regarding lag • Rear view on the display - Minimal lag, high resolution, flipped frames • 360 degree stitched view to present road around the car - Multiple cameras setup, correction of lens distortion, stitching of camera outputs into single panoramic video, minimal lag • Visualization of the driving details and some camera view merged on windshield - Merge of the dynamic/static data with actual camera view, minimal lag, medium FPS. • In car camera to view rear passenger area - Multiple cameras setup, medium or low FPS, medium lag. • One by one frame processing before display in various recognition cases such as faces, road signs or other objects recognition – low lag is preferred, but may not be critical ( depends on circumstance ), medium FPS as the recognition may take longer than single frame release time, no en(de)coding artifacts to achieve best recognition results Typical use cases
10.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 10 General architecture and types of SW Source Weston Underlying Middleware System/Kernel Services Kernel Card Drivers WebKit WRT Application WRT Plug-in JS API Wayland client D-BUS API Variety of Weston plug-ins, shell, compositor ProcessingPresentation A general architecture can be presented in the following way, where some of the layers can be excluded for particular cases.
11.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 11 • WebKit in this view is a sandbox where the app is running WRT Application is a combination of HTML/CSS/JS technologies, hence it can deal with the video frames with help of JS or leave it to the functions available in WebKit core. So, it can be Video tag, Pure Canvas, WebGL or JS MPEG-4 decoder. WRT Plug-in primary usage is control (over D-BUS or similar interface ) and secondary is bypass interface to the source of the video data (Underlying Middleware implementing access to particular System/Kernel services) With respect to webkit there are few more options to deal with particular camera options such as: Combination of webkitGetUserMedia and createObjectURL to get blob object referencing the stream taken from V4l2, hence applicable to USB and Video Capture Board options Make your own Blob object with help of JS and WRT Plug-in and feed it to video tag like the one from the above option • Weston plug-ins might be one of the options if you want to make overlay client where the WRT Application only controls the way and time video is presented. Video frame processing is then done outside WRT sandbox. Presentation layer
12.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 12 • Underlying Middleware ( Processing layer ) is a combination of several components: Tizen components such as GStreamer (or FFMPEG) to unify interface to the various video streaming devices. NB This is distinct from GStreamer as used directly inside Webkit. Custom SW components to control parameters of System components Or Custom SW components to deal with the device interfaces directly, considering specific details of each video streaming device • What is the purpose of Processing layer group? Essentially it is an abstraction layer that hides details of the various types of processing required by different video sources before that source is exposed to the WRT. • GStreamer can perform most such processing tasks, even if some of the options required are really complicated. Where little or no processing is required, then of course you may avoid any GStreamer overhead by creating Custom SW to deal with V4l2 or IP camera interface directly. However, if it is simply that GStreamer does not provide the processing you need, you should probably consider creating a GStreamer plug-in, which would then make all the GStreamer facilities available too. Processing layer (could include WRT or Weston Plug-in)
13.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 13 • Could be a Linux Video device in combination with V4l2 interfaces. Most applicable to USB and Video Capture board types of HW. Functionality and parameters available thru ioctl interface should be also considered as part of this layer group • Could be custom interfaces made to support specific features such as video in h.264 • Network interface to IP cameras or non-standard interfaces to the devices where you would need to use some intermediate layers to get access to the camera even thru IP or other type of connection. Source layer
14.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 14 • Webkit: video tag. Does buffering of the video if used with network stream set directly to SRC. Lag depends to video type and properties, but typical values to expect are from 3 to 5 seconds. Another complication is list of supported video formats. Webkit used in Tizen-IVI supports OGG and H.264/MPEG-4 Part 10. On the plus side, it’s possible to run up to 6-8 streams at the same time with good FPS and quality of decoding. • Webkit: pure canvas. As fast as can be expected for single camera set up and can offer single camera view with FPS close to 25-30, but strictly depends on implementation in JS and below, which provides byte array with frame. Typical lag is about 1 second or less. • Webkit: WebGL could give you some fancy features with additional processing of the video frames, but still might be comparable to the pure canvas solution. • Webkit: JS MPEG-4 decoder is fast enough to get smooth play of the video with very minimal lag, but might not work out of the box on Tizen IVI. • WRT plug-in: D-BUS control interface. Can be Corba or any other interface to the system or custom service running to serve the webkit with the video streams and control the cameras over ioctl and V4L2 interfaces. Specific of particular selections of SW and HW
15.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 15 • WRT plug-in: Combination with underlying MW to control and run Gstreamer, or to read the frames directly from the Linux video interface such as /dev/videoX. Good for intermediate processing of the frames, but relatively slow because can work only with Canvas or other direct draw method. • Webkit: webkitGetUserMedia and createObjectURL. Doesn’t work out of the box with Tizen-IVI as Webkit is using GStreamer 1.0 library, but still sends the parameters in the GStreamer 0.10 format. Hence some modifications would be required to bypass this issue. Meanwhile, this way does work with Crosswalk and result is very smooth. • Webkit: own blob object in combination with WRT plug-in to serve video tag source. Could be comparable to webkitGetUserMedia and createObjectURL method, but performance would depend entirely on the WRT Plug-in implementation. • Weston: shell or compositor. Fast and flexible as you are managing merge of the layers and surfaces directly. Can be helpful in complex layer merge scenarios such as Media Player with data shown on top of the video, but constraints are in the high knowledge entrance level and complexity of the system. Additionally it may impact whole system performance and stability, hence shouldn’t be used without proper assessment of the needs and risks. Specific of particular selections of SW and HW (cont.)
16.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 16 • GStreamer: Complex, but powerful solution which can read multiple sources (from files to IP cameras) and give multiple types of outputs with different conversions in between input and output. Main points to consider are: There will probably be more than one way to build a GStreamer pipe to get functionality you need. Different versions of Tizen may include different versions of GStreamer and its plug-ins leading to different and sometimes failing behavior in between releases Vaapi ( HW support ) is present, but it’s effectiveness depends on the Host system. SW encoders may fail under heavy system load and produce corrupted streams. Not everything is obvious . E.g. tcpserversink is good for streaming the data over the network, but you would need to create your own wrapper (proxy) to add HTTP headers if you try to use it with browser Video tag. Many useful plug-ins are still in “bad” and “ugly” packages meaning you have very minimal support for most of functionality outside mainstream usage. Specific of particular selections of SW and HW (cont.)
17.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 17 • FFMPEG: Simpler than GStreamer, but also limited. Upstream versions are not expected to be available in the Tizen IVI repo and please be aware of modules which are not even built for Tizen. • Custom SW: Needed to deal with devices directly, or to control device status while streaming is ongoing thru other components. Highly dependent on circumstances, but might be required for: Setting up Video Capture board parameters Controlling status and availability of the devices Matching devices versus some ID to identify exact cameras in multicamera setup. Specific of particular selections of SW and HW (cont.)
18.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 18 Let’s try to assemble combinations of SW for particular usecases defined before • Slow motion camera - Intermediate quality of the video, no requirements regarding lag In short this is the case to record/present status of object over the time rather than to display its immediate state. Simplest solution would be to use GStreamer and split the source into two identical outputs where the one saved to the file and second displayed to user. No lag requirement means variety of options to implement presentation layer • Rear view on the display - Minimal lag, high resolution, flipped frames For the USB and Analog cameras the simplest way would be a combination of webkitGetUserMedia and createObjectURL methods. A custom solution could be implemented, such as own blob object or Weston plug-in for overlay in combination with GStreamer. Last options are helpful especially if there is a need for camera multiplexing or IP camera usage. Flipping could be done by browser as layer property, hence no additional processing required. Solutions
19.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 19 • 360 degree stitched view to present road around the car - Multiple cameras setup, correction of lens distortion, stitching of camera outputs into single panoramic video, minimal lag Architecturally correct way would be custom GStreamer plug-in implementation and usage of OpenCV for different sources stitching, blending and lens corrections. Standard videomixer plug-in is good example for quick start with GStreamer plug-ins. Custom SW could implement the same functionality by reading the frames from Linux Video interfaces directly or using GStreamer as a library. For minimal lag Weston overlay or own blob object could be used to pass video. • Visualization of the driving details and some camera view merged on windshield - Merge of the dynamic/static data with actual camera view, minimal lag, medium FPS If data is primarily textual information, then browser layer capabilities could be used to merge it with video stream. Otherwise variety of GStreamer plug-ins could do it at the cost of some MW complication. With medium FPS browser Canvas is good option where the frames are read directly from device (or from GStreamer for IP cameras), processed in custom SW and then passed to Canvas thru WRT plug-in. Meanwhile webkitGetUserMedia and createObjectURL methods are still good if there is no need to deal with IP cameras. Solutions (cont.)
20.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 20 • In car camera to control taxi passenger area - Multiple cameras setup, medium or low FPS, medium lag. In general similar to “360 degree stitched view” case, but lower requirements for presentation part, hence Canvas or own blob object can be implemented to handle display. • One by one frame processing before display in various recognition cases such as faces, road signs or other objects recognition - Better no lag, but depends to the case, medium FPS as the recognition may take longer than single frame release time, no en(de)coding artifacts to achieve best recognition results. Custom SW to read frames directly from the device or GStreamer library and to process it further can be implemented. Meanwhile, depending on the processing required, it could be a Gstreamer plug-in doing analysis of the frames and sending the signals to upper layers. Canvas or blob object for Medium FPS and minimal lag. Weston overlay may overcomplicate the system, but technically possible to achieve minimal lag and best performance of the display. Solutions (cont.)
21.
Copyright © 2013
Symphony Teleca Corp. All rights reserved. CONFIDENTIAL AND PROPRIETARY 21 Questions? Last slide
Download now