“OpenCV: Past, Present and Future,” a Presentation from OpenCV.org
•
0 j'aime•1,397 vues
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2020/03/opencv-past-present-and-future-a-presentation-from-opencv-org/ For more information about edge AI and vision, please visit: http://www.edge-ai-vision.com Gary Bradski, the President and CEO of OpenCV.org, delivers the presentation “OpenCV: Past, Present and Future” at the Edge AI and Vision Alliance’s March 2020 Vision Industry and Technology Forum. Bradski shares the latest developments in the OpenCV open source library for computer vision and deep learning applications, as well as where OpenCV is heading.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à “OpenCV: Past, Present and Future,” a Presentation from OpenCV.org
Similaire à “OpenCV: Past, Present and Future,” a Presentation from OpenCV.org (20)
Plus de Edge AI and Vision Alliance
Plus de Edge AI and Vision Alliance (20)
Dernier
Dernier (20)
“OpenCV: Past, Present and Future,” a Presentation from OpenCV.org
- 2. 2 OpenCV is … • The most popular CV library in the world (at least until Deep Learning Era) used by many companies and organizations: >100,000,000 downloads and about 1.5M downloads/week • Implemented in C++ with interfaces in C++, Python, Java and JavaScript, includes thousands of functions, has modular structure (with modules at github in opencv, opencv_contrib, DNN, open_model_zoo, cvat, opencv_extra ) • Distributed under BSD license (free for any kind of use - personal, academic, commercial) • Developed since 1998, first at Intel, then at Itseez, now at Intel again and OpenCV.org, with active help from community • Runs everywhere:
- 3. OpenCV Past Enable Computer Vision
- 4. 4 Past Pre-2000 problem: • Computer Vision code was handcrafted, • difficult to reproduce from conference or journal publications, • full of bugs, • no general infrastructure. 1998+: Founded OpenCV to address the above: • Provide a universal, debugged, reliable infrastructure to: • Enhance reproducibility and • Speed up innovation • Promote commercial use by producing free (BSD) and open source code in CV
- 5. OpenCV Present Scale beneficial uses of CV in Society
- 6. OpenCV : 1.5M+ installs per week! Python: 2.5M/month, Built C++: 150K/month, Git: 360K/month + many linux distros, anaconda etc. Estimate: ~6M/downloads/month. Sourceforge
- 7. 7 OpenCV 4.2 Summary • Shift to full C++11 compliance • Addition of DNN – Deep Learning Inference, particularly for embedded/edge • Direct support for Myriad X, Intel, CUDA, can make plugins for any backend • Comes with extensive model zoos: • https://github.com/opencv/open_model_zoo/tree/master/models/public • https://github.com/opencv/open_model_zoo/tree/master/models/intel • Core is smaller and faster • Updated with more state of the art vision • Fiducial markers, dense optical flow, 3D scan fusion, Android camera support, improved tracking, robotics support, image quality assesment
- 8. 8 What’s new in OpenCV 4? (1/3) • OpenCV is now more than just OpenCV (see http://github.com/opencv): • opencv – the main OpenCV repository, essential, stable modules • opencv_contrib – experimental or obsolete functionality • cvat – Computer Vision Annotation Tool, reworked version of VATIC • dldt – Deep Learning Deployment Toolkit, a very fast Deep Learning Inference Engine and Model Optimizer/Converter tool • open_model_zoo – a set of deep learning models, including some created at Intel, for various computer vision tasks • training_toolbox_tensorflow – scripts for TensorFlow to retrain some of the models from open_model_zoo.
- 9. 9 What’s new in OpenCV 4? (1/3) • OpenCV is now more than just OpenCV (see http://github.com/opencv): • opencv – the main OpenCV repository, essential, stable modules • opencv_contrib – experimental or obsolete functionality • cvat – Computer Vision Annotation Tool, reworked version of VATIC • dldt – Deep Learning Deployment Toolkit, a very fast Deep Learning Inference Engine and Model Optimizer/Converter tool • open_model_zoo – a set of deep learning models, including some created at Intel, for various computer vision tasks • training_toolbox_tensorflow – scripts for TensorFlow to retrain some of the models from open_model_zoo. Also known as
- 10. 10 What’s new in OpenCV 4? (2/3) • C++ 11 library! • Basic functionality is ~100% compatible with OpenCV 3.0 • Emphasis on deep learning • Significantly extended and accelerated OpenCV DNN module • Started replacing some traditional algorithms in OpenCV with deep nets (e.g. face, object, text detection) • Obsolete functionality moved from opencv to opencv_contrib • Introduced graph API (G-API) for efficient image processing pipelines • Smaller and faster • 127 kernels (55 functions) have been accelerated for AVX2 via Wide Universal intrinsics ~ 1.5x acceleration when using CPU_BASELINE=AVX2; some functions optimized for NEON • Lower footprint. OpenCV 4.0 is ~20% smaller than OpenCV 3.x.
- 11. 11 • Initial FP16 support: CV_16F data type; basic functionality and DNN now support FP16. • Hardware-accelerated media decoding/encoding on Windows (via WMF) and Linux (via Gstreamer). • Several accuracy-critical image preprocessing functions have been converted to fixed-point and made bit- exact: color=>gray conversion, resize, gaussian blur etc. • Some new traditional CV functionality: realtime QR detector and decoder, optimized Kinect Fusion, Hand-Eye What’s new in OpenCV 4? (3/3)
- 12. DNN module (DeepLearning @ OpenCV) • Compact self-contained implementation in C++; inference only! • 5 importers (Caffe 1, TF, Torch, Darknet, ONNX) • 40+ layers, 50+ unit tests, 12 samples • Supports many popular topologies: image classification, object detection, semantic segmentation etc. • Easy-to-use C++, Python, Java and Javascript interface • Several execution backends that support various targets that users/companies can contribute to: 12 Backend CPU iGPU fp32 iGPU fp16 GPU VPU (NCS/N CS2) FPGA DNN_BACKEND_OPENCV + + + + – – DNN_BACKEND_INFERENCE_ENGINE + + + – + + DNN_BACKEND_HALIDE (Deprecated) + + – + – – DNN_BACKEND_VKCOM (Vulkan) – + – ? – – DNN_BACKEND_CUDA (Nvidia) – – – + – –
- 13. 13 Using OpenCV DNN with Intel Inference Engine (a.k.a. DLDT) original model Intel Model Optimizer path #1: load models in DLDT native format path #2: Load models as-is and construct DLDT graph using OpenCV DNN Caffe IR frame Caffe DLDT CPU VPUGPU https://github.com/opencv/dldt Optimized Modules can be created for any hardware. Risk-V, FPGA, TPU, NPU … We do this! OpenCV.ai Contact: GaryBradski@gmail.com
- 14. 14 Graph API (G-API): Overview ▪ A new separate module opencv_gapi (not a complete library rewrite): https://github.com/opencv/opencv/wiki/Graph-API ▪ Provides alternative “lazy” image processing functions, e.g. cv::Sobel => cv::gapi::Sobel: – Instead of immediate evaluation gapi:: functions construct expression subgraphs (GMat) and then you get a complete graph (GComputation) ▪ The produced graph is compiled (once) and then can be processed more efficiently than a sequence of direct function calls ▪ CPU and GPU backends are ready; more backends are in progress
- 15. Mask R-CNN with OpenCV
- 17. 17 Some 3D Context tracking
- 19. 19 My Recent Work in Medical Imaging Using OpenCV • Color Calibration for blood volume measurement • Signal boosting to image blood flow in tissue in real time
- 20. OpenCV Future Scale beneficial uses of CV in Society
- 21. 21 OpenCV OpenCV.org • Open Source Code • Courseware • Conferences • Standards • Certification • HW Program OpenCV.ai • Partnerships • Vision/Deepnet Contracting/Consulting • Smart Camera Verticals • Medical, Robotics, Agriculture, Security, Drones, Film, Factory/Industrial, Consumer, AR/VR GOAL: Drive the beneficial uses of computer vision in society Move the needle for robotics and medical
- 22. Future: Specialized Toolboxes For example: Calibration Toolboxes Multi-Camera Calibration Color Calibration Medical Robotics Motion Planning SLAM Manipulation Automotive/Autonomous Driving Industrial/Factory Film Production Agriculture Inverse rendering, research goal
- 23. 23 OpenCV.org HW Program • Partners for support decisions • Courseware, training • Contracting • Online Store with Certified: • Cameras • Processors • Sensors
- 24. 24 HW Program Memberships Individual Annual Membership: $100 ▪ Buy products and books at a discount from OpenCV Listed & Certified stores ▪ Buy Courseware offered at a discount ▪ Develop products for the AI marketplace ▪ Connect with global professionals, consultants, researchers, and professors Group Annual Membership: $3,000 ▪ Sell products at OpenCV Listed & Certified Stores – Registered devices – Certified devices: OpenCV Certified Logo; OpenCV Certified Tool, device driver release ▪ Branding: OpenCV websites, conferences, email lists, etc. Core Annual Membership: $12,000 ▪ Preview & comment on the future OpenCV HW release & programs ▪ Run certificate programs ▪ Candidacy for OpenCV HW Board Member Board Annual Membership: $100,000 ▪ Seat on OpenCV HW Partnership Committee ▪ Drive OpenCV HW roadmap & standards ▪ Co-develop annual Smart Vision Flagship Devices https://opencv.org/opencv-hardware-partnership-program/
- 25. 25 OpenCV.ai • Contracting and consulting in computer vision and deepnets • Expertise on optimizing to hardware • ARM, Raspberry PI, Myriad, GPU, NPU, CPU, RISK-V, NXP, Edge in general • Not just vision, any sort of medical/security/industrial detection and recognition • Partnered application development in verticals ... Talk with us Gary Bradski: garybradski@gmail.com