Inferência de IA em edge devices - A câmera é o novo sensor universal

1. Jomar Silva
Technical Evangelist

2. Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for
optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and
SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or
effectiveness or any optimization on microprocessors not manufactured by Intel. Microprocessor-
dependent optimizations in this product are intended for use with Intel microprocessors. Certain
optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Refer to the
applicable product User and Reference Guides for more information regarding the specific instruction
sets covered by this notice. Notice Revision #20110804.
2

3. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software, or service activation.
Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system
manufacturer or retailer or learn more at www.intel.com.
Performance estimates were obtained prior to implementation of recent software patches and firmware updates intended to address
exploits referred to as "Spectre" and "Meltdown." Implementation of these updates may make these results inapplicable to your device or
system.
Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and
configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost
reduction.
This document contains information on products, services, and processes in development. All information provided here is subject to
change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.
Any forecasts of goods and services needed for Intel’s operations are provided for discussion purposes only. Intel will have no liability to
make any purchase in connection with forecasts published in this document.
Arduino* 101 and the Arduino infinity logo are trademarks or registered trademarks of Arduino, LLC.
Altera, Arria, the Arria logo, Intel, the Intel logo, Intel Atom, Intel Core, Intel Nervana, Intel Xeon Phi, Movidius, Saffron, and Xeon are
trademarks of Intel Corporation or its subsidiaries in the United States and other countries.
*Other names and brands may be claimed as the property of others.
Copyright ® 2018 Intel Corporation. All rights reserved.
3

4. This document contains information on products, services, and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to
obtain the latest forecast, schedule, specifications, and roadmaps. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software, or service
activation. Learn more at intel.com, or from the OEM or retailer.
No computer system can be absolutely secure.
Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of
information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit www.intel.com/performance.
Cost-reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost
savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.
Statements in this document that refer to Intel’s plans and expectations for the quarter, the year, and the future are forward-looking statements that involve a number of risks and uncertainties.
A detailed discussion of the factors that could affect Intel’s results and plans is included in Intel’s SEC filings, including the annual report on Form 10-K.
The products described may contain design defects or errors, known as errata, which may cause the product to deviate from published specifications. Current characterized errata are available on
request.
Performance estimates were obtained prior to implementation of recent software patches and firmware updates intended to address exploits referred to as "Spectre" and "Meltdown."
Implementation of these updates may make these results inapplicable to your device or system.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are
accurate.
Intel, the Intel logo, Pentium, Celeron, Atom, Core, Xeon, Movidius, Saffron, and others are trademarks of Intel Corporation in the United States and other countries.
*Other names and brands may be claimed as the property of others.
Copyright © 2018, Intel Corporation. All rights reserved.
4

5. 1. Amalgamation of analyst data and Intel analysis.
2. IDC FutureScape: Worldwide Internet of Things 2017 Predictions (link)
3. IDC FutureScape: Worldwide Internet of Things 2015 Predictions (link)
50%
45%
of data will be stored,
analyzed, and acted
on at the edge22018
2019
of IoT deployments will be
network constrained3
By2020
Average
internetuser 1.5GB data/day
Smart
hospital 3TB data/day
Autonomous
automobile 4TB data/day
Connected
airplane 40TB data/day
Smart
factory 1PB data/day
5
5

6. Things Network
Infrastructure
Data
Center/CloudEdgeCompute
6

7. AutonomousVehicles ResponsiveRetail Manufacturing
EmergencyResponse Financialservices MachineVision Cities/transportation
Publicsector
7

8. “Computer vision is concerned with the automatic
extraction, analysis and understanding of useful
information from a single image or a sequence of
images.”
Source: http://www.bmva.org/visionoverview

9. • Image quality (low quality)
• Light
• Viewpoint / Orientation
• Movement deformation
• Occlusion
• Object variation
• Camouflage
• Optical Illusions

10. • In a nutshell: a color image is a three dimensional array (one element for each
pixel, one layer for each color)
Computational image manipulation is
basically array-based math

11. • Open and loose definition: “an interesting part of an image”
• Some OpenCV algorithms from version 2.x are patented (removed on OpenCV
3.x)

12. 1
 Based on selection and connections of computational filters to
abstract key features and correlating them to an object.
 Works well with well defined objects and controlled scene.
 Difficult to predict critical features in larger number of objects or
varying scenes.
Traditional Computer Vision
 Based on application of a large number of filters to an image to
extract features.
 Features in the object(s) are analyzed with the goal of associating
each input image with an output node for each type of object.
 Values are assigned to output node representing the probability
that the image is the object associated with the output node.
Deep Learning Computer Vision

13. SOURCE: Gradient Based Learning Applied to Document Recognition - http://yann.lecun.com/exdb/publis/psgz/lecun-98.ps.gz
Feature extraction Classification

14. SOURCE: http://setosa.io/ev/image-kernels/

15. SOURCE: http://scs.ryerson.ca/~aharley/vis/conv/

16. © 2019 Intel Corporation
AIIsthedrivingforce
Foresight
Predictive
Analytics
Forecast
Prescriptive
Analytics
Act/adapt
Cognitive
Analytics
Hindsight
Descriptive
Analytics
insight
Diagnostic
Analytics
AnalyticsCurveDatadeluge(2019)
25GBper month
InternetUser
1
50GBper day
SmartCar
2
3TB per day
SmartHospital
2
40TBper day
AirplaneData
2
1pBper day
SmartFactory
2
50PBper day
CitySafety
2
WhyAInow?
1. Source: http://www.cisco.com/c/en/us/solutions/service-provider/vni-network-traffic-forecast/infographic.html
2. Source: https://www.cisco.com/c/dam/m/en_us/service-provider/ciscoknowledgenetwork/files/547_11_10-15-DocumentsCisco_GCI_Deck_2014-2019_for_CKN__10NOV2015_.pdf
16
Insights
Business
Operational
Security

17. Source: Forrester Research – Artificial Intelligence: Fact, Fiction. How Enterprises Can Crush It; What’s Possible for Enterprises in 2017
Aiadoptionisjustbeginning
58%
of business and technology
professionals said they're
researching AI, but only…
12%said they are currently
using AI systems.
In a recent Forrester Research survey…

18. Machine
Learning
How do you
engineer the best
features?
Machinelearning
𝑁 × 𝑁
Arjun
NEURAL NETWORK
𝒇 𝟏, 𝒇 𝟐, … , 𝒇 𝑲
Roundness of face
Dist between eyes
Nose width
Eye socket depth
Cheek bone structure
Jaw line length
…etc.
CLASSIFIER
ALGORITHM
SVM
Random Forest
Naïve Bayes
Decision Trees
Logistic Regression
Ensemble methods
𝑁 × 𝑁
Arjun
DeepLearning
How do you guide
the model to find
the best features?

19. Deeplearning:Trainingvs.inference
Lots of
labeled data!
Training
Inference
Forward
Backward
Model weights
Forward
“Bicycle”?
“Strawberry”
“Bicycle”?
Error
Human
Bicycle
Strawberry
??????
Data set size
Accuracy
Didyouknow?
Training requires a very large
data set and deep neural
network (i.e. many layers) to
achieve the highest accuracy
in most cases

20. The complexity of the problem (data set) dictates the network structure. The
more complex the problem, the more ‘features’ required, the deeper the
network.

21. © 2019 Intel Corporation
SpeedupdevelopmentUsingopenAIsoftware
21
1 An open source version is available at: 01.org/openvinotoolkit *Other names and brands may be claimed as the property of others.
Developer personas show above represent the primary user base for each row, but are not mutually-exclusive
All products, computer systems, dates, and figures are preliminary based on current expectations, and are subject to change without notice.
TOOLKITS
App
developers
libraries
Data
scientists
Kernels
Library
developers
Intel® Distribution of
OpenVINO™ Toolkit1
Nauta (Beta)
Deep learning inference deployment
on CPU/GPU/FPGA/VPU for
Caffe*, TensorFlow*, MXNet*, ONNX*, Kaldi*
Open source, scalable, and extensible
distributed deep learning platform
built on Kubernetes
Intel-optimized Frameworks
And more framework
optimizations underway
including PaddlePaddle*,
CNTK* & others
Python R Distributed
• Scikit-
learn
• Pandas
• NumPy
• Cart
• Random
Forest
• e1071
• MlLib (on Spark)
• Mahout
Intel®
Distribution
for Python*
Intel®
DAAL
Intel distribution
optimized for
machine learning
Intel® Data Analytics
Acceleration Library
(incl machine learning)
Intel® nGraph™ Compiler (Beta)
Open source compiler for deep learning model
computations optimized for multiple devices (CPU, GPU,
NNP) from multiple frameworks (TF, MXNet, ONNX)
Intel® Math Kernel
Library for Deep Neural
Networks (Intel® MKL-DNN)
Open source DNN functions for
CPU / integrated graphics
Visit: www.intel.ai/technology
Machinelearning Deeplearning
*
*
*
*
*

22. Today
DigitalSecurity&Surveillance
2018+
ScalingtoIndustrial&Retail,EnabledbySWTools
OpenVINO™
22

23. What’sInsideIntel®DistributionofOpenVINO™toolkit
OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc.
OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos
Intel® Architecture-Based
Platforms Support
OS Support: CentOS* 7.4 (64 bit), Ubuntu* 16.04.3 LTS (64 bit), Microsoft Windows* 10 (64 bit), Yocto Project* version Poky Jethro v2.0.3 (64 bit), macOS* 10.13 & 10.14 (64 bit)
Intel® Deep Learning Deployment Toolkit Traditional Computer Vision
Model Optimizer
Convert & Optimize
Inference Engine
Optimized InferenceIR OpenCV* OpenVX*
Optimized Libraries & Code Samples
IR = Intermediate Representation file
For Intel® CPU & GPU/Intel® Processor Graphics
Increase Media/Video/Graphics Performance
Intel® Media SDK
Open Source version
OpenCL™
Drivers & Runtimes
For GPU/Intel® Processor Graphics
Optimize Intel® FPGA (Linux* only)
FPGA RunTime Environment
(from Intel® FPGA SDK for OpenCL™)
Bitstreams
Samples
An open source version is available at 01.org/openvinotoolkit (some deep learning functions support Intel CPU/GPU only).
Tools & Libraries
Intel® Vision Accelerator
Design Products &
AI in Production/
Developer Kits
Open Model Zoo
(30+ Pre-trained Models)
Samples

24. Expedite development, accelerate deep learning inference performance, and speed production deployment.
24
Pretrained Models in Intel® Distribution of OpenVINO™ toolkit
Binary Models
 Face Detection Binary
 Pedestrian Detection Binary
 Vehicle Detection Binary  ResNet50 Binary
 Age & Gender
 Face Detection–standard & enhanced
 Head Position
 Human Detection–eye-level
& high-angle detection
 Detect People, Vehicles & Bikes
 License Plate Detection: small & front facing
 Vehicle Metadata
 Human Pose Estimation
 Action recognition – encoder & decoder
 Text Detection & Recognition
 Vehicle Detection
 Retail Environment
 Pedestrian Detection
 Pedestrian & Vehicle Detection
 Person Attributes Recognition Crossroad
 Emotion Recognition
 Identify Someone from Different Videos–standard &
enhanced
 Facial Landmarks
 Gaze estimation
 Identify Roadside objects
 Advanced Roadside Identification
 Person Detection & Action Recognition
 Person Re-identification–ultra small/ultra fast
 Face Re-identification
 Landmarks Regression
 Smart Classroom Use Cases
 Single image Super Resolution
(3 models)
 Instance segmentation
 and more…

25. 25
Use Model Optimizer & Inference Engine for public models & Intel pretrained models
• Object Detection
• Standard & Pipelined Image Classification
• Security Barrier
• Object Detection SSD
• Neural Style Transfer
• Object Detection for Single Shot Multibox
Detector using Asynch API+
• Hello Infer Classification
• Interactive Face Detection
• Image Segmentation
• Validation Application
• Multi-channel Face Detection

26. Optimize/
Heterogeneous
Inference engine
supports multiple
devices for
heterogeneous flows.
(device-level
optimization)
Prepare
Optimize
Model optimizer:
 Converting
 Optimizing
 Preparing to
inference
(device agnostic,
generic optimization)
Inference
Inference engine
lightweight API
to use in
applications for
inference.
MKL-
DNN
cl-DNN
CPU: Intel®
Xeon®/Intel®
Core™/Intel Atom®
GPU
FPGA
Myriad™ 2/X
DLA
Intel®
Movidius™
API
Train
Train a DL model.
Currently supports:
 Caffe*
 Mxnet*
 TensorFlow*
 ONNX*
Extend
Inference engine
supports
extensibility
and allows
custom kernels
for various
devices.
Extensibility
C++
Extensibility
OpenCL™
Extensibility
OpenCL™/TBD
Extensibility
TBD
ApplicationdevelopmentwithOpenVINO™Toolkit

27. • Simple and unified API for inference
across all Intel® architecture
• Optimized inference on large Intel®
architecture hardware targets
(CPU/GEN/FPGA)
• Heterogeneous support allows
execution of layers across hardware
types
• Asynchronous execution improves
performance
• Futureproof/scale development for
future Intel® architecture processors
Inference Engine Common API
PluginArchitecture
Inference
Engine
Runtime
Intel®
Movidius™ API
Intel® Movidius™
Myriad™ 2
DLA
Intel Integrated
Graphics (GPU)
CPU: Intel® Xeon®/Intel®
Core™/Intel Atom®
clDNN Plugin
Intel® MKL-DNN
Plugin
OpenCL™Intrinsics*
FPGA Plugin
Applications/Service
Intel® Arria®
10 FPGA
Intel® Movidius™
Plugin

28. • Power/Performance Efficiency Varies
– Running the right workload on the right
piece of hardware  higher efficiency
– Hardware acceleration is a must
– Heterogeneous computing?
• Tradeoffs
– Power/performance
– Price
– Software flexibility, portability
PowerEfficiency
Computation Flexibility
Dedicated
Hardware
GPU
CPU
X1
X10
X100 Vision Processing
Efficiency
Vision DSPs
FPGA

29. 29
HighPerformance&LowPowerforAIInference
Intel®neuralcomputestick2
8X¹HIGHER
PERFORMANCE
On deep neural networks compared to Intel®
Movidius™ Neural Compute Stick
UPTO
+
Intel®
Movidius™
Myriad™ X
VPU
Intel® Distribution of
OpenVINO™ toolkit
Optimized by
Powered by
MORE CORES. MORE AI INFERENCE.
 Start quickly with plug-and-play simplicity
 Develop on common frameworks and
out-of-box sample applications
 Prototype on any platform with a USB port
 Operate without cloud compute dependence
Boost
productivity
Simplify
prototyping
Discover
efficiencies

30. 30

31. Intel Distribution for OpenVINO: http://software.intel.com/openvino-toolkit
OpenVINO OpenSource: http://01.org/openvinotoolkit
Implementações de referência: https://intel.ly/2SwRigI
OpenVINO Workshop: https://bit.ly/2Eb6k3e
31

32. 32
Obrigado!
Twitter/Facebook: @homembit