“Market Analysis on SoCs for Imaging, Vision and Deep Learning in Automotive and Mobile Markets,” a Presentation from Yole Développement

From Technologies to Markets
© 2020
Computing for AI
in Automotive and
Smartphone,
the leading edge
applications
John Lorenz
Yole Développement
Vision Industry andTechnology Forums
March 10th, 2020

22
Frame processing +
other sensors
Fusion platform
FROM IMAGE SIGNAL PROCESSOR TO FUSION PLATFORM
Vision processor
from Mobileye
Frame processing
Vision processor
• Amount of data processed
• Performance
• Consumption
Computer vision and AI algorithms
Price
per unit
> $1000
$10
< $1
Set of pixels processing
Image Signal Processor
Image processing
algorithms
Standalone ISP from Altek
Fusion platform from NVIDIA
Algorithms
complexity
$100
Sensing Processing Unit – ISP
stacked with CIS
Computing for AI in automotive and smartphone, the leading edge applications | Yole Développement | VITF | March 2020

33
THEVISION PROCESSOR, IMAGING-DEDICATED HARDWARE FOR AI
Two different
architectures
for vision
processor
Standalone chip Unit(s) embedded in a SoC
NXP S32V234
Automotive
The chip is fully dedicated to
processing algorithms for imaging.
In a single system-on-chip (SoC), multiple
units combine to form the vision processor.
These units include ISP, CPU, memory, and
even a dedicated unit for inference
acceleration.
Algorithms for analyzing images are run in a
dedicated unit and can be assisted by other
units that form the SoC, i.e. GPU, CPU, and
memory.
The ISP can also be embedded as a unit.These algorithms are
generally computer vision algorithms. For AI, a dedicated unit
for inference acceleration can be found too in the SoC.
Qualcomm Snapdragon
Smartphone

44Computing for AI in automotive and smartphone, the leading edge applications | Yole Développement | VITF | March 2020
COMPUTING HARDWARE FOR AI SOLUTIONS LANDSCAPE
Ambarella CV2
ARM ML
Cadence DNA 100
Cadence Vision P6 DSP
Hailo Hailo-8 DL
Imagination PowerVR AI
Intel Mobileye EyeQ4
Kalray MPPA 3
KortiQ AIScale
NVIDIA Xavier
NXP S32V234
Renesas Renesas R-Car H3
Synopsys DesignWare EV
Tesla FSD
Texas Instruments Jacinto TDA3
Toshiba Visconti 4
Xilinx Zynq Ultrascale+ series
Google TPUv2
Intel Nervana
Xilinx Virtex Ultrascale+
Canaan Kendryte K210
CEVA NeuPro
Google Coral Edge TPU
Greenwaves GAP8
Intel Movidius
Lattice iCE40
NVIDIA Jetson Nano
NVIDIA Jetson TX2
Rockchip RK3399Pro
STMicroelectronics STM32 series
Bitmain Sophon series
Gyrfalcon Lightspeeur series
Apple A12
HiSilicon Kirin 980
Mediatek Helio P65
Qualcomm
Snapdragon 855
Samsung Exynos 9820
0.01
0.1
1
10
100
1000
0.01 0.1 1 10 100
Consumption(W)
Performance (TOPS)
Edge computing
Battery-powered devices
Autonomous machines
ADAS vehicles
High Performance
Data center - Robotic vehicles
Mobile
Smartphones with
neural engines
Specific players target specific segments.
It is complicated for one player to
propose a product for each segment,
since performance and consumption
requirements are very different.

5
Taiwan
USA
IMAGING AI ON THE EDGE MAIN HARDWARE PLAYERS
Europe
China
Japan
Non-exhaustive list

Automotive

7
AUTONOMOUSVEHICLES -THE DISRUPTION CASE
Two distinctive paths for autonomous vehicles
2020 should see
the first
commercial
implementation
of autonomous
vehicles
1880 1960 2000 2020 2030 2035
Technology x Market Penetration
Acceleration :The speed of technology change doubles every technology shift
Improvement
of cars as we
know
5 years10 years20 years40 years80 years
Yole Développement
© August 2020
Below expectation
“cars” fulfilling needs
in a new plane of
consumption
Disruption ?
Electronics
Invades cars
Electric car
matures
Industrialization
phase
New use cases
Automated
driving
Autonomous
vehicles
Robotic cars
ADAS vehicles Robotic vehicles

8
MARKET BREAKDOWN – ORDERS OF MAGNITUDE
2018 – Main automotive imaging applications
TOTAL
automotive
imaging
revenue is
$4.1B
x1 $30 « ADAS » cameras
x1 $40Vision processing
x1 $70Vision processing
X1 Million systemsx20 Million systems
$900M
$300M
$60M
$70M
x1 $22.5 «Viewing » camera
x1 $7.5 ISP board
$945M
$315M
x42 Million systems
Cameras
Processing
x4 $22.5 « for display » cameras
x1 $30 ISP board or x4 $7.5 ISP
x10 Million systems
$600M
$800M
$30 Rear
view
Viewing
$70
Forward
ADAS
Sensing
TOTAL Automotive Imaging $4B
~$2,500M
~$1,500M
TOTAL Cameras module
TOTALVision processing
$140
Surround
view
Viewing
$130
Forward
ADAS
Sensing

9
SENSOR MODULE ASP FOR EACH AUTOMATION LEVEL
A level-2+
car will
have $500
worth of
embedded
sensors for
AD
9
16
18
22
28
$260
$405
$500
$1,906
$1,758
Level 1
Level 2
Level 2+
Level 3
Level 4/5
$BOM Sensor Module Count
x1 x4 Radar SRR
x1 In-cabin/Driver camera
x1 µbolometer
x1 x2 x4 LIDAR
x1 Dead reckoning
x1 Event-based camera
x1 Radar LRR
x1 x3 Forward camera
x4 Camera surroundBackup camera x1
x6 x8 Ultrasonic
Today
Tomorrow

VISION PROCESSORS IN AUTOMOTIVE
VP
Ultrasonic
Radar
Forward Camera
Surround Camera
Driver Camera
LiDAR
Fusion
Fusion
Functionalities
Level 1
ACC: Automatic Cruise Control
AEB: Advanced Emergency Braking
CTA: Cross Traffic Alert
TJA: Traffic Jam Assist
PA: Park Assist
LKA: Lane Keeping Assist
DM: Driver Monitoring
HP: Highway Pilot
AP: Auto Pilot
ACC Level 2
PALKA
ACC
TJA
Level 2+/3
PALKA
ACC
TJA
AEB DM TJA
Level 4
PALKA
ACC
TJA
AEB DM TJA
HP
Level 5
PALKA
ACC
TJA
AEB DM TJA
AP
MCU: Micro-Controller
FPGA: Field-Programmable Gate Array
VP: Vision Processor Unit
CPU: Central Processing Unit/Processor
Fusion
Fusion of camera inputs is made through a VP
(Mobileye EyeQ3) or a FPGA (Xilinx solutions) or
fusion platform (Renesas R-Car H3)
Fusion of camera, radar and LiDAR
inputs is made through a fusion
platform (like NVidia solutions) with
FPGA support for preprocessing
MCU FPGA Fusion platform
Fusion
Fusion of different types of’ inputs for Level 2+ and Level 3 through
VP (Mobileye EyeQ4/5), Renesas NextGen and Nvidia platform
Technology
penetration

1111
EXAMPLE OF A FAMOUSVISION PROCESSOR: MOBILEYE EYEQ4
Description
of the units of
the Mobileye
EyeQ4
Source: Mobileye EyeQ4 Processor Family – System Plus Consulting
28nm CMOS
2.5TOPs @ 3W
• EyeQ4-High and -Mid processors,
are found in the ZF S-Cam4 Tri-
cam and Mono-cam cameras
• They integrate multi-threaded
Microprocessor from MIPS.
• These cores are coupled with the
new generation of Mobileye's
Vector Micro-code Processors
(VMP), Multithreaded Processing
Cluster (MPC) cores and
Programmable Macro Array (PMA)
cores
• Ability to manage up to three
cameras at the same time.

AUTOMOTIVE ADAS PROCESSING PLATFORM – LEVEL 3?
Courtesy of Audi
ASIC SoC
GPU
FPGA
CPU
Audi zFAS of the
Audi A8
AD computing platforms are using the full spectrum of computing architecture

13
2018 2019e 2020e 2021e 2022e 2023e 2024e 2025e 2026e 2027e 2028e CAGR
Robotic Vehicle $ 156 $ 560 $ 945 $1 424 $2 057 $3 097 $4 065 $4 898 $6 011 $7 388 $9 431 51%
ADAS Level 5 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 104 -
ADAS Level 4 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 144 $ 293 $ 893 $1 513 119%
ADAS Level 3 $ 0 $ 18 $ 48 $ 96 $ 220 $ 392 $ 689 $ 889 $1 092 $1 304 $1 674 65%
ADAS Level 2 $ 0 $ 45 $ 87 $ 184 $ 399 $ 431 $ 384 $ 399 $ 403 $ 400 $ 401 27%
ADAS Level 1 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 $ 0 -
Total $ 156 $ 623 $1 080 $1 704 $2 676 $3 920 $5 138 $6 329 $7 799 $9 986 $13 122
$ 0
$2 000
$4 000
$6 000
$8 000
$10 000
$12 000
$14 000
$16 000
Revenue($M)
AUTONOMOUS CARS AI-RELATED COMPUTING HARDWARE MARKET REVENUE IN $M BY LEVEL OF AUTONOMY
Forecast for 2018 to 2028
AI-related
hardware will
be driven by
robotic
vehicles to
reach $13b
revenue in
2028
56%
Includes revenues from sales of all
types of computing hardware that
run AI algorithms:
VPU, FPGA, fusion platforms

1414
COMPUTING HARDWARE FOR AUTONOMOUS DRIVING
Ambarella CV2
Ambarella CV22
Hailo Hailo-8 DL
Kalray Coolidge
NVIDIA Drive PX 2
NVIDIA Drive PX Xavier
NVIDIA Drive PX Pegasus
NVIDIA Drive PX Orin
NVIDIA Drive PX Orin x2
NXP S32V234
Qualcomm Snapdragon Ride
Qualcomm Snapdragon Ride
Accelerators x2
Renesas R-Car H3
Tesla FSD
TI Jacinto TDA3
Toshiba Visconti 4
Xilinx Zynq Ultrascale+ EV
1
10
100
1000
0.1 1 10 100 1000
Log Scale
Performance (TOPS)
Log Scale
Power dissipation (W)
Level 1-2
Level 2+
Level 3
Level 2++
Robotic vehicles
are using chips in
>100W range
ADAS computing
is using chips in
the 2W to 20W
range
1Petaflop
Next battleground
for the ADAS industry
SiP
The use of accelerators
in SoC or as
coprocessors allow to
increase performance
faster than consumption
Level 4-5
5 years 5 years 5 years
~100Tops/W~10Tops/W~1Tops/W~0.1Tops/W
Robotic
ADAS computing race :
higher performance for
minimum consumption

15
ECOSYSTEMS FOR AUTONOMOUS DRIVING
Key points
• Because the technologies are different, ecosystems and supply chains for ADAS and robotic cars are different;
• In both of these ecosystems, the supply chains are organizing;
• ADAS ecosystems are built around historical automotive OEMs, though with classical supply chains going less and
less throughTier-1
• Robotic vehicles ecosystems are built around full stack solution partnerships such as proposed by NVidia or Apollo
and are not exclusive to each other
• Because the path to full autonomy through robotic cars is tough, a lot of companies have made the choice to
be part of these shared and open ecosystems for software (AI, simulations, mapping,…) and hardware
(sensors, computing, shuttle/robotaxi)
Hardware for ADAS is
lead by Mobileye but
competition is tough
OEM
Tier-1
2 main ecosystems with NVidia leading the
computing hardware ecosystem thanks to their
product quality and open software stacks
Apollo ecosystem is huge and
very promising with a clear and
precise roadmap that, however,
seems a bit optimistic

EVOLUTION OF SMARTPHONE TOWARDS AI
2000 2001 2007 2008 2013 2017 2018
2000
Nokia 3310
Texas Instruments
MDA2WDIBasic tasks
Snake
Graphic
applications
1 year 6 years 5 years 4 years1 year 1 year
2000
SharpJ SH04
1st photo
phone
One of the first photos
taken by a phone
2001
Siemens SL45
1st mp3
phone
Embedded
AI
Applications
2007
iPhone
1st
touchscreen
phone
Touchscreen in the
heart of utilization
Samsung
ARM1176JZ(F)-S
V1.0
CPU, GPU &
memory in a
single chip
2008
HTC Tattoo
Snapdragon S1
MSM7225
Nice notification
display
2013
Galaxy S4
Samsung
Exynos 5
High-resolution games
More & more functions integrated:
DSP, connectivity, VPU, ISP
2017
iPhone X
Apple
A11Bionic
Integration of AI
Facial ID
Biometry
Apple
A12Bionic2018
iPhone XS
HiSilicon
Kirin 9802018
Huawei Mate 20 Pro
AR/VR
CPU CPU & GPU “1st APU”, as we
call it today
Neural engine
Several distinct
components
Progressive SoCs
appear
SoCs
90 nm
65 nm
28 nm
10 nm
7 nm
Node size
20191 year
Apple
A13Bionic
Photography
180 nm

1818
• Since the advent of application processors for
mobile, the ISP has been embedded as a
dedicated unit to treat data from the camera;
• Some players like Sony want/try to stack the
CMOS sensors with the ISP, however it is not
something with a high value added and as
cameras are more and more numerous and
with more data to handle, it is easier to
embed it in the APU.
IMAGE SIGNAL PROCESSOR IN MOBILE
The dream of embedded ISP with the sensor
Samsung Exynos 9
Courtesy of Samsung
Snapdragon 845
Courtesy of Qualcomm
Apple A12
Courtesy of Apple

IMAGE SIGNAL PROCESSOR IN MOBILE – STACKED WITH CMOS
Cost and average selling price assumptions
ISP cost is around $1.50
Sony technology is advanced and we will assume
that the ASP of ISP for smartphones is equivalent
to this cost
Sony Xperia Z

20
SMARTPHONE APPLICATION PROCESSORS
Why develop a dedicated unit to compute AI applications on the edge?
Processing AI
on the edge
makes data
handling
easier
• Processing AI applications with a dedicated unit is faster and
consumes less energy.
• No dependence on internet connection: run applications
anywhere, anytime.
• Improvement of privacy: data are not sent to the Cloud and stay on
the device;
• Possible to use personal data to improve habits of use
• Less latency for critical applications like authentication
AI’s huge requirements
• High computational need
• Real-time
• Always-on
• Huge neural network
Mobile environment constraints
• Thermal efficiency
• Low consumption for long battery life
• Memory limitations
AI-accelerator dedicated unit
embedded in the AP

CENTRALIZATION AND SPECIALIZATION – APPLE AP EVOLUTION
Annotated A4 die photo
(source: MuAnalysis)
(source: Chipworks)
(source: TechInsights)Adding more and more elements
inside the same chip, and
introducing specialized
computing units

22
2017 2018 2019e 2020e 2021e 2022e 2023e 2024e
Total smartphone shipments 1466.7 1428.9 1363.1 1331.7 1384.4 1414.0 1429.4 1423.9
Total smartphone with AI shipments 166.3 299.8 475.1 599.3 761.4 862.5 929.1 996.7
Penetration rate 11.3% 21.0% 34.9% 45.0% 55.0% 61.0% 65.0% 70.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
0.0
200.0
400.0
600.0
800.0
1000.0
1200.0
1400.0
1600.0
Penetrationrate
VolumeinMunits
Application Processors with AI-dedicated unit volume shipments and penetration
rate
• AI penetration in
smartphones is getting
very high, with a 50%
rate expected for mid-
2020
APPLICATION PROCESSOR WITH AI-DEDICATED UNIT
A five years forecast
Clear risk for some competitors
to be kicked out of the AP
market by not integrating AI,
following the first wave with
Apple and Huawei and the 2nd
wave with Samsung, Qualcomm
→ One way to catch-up is to
focus on audio AI by integrating
a dedicated unit in the AP

Computing for AI in automotive and smartphone, the leading edge applications | Yole Développement | VITF | March 2020 23
• Revenues expected to grow modestly as the cost to manufacture APUs grows
while foundries and designers set prices to maintain margin
• APUs containing embedded AI accelerating hardware skew towards higher range of
ASP, so expect >$32B of 2024 revenue to be associated with AI-capable hardware
APPLICATION PROCESSOR REVENUE: GROWS TO $46B IN 2024
-
2
4
6
8
10
12
14
16
revenue($b)
Apple Samsung Qualcomm HiSilicon MediaTek Spreadtrum Other Forecast
“Revenue” is APU designers’ revenue
2019
$32
2024
$46

THEVALUE CHAIN FOLLOWS THE DATA FLOW
Sense
Sensor $0.1 - $1
Process
Hardware $1-$10
Skiing
99%
Compute
Hardware $10-$100
IP
License/Royalties
Analyze
Hardware >$1000
The output of the
Process step is of the
same type than the
input. Processing value is
measured through how
the Compute step is
facilitated
On top of the image/sound,
information are provided.The
quality and precision of this
information as a function of
the computing power defines
the value of the Compute
step
Maximum level of value is reached here, the Analyze step,
with dedicated information that are used for understanding
habits, center of interests,… for targeted ads

2626
• Computing hardware for AI organizes around power consumption and performance
requirements
• Edge devices occupy the lower bands, and Automotive, HPC, and Robotics pushing the higher
limits of power and performance
• Imaging and AI in Automotive: Autonomy level correlates with computational
requirements
• Continued march toward higher levels of autonomy, but organized around different approaches
• ADAS solutions incrementally automate more driving sub-tasks, living within traditional
Auto ecosystem
• Robotic vehicles integrating the full stack, as a market-disrupting approach
• AI-related hardware generating ~$1B revenue in 2020, expecting >$13B in 2028, led by robotic
vehicles
• The next battleground for AD computing should see solutions with 10-50Tops at ~1Tops/W
• Imaging and AI in Smartphones: Improved AI/VP making its mark in the Silicon
• Roughly half of today’s smartphone application processors contain an embedded unit dedicated
to AI, growing to >70% in 2024, representing more than $32B in APU designer revenue
KEY TAKEAWAYS
What does the future hold?

“Market Analysis on SoCs for Imaging, Vision and Deep Learning in Automotive and Mobile Markets,” a Presentation from Yole Développement

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à “Market Analysis on SoCs for Imaging, Vision and Deep Learning in Automotive and Mobile Markets,” a Presentation from Yole Développement

Similaire à “Market Analysis on SoCs for Imaging, Vision and Deep Learning in Automotive and Mobile Markets,” a Presentation from Yole Développement (20)

Plus de Edge AI and Vision Alliance

Plus de Edge AI and Vision Alliance (20)

Dernier

Dernier (20)

“Market Analysis on SoCs for Imaging, Vision and Deep Learning in Automotive and Mobile Markets,” a Presentation from Yole Développement