For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/mythic/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit For more information about embedded vision, please visit: http://www.embedded-vision.com Mike Henry, CEO and Founder of Mythic, presents the "Pioneering Analog Compute for Edge AI to Overcome the End of Digital Scaling" tutorial at the May 2019 Embedded Vision Summit. AI inference at the edge will continue to create insatiable demand for compute performance in power- and cost-constrained form factors. Taking into account past trends, continuous scale-up of algorithms and the real economic value now being generated by AI at the edge, a demand for 1000x more compute over the next 10 years is not out of the question. Mythic is a pioneer in analog compute, a key technology that will take us well beyond the end of Moore’s Law and deliver powerful, easy to use compute at the edge to meet this demand. In this presentation, Henry discusses Mythic’s unique IPU architecture that combines analog compute with compute-in-memory, delivering an unparalleled combination of AI inference performance and energy efficiency. He also highlights the advantages of using the Mythic architecture in edge applications such as DNN-enabled video surveillance cameras.

1. © 2019 Mythic, Inc.
Pioneering Analog Compute for
Edge AI to Overcome the End of
Digital Scaling
Mike Henry – CEO/Founder
Mythic
May 2019

2. © 2019 Mythic, Inc.
The Trends are Clear:
• Insatiable demand for DNN compute
at the edge will only increase
• Models like ResNet-50 are considered
“toys” in production systems
• Developers need Bigger models that
handle more edge cases
• Applications run many concurrent
models with tough latency requirements
2
The Largest Problems you See Should be the
Smallest Problem you Think About

3. © 2019 Mythic, Inc.
Compute Needs to Keep Up
Faster training
and AI-focused
compute will drive
further scientific
breakthroughs
Inference
hardware will
allow rapid scale-
up of powerful
algorithms
Semiconductor TAM for AI ($B) – Barclay’s Research
0
5
10
15
20
25
30
2016 2017 2018 2019 2020 2021 2022 2023
Training Data center inference Edge inference

4. © 2019 Mythic, Inc.
Mythic

5. © 2019 Mythic, Inc.
Mythic Focused on Scaling Inference
Training Inference
• Cost
• Power
• Easy Software for Deployment
• Maximize Compute and Memory
• Keep the Two Close
• Few Other Limits

6. © 2019 Mythic, Inc.
Decreasing Cost,
Exponentially Increasing
Market Size Defines
Semiconductors

7. © 2019 Mythic, Inc.
Moore’s Law is Over
GlobalFoundries Stops All 7nm DevelopmentMoore’s Law Comes Up Short

8. © 2019 Mythic, Inc.
AI compute Needs Something that Resets the Clock
Just like these did to their predecessors
NAND Flash RF CMOS
CMOS Imaging

9. © 2019 Mythic, Inc.
Targeting High-Value Inference Applications
Cloud edge and
datacenter
On premise
aggregation
High value edge
▪ AI applications shifting in
large scale to DNNs
▪ Offers compute density
unlike anything on the market
▪ Targets for Mythic
– On-premise aggregation
– Cloud edge/ datacenter
– High-value edge
▪ Roadmap envisions mobile,
consumer versions

10. © 2019 Mythic, Inc.
Mythic IPU – Analog DNN Inference Co-processor
Host
PCIe
▪ Co-processor on the PCIe bus
▪ Weights stored in on-chip flash
(INT8)
▪ Compute with analog technology
▪ 19x19mm BGA package
▪ Multiple chips can be tiled on a
single PCIe card or in a system

11. © 2019 Mythic, Inc.
Mythic Enables High Compute in Small Form-Factors
11

12. © 2019 Mythic, Inc.
Mythic Features In-Memory DNN Compute
In-Memory DNN Tile Array of Tiles
Matrix Multiplication Accelerator (MMA)

13. © 2019 Mythic, Inc. 13
Neural networks are largely matrix multiplies
Primary DNN Calculation is Input Vector * Weight Matrix = Output Vector
Input Data
Neuron Weights Outputs Equations
𝑋0 𝑋1 ⋯ 𝑋 𝑁 ∗
𝐴0 𝐵0 𝐶0
𝐴1 𝐵1 𝐶1
⋯ ⋯ ⋯
𝐴 𝑁 𝐵 𝑁 𝐶 𝑁
=
𝑌𝐴 = 𝑋0 𝐴0 + 𝑋1 𝐴1 + 𝑋2 𝐴2
𝑌𝐵 = 𝑋0 𝐵0 + 𝑋1 𝐵1 + 𝑋2 𝐵2
𝑌𝐶 = 𝑋0 𝐶0 + 𝑋1 𝐶1 + 𝑋2 𝐶2
𝑇
In-Memory Compute

14. © 2019 Mythic, Inc. 14
Analog circuits give us the MAC we need

15. © 2019 Mythic, Inc.
Single Chip, Multiple Apps
(e.g. 1080P30 SSD & Classify)
PCIe
Switch
Multiple Chips, Cascaded via PCIe
(e.g. 4K30 SSD & Classify)
Flexible, Scalable Architecture

16. © 2019 Mythic, Inc.
Low Power DNN Inference Solution
16
Nvidia Jetson TX2
Qualcomm835
Nvidia Tesla T4
Habana Goya HL-100
Mythic IPU
Frames Per Second/Watt
Inference Capability per Watt for Common DNN Processors
ResNet-50, INT8, and Batch=1

17. © 2019 Mythic, Inc.
Training
(In the cloud)
Optimize & Compile
TF and
ONNX
Deploy
(On Mythic’s Compiler)
Performance
Estimate
Deploy to Silicon
• Firmware
• Linux and Android
Drivers
• Quantization +
Compression
• Mapping to processing
and storage tiles
• Machine code
generation
Automatic:
(On the system)
Develop with Latest Networks and Frameworks

18. © 2019 Mythic, Inc.
Simple Run-Time API to Load DNN & Run Inference
Host Processor
PCIe

19. © 2019 Mythic, Inc.
Mythic Solutions for Security and Surveillance
Single chip in
camera
On premise NVR
SoC
Camera+Lidar
Lidar
Power
1-2W
4-32W
4-32W
Use cases
Low bandwidth to host (e.g.
radio link, remote install)
Large number of cameras
Heavy processing (e.g.
video)
High risk pedestrian,
object detection
SoC
SoC

20. © 2019 Mythic, Inc.
Mythic’s analog compute lets us deliver what no else can
• Large, powerful models entirely on chip and instantly computable
• Unmatched performance and power efficiency for a given
solution cost
• Rich product roadmap that will far surpass all-digital approaches
as Moore’s Law and DRAM bottlenecks tighten
• Powerful SDK that makes quantization and compilation easy
20
Video Redaction & PrivacyObject Tracking & Identification Image Enhancement and Compression

21. © 2019 Mythic, Inc.
Conclusions
• Insatiable demand for DNN edge compute will only grow:
• Bigger models
• Huge memory requirements
• Ultra-low latency
• Mythic’s analog compute is the offramp from the End of Moore’s Law
• Keep a close eye on us this year!
21

22. © 2019 Mythic, Inc.
Resources
• info@mythic-ai.com
• www.mythic-ai.com
Mythic @ Hot Chips 2018
https://medium.com/mythic-ai/mythic-hot-chips-2018-637dfb9e38b7
A Peek Into Software Engineering at Mythic
https://medium.com/mythic-ai/a-peek-into-software-engineering-at-mythic-
1b0ca5522868
22