As Machine learning reaches the mainstream, new tools available to developers makes it possible to implement machine-learning features—voice, face, and image recognition; personalized recommendations; and more—in a mobile context. TensorFlow Lite applies many techniques for achieving low latency; optimizing the kernels for mobile apps, pre-fused activations, and quantized kernels that allow smaller and faster (fixed-point math) models.

1. Machine Learning for Mobile Developers:
Tensorflow Lite Framework
Avid Farhoodfar, PhD, MSSW
Artificial intelligence applications in
Consumer Electronic Devices
1

2. Index
● Why Machine Learning directly on-device is important &
how it is different than what you may do on the server.
● What has been built with TensorFlow Lite.
● Some Demo
2

3. What is Machine Learning ?
3

4. What’s a machine learning?
Use algorithms to learn from data (a.k.a Training)
Algorithms are known as models
Models perform prediction (a.k.a inference) Model
Output
“cat”
4

5. What’s a machine learning?
Use labeled data to improve models
labeled data = Input data + predictions
Errors used to improve the model
We need a Framework to make machine
learning predictions easier
Model
Output
“cat”
“cat”
Error
5

6. TensorFlow
TensorFlow is google’s framework for
machine learning.
It makes it easy to build and train
neural networks.
It is cross platform, works with CPUs, GPUs,
TPUs, as well as Mobile devices, and
Embedded Platforms.
Model
Output
“cat”
“cat”
Error
6

7. ML runs in many places
7

8. ML runs in many places
● Access to more data
8

9. ML runs in many places
● Access to more data
● Fast and closely knit interactions
9

10. ML runs in many places
● Access to more data
● Fast and closely knit interactions
● Privacy preserving
10

11. On-device ML allows building
new types of products!
11

12. On-device ML is hard
● Produced computer power
12

13. On-device ML is hard
● Produced computer power
● Limited memory
13

14. On-device ML is hard
● Produced computer power
● Limited memory
● Battery Constraints
14

15. Simplifying ML on-device
TensorFlow Lite makes these challenges much easier!
15

16. What can I do with it?
16

17. Many use cases
Speech Content
Classification
Prediction
Recognition
Text to Speech
Speech to Text
Object detection
Object Location
OCR Gesture
recognition
Facial modelling
Segmentation
Clustering
Compression
Super Resolution
Translation
Voice Synthesis
Video generation
Text generation
Audio generation
Text Image Audio
17

18. Who is using it?
18

19. >2B mobile devices
Have TensorFlow Lite deployed on them in production
19

20. Some of the users ...
20

21. 21

22. 22

23. Why Did They Migrate to TFLite?
their
their
23

24. Where are we at?
Where are we going?
24

25. 25

26. 26

27. 27

28. 28

29. 29

30. 30
GPU vs CPU Performance
At Google, they use
new GPU backend
which is accelerating
compute intensive
networks that enable
vital use cases for
the users.

31. 31

32. 32

33. 33

34. 34

35. 35

36. 36

37. 37

38. 38

39. 39

40. DEMO TIME
TensorFlow Lite for
Microcontrollers
Smaller, cheaper & wider range of devices
40

41. This is about
Tiny models on tiny computers!
● Microcontrollers are everywhere
● Speech researchers were pioneers
● Models just tens of kilobytes
41

42. Here’s one I have in my pocket
Get ready for a live demo!
https://www.sparkfun.com/products/15170
384KB RAM, 1MB Flash, $15
Low single-digit milliwatt power usage
Days on a coin battery!
42

43. Why is this important?
(1) This is running entirely locally on the embedded chip.
We don’t need to have any internet connection
(2) The model itself is not quite 13 KB but it takes 20KB flash
storage on this device
(3) And the footprint of TensorFlow Lite for
Microcontrollers is only another 25 KB
43