With iOS 10, Apple added tools for creating neural networks on iOS, but how can you use these tools to solve real-world problems? In this talk will examine a real-world problem that can be solved using neural networks and machine learning. From training a neural network on industry standard platforms, through to migrating the trained model to iOS, this talk will show how machine learning can be used to create the next generation of intelligent apps. Some knowledge of the basic concepts of Machine Learning would be helpful but is not required.

2. Keith Moon - Senior iOS Developer
Machine Learning for
iOS Developers
UnitedDefConf - Minsk
April 2017

3. 3
• iOS Developer since 2010
• Worked with BBC News, Hotels.com and Travelex
• “Swift 3 Cookbook” to be published by Pakt
Who am I?
@keefmoon

4. 4
• Help users discover great local food
• Make it quick and easy to order from a wide variety of takeaways
• Available on:
–Web
–iOS
–tvOS
–Android
–Amazon Echo
What is Just Eat?
3

5. 5
• Australia
• Brazil
• Canada
• Denmark
• France
• Ireland
• Italy
• Mexico
What is Just Eat?
Global Business
• New Zealand
• Norway
• Spain
• Switzerland
• UK

6. 8
What this talk isn’t?
• Not a ML expert
• Not a Python expert
• Not great at Maths
• Not a deep dive into ML
So what is this talk?

7. So what is this talk?
9
• ML from an iOS developer’s point of view
• High level overview of ML
• What features are appropriate uses of ML?
• How can you use ML in your apps?
• Current state of ML tools
• What does the future hold?

8. Glossary
10
Machine Learning
Artificial Intelligence Neural Network
Deep Learning

9. Glossary
11
Machine Learning
Artificial Intelligence
Neural Network
Deep Learning

10. Artificial Intelligence
Glossary
12
Machine Learning
Neural Network
Deep Learning

11. Artificial Intelligence
Glossary
13
Machine Learning
Neural Network
Deep Learning

12. Artificial Intelligence
Glossary
14
Machine Learning
Deep Learning
Neural Network

13. Machine Learning Examples
15

14. ML Use Cases
16
• Spam
• Recommendations
• Handwriting recognition
• Speech recognition
• Face Detection
• Entity extraction
• Facial Recognition
• Object Recognition
• Text Prediction
• Sentiment Analysis
• Image Style transfer

15. Machine Learning Goal
17
Neural Network Output Answer
Training Input
Input Question

16. Classifier
18
Untrained
Neural Network
Training Input
Data
Data
Data
Input to categorise
Category 1
Category 2
Category 3
1
1
2
2
3
3

17. Classifier
19
Trained
Neural Network
Training Input
Input to categorise
Category 1
Category 2
Category 3

18. Classifier
20
Neural Network
Input
Input to categorise
Hidden
Training Input
Data
Data
Data
1
2
3
Category 1
Category 2
Category 3
1
2
3
Output

19. ⨍( )
Training the model
21
w1
w2
w3
∑i xi * wi + bx1
x2
x3
b
Forward Propagation
eg. Softmax

20. Training the model
22
w1
w2
w3
x1
x2
x3
b
Back Propagation
g = gradient
The extent to which
changing the value
reduces the error
g
Difference between
expected and actual
= error
g
g
g

21. Classifier
23
Neural Network
Input
Input to categorise
Hidden
Training Input
Data
Data
Data
1
2
3
Category 1
Category 2
Category 3
1
2
3
Output

22. Handwriting Image Input
26
Input
?
?
?
?
from the MNIST dataset

23. Handwriting Image Input
27
Input
?
?
?
?
14 pixels x 14 pixels 14 x 14 = 196 values
between 0 and 1

24. Handwriting Image Input
28
Input
14 pixels x 14 pixels 14 x 14 = 196 values
between 0 and 1
…
px 1
px 2
px 3
px 4
px 5
px 6
px 196

25. Your
App
API
Adding a ML Feature
29
What are my options?
Train
Use
1) Managed Machine Learning driven API

26. 30
• No ML knowledge required
• Simple to implement
• Light on resources
• Only solves common ML problems
• Third-Party dependancy
• Needs connectivity
• Data ownership issues
• No control over model improvement
Adding a ML Feature What are my options?
1) Managed Machine Learning driven API

27. 31
Adding a ML Feature What are my options?
2) Custom Model Trained and Used on Server
Your
App
API
Train
Use

28. 32
• Can customise the model to your
needs
• Make use of open source models
• Light on mobile resources
• You control the data
• You control model improvement
• Knowledge of ML tools / Python
needed
• Server management overhead
• App friendly API needed
• Needs connectivity
Adding a ML Feature What are my options?
2) Custom Model Trained and Used on Server

29. 33
Adding a ML Feature What are my options?
3) Custom Model Trained on Server. Used on Phone.
API
Train
Trained ModelAccelerate
Framework
Metal
Framework
GPU
CPU

30. 34
• Can customise the model to your
needs
• Make use of open source models
• User can control the data
• Can work offline
• Knowledge of ML tools / Python needed
• Server management overhead
• Need to transfer complex model to phone
• May limit the scope of model
improvement
Adding a ML Feature What are my options?
3) Custom Model Trained on Server. Used on Phone.

31. 36
Adding a ML Feature What are my options?
3) Custom Model Trained on Server. Used on Phone.
2) Custom Model Trained and Used on Server
1) Managed Machine Learning driven API

32. Future of Machine Learning on iOS
● Easier model transfer from server to phone
● Trainable network APIs from Apple
● Ability to plug and play ML models together
● Further development of Swift ML tools
37

33. References
Machine Learning APIs:
Google Prediction: https://cloud.google.com/prediction
Google Natural Language: https://cloud.google.com/natural-language
Microsoft Cognitive Services: https://www.microsoft.com/cognitive-services
Amazon ML: https://aws.amazon.com/documentation/machine-learning
IBM Watson: https://www.ibm.com/watson/developercloud
Open Source Model:
Tensor Flow Models: https://github.com/tensorflow/models
FaceNet for TensorFlow: https://github.com/davidsandberg/facenet
38

34. References
Machine Learning Frameworks:
Torch: http://torch.ch
TensorFlow: https://www.tensorflow.org
Caffe: https://github.com/BVLC/caffe
Awesome Machine Learning resources: https://github.com/josephmisiti/awesome-machine-learning
Hosting:
Amazon Web Services: https://aws.amazon.com
Amazon Deep Learning AMI - Ubuntu Edition https://aws.amazon.com/marketplace/pp/B06VSPXKDX
Digital Ocean https://www.digitalocean.com
39

35. References
Machine Learning Frameworks on iOS:
Torch: https://github.com/clementfarabet/torch-ios
TensorFlow: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/ios_examples
http://www.mattrajca.com/2016/11/25/getting-started-with-deep-mnist-and-tensorflow-on-ios.html
Caffe: https://github.com/noradaiko/caffe-ios-sample
Using Metal Performance Shaders with a TensorFlow trained model:
https://developer.apple.com/library/content/samplecode/MPSCNNHelloWorld
Neural Networks and Accelerate: https://developer.apple.com/videos/play/wwdc2016/715
BNNS in Accelerate: https://developer.apple.com/reference/accelerate/bnns
List of ML resources for iOS: https://github.com/alexsosn/iOS_ML
40

36. References
Open Source:
Face Entry Example:
https://github.com/keefmoon/faceentry
Just Eat Open Source:
https://github.com/justeat
41

37. Thanks! @keefmoon
keith.moon@just-eat.com
keefmoon