Machine Learning at the Edge: An Introduction to AI/ML Options on Mobile Devices | AWS Developer Workshop - Web Summit 2018 Machine Learning at the edge is about making your apps smarter with real-time object detection, face recognition, offline predictions, and protecting user privacy. In this session, we'll explore and demo real solutions to handling prediction, inference, and decision making in modern apps. Speaker: Dennis Hills - Developer Advocate, AWS Mobile Applications

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Dennis Hills – Mobile Developer Advocate
November 8, 2018
Machine learning at the edge: An introduction
to AI/ML options on mobile devices
@dmennis

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Decision Time – 4 Ways to ML on Mobile
Call API-driven Managed Application Services
Build/Train in Cloud – Host Model Behind API
Build/Train in Cloud – Deploy Model to Device (edge)
Utilize Platform APIs
1
2
3
4

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
App requirements – How to ML?
Online Offline
Pre-built
ML
Custom
ML
Managed API-driven Services1 Train model in
the cloud
Deploy model
to device (edge)
Train model in the cloud
Host model behind API
HTTPs endpoint
Apple®
Vision.framework
Android.speech
Platform APIs
2
3 4

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Managed API-driven Machine Learning Services
1

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
API-driven Managed Service
Use Case
Amazon Rekognition
§ Celebrity recognition mobile app
Amazon Translate
§ Translate English text to Portuguese
1

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Build/Train in Cloud – Host Model Behind API
ü Train a model in SageMaker (or bring your own)
ü Deploy model to a prediction endpoint
ü Invoke the HTTPS endpoint from your application
2

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Build/Train in Cloud - Host Model Behind API
Use Case
2
Best when
Devices are not powerful enough for local inference.
Models can’t be easily deployed to mobile or IoT devices.
Additional cloud-based data is required for prediction.
Prediction activity must be centralized.

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
3
Build/Train in Cloud – Deploy Model to Device
ü Train a model in SageMaker or use Deep Learning AMIs
ü Deploy model to the device (edge) and get real-time prediction offline…
iOS =>
ü Use Core ML to access a local on-device trained model.
Android =>
ü Use TensorFlow Lite to interact with the ML Model on the device
Web =>
ü Use TensorFlow, Python, & modern JavaScript to interact with the ML model
within the browser

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Build/Train in Cloud – Deploy Model to Device
Use Case
3
Real-time object detection
§ Using a model (pre-trained in the cloud) deployed to an iOS
device and using Core ML to provide prediction on 1,000’s of
objects in real-time and offline.

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
Utilize Platform APIs (Mobile & IoT)
The Vision framework from Apple performs face and face landmark detection, text
detection, barcode recognition, image registration, and general feature tracking.
Vision also allows the use of custom Core ML models for tasks like classification or
object detection.
Use Apple’s Natural Language framework to perform tasks like language and script
identification, tokenization, lemmatization, parts-of-speech tagging, and named
entity recognition.

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
Utilize Platform APIs (Mobile & IoT)
Use Case
Voice Translate app for iOS
§ Using Apple Speech API for voice-to-text, completely offline.
Demonstrate real-time face detection
§ Using Apple’s Vision Framework for iOS

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recap – 4 Ways to ML on Mobile
Call API-driven Managed Application Services
Build/Train in Cloud – Host Model Behind API
Build/Train in Cloud – Deploy Model to Device (edge)
Utilize Platform APIs
1
2
3
4

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
@dmennis
Thank You!
Dennis Hills