Norman Sasono Presentation for Agile Impact Conference 2018 Day 1. "Incorporating AI/ML into Your Application Architecture"

1. Incorporating AI/ML
into Your Application Architecture
Norman Sasono
CTO & Co-Founder, bizzy.co.id
@nsasono /in/normansasono

2. AI/ML can do wonders.
But it has been too hyped up.
As Architects/Developers, we need to cut through the hype,
and understand how to incorporate AI/ML into our App Architecture,
and solve real problems.
As Data Scientists, we need to understand how Architects/
Developers will use your Model.

3. What has happened?
AI/ML is now accessible to many more people
Back Then: A few highly specialized individuals Today: Every Developer and Data Scientist

4. Why now?
AI/ML has been democratized
Convergence of:
• Algorithmic Advancements
• Data Explosion
• Cloud Computing (Computing Power and Storage)

5. AI
The ability of a machine to perform cognitive functions we associate with
human minds, such as perceiving, reasoning, learning, interacting with
the environment, problem solving, and even exercising creativity.
Examples of technologies that enable AI to solve business problems are
robotics and autonomous vehicles, computer vision, language, virtual
agents, and machine learning

6. ML
Machine-learning algorithms detect patterns and learn how to make
predictions and recommendations by processing data and experiences,
rather than by receiving explicit programming instruction.
The algorithms also adapt in response to new data and experiences to
improve efficacy over time.

7. AI > ML
AI
ML

8. ML is a “Function”
Creating Algorithms by training those Algorithms with data. The training
will result Predictive Model that provides an estimated output based on
given input.
The techniques in ML can create decision logic that would be impractical or
impossible to build using traditional application development tools and
algorithms.

9. Types of Analytics

10. ML: Supervised Learning
• An algorithm uses training data and
feedback from humans to learn the
relationship of given inputs to a given
output
• Linear Regression
• Logistic Regression
• Linear Quadratic/Discriminant Analysis
• Decision Tree
• Naive Bayes
• Support Vector Machine
• Random Forest
• AdaBoost
• Gradient-Bossting Trees
• Neural Network

11. ML: Unsupervised Learning
• An algorithm explores input data without
being given an explicit output variable (eg,
explores customer demographic data to
identify patterns)
• K-Means Clustering
• Gaussian Mixture Model
• Hierarchichal Clustering
• Recommender System

12. ML: Deep Learning
Convolutional Neural Network:
A multilayered neural network with a special architecture
designed to extract increasingly complex features of the
data at each layer to determine the output
Recurrent Neural Network:
A multilayered neural network that can store information
in context nodes, allowing it to learn data sequences
and output a number or another sequence

13. Software Development Cycle v ML Development Cycle
Note that the ML-Model still needs to be implemented in Production Grade

14. ML Development requires Data Scientist.
It requires skills in Data Analysis & Manipulation,
Mathematics, Statistics, ML Algorithms and Patterns.
Not just Software Development.

15. Architects/Developers,
let’s say now you need ML in your App.
What can you do?

16. Treat ML-based functionality like any other component:
• Modularize it
• Keep it loosely coupled

17. The scope of ML models in app architectures is commonly very localized.
They perform specific functions.
The ML models may be incredibly complex. However, from an app
architect's perspective, they can be encapsulated by simple interfaces that
take the input data to be processed and return a prediction from the model.
Hence, API (in module runtime or remote API).

18. Some Examples of Integrating ML into App Architecture
Your App
Provider-
Trained ML
Model
Cloud
ML Service
API
Your App
Your-
Trained ML
Model
Your Own
ML Service
API
ML Serving
Framework
Your-
Trained ML
Model
Your App
ML Serving
Framework
Using General Purpose
ML-Based APIs
Using Custom ML APIs
Embed Model in App
At Build
Load/Update Model
At Runtime
Your-
Trained ML
Model
Your App
ML Serving
Framework
Your-
Trained ML
Model

19. Options for Sourcing ML-Based Capabilities
Use ML-
Based APIs
Use 3rd Party
Software with
Embedded ML
Capabilities
Use Pre-
Trained ML
Model
Refine/Re-
Traine Pre-
Trained ML
Model
Create & Train
New Model

20. Options for Sourcing ML-based Capabilities
• Use ML-based APIs - the ML behind the APIs are managed by the ML Provider (Microsoft,
Amazon, Google, etc, for NLP, Image Processing, Audio Processing, Voice Processing, etc)
• Use 3rd Party Software with Embedded ML Capability - a bundled ML capabilities in
Software or Open Source Project (ML capabilities in Mobile Phones, CRM Tools,
Productivity Tools, etc)
• Using a Pre-Trained Model (with an ML Framework) - reuse pre-trained model
(Tensorflow Repo, Keras Repo, etc)
• Refining/Re-Train Pre-Built Model (with an ML Framework) - retrain a model with a
specific Data Sets
• Create & Train New Model (with an ML Framework) - create and train custom ML
Model if your data and reqs are proprietary

24. Data Architecture

25. Key Takeaways
• Modularize and decouple ML Models as discrete services or components, be it
consuming 3rd party ML-based APIs or your own, then compose these into your
architecture.
• Learn and understand basic ML vocabularies.
• Start with consuming available off-the-shelf ML-based APIs, then custom build
your own model later as needed.
• If you really going to need train an ML Model or create your own custom ML
Model, start to build your Data Science team. Data Scientists will work with
Software Engineers too for Production Grade model Implementation.
• Start with Regression, not Deep Neural Network :)

26. Thank You
@nsasono /in/normansasono
From various sources: Gartner, McKinsey, Microsoft, AWS, etc