Inteligencia artificial, open source e IBM Call for Code

Open Source @ IBM
Inteligência Artificial,
Open Source e
IBM Call for Code
2018 / © 2018 IBM Corporation 1
Luciano Resende
Data Science Platform Architect

About me - Luciano Resende
2
Data Science Platform Architect – IBM – CODAIT
• Have been contributing to open source at ASF for over 10 years
• Currently contributing to : Jupyter Notebook ecosystem, Apache Bahir, Apache Toree,
Apache Spark among other projects related to AI/ML platforms
lresende@apache.org
https://www.linkedin.com/in/lresende
@lresende1975
https://github.com/lresende

3
Learn
Open Source @ IBM
Program touches
78,000
IBMers annually
Consume
Virtually all
IBM products
contain some
open source
• 40,363 pkgs
Per Year
Contribute
• >62K OS Certs
per year
• ~10K IBM
commits per
month
Connect
> 1000
active IBM
Contributors
Working in key OS
projects
2018 / © 2018 IBM Corporation
Open Source participation and usage is simpler than ever

4
Open Source is essential to Developer Advocacy
IBM generated open source innovation
• 137 Code Open (dWO) projects w/1000+ Github projects
• 4 graduates: Node-Red, OpenWhisk, SystemML,
Blockchain fabric to full open governance in the last year
• developer.ibm.com/code/open/code/
Community
• IBM focused on 18 strategic communities
• Drive open governance in “Centers of Gravity”
• IBM Leaders drive key technologies and assure freedom
of action
The IBM OS Way is now open sourced
• Training, Recognition, Tooling
• Organization, Consuming, Contributing

5
IBM’s history of strong AI leadership
1997: Deep Blue
• Deep Blue became the first machine to beat a world chess
champion in tournament play
2011: Jeopardy!
• Watson beat two top
Jeopardy! champions
1968, 2001: A Space Odyssey
• IBM was a technical
advisor
• HAL is “the latest in
machine intelligence”
2018: Open Tech, AI & emerging
standards
• New IBM centers of gravity for AI
• OS projects increasing exponentially
• Emerging global standards in AI

AI Scenarios Today

Home Automation & Security
- Multiple connected or
standalone devices
- Controlled by Voice
- Amazon Echo (Alexa)
- Google Home
- Apple HomePod (Siri)
9

TESLA connected cars
CONNECTED VEHICLES.
It’s not just about Google Maps
in cars. When Tesla finds a
software fault with their vehicle
rather than issuing an expensive
and damaging recall, they
simply updated the car’s
operating system over the air.
[hcp://www.wired.com/2014/02/te
slas- air-fix-best-example-yet-
internet-things/]
10

AMAZON Go
AMAZON GO – No lines, no
checkout, just grab and go
11

Model Asset eXchange
Enabling domain experts to
use deep learning in the
enterprise

Q: What is deep learning?
A: Machine learning using
deep neural networks.
InceptionV3 Convolutional Neural Net
(A “medium-sized” deep learning model)
Image Source:
https://github.com/tensorflow/models/blob/master/research/inception/g3doc
/inception_v3_architecture.png

Characteristics of Deep
Learning (1)
14
State-of-the-Art prediction
quality in many domains
– Image classification
– Machine translation
– Facial recognition
– Time series prediction
– Many more

Learning (2)
15
Large, complex models
– Model size generally determined by “how big a
model can you fit on your device?”
Each box ≈ between
32 and 768 linear
regression models

Learning (3)
16
Poorly understood today
…even by experts
– Why do the models converge?
– Why do the models converge with low loss?
– Why do the models generalize?

Focus of this Talk
17
Incorporating well-understood
deep learning models into
enterprise applications.

Sounds easy!

“cat”
The Components of a Deep
Learning Model
Dense
(3×8)
Dense
(8×6)
Input
(3)
Output
(2)Dense
(6×4)
Dense
(4×2)
Neural Network
Graph
Weights
(not to scale)
Driver Program

Example: Get an Image Classifier
20
Step 1: Find a suitable neural
network graph.
– Need to read some papers

21
Step 2: Find code to generate
the neural network graph
TensorFlow code to build ResNet50 neural network graph

22
Step 3: Find some pre-trained
weights for your graph
Caffe2 ResNet50 model weights

23
Step 4: Find example code
that performs model inference
TensorFlow code for training and batch inference on ResNet50

24
Step 5: Write your own code to
perform model inference on one
image at a time
Step 6: Package your inference
code, graph creation code, and pre-
trained weights together
Step 7: Deploy your package

Model Marketplaces
25
Collections of well-understood
deep learning models
Provide a central place to find
known-good implementations
of these models

IBM Model Asset eXchange
MAX is a one-stop shop open source
ecosystem for data scientists and AI
developers to share and consume models that
use machine learning engines, such
as TensorFlow, PyTorch and Caffe2.
It also provides a standard approach to classify,
annotate, and deploy these models for
prediction and inferencing.
MAX
https://developer.ibm.com/cod
e/exchanges/models/

Demo!
https://developer.ibm.com/code/exchanges/models/
https://developer.ibm.com/code/patterns/create-web-app-interact-machine-learning-generated-image-captions/

Summary
28
Free, open-source models.
Wide variety of domains.
Multiple deep learning frameworks.
Vetted and tested code and IP.
Build and deploy a web service in 30
seconds.
Start training on Watson Studio in
minutes.

MAX: Future Plans
29
Many more models
– Train with Watson Studio/DLaaS
– Run inference on IBM infrastructure
Revamped website
Integration with Watson Catalog
IBMer-uploaded models
More IBM Code code patterns showing usage
https://developer.ibm.com/code/exchanges/models/

Click to edit Master title style
FfDL
Fabric for Deep Learning
FfDL provides a scalable,
resilient, and fault tolerant
deep-learning framework

https://github.com/IBM/FfDL
FfDL provides a scalable, resilient, and fault
tolerant deep-learning framework
FfDL Github Page
FfDL dwOpen Page
https://developer.ibm.com/code/open/projects/fabri
c-for-deep-learning-ffdl/
FfDL Announcement Blog
http://developer.ibm.com/code/2018/03/20/fabric-
for-deep-learning
FfDL Technical Architecture Blog
http://developer.ibm.com/code/2018/03/20/democr
atize-ai-with-fabric-for-deep-learning
Deep Learning as a Service within Watson Studio
https://www.ibm.com/cloud/deep-learning
Research paper: “Scalable Multi-Framework
Management of Deep Learning Training Jobs”
http://learningsys.org/nips17/assets/papers/paper_
29.pdf
• Fabric for Deep Learning or FfDL (pronounced as ‘fiddle’) is an open source
project which aims at making Deep Learning easily accessible to the people
it matters the most i.e. Data Scientists, and AI developers.
• FfDL Provides a consistent way to deploy, train and visualize Deep Learning
jobs across multiple frameworks like TensorFlow, Caffe, PyTorch, Keras etc.
• FfDL is being developed in close collaboration with IBM Research and IBM
Watson. It forms the core of Watson`s Deep Learning service in open
source.
FfDL
31

FfDL is built using Microservices architecture
on Kubernetes
• FfDL platform uses a microservices architecture to offer
resilience, scalability, multi-tenancy, and security without
modifying the deep learning frameworks, and with no or minimal
changes to model code.
• FfDL control plane microservices are deployed as pods on
Kubernetes to manage this cluster of GPU- and CPU-enabled
machines effectively
• Tested Platforms: Minikube, IBM Cloud Public, IBM Cloud
Private, GPUs using both Kubernetes feature gate Accelerators
and NVidia device plugins

source code
training
definition
Auto-allocation means infrastructure is used only when needed
Kubernetes container
training
artifacts
compute cluster
NVIDIA Tesla K80, P100, V100
Cloud Object Storage
Training assets are
managed and tracked.
Access to elastic compute leveraging Kubernetes

NVIDIA GPUs
Kubernetes
container orchestration
training runs
containers
server cluster
dataset
Cloud Object Storage
Model training distributed across containers

35
FfDL: Architecture

36
https://arxiv.org/abs/1709.05871
FfDL: Research Papers

Click to edit Master title style
Jupyter
Enterprise
Gateway
Provides multi-tenant,
scalable and secure remote
Jupyter Notebook kernels

Jupyter Notebooks
© 2018 IBM Corporation 39
Notebooks are interactive
computational
environments, in which
you can combine code
execution, rich text,
mathematics, plots and
rich media.

Jupyter Notebooks
• Notebook UI runs on the browser
• The Notebook Server serves the
’Notebooks’
• Kernels interpret/execute cell contents
– Are responsible for code execution
– Abstracts different languages

Building a
Data Science
Analytical Platform

Building an Data Science Platform
© 2018 IBM Corporation
Large pool of shared computing resources
• Enterprise Cloud, Public Cloud or Hybrid
• Data in the cloud (Data Lakes/Object Storage)
Distributed Consumers
• Notebooks running local (users laptop)
or as a service (e.g. Jupyter Hub)
Different Resource Utilization Patterns
• High number of idle resources

Vanilla Jupyter Notebooks
Gather
Data
Analyze
Data
Machine
Learning
Deep
Learning
Deploy
Model
Maintain
Model
Python
Data Science
Stack
Fabric for
Deep Learning
(FfDL)
Mleap +
PFA
Scikit-LearnPandas
Apache
Spark
Apache
Spark
Jupyter
Model
Asset
eXchange
Keras +
Tensorflow
43
8 8 8 8
0
10
20
30
40
50
60
70
80
4 Nodes 8 Nodes 12 Nodes 16 NodesMaxKernels(4GBHeap)
Cluster Size (32GB Nodes)
MAXIMUM NUMBER OF
SIMULTANEOUS KERNELS
Kernel
Kernel
Kernel
Kernel
Limitations of Jupyter Notebook Stack
• Security limitations
• Single user sharing the same privileges
• Users can see and control each other process
using Jupyter administrative utilities
• Scalability limitations
• Jupyter Kernels running as local process
• Resources are limited by what is available
on the one single node that runs all Kernels
and associated Spark drivers
Kernel

Jupyter Enterprise
Gateway
Jupyter Enterprise Gateway at IBM Code
https://developer.ibm.com/code/openprojects/jupyter-enterprise-gateway/
Jupyter Enterprise Gateway source code at GitHub
https://github.com/jupyter-incubator/enterprise_gateway
Jupyter Enterprise Gateway Documentation
http://jupyter-enterprise-gateway.readthedocs.io/en/latest/
Supported Kernels
Supported Platforms
45
A lightweight, multi-tenant, scalable and
secure gateway that enables Jupyter
Notebooks to share resources across an
Apache Spark or Kubernetes cluster for
Enterprise/Cloud use cases
Spectrum Conductor
+

Jupyter Enterprise Gateway
Gather
Data
Analyze
Data
Machine
Learning
Deep
Learning
Deploy
Model
Maintain
Model
Python
Data Science
Stack
Fabric for
Deep Learning
(FfDL)
Mleap +
PFA
Scikit-LearnPandas
Apache
Spark
Apache
Spark
Jupyter
Model
Asset
eXchange
Keras +
Tensorflow
46
16
32
48
64
0
10
20
30
40
50
60
70
80
4 Nodes 8 Nodes 12 Nodes 16 NodesMaxKernels(4GBHeap)
Cluster Size (32GB Nodes)
MAXIMUM NUMBER OF
SIMULTANEOUS KERNELS
Kernel
Kernel
KernelKernel
Optimized Resource Allocation
– Utilize resources on all cluster nodes by running kernels as Spark
applications in YARN Cluster Mode.
– Pluggable architecture to enable support for additional Resource Managers
Enhanced Security
– End-to-End secure communications
• Secure socket communications
• Encrypted HTTP communication using SSL
Multiuser support with user impersonation
– Enhance security and sandboxing by enabling user impersonation when
running kernels (using Kerberos).
– Individual HDFS home folder for each notebook user.
– Use the same user ID for notebook and batch jobs.
KernelKernel
Kernel

Jupyter Enterprise Gateway – YARN
YARN Cluster
YARN
Workers
Gateway Node
Jupyter Enterprise Gateway
• Multitenancy
• Remote kernel lifecycle management via process proxies
Spark Executors
Spark Executors
Spark Executors
Yarn Container
Jupyter Kernel
Spark Driver
Impersonation: Alice’s
kernel runs under
Alice’s user ID.
Spark Executors
Spark Executors
Spark Executors
Yarn Container
Jupyter Kernel
Spark Driver
SecurityLayer
nb2kg
nb2kg
Spark Executors
Spark Executors
Spark Executors
Yarn Container
Jupyter Kernel
Spark Driver
Bob
Alice

Enterprise Gateway & Kubernetes
Supported Platforms
Kernel
Kernel
Kernel
Kernel
Before Jupyter Enterprise Gateway …
• Scalability limitations
• Resources are limited and the amount
required to all kernels needs to be allocated
during Notebook Server pod creation.
• Resources are limited by what is available
on the one single node that runs all Kernels
and associated Spark drivers
Kernel
KernelKernel

Jupyter Enterprise Gateway - Kubernetes
Container images defined in kernelspec
Community image
Kernel
Spark on K8
Kernel
Distributed
File
System
Vanilla Kernels
Spark based kernels
Gateway
nb2kg
nb2kg

Summary

Summary
• Model Asset Exchange
• Curated set of models ready to use or embedded in your
application or solution
• Fabric for Deep Learning
• Provides a consistent way for AI developers and
Data Scientists to train their models
• Jupyter Enterprise Gateway
• Enables your Jupyter Notebook stack to scale in
order to build Machine Learning and AI Models
more resource effectively
MAX
https://developer.ibm.com/cod
e/exchanges/models/

Inteligencia artificial, open source e IBM Call for Code

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Inteligencia artificial, open source e IBM Call for Code

Similaire à Inteligencia artificial, open source e IBM Call for Code (20)

Plus de Luciano Resende

Plus de Luciano Resende (20)

Dernier

Dernier (20)

Inteligencia artificial, open source e IBM Call for Code

Notes de l'éditeur