2018-11-05 Intro to AI

Intro to AI
Jeff Boleman, M.S.C.S.
jkbolema@us.ibm.com
@uid0_and_beyond

AI History: Before the birth of AI
August 2018 / © 2018 IBM Corporation
§ 1833 Charles Babbage – Difference Engine lead to Analytical Engine.
§ 1900’s (early) saw Analog computers (electrical, mechanical, hydraulics) for
maritime use, firing solutions and other phenomena.
§ 1936 – Alan Turing publishes paper “On Computable Numbers”
§ 1940’s Turing Machine breaks Enigma Coding based on inputs.
§ Mid 1940 Vacuum tubes
§ 1945 Zeus Z4 – 1st Turing Complete computer
§ 1946 Eniac
§ 1951 Marvin Minsky and Dean Edmunds build SNARC (Stochastic Neural
Analog Reinforcement Calculator), the first artificial neural network, using
3000 vacuum tubes to simulate a network of 40 neurons.
§ 1955 Transistor based computers

AI History: The Birth of AI (1955)

AI History: AI up to the 1980s

AI History: AI Winter 70s-90s
1966: failure of machine translation
1970: abandonment of connectionism
Period of overlapping trends:
1971–75: DARPA's frustration with the Speech Understanding Research program at
Carnegie Mellon University
1973: large decrease in AI research in the United Kingdom in response to the Lighthill
report
1973–74: DARPA's cutbacks to academic AI research in general
1987: collapse of the Lisp machine market
1988: cancellation of new spending on AI by the Strategic Computing Initiative
1993: expert systems slowly reaching the bottom
1990s: quiet disappearance of the fifth-generation computer project's original goals
Reference: https://en.wikipedia.org/wiki/AI_winter

AI History: AI Spring
§ 1989 Yann LeCun and other researchers at AT&T
Bell Labs successfully apply a backpropagation
algorithm to a multi-layer neural network,
recognizing handwritten ZIP codes. Given the
hardware limitations at the time, it took about 3
days (still a significant improvement over earlier
efforts) to train the network.
§ 1998 Yann LeCun, Yoshua Bengio and others
publish papers on the application of neural
networks to handwriting recognition and on
optimizing backpropagation.
§ 2006 Geoffrey Hinton publishes “Learning Multiple
Layers of Representation,” summarizing the ideas
that have led to “multilayer neural networks that
contain top-down connections and training them to
generate sensory data rather than to classify it,”
i.e., the new approaches to deep learning.

AI History: Timeline of modern AI

Why the AI explosion?
Lots of automated collection of data (examples)
§ Smart Watches – Fitbit, Apple Watch and Garmin for heart rate, steps, activity, etc…
§ Internet of Things (IoT) – cheap devices collecting data such as manufacturing QA
§ Cameras and storage – everything is being recorded with high quality
§ Nest devices – thermostat data collected/managed to reduce spikes on power plants
§ Logging of physical devices such as cars, braking, acceleration, use, etc…
§ Customized online experienced based on usage (i.e. Facebook suggesting
products you like)
§ Inexpensive devices using technology such as ARM and FPGA do inference and
also drive intelligent collection of data

Lots of Compute (GPUs)
§ Deep Learning
§ Scalar Vector Machines
§ Matrix Multiplication
§ 5120 KUDA Cores
§ 640 Tensor Cores
§ 16-32 GB memory
§ 9000 GB/s memory bandwidth
§ Up to 6 in a liquid cooled 2 socket 2U IBM POWER9
Why the AI explosion?

AI use cases have endless possibilities…
Retail Analytics &
Theft Detection
Worker Safety
Compliance
Drone Video
Analytics
Warehouse
Automation
Self-Driving
Cars
Remote Inspection &
Asset Management

AI 101: Decision Trees
Simple Question: How can we decide if we are productive?

AI 101: Rule Based Systems
These expert systems started to hit their stride in the 1970s and were applied to
speech recognition, planning and control, and disease identification.

AI 101: Machine Learning vs. Deep Learning

AI 101: Machine Learning vs Deep Learning
Reflector areas stamped into fenders, grill treatments, side scoops,
engine mounts, steering column, gear boxes, wiring, dash panels,
seats, shoulder belt mounts, engine options are all different
These differences are subtle but avid Mustang enthusiasts can
instinctively identify the year on a passing car. They just know.
How do you teach a computer to differentiate between a
1967 and 1968 Mustang?

AI 101: Bias Mitigation – Husky or Wolf?

August 2018 / © 2018 IBM CorporationAugust 2018 / © 2018 IBM Corporation
AI 101: A Macro Look
Artificial
Intelligence
Mimic Humans
Machine
Learning
Learn with
Experience
Deep Learning
(Neural Networks)
Self-Learn with
More Data

Deep Learning Has Revolutionized
Machine Learning
Data
Accuracy
Deep
Learning
Traditional
Machine
Learning
Modern AI Methods Require
Accelerated Servers
Deep Learning Fueling Growth of AI
Lots of Data & Lots of Compute
Requires Systems with CPUs along with
GPU Hardware Accelerators
AI 101: ML/DL

AI 101: Three Major ML/DL Networks

AI 101: High Level ML/DL Process Flow

AI 101: Deep Learning Frameworks

AI 101: TensorFlow Use Cases

AI 101: AI Challenges People and Technology

Watson
Public Cloud
Enterprise AI Platform
PowerAI
On-Premise Private Cloud
AI Models
Built by Client Data Science Team Builds
PowerAI
Machine / Deep Learning Software
+ Cluster Orchestration
Power AI Systems Data Stores
IBM Cloud Private
Watson APIs
(Pre-built AI Models)
Watson Machine & Deep Learning Infrastructure
Power Systems Data Stores
Bluemix Software Stack
In-House AI APIs
(Client AI Models)
IBM Cloud
There are IBM AI Solutions for Cloud & On-Premise

On Premise: Data Gravity Influence
Quite simply, it is easier to move compute to the data. This is why so much AI/ML/DL is done on premise today.

Enterprises Embracing
Open-Source AI Software
Enterprises building
machine learning teams
Most using Open-Source software:
TensorFlow is most popular
IDC 2021 Market Size Projection:
$14B for AI Servers
57%
AI Developed
On-Premise
42%
Cloud Hosted
Modeling
57%
Developed
using on-premise
resources
On Premise: AI Development

Best Infrastructure for Enterprise AI
IBM Power Systems AC922
POWER9: AC922

4 GPUs @150GB/s
CPU ßà GPU bandwidth
6 GPUs @100GB/s
CPU ßà GPU bandwidth
Coherent access to system memory
PCIe Gen 4 and CAPI 2.0 to InfiniBand
Air and Water cooled options
Coherent access to system memory
PCIe Gen 4 and CAPI 2.0 to InfiniBand
Water cooled only
NVLink
100GB/s
NVLink
100GB/s
NVDIA V100
Coherent
access to
system memory
(2TB)
NVLink
100GB/s
NVLink
100GB/s
NVLink
100GB/s
170GB/s
CPU
PCIe Gen 4
CAPI 2.0
NVDIA V100NVDIA V100
DDR4
IB
Coherent
access to
system memory
(2TB)
NVLink
150GB/s
NVLink
150GB/s
170GB/s
CPU
PCIe Gen 4
CAPI 2.0
NVLink
150GB/s NVDIA V100NVDIA V100
DDR4
IB
POWER9: NVLink CPU to GPU Throughput

200-petaflops system
most powerful & smartest
supercomputer for science
IN THE WORLD
“...unprecedented computing power for research
in energy, advanced materials, AI, and more
enabling scientific discoveries that were
previously impractical or impossible.”
+++
POWER9: The fastest computer in the world

Computational Horsepower: 200 Petaflops
200 Quadrillion calculations, per sec. Each person on earth doing
arithmetic problem at once, 33X over every second.
Data Throughput: ~2.5 TB/sec
Moves equivalent of every bit of new data ingested into Facebook
today by 1 billion users—
every 30 minutes
Greenest & Most Effective HPC Performance
Will nearly top the Green 500 & HPCG rankings— today, 2 of the Top
15 Clusters in 2017 Green 500 are POWER8 with NVLink + Tesla P100-
based
CORAL
POWER9: Summit’s Architecture

Deep Learning ML and DL Machine Learning
Power AI Base Power AI Enterprise AI Vision Data Science Experience H2O Driverless AI
Offering
Description Deep Learning Deep Learning for the Enterprise Deep Learning with Video tools
Notebook oriented development
environment for ML and DL
Automated Machine learning
Pricing Model Free download Commercial Commercial Commercial Commercial
Support Available from IBM IBM L 1-3 Included IBM L1-3 Included Available from IBM H2O L 1-3
Seller Compensation Server, Support Server, Gross SW License Server, Gross or Term SW License Server Server, Net SW License
Applications
Text & Numeric Yes Yes No Yes Yes
Images Yes Yes Yes Yes No
Video - Optional add-on Yes No
Primary Persona Data Scientist Data Scientist Line of Business Data Scientist Data Scientist
Second persona IT IT IT IT Line of Business
User Skill Level High Medium to high Low Medium to high Low to Medium
Strengths
Rapid deployment, high
performance, scale
enterprise grade, High
performance, rapid Deployment
Rapid deployment, simple GUI
high performance
Notebook based development
environment, strong
collaboration, model management
Simplified deployment, intuitive user
interface, automatic pipelines,
"explainability" for models, end to
end automation
Platform
Distributed DL (DDL) 1-4 nodes 1-thousands of nodes Coming Coming -
Large Model Support Yes Yes Coming Coming -
Server(s) S822LC or AC922 S822LC or AC922 S822LC or AC922 S822LC or AC922, LC922 S822LC, AC922, LC921/922
IBMProducts
Spectrum MPI (DDL) Limited to 4 nodes Included ? Optional add-on
Spectrum Conductor DLI
Optional add-on Included ? Optional Add On Optional add-on
IBM DSX Local Optional add-on Optional add-on No Optional add-on
Cloud
IBM Cloud Public Yes No Trial only Watson Studio ?
IBM Cloud Private
Yes Coming Yes Yes Coming
Software: AI offerings of Power

PowerAI
Open-Source Based
Open Source Frameworks:
Supported Distribution
Developer Ease-of-Use Tools
Faster Training Times via
HW & SW Performance Optimizations
Integrated & Supported AI Platform
3-4x Speedup for AI Training
Ease of Use Tools for Data Scientists
GPU-Accelerated Power
Servers
Storage
Caffe
SnapML
Software: PowerAI

PowerAI
Deep Learning Impact
(DLI) Module
Data & Model
Management, ETL,
Visualize, Advise
IBM Spectrum Conductor with Spark
Cluster Virtualization, Elastic Training
PowerAI: Open Source ML Frameworks
Large Model Support (LMS)
Distributed Deep Learning
(DDL)
Auto-Hyper
Parameter Tuning
PowerAI
Enterprise
Auto-ML for Images & Video
PowerAI
Vision Label Train Deploy
Accelerated
Infrastructure
Accelerated Servers Storage
SnapML
Software: PowerAI

Label Image or
Video Data
Auto-Train AI
Model
Package &
Deploy
AI Model
Software: PowerAI Vision
Auto-Deep Learning for Images and Video

Software: PowerAI Enterprise
Arch. | Env. Stack | Packaging View | Bundle Contents & Providers
Server (Power) GPU
Operating System (RHEL)
NVIDIA Driver
CUDA
cuDNN
Spectrum Conductor
Spectrum Conductor Deep
Learning
Impact
Elastic Distributed
Training
Apps
Hardware
PowerAI
TensorFlow Caffe
...
LMS*
Distributed
TensorFlow
DDL
DDL hooks DDL hooks
Bring Your Own
Framework
File
System
(client)
DataStore Servers
File
System
(server)
Disk
add-on
libs
Anaconda-
sourced libs
libs
Anaconda-
sourced libs
libs
Anaconda-
sourced libs
PowerAI Branded
UI
Client or other third party
IBM – PowerAI
NVIDIA
Spectrum
MPI
OFED
IBM – Spectrum MPI
Anaconda
Red HatRed Hat
EPEL
Mellanox
IBM – Spectrum Conductor
IBM - PowerAI Enterprise
[Software Bundle]
Various, e.g. IBM – Spectrum Scale
Including the NVIDIA stack in the PowerAI Enterprise software
bundle is under consideration, but for the initial release, it is not
packaged in the software bundle.

Data Processes | Application Layer | Training Flow
Data – Training - Original
Data – Training - Scrubbed
Model – Training Weights (intermediate)
Transform
Data – Training - Datasets
Import
Data – Training – Model Ready
Train (Data Set Up)
Weights (Trained)
Train (Proper)
Key:
stored
on Data Store - Originals
processed
external to Cluster
stored
on Data Store - Working
processed
on Cluster
stored
on Compute Nodes (temporal)

Data Processes | Application Layer | Inferencing Flow (local)
Data – Inference - Original
Data – Inference - Scrubbed
Model – Inference
Transform
Data – Inference – Datasets
Import
Data – Inference – Model Ready
Inference (Data Set Up)
Weights (Trained)
Inference (Proper)
Model – Training
Create Inference Model
Inference Results
Key:
stored
on Data Store - Originals
processed
external to Cluster
stored
on Data Store - Working
processed
on Cluster
stored
on Compute Nodes (temporal)

Import
Train (Data Set Up)
Train (Proper)
Shared
Storage
Domain
Data – Training - Datasets
Data Flow | Training (New Model, generic)
Cluster Storage
Network
(public
IB or EN)
Client Uplink
(Access
Network)
Existing client environment
Solution environment
Access
Network
(public EN
10Gb+)
VLAN
Users
DataStore
- Working
DataStore
- Originals
Client Uplink
(Storage
Network)
Data – Training - Original
Data – Training - Scrubbed
Model – Training
Weights (Trained)
Data – Training – Locally Cached
Transform
All inter-element data transfers flow
over the Storage Network

Reference Design A2 – Recommended Node Specifications
System Mgmt Node Master Node
Cluster Type All All Performance Production PoC/Development
Server Model 1U LC921 (Boston) 1U LC921 (Boston) 2U AC922 (Newell) 2U AC922 (Newell) 2U AC922 (Newell)
Count (Min/Max) 0 / 1 0 / Any (typ. 0..2) 1 / Any 1 / Any 1 / Any
CPU 1x 16c (16c) ___ GHz 2x 22c (44c) ___ GHz 2x 20c (40c) 3.0 GHz 2x 20c (40c) 3.0 GHz 2x 16c (32c) 3.3 GHz
Memory 64GB 256GB 1024GB 512GB 256GB
GPU - - 4x GV100 4x GV100 4x GV100
Storage - HDD 2x 4TB [SATA] 2x 4TB [SATA] - - -
Storage - SSD - - 2x 3.8TB [SATA] 2x 3.8TB [SATA] 2x 1.9TB [SATA]
Storage Controller Marvell (internal) Marvell (internal) Marvell (internal) Marvell (internal) Marvell (internal)
Network - 1GbE Internal (4 ports OS) Internal (4 ports OS) External (2 ports OS) External (2 ports OS) External (2 ports OS)
Cables - 1GbE 3 (2 OS + 1 BMC) 3 (2 OS + 1 BMC) 3 (2 OS + 1 BMC) 3 (2 OS + 1 BMC) 2 (1 OS + 1 BMC)
Network - 10GbE 1x 2-port Intel (2 ports) 1x 2-port Intel (2 ports) 1x 2-port Intel (2 ports) 1x 2-port Intel (2 ports) 1x 2-port Intel (2 ports)
Cables - 10GbE 2 cables 2 cables 2 cables 2 cables 1 cable
Network - 100GbIB - - 1x 2-port Mellanox (2 ports) 1x 2-port Mellanox (2 ports) 1x 2-port Mellanox (2 port)
Cables - 100GbIB - - 2 cables 2 cables 1 cable
Compute Node

Get Started Today with Machine
& Deep Learning
40
Build a Data Science Team
Your Developers Can Learn
http://cognitiveclass.ai
Identify a Low Hanging Use Case
Figure Out Data Strategy
Consider Pre-Built AI APIs
Hire Consulting Services
Get Started Today at
www.ibm.biz/poweraideveloper

Software: H2O.ai is a Leader in the 2018 Gartner Data Science
and Machine Learning Platforms Magic Quadrant
• Technology leader with most
completeness of vision
• Recognized for the mindshare,
partner network and status as a
quasi-industry standard for machine
learning and AI
• H2O.ai customers gave the highest
overall score among all the vendors
for sales relationship and account
management, customer support
(onboarding, troubleshooting, etc.)
and overall service and support
Get the
Gartner
Magic
Quadrant
here

IBM Power AI delivers
Deep Learning for Images
Sensor
Log
Transactional
H2O Driverless AI is
Automatic Machine Learning
Image
Software: H2O Driverless AI complements PowerAI & Vision

Thank you, humans.
Cyberdyne Systems Series T-800 Model 101 Terminator
Jeff Boleman, M.S.C.S.
jkbolema@us.ibm.com
@uid0_and_beyond

Benchmark Details
45
Large Model Support benchmark Details
• Hardware: Power AC922; 40 cores (2 x 20c chips), POWER9 with NVLink 2.0; 2.25 GHz, 1024 GB memory, 4xTesla V100 GPU Pegas 1.0. Competitive
stack: 2x Xeon E5-2640 v4; 20 cores (2 x 10c chips) / 40 threads; Intel Xeon E5-2640 v4; 2.4 GHz; 1024 GB memory, 4xTesla V100 GPU, Ubuntu 16.04.
• Chainer: IBM Internal Measurements running 1000 iterations of Enlarged GoogleNet model on Enlarged Imagenet Dataset (2240x2240) .
• Software: Chainverv3 /LMS/Out of Core with CUDA 9 / CuDNN7 with patches found at https://github.com/cupy/cupy/pull/694 and
https://github.com/chainer/chainer/pull/3762
• Caffe Results: IBM Internal Measurements running 1000 iterations of Enlarged GoogleNet model (mini-batch size=5) on Enlarged Imagenet Dataset
(2240x2240) .
• Software: IBM Caffe with LMS Source code: https://github.ibm.com/TUNG/trlcaffe/tree/1.0-ibm-blc-bm-fix-hang+-p9collateral based on the
branch "1.0-ibm-blc-bm-fix-hang+" (base for PowerAI R4) and a PR#5972 from BVLC/Caffe (for supporting cudnn7).

Large AI Models Train
~4 Times Faster
POWER9 Servers with NVLink to GPUs
vs
x86 Servers with PCIe to GPUs
46
3.1 Hours
49 Mins
0
2000
4000
6000
8000
10000
12000
Xeon x86 2640v4 w/ 4x
V100 GPUs
Power AC922 w/ 4x V100
GPUs
Time(secs)
Caffe with LMS (Large Model Support)
Runtime of 1000 Iterations
3.8x Faster
GoogleNet model on Enlarged
ImageNet Dataset (2240x2240)
Detailed Benchmark Information in Back

47
libGLM (C++ / CUDA
Optimized Primitive Lib)
Distributed Training
Logistic Regression Linear Regression
Support Vector
Machines (SVM)
Distributed Hyper-Parameter
Optimization
More Coming Soon
APIs for Popular ML
Frameworks
Snap ML
Distributed GPU-Accelerated Machine Learning Library
(coming
soon)
Snap Machine Learning (ML) Library

Distributed Deep Learning (DDL):
Reduce training time: Days to Hours
Deep Learning has limited scaling to multiple
servers: IBM DDL solves this limitation
1
2
4
8
16
32
64
128
256
4 16 64 256
Speedup
Number of GPUs
Ideal Scaling
DDL Actual Scaling
95%Scaling with
256 GPUS
ResNet-50, ImageNet-1K
Caffe with PowerAI DDL, Running on Minsky (S822Lc)
Power8 System
16 Days 7 Hours
On-Prem Open AI Platform for
Ecosystem Partners
Integrated with ICP for Data, Data Science
Experience (DSX), Intelligent Video
Analytics (IVA), ….
Partnering with Data Store
& AI Software Providers:
H2O, Anaconda, HortonWorks
Ecosystem of SIs (GBS, Tech Mahindra,
TCS, Accenture) to build Client Solutions
using PowerAI
Many More Advantages of PowerAI48

46x faster than previous
record set by Google
Workload: Click-through rate
prediction for advertising
Logistic Regression Classifier in
Snap ML using GPUs vs
TensorFlow using CPU-only
49
Snap ML: Training Time Goes
From An Hour to Minutes
Logistic Regression in Snap ML (with
GPUs) vs TensorFlow (CPU-only)
1.1 Hours
1.53
Minutes
0
20
40
60
80
Google
CPU-only
Snap ML
Power + GPU
Runtime(Minutes)
46x Faster
Dataset: Criteo Terabyte Click Logs
(http://labs.criteo.com/2013/12/download-terabyte-click-logs/)
4 billion training examples, 1 million features
Model: Logistic Regression: TensorFlow vs Snap ML
Test LogLoss: 0.1293 (Google using Tensorflow), 0.1292 (Snap ML)
Platform: 89 CPU-only machines in Google using Tensorflow versus
4 AC922 servers (each 2 Power9 CPUs + 4 V100 GPUs) for Snap ML
Google data from this Google blog
90 x86 Servers
(CPU-only)
4 Power9 Servers
With GPUs

Data
DNN Model
Monitor & Prune
Select Best
Hyperparameters
Job n
Job 2
Job 1
Auto Hyper-Parameter Tuning in PowerAI
IBM Spectrum Conductor with Spark
GPU-Accelerated Power9
Servers
• Data scientists run 100s of jobs with different Hyper-parameters
• Learning rate, Decay rate, Batch size, Optimizers (GradientDecedent, Adadelta, Momentum, RMSProp, ..)
• Auto-Tuner searches for good hyper-parameters by launching 10s of jobs & selecting the best ones
• 3 search approaches: Random, Tree-based Parzen Estimator (TPE), Bayesian
PowerAI Auto-Tuner (DL Insight)

August 2018 / © 2018 IBM CorporationAugust 2018 / © 2018 IBM Corporation 51
Simplicity: Integrated
Platform that Just Works
Curate, Test, and Support
Fast Moving Open Source
Provide Enterprise
Distribution on RedHat
Easy to deploy Enterprise
AI Platform
Ease of Use, Unique
Capabilities
Faster Model
Training Time
Large data & model
support due to NVLink
Acceleration of Analytics
& ML
AutoML: PowerAI Vision
Elastic Training: Scale
GPUs as Required
Faster Training Times in
Single Server
Scalability to 100s of
Servers (Cluster level
Integration)
Leads to Faster Insights
and Better Economics
Platform that Partners
can build on
Software Partners: H2O,
IBM, Anaconda
SIs, Solution Vendors
& Accelerator Partners
Open AI Platform w/
Ecosystem Partners
Power9
CPU
GPU
PowerAI
IBM
SW
ISV
SW
Solution
SIs
Top Reasons to Choose PowerAI

52
Transform & Prep
Data (ETL)
AI Infrastructure Stack
Applications / Micro-Services
AI APIs
(Eg: Watson)
In-House APIs
(Custom models)
Machine & Deep Learning
Libraries & Frameworks
Distributed Computing &
Cluster Orchestration
Data Lake & Data Stores
Segment Specific:
Finance, Retail, Healthcare
Speech, Vision,
NLP, Sentiment
TensorFlow, H2O,
pyTorch, SparkML
Spark, MPI
Hadoop HDFS,
NoSQL DBs
Accelerated
Infrastructure
Accelerated Servers Storage
PowerAI
+DSX

2018-11-05 Intro to AI

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (18)

Similaire à 2018-11-05 Intro to AI

Similaire à 2018-11-05 Intro to AI (20)

Dernier

Dernier (20)

2018-11-05 Intro to AI