Machine Intelligence at Google Scale: Tensor Flow and Cloud Machine Learning: The biggest challenge of Deep Learning technology is the scalability. As long as using single GPU server, you have to wait for hours or days to get the result of your work. This doesn’t scale for production service, so you need a Distributed Training on the cloud eventually. Google has been building infrastructure for training the large scale neural network on the cloud for years, and now started to share the technology with external developers. In this session, we will introduce new pre-trained ML services such as Cloud Vision API and Speech API that works without any training. Also, we will look how TensorFlow and Cloud Machine Learning will accelerate custom model training for 10x – 40x with Google’s distributed training infrastructure.
3. What we’ll cover
What is Neural Network and Deep Learning
Machine Learning use cases at Google services
Externalizing the power with ML APIs
TensorFlow: the open source library for ML
TensorFlow in the Wild
Distributed training and prediction with Cloud ML
22. We need to go deeper neural network
From: Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee et al.
23. From: mNeuron: A Matlab Plugin to Visualize Neurons from Deep Models, Donglai Wei et. al.
25. 25
signal
for Search ranking,
out of hundreds
improvement
to ranking quality
in 2+ years
#3 #1
Search
machine learning for search engines
RankBrain: a deep neural network for search ranking
34. TensorFlow Cloud Machine Learning ML API
Easy-to-Use, for non-ML engineers
Customizable, for Data Scientists
Machine Learning products from Google
35. Image analysis with pre-trained models
No Machine Learning skill required
REST API: receives an image and returns a JSON
General Availability
Cloud Vision API
36. Confidential & ProprietaryGoogle Cloud Platform 36
Faces
Faces, facial landmarks, emotions
OCR
Read and extract text, with
support for > 10 languages
Label
Detect entities from furniture to
transportation
Logos
Identify product logos
Landmarks & Image Properties
Detect landmarks & dominant
color of image
Safe Search
Detect explicit content - adult,
violent, medical and spoof
38. Pre-trained models. No ML skill required
REST API: receives audio and returns texts
Supports 80+ languages
Streaming or non-streaming
Public Beta - cloud.google.com/speech
Cloud Speech API
39. Confidential & ProprietaryGoogle Cloud Platform 39
Features
Automatic Speech Recognition (ASR)
powered by deep learning neural
networking to power your
applications like voice search or
speech transcription.
Recognizes over 80
languages and variants
with an extensive
vocabulary.
Returns partial
recognition results
immediately, as they
become available.
Filter inappropriate
content in text results.
Audio input can be captured by an application’s
microphone or sent from a pre-recorded audio
file. Multiple audio file formats are supported,
including FLAC, AMR, PCMU and linear-16.
Handles noisy audio from many
environments without requiring
additional noise cancellation.
Audio files can be uploaded in the
request and, in future releases,
integrated with Google Cloud
Storage.
Automatic Speech Recognition Global Vocabulary Inappropriate Content
Filtering
Streaming Recognition
Real-time or Buffered Audio Support Noisy Audio Handling Integrated API
41. Pre-trained models. No ML skill required
REST API: receives text and returns analysis results
Supports English, Spanish and Japanese
Public Beta - cloud.google.com/natural-language
Cloud Natural Language API
42. Confidential & ProprietaryGoogle Cloud Platform 42
Features
Extract sentence, identify parts of
speech and create dependency parse
trees for each sentence.
Identify entities and label by types such
as person, organization, location, events,
products and media.
Understand the overall sentiment of a
block of text.
Syntax Analysis Entity Recognition
Sentiment Analysis
45. Google's open source library for
machine intelligence
tensorflow.org launched in Nov 2015
Used by many production ML projects
What is TensorFlow?
46. Sharing our tools with researchers and developers
around the world
repository
for “machine learning”
category on GitHub
#1
Released in Nov. 2015
From: http://deliprao.com/archives/168
47. Before
Hire Data Scientists
↓
Understand the math model
↓
Impl with programming code
↓
Train with single GPU
↓
Build a GPU cluster
↓
Train with the GPU cluster
↓
Build a prediction server
or Impl mobile/IoT prediction
After
Easy network design and impl
↓
Train with single machine
↓
Train on the cloud
↓
Prediction on the cloud
or mobile/IoT devices
many people
stuck here
48. # define the network
import tensorflow as tf
x = tf.placeholder(tf.float32, [None, 784])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x, W) + b)
# define a training step
y_ = tf.placeholder(tf.float32, [None, 10])
xent = -tf.reduce_sum(y_*tf.log(y))
step =
tf.train.GradientDescentOptimizer(0.01).minimize(xent)
62. The Challenge: Computing Power
DNN requires large training datasets
Large models doesn't fit into a GPU
Requires try-and-errors to find the
best design, configs and params
↓
Need to spend a few days or
weeks to finish a training
63. GPUs run at nanoseconds
GPU cluster needs microsec network
69. ● CPU/GPU scheduling
● Communications
○ Local, RPC, RDMA
○ 32/16/8 bit quantization
● Cost-based optimization
● Fault tolerance
Distributed Systems for Large Neural Network
70. What's the scalability of Google Brain?
"Large Scale Distributed Systems for Training Neural
Networks", NIPS 2015
○ Inception / ImageNet: 40x with 50 GPUs
○ RankBrain: 300x with 500 nodes
71. Fully managed distributed training and prediction
Supports custom TensorFlow graphs
Integrated with Cloud Dataflow and Cloud Datalab
Limited Preview - cloud.google.com/ml
Cloud Machine Learning (Cloud ML)
72. 7272
Ready to use Machine
Learning models
Use your own data to
train models
Cloud
Vision API
Cloud
Speech API
Cloud
Translate API
Cloud Machine Learning
Develop - Model - Test
Google
BigQuery
Stay
Tuned….
Cloud
Storage
Cloud
Datalab
NEW
Alpha
GA BetaGA
Alpha
GA
GA
73. Tensor Processing Unit
ASIC for TensorFlow
Designed by Google
10x better perf / watt
latency and efficiency
bit quantization
76. Links & Resources
Large Scale Distributed Systems for Training Neural Networks, Jeff Dean and
Oriol Vinals
Cloud Vision API: cloud.google.com/vision
Cloud Speech API: cloud.google.com/speech
TensorFlow: tensorflow.org
Cloud Machine Learning: cloud.google.com/ml
Cloud Machine Learning: demo video