행사 사진 : https://goo.gl/LB6eEp

1. Machine Intelligence at Google Scale
ML APIs, TensorFlow and Cloud ML

2. +Kazunori Sato
@kazunori_279
Kaz Sato
Staff Developer Advocate
Tech Lead for Data & Analytics
Cloud Platform, Google Inc.

3. What we’ll cover
What is Neural Network and Deep Learning
Machine Learning use cases at Google services
Externalizing the power with ML APIs
TensorFlow: the open source library for ML
TensorFlow in the Wild
Distributed training and prediction with Cloud ML

4. What is Neural Network
and Deep Learning

5. Neural Network is a function that can learn

6. xn
> b?
w1
wn
x2
x1
Inspired by the behavior of biological neurons

7. How do you
classify them?

8. weights
bias
(threshold)
Programmers need to specify the parameters

9. Let’s see how neural network solves the problem

10. The computer tries to find
the best parameters
A neuron classifies a data point into two kinds

11. Gradient Descent: adjusting the params
gradually to reduce errors

12. How do you
classify them?

13. What we see What the computer “sees”

14. 28 x 28 gray scale image =
784 numbers

15. input vector
(pixel data)
output vector
(probability)

16. How do you
classify them?

17. More neurons = More features to extract

18. Hidden Layers:
mapping inputs to
a feature space,
classifying with
a hyperplane
From: Neural Networks, Manifolds, and Topology, colah's blog

19. From: Neural Networks, Manifolds, and Topology, colah's blog

20. How about this?

21. More hidden layers = More hierarchies of features

22. How about this?

23. We need to go deeper neural network
From: Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee et al.

24. From: mNeuron: A Matlab Plugin to Visualize Neurons from Deep Models, Donglai Wei et. al.

25. How CNN
works

26. Machine Learning use cases
at Google services

27. 27
signal
for Search ranking,
out of hundreds
improvement
to ranking quality
in 2+ years
#3 #1
Search
machine learning for search engines
RankBrain: a deep neural network for search ranking

29. WaveNet by
DeepMind

30. 30
[glacier]
Google Photos
30

31. 31
Smart reply
in Inbox by Gmail
10%
of all responses
sent on mobile

32. Google Translate with Neural Machine Translation

33. Saved Data Center cooling energy for 40%
Improved Power Usage Effectiveness (PUE) for 15%

34. 34
Android
Apps
Gmail
Maps
Photos
Speech
Search
Translation
YouTube
and many others ...
Used across products:
2012 2013 2014 2015
Deep Learning usage at Google

35. Externalizing the power
with ML APIs

36. TensorFlow Cloud Machine Learning ML API
Easy-to-Use, for non-ML engineers
Customizable, for Data Scientists
Machine Learning products from Google

37. Image analysis with pre-trained models
No Machine Learning skill required
REST API: receives an image and returns a JSON
$1.50 per 1,000 units
GA - cloud.google.com/vision
Cloud Vision API

38. Confidential & ProprietaryGoogle Cloud Platform 38
Faces
Faces, facial landmarks, emotions
OCR
Read and extract text, with
support for > 10 languages
Label
Detect entities from furniture to
transportation
Logos
Identify product logos
Landmarks & Image Properties
Detect landmarks & dominant
color of image
Safe Search
Detect explicit content - adult,
violent, medical and spoof

39. 3939
Demo

40. Pre-trained models. No ML skill required
REST API: receives audio and returns texts
Supports 80+ languages
Streaming or non-streaming
Public Beta - cloud.google.com/speech
Cloud Speech API

41. Confidential & ProprietaryGoogle Cloud Platform 41
Features
Automatic Speech Recognition
(ASR) powered by deep learning
neural networking to power your
applications like voice search or
speech transcription.
Recognizes over 80
languages and variants
with an extensive
vocabulary.
Returns partial
recognition results
immediately, as they
become available.
Filter inappropriate
content in text results.
Audio input can be captured by an application’s
microphone or sent from a pre-recorded audio
file. Multiple audio file formats are supported,
including FLAC, AMR, PCMU and linear-16.
Handles noisy audio from many
environments without requiring
additional noise cancellation.
Audio files can be uploaded in the
request and, in future releases,
integrated with Google Cloud
Storage.
Automatic Speech Recognition Global Vocabulary Inappropriate Content
Filtering
Streaming Recognition
Real-time or Buffered Audio Support Noisy Audio Handling Integrated API

42. 4242
Demo

43. Pre-trained models. No ML skill required
REST API: receives text and returns analysis results
Supports English, Spanish and Japanese
GA - cloud.google.com/natural-language
Cloud Natural Language API

44. Confidential & ProprietaryGoogle Cloud Platform 44
Features
Extract sentence, identify parts of
speech and create dependency parse
trees for each sentence.
Identify entities and label by types such
as person, organization, location, events,
products and media.
Understand the overall sentiment of a
block of text.
Syntax Analysis Entity Recognition
Sentiment Analysis

45. 4545
Demo

46. Pre-trained models. No ML skill required
REST API: receives text and returns translated text
8 languages: English to Chinese, French, German,
Japanese, Korean, Portuguese, Spanish, Turkish
Public Beta - cloud.google.com/translate
Cloud Translation API Premium

47. 4747
Demo

48. TensorFlow:
An open source library for
Machine Intelligence

49. Google's open source library for
machine intelligence
tensorflow.org launched in Nov 2015
Used by many production ML projects
What is TensorFlow?

50. 50
Sharing our tools with researchers and developers
around the world
repository
for “machine learning”
category on GitHub
#1
Released in Nov.
2015
From: http://deliprao.com/archives/168

51. 51
Before
Hire Data Scientists
↓
Understand the math model
↓
Impl with programming code
↓
Train with single GPU
↓
Build a GPU cluster
↓
Train with the GPU cluster
↓
Build a prediction server
or Impl mobile/IoT prediction
After
Easy network design and impl
↓
Train with single machine
↓
Train on the cloud
↓
Prediction on the cloud
or mobile/IoT devices
many people
stuck here

52. # define the network
import tensorflow as tf
x = tf.placeholder(tf.float32, [None, 784])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x, W) + b)
# define a training step
y_ = tf.placeholder(tf.float32, [None, 10])
xent = -tf.reduce_sum(y_*tf.log(y))
step =
tf.train.GradientDescentOptimizer(0.01).minimize(xent)

53. TensorBoard: visualization tool

54. Portable and Scalable
Training on:
Mac/Windows
GPU server
GPU cluster / Cloud
Prediction on:
Android and iOS
RasPi and TPU

55. Distributed Training with TensorFlow

56. TensorFlow in the Wild
(or democratization of deep learning)

58. TensorFlow powered Cucumber Sorter
From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/

59. TensorFlow powered Cucumber Sorter

60. TensorFlow
powered
Fried Chicken
Nugget Server
From: http://www.rt-net.jp/karaage1/

61. TV popstar face
generator with DCGAN
From: http://memo.sugyan.com/entry/2016/10/12/084751

62. TensorFlow +
Drones
for counting trucks
From: http://www.brainpad.co.jp/news/2016/09/02/3454

63. TensorFlow＋
RasPi for
sorting garbages
From: https://techcrunch.com/2016/09/13/auto-
trash-sorts-garbage-automatically-at-the-techcrunch-
disrupt-hackathon/

64. Autonomous
Driving
of RasPi car
with Inception 3
on TensorFlow
From: https://github.com/zxzhijia/GoPiGo-
Driven-by-Tensorflow

65. From: http://otoro.net/
Generative Arts with TensorFlow

66. Distributed Training and
Prediction with Cloud ML

67. From: Andrew Ng
The Bigger, The Better

68. The Challenge: Computing Power
DNN requires large training datasets
Large models doesn't fit into a GPU
Requires try-and-errors to find the
best design, configs and params
↓
Need to spend a few days or
weeks to finish a training

69. GPUs run at nanoseconds
GPU cluster needs microsec network

71. Enterprise
Google Cloud is
The Datacenter as a Computer

72. Jupiter network
10 GbE x 100 K = 1 Pbps
Consolidates servers with
microsec latency

73. Borg
No VMs, pure containers
10K - 20K nodes per Cell
DC-scale job scheduling
CPUs, mem, disks and IO

74. Distributed Training
with TensorFlow
by data parallelism
split data,
share model

75. CPU/GPU scheduling
Communications
Local, RPC, RDMA
32/16/8 bit quantization
Cost-based optimization
Distributed Systems for Large Neural Network

76. Distributed Training with TensorFlow
on Google Cloud
"Large Scale Distributed Systems for Training Neural
Networks", NIPS 2015
Inception / ImageNet: 40x with 50 GPUs
RankBrain: 300x with 500 nodes

77. Fully managed distributed training and prediction
Supports custom TensorFlow graphs
HyperTune for hyper-parameter tuning automation
Integrated with Cloud Dataflow and Cloud Datalab
Public Beta - cloud.google.com/ml
Cloud Machine Learning (Cloud ML)

78. Cloud ML at Work: AUCNET
The largest real-time car auction service in Japan
For 30K used car dealers
The auction volume overs $3.7B every year
Problem: auction entry is time consuming task for dealers
Classifying parts of car for thousands of photos
Identifying the exact car model

79. Solution: Custom Image Classification with TensorFlow/Cloud ML
Used 5,000 training images for 500 car models
Inception v3 + Transfer Learning
Cloud ML: increased training performance for 6x faster

80. Predicting "large loss" cases in car insurance
Old method (Random Forest): 38% accuracy
New method (TensorFlow): 73% accuracy
A Global Insurance Firm

81. 8181
Demo

82. 8282
Ready to use Machine
Learning models
Use your own data to
train models
Cloud
Vision API
Cloud
Speech API
Cloud
Translation API
Cloud Machine Learning
Develop - Model - Test
Google
BigQuery
Cloud
Storage
Cloud
Datalab
Beta
GA BetaGA
Beta
GA
GA GA
Cloud Natural
Language API

83. Tensor Processing Unit
ASIC for TensorFlow
Designed by Google
10x better perf / watt
latency and efficiency
bit quantization

84. TPU on Production
RankBrain
AlphaGo
Google Photos
Speech
and more

85. Thank you!