Contenu connexe Similaire à Introduction To TensorFlow | Deep Learning Using TensorFlow | TensorFlow Tutorial | Edureka (20) Introduction To TensorFlow | Deep Learning Using TensorFlow | TensorFlow Tutorial | Edureka1. Agenda
▪ Difference Between Machine Learning and Deep Learning
▪ What is Deep Learning?
▪ What is TensorFlow?
▪ TensorFlow Data Structures
▪ TensorFlow Use-Case
2. Agenda
▪ Difference Between Machine Learning and Deep Learning
▪ What is Deep Learning?
▪ What is TensorFlow?
▪ TensorFlow Data Structures
▪ TensorFlow Use-Case
3. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Machine Learning vs Deep Learning
Let’s see what are the differences between Machine Learning and Deep Learning
4. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Machine Learning vs Deep Learning
Machine Learning Deep Learning
High performance on less data Low performance on less data
Deep Learning Performance
Machine Learning Performance
5. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Machine Learning vs Deep Learning
Machine Learning Deep Learning
Can work on low end machines Requires high end machines
6. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Machine Learning vs Deep Learning
Machine Learning Deep Learning
Features need to be hand-coded
as per the domain and data type
Tries to learn high-level features
from data
7. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
What is Deep Learning?
Now is the time to understand what exactly is Deep Learning?
8. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
What is Deep Learning?
Input Layer
Hidden Layer 1
Hidden Layer 2
Output Layer
A collection of statistical machine learning techniques used to learn feature hierarchies often based on
artificial neural networks
9. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
What are Tensors?
Let’s see what are Tensors?
10. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
What Are Tensors?
Tensors are the standard way of representing data in TensorFlow (deep learning).
Tensors are multidimensional arrays, an extension of two-dimensional tables (matrices) to data
with higher dimension.
Tensor of
dimension[1]
Tensor of
dimensions[2]
Tensor of
dimensions[3]
11. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Tensors Rank
Rank Math Entity Python Example
0 Scalar (magnitude
only)
s = 483
1 Vector (magnitude
and direction)
v = [1.1, 2.2, 3.3]
2 Matrix (table of
numbers)
m = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
3 3-Tensor (cube of
numbers)
t =
[[[2], [4], [6]], [[8], [10], [12]], [[14], [16], [18
]]]
n n-Tensor (you get
the idea)
....
12. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Tensor Data Types
In addition to dimensionality Tensors have different data types as well, you can assign any one of
these data types to a Tensor
13. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
What Is TensorFlow?
Now, is the time explore TensorFlow.
14. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
What Is TensorFlow?
TensorFlow is a Python library used to implement deep networks.
In TensorFlow, computation is approached as a dataflow graph.
3.2 -1.4 5.1 …
-1.0 -2 2.4 …
… … … …
… … … …
Tensor Flow
Matmul
W X
Add
Relu
B
Computational
Graph
Functions
Tensors
15. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
TensorFlow Code-Basics
Let’s understand the fundamentals of TensorFlow
16. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
TensorFlow Code-Basics
TensorFlow core programs consists of two discrete sections:
Building a computational graph Running a computational graph
A computational graph is a series of TensorFlow
operations arranged into a graph of nodes
17. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
TensorFlow Building And Running A Graph
Building a computational graph Running a computational graph
import tensorflow as tf
node1 = tf.constant(3.0, tf.float32)
node2 = tf.constant(4.0)
print(node1, node2)
Constant nodes
sess = tf.Session()
print(sess.run([node1, node2]))
To actually evaluate the nodes, we must run
the computational graph within a session.
As the session encapsulates the control and
state of the TensorFlow runtime.
18. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Tensorflow Example
a
5.0
Constimport tensorflow as tf
# Build a graph
a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
# Launch the graph in a session
sess = tf.Session()
# Evaluate the tensor 'C'
print(sess.run(c))
Computational Graph
19. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Tensorflow Example
a
b
5.0
6.0
Const
Constimport tensorflow as tf
# Build a graph
a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
# Launch the graph in a session
sess = tf.Session()
# Evaluate the tensor 'C'
print(sess.run(c))
Computational Graph
20. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Tensorflow Example
a
b c
5.0
6.0
Const Mul
Constimport tensorflow as tf
# Build a graph
a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
# Launch the graph in a session
sess = tf.Session()
# Evaluate the tensor 'C'
print(sess.run(c))
Computational Graph
21. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Tensorflow Example
a
b c
5.0
6.0
Const Mul
30.0
Constimport tensorflow as tf
# Build a graph
a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
# Launch the graph in a session
sess = tf.Session()
# Evaluate the tensor 'C'
print(sess.run(c))
Running The Computational Graph
22. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Graph Visualization
23. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Graph Visualization
For visualizing TensorFlow graphs, we use TensorBoard.
The first argument when creating the FileWriter is an output directory name, which will be created
if it doesn't exist.
File_writer = tf.summary.FileWriter('log_simple_graph', sess.graph)
TensorBoard runs as a local web app, on port 6006. (this
is default port, “6006” is “ ” upside-down.)oo
tensorboard --logdir = “path_to_the_graph”
Execute this command in the cmd
24. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Constants, Placeholders and Variables
Let’s understand what are constants, placeholders and variables
25. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Constant
One type of a node is a constant. It takes no inputs, and it outputs a value
it stores internally.
import tensorflow as tf
node1 = tf.constant(3.0, tf.float32)
node2 = tf.constant(4.0)
print(node1, node2)
Constant nodes
Constant
Placeholder
Variable
26. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Constant
One type of a node is a constant. It takes no inputs, and it outputs a value
it stores internally.
import tensorflow as tf
node1 = tf.constant(3.0, tf.float32)
node2 = tf.constant(4.0)
print(node1, node2)
Constant nodes
Constant
Placeholder
Variable
What if I want the
graph to accept
external inputs?
27. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Placeholder
Constant
Placeholder
Variable
A graph can be parameterized to accept external inputs, known as placeholders.
A placeholder is a promise to provide a value later.
28. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Placeholder
Constant
Placeholder
Variable
A graph can be parameterized to accept external inputs, known as placeholders.
A placeholder is a promise to provide a value later.
How to modify the
graph, if I want new
output for the same
input ?
29. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Variable
Constant
Placeholder
Variable
To make the model trainable, we need to be able to modify the graph to get
new outputs with the same input. Variables allow us to add trainable
parameters to a graph
30. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Let Us Now Create A Model
31. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Simple Linear Model
import tensorflow as tf
W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W * x + b
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
print(sess.run(linear_model, {x:[1,2,3,4]}))
We've created a model, but we
don't know how good it is yet
32. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
How To Increase The Efficiency Of The Model?
Calculate the loss
Model
Update the Variables
Repeat the process until the loss becomes very small
A loss function measures how
far apart the current model is
from the provided data.
33. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Calculating The Loss
In order to understand how good the Model is, we should know the loss/error.
To evaluate the model on training data, we need a y i.e. a
placeholder to provide the desired values, and we need to
write a loss function.
We'll use a standard loss model for linear regression.
(linear_model – y ) creates a vector where each element is
the corresponding example's error delta.
tf.square is used to square that error.
tf.reduce_sum is used to sum all the squared error.
y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x:[1,2,3,4], y:[0,-1,-2,-3]}))
34. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Reducing The Loss
Optimizer modifies each variable according to the magnitude of the derivative of loss with
respect to that variable. Here we will use Gradient Descent Optimizer
How Gradient Descent Actually
Works?
Let’s understand this
with an analogy
35. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Reducing The Loss
• Suppose you are at the top of a mountain, and you have to reach a lake which is at the lowest
point of the mountain (a.k.a valley).
• A twist is that you are blindfolded and you have zero visibility to see where you are headed. So,
what approach will you take to reach the lake?
36. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Reducing The Loss
• The best way is to check the ground near you and observe where the land tends to descend.
• This will give an idea in what direction you should take your first step. If you follow the
descending path, it is very likely you would reach the lake.
Consider the length of the step as learning rate
Consider the position of the hiker as weight
Consider the process of climbing down
the mountain as cost function/loss
function
37. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Reducing The Loss
Global Cost/Loss
Minimum
Jmin(w)
J(w)
Let us
understand the
math behind
Gradient
Descent
38. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Batch Gradient Descent
The weights are updated
incrementally after each
epoch. The cost function J(⋅),
the sum of squared errors
(SSE), can be written as:
39. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Batch Gradient Descent
The weights are updated
incrementally after each
epoch. The cost function J(⋅),
the sum of squared errors
(SSE), can be written as:
The magnitude and direction
of the weight update is
computed by taking a step in
the opposite direction of the
cost gradient
40. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Batch Gradient Descent
The weights are updated
incrementally after each
epoch. The cost function J(⋅),
the sum of squared errors
(SSE), can be written as:
The magnitude and direction
of the weight update is
computed by taking a step in
the opposite direction of the
cost gradient
The weights are then updated
after each epoch via the
following update rule:
41. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Batch Gradient Descent
The weights are updated
incrementally after each
epoch. The cost function J(⋅),
the sum of squared errors
(SSE), can be written as:
The magnitude and direction
of the weight update is
computed by taking a step in
the opposite direction of the
cost gradient
The weights are then updated
after each epoch via the
following update rule:
Here, Δw is a vector that
contains the weight
updates of each weight
coefficient w, which are
computed as follows:
42. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Reducing The Loss
Suppose, we want to find the best parameters (W) for our learning algorithm. We can apply the
same analogy and find the best possible values for that parameter. Consider the example below:
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
sess.run(init)
for i in range(1000):
sess.run(train, {x:[1,2,3,4], y:[0,-1,-2,-3]})
print(sess.run([W, b]))
43. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
TensorFlow Use-Case
44. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Long Short Term Memory Networks Use-Case
We will feed a LSTM with correct sequences from the text of 3 symbols as inputs and 1 labeled
symbol, eventually the neural network will learn to predict the next symbol correctly
had a general
LSTM
cell
Council
Prediction
label
vs
inputs
LSTM cell with
three inputs and
1 output.
45. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Long Short Term Memory Networks Use-Case
long ago , the mice had a general council to consider what measures
they could take to outwit their common enemy , the cat . some said
this , and some said that but at last a young mouse got up and said he
had a proposal to make , which he thought would meet the case . you
will all agree , said he , that our chief danger consists in the sly and
treacherous manner in which the enemy approaches us . now , if we
could receive some signal of her approach , we could easily escape from
her . i venture , therefore , to propose that a small bell be procured , and
attached by a ribbon round the neck of the cat . by this means we
should always know when she was about , and could easily retire while
she was in the neighborhood . this proposal met with general applause ,
until an old mouse got up and said that is all very well , but who is to
bell the cat ? the mice looked at one another and nobody spoke . then
the old mouse said it is easy to propose impossible remedies .
How to
train the
network?
A short story from Aesop’s Fables
with 112 unique symbols
46. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Long Short Term Memory Networks Use-Case
A unique integer value is assigned to each symbol because
LSTM inputs can only understand real numbers.
20 6 33
LSTM
cell
LSTM cell with
three inputs and
1 output.
had a general
.01 .02 .6 .00
37
37
vs
Council
Council
112-element
vector
47. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Session In A Minute
Machine Learning vs Deep Learning What is Deep Learning? What is TensorFlow?
TensorFlow Code-Basics Simple Linear Model TensorFlow Use-Case