Deep Learning Tutorial | Deep Learning Tutorial for Beginners | Neural Networks | Edureka

Deep Learning Using TensorFlow

Copyright © 2017, edureka and/or its affiliates. All rights reserved.
Agenda
• What Is Deep Learning?
• How Deep Learning Works?
• Single Layer Perceptron (Early Deep Learning Models)
• Single Layer Perceptron Examples
• Limitations Of Single Layer Perceptron
• Multi Layer Perceptron
• Multi Layer Perceptron Examples

Module
1
Module
2
Module
3
Module
4
Module
5
Module
6
Module
7
Course Outline
Introduction to Deep Learning
Fundamentals of Neural Networks
Fundamentals of Deep Networks
Introduction to TensorFlow
Convolutional Neural Networks
Recurrent Neural Networks
RBM and Autoencoders

Session In A Minute
Why Artificial Intelligence ? What is Artificial Intelligence? Subsets Of Artificial Intelligence
Machine Learning to Deep Learning What is Deep Learning? Deep Learning Applications

What Is Deep Learning?
Dog
Cat
Dog
Cat
How will a
machine
identify
Humans have seen so many dogs
and cats, they can identify the
difference between the two

Training
Input Image
Feature
extraction
Machine
Learning Model
Predict what
the object is

Deep Learning skips the manual steps of extracting features, you can directly feed images to the deep
learning algorithm, which then predicts the object.
Input Image
Learned Features
95%
3%
2%
Predict what
the object is

How It Works?

How Deep Learning Works?
Deep learning is a form of machine learning that uses a model of computing that's very much inspired
by the structure of the brain, so lets understand that first.
Neuron
Dendrite: Receives signals from
other neurons
Cell Body: Sums all the inputs
Axon: It is used to transmit
signals to the other cells

Why We Need Artificial Neurons?
We can understand this with an example

Why We Need Artificial Neuron?
I have a dataset about flowers.
It includes:
 Sepal Length
 Sepal Width
 Petal Length
 Petal Width
I want to classify the type of flower on the
basis of this dataset.

We Need A System To Separate The
Two Species

With the help of an Artificial Neuron we can separate the two species of flowers
Artificial Neuron

Perceptron Learning Algorithm
To get started, I'll explain a type of artificial neuron called a perceptron.
X1
X2
X3
Xn
W1
W2
W3
Wn
Transfer
Function
Activation
Function
Schematic for a neuron in a neural net

Initialize the weights
and threshold
1
Wj – initial Weight

Provide the input and
calculate the output
and threshold
1 2
X – Input
Y - Output

Provide the input and
calculate the output
and threshold
Update the weights Repeat step 2 and 3
1 2 3 4
Wj (t+1) = Wj (t) + n (d-y) x
Wj (t+1) – Updated Weight
Wj (t) – Old Weight
d – Desired Output
y – Actual Output
x - Input

Types Of Activation Functions

Activation Function
Step Function Sigmoid Function
Sign Function

Applications Of Single Layer
Perceptron

Applications
It is used to classify any linearly separable set of inputs.
Error = 2 Error = 1 Error = 0

Applications
It can be used to implement Logic Gates.
OR
X1 X2 Y
0 0 0
0 1 1
1 0 1
1 1 1
AND
X1 X2 Y
0 0 0
0 1 0
1 0 0
1 1 1

Applications
OR
X1 X2 Y
0 0 0
0 1 1
1 0 1
1 1 1
t = 0.5
W = 1
W = 1
X1 X2
0 0
0 1
1 0
1 1
1
0 1
X1
X2

Applications
X1 X2 Y
0 0 0
0 1 0
1 0 0
1 1 1
t = 1.5
W = 1
W = 1
X1 X2
0 0
0 1
1 0
1 1
1
0 1
X1
X2
AND

Single Layer Perceptron – Use Case
Let's now see how to implement a single layer neural network for an image classification problem
using TensorFlow.
MNIST Dataset
10,000 testing images55,000 training images

What Is TensorFlow?

What Is Tensorflow?
 Tensors are the standard way of representing data in deep learning.
 Tensors are just multidimensional arrays, an extension of two-dimensional tables (matrices) to data
with higher dimension.
Tensor of
dimensions[6]
Tensor of
dimensions[6,4]
Tensor of
dimensions[6,4,2]

What Is Tensorflow?
In Tensorflow, computation is approached as a dataflow graph
Tensor Flow
3.2 -1.4 5.1 …
-1.0 -2 2.4 …
… … … …
… … … …
Matmul
W X
Add
Relu
B

TensorFlow Code Basics

TensorFlow Core programs consists of two discrete sections:
Building a computational graph Running a computational graph

A computational graph is a series of TensorFlow operations arranged into a graph of nodes
 Let's build a simple computational graph.
 Each node takes zero or more tensors as inputs and produces a tensor as an output.
import tensorflow as tf
node1 = tf.constant(3.0, tf.float32)
node2 = tf.constant(4.0)
print(node1, node2)
Building a computational graph
Constant nodes

 To actually evaluate the nodes, we must run the computational graph within a session.
 A session encapsulates the control and state of the TensorFlow runtime.
sess = tf.Session()
print(sess.run([node1, node2])) Running a computational graph
It always produces a constant result

Tensorflow Example
a = tf.constant(5)
b = tf.constant(2)
c = tf.constant(3)
d = tf.multiply(a,b)
e = tf.add(c,b)
f = tf.subtract(d,e)
sess = tf.Session()
outs = sess.run(f)
sess.close()
print ("outs = {}".format(outs))
c
b
a
e
d f
3
2
2
5
Const
Const
Const
Add
Mul
5
10 5
Sub

 A graph can be parameterized to accept external inputs, known as placeholders.
 A placeholder is a promise to provide a value later.
a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
adder_node = a + b
sess = tf.Session()
print(sess.run(adder_node, {a: [1,3], b: [2, 4]}))
Placeholder

 To make the model trainable, we need to be able to modify the graph to get new outputs with the
same input. Variables allow us to add trainable parameters to a graph.
W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W * x + b
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
print(sess.run(linear_model, {x:[1,2,3,4]}))
We've created a model, but we
don't know how good it is yet

What Is A Computational Graph?
A loss function measures how far apart the current model is from the provided data.
To evaluate the model on training data, we need a y i.e.
a placeholder to provide the desired values, and we
need to write a loss function.
We'll use a standard loss model for linear regression.
(linear_model – y ) creates a vector where each
element is the corresponding example's error delta.
tf.square is used to square that error.
tf.reduce_sum is used to sum all the squared error.
y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x:[1,2,3,4], y:[0,-1,-2,-3]}))

TensorFlow provides optimizers that slowly change each variable in order to minimize the loss function
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
sess.run(init)
for i in range(1000):
sess.run(train, {x:[1,2,3,4], y:[0,-1,-2,-3]})
print(sess.run([W, b]))

Limitations Of Sigle Layer
Perceptron

Limitations Of Single Layer Perceptron
Let us understand this with an example:
How can I implement an XOR gate using Single Layer Perceptron?
X1 X2 Y
0 0 0
0 1 1
1 0 1
1 1 0
XOR

Limitations Of Single Layer Perceptron
Now, how will I separate the two set of outputs?
Can we use
two neurons?
For Solving this problem, a Multilayer Perceptron with
backpropagation can be used

Multilayer Perceptron
A Multi-layer Perceptron has the same structure of a single layer perceptron but with one or more
hidden layers and is thus considered a deep neural network.
𝑥1
𝑥2
𝑥3
𝑦1
𝑦2
𝑦3
𝑧
bias
bias
𝑥1
𝑥2
𝑥 𝑛
𝑤1
𝑤2
𝑤 𝑛
𝒇(𝒔)𝑺
Summation:
𝑺 = 𝑤𝑖 ∗ 𝑥1
𝑖 = 1
𝑛
Transformation:

How It Works?
• The weights between the units are the primary means of long-term information storage in neural
networks
• Updating the weights is the primary way the neural network learns new information
Class1
Class2
A set of inputs is passed to the first hidden layer, the activations from that layer are passed to the next
layer and so on, until you reach the output layer

What Is Backpropagation?
Leads generated from
various sources
Classify the leads on
the basis of priority
Input Layer
Hidden Layer
Output Layer

What Is Backpropagation?
In order to classify the leads on the basis of priorities, we need to provide the maximum weight to
the most important lead.
For that we can calculate the difference between the actual output and the desired output.
According to that difference we can update the weights.
The Backpropagation algorithm is a supervised learning method for Multilayer Perceptron.

Backpropagation – Learning Algorithm

i1
i2
h1
h2 o2
o1
1 1
b1 .35 b2 .60
.10
.05
.15 w1
.20 w2
.25 w3
.55 w8
.50 w7
.45 w6
.40 w5
.01
.99..30 w4
Consider the following neural network with:
• two inputs
• two hidden neurons
• two output neurons
• two biases

net h1 = w1*i1 + w2*i2 + b1*1 net h1 = 0.15*0.05 + 0.2*0.1 + 0.35*1 = 0.3775
Net Input For h1:
Output Of h1:
out h1 = 1/1+e-net h1
1/1+e = 0.593269992
.3775
Output Of h2:
out h2 = 0.596884378

We repeat this process for the output layer neurons, using the output from the hidden layer
neurons as inputs.
Output For o1:
net o1 = w5*out h1 + w6*out h2 + b2*1 0.4*0.593269992 + 0.45*0.596884378 + 0.6*1 = 1.105905967
Out o1 = 1/1+e-net o1
1/1+e = 0.75136507-1.105905967
Output For o2:
Out o2 = 0.772928465

Error For o1:
E o1 = Σ1/2 𝑡𝑎𝑟𝑔𝑒𝑡 − 𝑜𝑢𝑡𝑝𝑢𝑡 2 ½ (0.01 – 0.75136507)2
= 0.274811083
Error For o2:
E o2 = 0.023560026
Total Error:
Etotal = E o1 + E o2 0.274811083 + 0.023560026 = 0.298371109

Update each of the weights in the network so that they cause the actual output to be closer the target
output.
Consider w5, we will calculate the change in total error w.r.t w5:
𝛿𝐸𝑡𝑜𝑡𝑎𝑙
𝛿𝑤5
𝛿𝑤5
=
𝛿𝑜𝑢𝑡 𝑜1
*
𝛿𝑛𝑒𝑡 𝑜1
𝛿𝑤5
* net o1 out o1 E totalout h1
1
out h2 w5
w6
b2

How much does the total error change with respect to the output?
Etotal = 1/2(target o1 – out o1)2
+ 1/2(target o2 − out o2)2
= -(target o1 – out o1) = -(0.01 – 0.75136507) = 0.74136507
How much does the output o1 change w.r.t its total net input?
out o1 = 1/1+e−𝑛𝑒𝑡𝑜1
= out o1 (1 - out o1) = 0.75136507 (1 – 0.75136507) = 0.186815602

How much does the total net input of o1 changes w.r.t w5?
net o1 = w5 * out h1 + w6 * out h2 + b2 * 1
𝛿𝑤5
= 1 * out h1 𝑤5(1−1)
+ 0 + 0 = 0.593269992
Putting all these values together
𝛿𝑤5
=
*
𝛿𝑤5
* 0.082167041

Decrease The Error:
w5+
= w5 – n
w5+
= 0.4 – 0.5 * 0.082167041
𝛿𝑤5
Similarly, we can calculate the other weights as well.
Updated w5 0.35891648

Multilayer Perceptron Use-Case
We will take the same MNIST dataset. By using Multilayer Perceptron the efficiency
can be increased.
Input Image
(28x28)
Filter Weights
(5x5 pixels)
(14x14 pixels)
(16 channels)
(7x7 pixels)
(36 channels)
Filter-Weights
(5x5 pixels)
16 of these …
0.0
0.0
0.1
0.0
0.0
0.1
0.0
0.8
0.0
0.0
0
1
2
3
4
5
6
7
8
9
Fully-Connected
Layer
Output
Layer Class
Convolutional Layer 1
Convolutional Layer 2
(128 features) (10 features)

Multi-Layer Perceptron
What Is Deep Learning How Deep Learning Works? Single Layer Perceptron
Introduction To TensorFlow Limitations Of Sigle Layer
Perceptron
Multi Layer Perceptron

Deep Learning Tutorial | Deep Learning Tutorial for Beginners | Neural Networks | Edureka

Deep Learning Tutorial | Deep Learning Tutorial for Beginners | Neural Networks | Edureka

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Deep Learning Tutorial | Deep Learning Tutorial for Beginners | Neural Networks | Edureka

Similaire à Deep Learning Tutorial | Deep Learning Tutorial for Beginners | Neural Networks | Edureka (20)

Plus de Edureka!

Plus de Edureka! (20)

Dernier

Dernier (20)

Deep Learning Tutorial | Deep Learning Tutorial for Beginners | Neural Networks | Edureka