Development of Deep Learning Architecture

Development of Deep
Learning Architecture
Organized by
Pantech Solutions
&
The Institution of Electronics and
Telecommunication

All Participants, Please fill the
Attendance form, Link given in
the description
Instruction for Attendees
www.pantechsolutions.net

Founded in 1953|1,25,000 members | 64 Centers
Profile
IETE conducts Technical Meetings | Conferences | Symposia
| Exhibitions | delivering career advancement opportunity
and publishes Technical journals
IETE Focus on
Computer Science & Engineering and Information
Technology streams and Diploma DIPIETE Exams|
ALCCS Post graduate source
IETE Exams
About IETE

Established on 2004 | 7 Branches | 100+ Team
Profile
Manufacturer of Lab equipment‘s & Development boards |
Industrial &Funded projects | Online retail store of
engineering products and projects
R & D
Lab equipment, Engineering Kits, Components, Sensors
and All level Projects
Online Retail Store
1500+ Workshops | 250+ FDP | 100+ Seminars
Training
About Pantech

Agenda
01
General talk on AI
02 About Deep Learning & Libraries
03
Deep Learning Algorithm
ANN | RNN | CNN
04 Application of Deep Learning
05
Demo on Character recognition &
Emotion Recognition www.pantechsolutions.net

What is AI
Intelligence is given by
feeding the experience
of human to machines
in the form of data
Intelligence demonstrated
by the Machines in
contrast to Natural
Intelligence delivered by
Humans
“AI WILL BE THE BEST OR
WORST THING EVER FOR
HUMANITY.” – ELON MUSK
“SUCCESS IN CREATING AI
WOULD BE THE BIGGEST EVENT
IN HUMAN HISTORY.
UNFORTUNATELY, IT MIGHT
ALSO BE THE LAST, UNLESS WE
LEARN HOW TO AVOID THE
RISKS.” – STEPHEN HAWKING www.pantechsolutions.net

Deep Learning
Subset of Machine learning, Algorithms inspired by the
structure and function of Human Brain
Brain
Vision
Recognition
Dataset Model Recognition
Training

Why Deep Learning ?
Input Data
Its an Apple
Prediction
Model
AI
Automatic Feature
Extraction

Why GPU
High Bandwidth | Thread Parallelism |
Easily programmable registers |
Bandwidth Optimized
GPU – Graphical Processing Unit
100%
50%Less Bandwidth | Latency Optimised |
Not suitable for Real time application
where performance is the main Factor
CPU –Central Processing Unit
“It Can train more number
of data in short time
period & Better for real
time AI based application”

Deep Learning Libraries ?
Numerical computation using data flow
graphs | Backend for Keras |
Distributed computing on multiple GPU
Tensor Flow
Framework for Deep learning | Same
code for CPU & GPU | Uses Theano/TF
as Backend | CNN | not support multi
GPU
Keras
Very high performance | Tuning Hyper
parameters | CNN & RNN(Facebook
extends) | | CPU & GPU
Caffe
Old ML & DL library | Supports
CUDA for parallel computation |
Supervised image problem with
CNN
Torch
Basic operation such as Sorting,
Reshaping, indexing | Scientific
computing | Most new feature
belong in SciPy rather than
Numpy
Numerical Python & Scientific
Python
More number of Language
Bindings | Distributed computing
Mxnet

Neuron & Activation Fn.
• Like a Human Brain, here a Neuron takes input and do some
function to give the output
• Function going to be the Mathematical function
• Those Function is known as ACTIVATION function
Neuron
Activation Function
• Step Function
• Sigmoid Function
• Tanh Function
• ReLU Function
Dendrites
Cell Body
Axon
Synapse
Neuron
OUTPUT
Activation Fn.
Inputs

Activation Function
STEP Function
• If value of X is greater than or equal to 0, then output is 1,
If value of X is less than 0, then output is 0
• NN uses back propagation & Gradient descent method to
calculate weight of different layers
• Since step Function is non differentiable to zero, it can’t do
the gradient descent method, so it can’t update weights.
SIGMOID Function
• If value of X is infinity, then output is 1,
If value of X is negative infinity, then output is 0
• It captures non-linearity in the data
• It can use Gradient descent & Back propagation method to
calculate weights.
• Output range [0,1]
Image source from Towards Data Science

Activation Function
Tanh Function
• Rescaled of Sigmoid Function
• Output range [-1,1]
• Better learning rate requires, higher gradient. In some times,
for the data is centred around 0, derivatives are higher.
ReLU Function
• Rectified Linear Unit. if any negative input is detected, it
returns 0, otherwise it returns the value back.
Leaky ReLU Function
• Same as ReLU, like returning same for Positive. But for
negative values instead of returning zero, it has constant slope

Activation Function
SOFTMAX Function
• Softmax activation function will be applied in the last layer
of Neural network, instead of ReLU, tanh, Sigmoid.
• It is used to map the non-normalized output of a network to
a probability distribution over predicted output class. That is
it converts output of last layer into a essential probability
distribution.

Deep Learning Algorithm
ARTIFICIAL NEURAL
NETWORK (ANN)01
RECURRENT NEURAL
NETWORK (ANN)02
CONVOLUTIONAL NEURAL
NETWORK (ANN)03

Vanishing & Exploding Gradient
• It is very common problem in every Neural Network, which is associated with Backpropagation.
• Weights of network are updated through backpropagation by finding gradients.
• When the number of hidden layer is high, then the gradient vanishes or explodes as it
propagates backward. It leads instability in network, unable to learn from training
• The explosion occurs through exponential growth by repeatedly multiplying gradients through
the network layers that have values larger than 1.0
• It can be fixed by redesigning the network, using Long Short Term Memory networks, Gradient
clipping, etc.

ANN
• Learns any Non-Linear Function, It is known as Universal
Function Approximators
• Activation Function introduce non linear property to network,
so it will identify complex relationship between input &
output
• Output of each neuron is the activation of weighted sum of
Input, If there is no Activation function, network can't learn
non-linear function
• Feed Forward Neural Network – Input processed in one
direction, When hidden layer is more than one, that is Deep
Neural Network
Input
Hidden
Output

RNN
• Looping system in hidden layer of ANN is known as RNN
• It captures sequential info of input data, that is dependency
between words to make prediction. Whereas, ANN cannot
capture sequential information
• RNN shares parameters across different time steps, so that
there will be few parameter to train
• It is the time series version of ANN. Common Recurrent
layers are LSTM(Long Short Term Memory) & GRU (Grated
Recurrent Units)
• GRU is used to how much pass data needed to flow through
model
• It is mostly used in NLP (Natural Language Processing)
Input
Hidden
Output

CNN
• CNN learns the filter automatically to extract the right features from the data
• It captures spatial features (Arrangement of pixels) whereas ANN can’t.
• It also follows parameter sharing like RNN, applies single filter in different part of single image.
Whereas ANN can’t.
• It don’t have recurrent connections like RNN, instead it has convolution type of hidden layers
• Convolution and pooling functions are used as activation functions
• CONVOLUTION: Input image and other as Filter on input image(Kernel) produces output image.
• POOLING: picking maximum value from selected region is Max pooling and vice versa.

CNN Architecture
n3 units
Output
.
.
.
.
.
.
.
.
.
.
.
.
.
Flattened
n2 channels
(4 x 4 x n2)
n2 channels
(8 x 8 x n2)
n1 channels
(12 x 12 x n1)
n1 channels
(24 x 24 x n1)
Input
28x28x1
0
1
2
9
Conv_1
Convolution
(5x5)
Max-pooling
(2x2)
Conv_2
Convolution
(5x5)
Max-pooling
(2x2)
FC_3
Fully Connected
ReLU Activation
FC_4
Fully Connected
Neural Network

Simple Softmax Classification
Input
28x28x1
784 Pixels
….
.
.
.
.
.
.
.
.
0 1 3 92

100 image at a time
…
.
…
.
…
.
.
.
.
…
.
100x784 (100 images Flattened)
W0,0 W0,1 W0,2 ……W0,9
W1,0 W1,1 W1,2 ……W1,9
W2,0 W2,1 W2,2 ……W2,9
W3,0 W3,1 W3,2 ……W3,9
W4,0 W4,1 W4,2 ……W4,9
.
.
.
W783,0 W783,1 W783,2 ……W783,9
784x10 Matrix
Input Image
Weights
L0,0 W0,1 W0,2 ……W0,9
L1,0 W1,1 W1,2 ……W1,9
L2,0 W2,1 W2,2 ……W2,9
L3,0 W3,1 W3,2 ……W3,9
L4,0 W4,1 W4,2 ……W4,9
.
.
.
L99,0 L99,1 L99,2 ……W99,9
X
X
+
b0 b1 b2 b3 ….. b9
1x10 Matrix
Bias
100x10 Matrix

In Tensor Flow
Y =
tf.nn.softmax(tf.matmul(X,W)+b)
W[784x10]
X[100x784]
b[10]
Y[100x10]
Cross Entropy
0 0 1 0 0 0 0 0 00
0.2 0.9 0.5 0.3 0.1 0.2 0.1 0.3 0.10.1
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
Actual Probability
Computed Probability

Application of Deep Learning
It is kind of machine analyses the symptoms and cause and
suggesting the Medicine for the patient. Those knowledge is fed by
tons of Medical data, based on the applications.
AI Doctor
It is the application uses medical images, to classify the diseased and
healthy like Diabetic Retinopathy
Prediction of Disease from Medical Image
ALS (amyotrophic lateral sclerosis) is a kind of disease which causes speech
problems, the voice of the patients completely non understandable. AI helps
to train the voice of ALP patients to deliver voice based assistance for them.
Voice recognition for ALS Patient

Application of Deep Learning
After cultivation, Fruits and Vegetables needed to be segregated based
on the quality like Rotten or Healthy. AI application will done this with
great accuracy
Fruit & vegetable classification
AI in Autonomous Vehicle
Now AI is used for studying and identifying the pattern of molecular structure of
Food Items to deliver a new dish recipes.
AI in Cooking
AI in voice Assistance

THANK YOU
May 30 – Conversational Bot design
April 2 – Brain Computer Interface
Upcoming Session

Development of Deep Learning Architecture

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Development of Deep Learning Architecture

Similaire à Development of Deep Learning Architecture (20)

Plus de Pantech ProLabs India Pvt Ltd

Plus de Pantech ProLabs India Pvt Ltd (20)

Dernier

Dernier (20)

Development of Deep Learning Architecture