2. Course Objective
To understand, successfully apply
and evaluate Neural Network
structures and paradigms for
problems in Science, Engineering and
It is expected that, the audience has a flair
to understand algorithms and basic
knowledge of Mathematics, Logic gates
How the human brain learns
Different types of Neural Networks
Network Layers and Structure
Training a Neural Network
Application of ANN
5. Introduction: Artificial Intelligence
Artificial Intelligence techniques such as Neural
networks, genetic algorithms and fuzzy logic are
among the most powerful tools available for
detecting and describing subtle relationships in
massive amounts of seemingly unrelated data.
Neural networks can learn and are actually
taught instead of being programmed.
Teaching mode can be supervised or
Neural Networks learn in the presence of noise
6. How the Human Brain learns
In the human brain, a typical neuron collects signals from others through a host
of fine structures called dendrites.
The neuron sends out spikes of electrical activity through a long, thin stand
known as an axon, which splits into thousands of branches.
At the end of each branch, a structure called a synapse converts the activity from
the axon into electrical effects that inhibit or excite activity in the connected
7. A Neuron Model
When a neuron receives excitatory input that is sufficiently large
compared with its inhibitory input, it sends a spike of electrical activity
down its axon. Learning occurs by changing the effectiveness of the
synapses so that the influence of one neuron on another changes.
We conduct these neural networks by first trying to deduce the essential
features of neurons and their interconnections.
We then typically program a computer to simulate these features.
8. A Simple Neuron
An artificial neuron is a device with many inputs and one output.
The neuron has two modes of operation;
the training mode and
the using mode.
9. The McCulloch-Pitts model
Neurons work by processing information. They receive and provide
information in form of spikes.
10. Properties for Mc Culloch and Pitts Model
Input is 0 or 1
Weights are -1, 0 or +1
Threshold is an integer
Output is 0 or 1
Output is 1 if multiplication of weight and input is more than the threshold
else Outputs 0
Represent the gates NOT, OR and AND with the help of this model
16. Summary of the simple networks
Single layer nets have limited representation
power (linear separability problem)
Error drive seems a good way to train a net
Multi-layer nets (or nets with non-linear hidden
units) may overcome linear inseparability
problem, learning methods for such nets are
Threshold/step output functions hinders the
effort to develop learning methods for multi-
17. Types of Problems
Mathematical Modeling (Function Approximation)
18. Training/ Learning
Learning can be of one of the following forms:
The patterns given to classifier may be on:
Non- Parametric Estimation
19. Machine Learning in ANNs
Supervised Learning − It involves a
teacher that is scholar than the ANN itself.
For example, the teacher feeds some
example data about which the teacher
already knows the answers.
20. Machine Learning in ANNs
Unsupervised Learning − It is required
when there is no example data set with
known answers. For example, searching
for a hidden pattern. In this case, clustering
i.e. dividing a set of elements into groups
according to some unknown pattern is
carried out based on the existing data sets
21. Machine Learning in ANNs
Reinforcement Learning − This strategy
built on observation. The ANN makes a
decision by observing its environment. If
the observation is negative, the network
adjusts its weights to be able to make a
different required decision the next time.
22. Unsupervised Learning: why?
Collecting and labeling a large set of sample patterns can
Train with large amounts of unlabeled data, and only then
use supervision to label the groupings found.
In dynamic systems, the samples can change slowly.
To find features that will then be useful for categorization.
To provide a form of data dependent smart processing or
smart feature extraction.
To Perform exploratory data analysis, to find structure of
data, to form proper classes for supervised analysis.
23. Measure of Dissimilarity:
Define a metric or distance function d on the vector space λ as
a real-valued function on the Cartesian product λX λ such that:
0 < d(x,y) < ∞ for x,y ελ and d(x,y)=0 if and only if x=y
d(x,y) = d(y,x) for x,y ελ
d(x,y) = d(x,z) + d(y,z) for x,y,z ελ
Invariance or distance function: d(x+z,y+z) = d(x,y)
25. No Free Lunch Theorem
No classification method is inherently superior
to any other.Classifier to be decide on the
grounds of type of problem, prior distribution of
samples, training data, cost function.
Ugly Duckling Theorem
There is no privileged or ‘best’ feature
representation, and that even the notion of
similarity between patterns depends implicitly
on assumption that may or may not be correct.
26. Neural networks have performed
successfully where other methods have
not, predicting system behavior,
recognizing and matching complicated,
vague, or incomplete data patterns.
Apply ANNs to pattern recognition,
interpretation, prediction, diagnosis,
planning, monitoring, debugging, repair,
Biomedical Signal Processing
Neural Network Applications