1. Artificial Neural
Networks
14.3.2016
Prof. Neeta Awasthy
Director, School of Engineering and Technology
NOIDA INTERNATIONAL UNVERSITY,
Greater Noida

2. Course Objective
 To understand, successfully apply
and evaluate Neural Network
structures and paradigms for
problems in Science, Engineering and
Business.

3. PreRequisites
 It is expected that, the audience has a flair
to understand algorithms and basic
knowledge of Mathematics, Logic gates
and Programming

4. Outline
 Introduction
 How the human brain learns
 Neuron Models
 Different types of Neural Networks
 Network Layers and Structure
 Training a Neural Network
 Application of ANN

5. Introduction: Artificial Intelligence
 Artificial Intelligence techniques such as Neural
networks, genetic algorithms and fuzzy logic are
among the most powerful tools available for
detecting and describing subtle relationships in
massive amounts of seemingly unrelated data.
 Neural networks can learn and are actually
taught instead of being programmed.
 Teaching mode can be supervised or
unsupervised
 Neural Networks learn in the presence of noise

6. How the Human Brain learns
 In the human brain, a typical neuron collects signals from others through a host
of fine structures called dendrites.
 The neuron sends out spikes of electrical activity through a long, thin stand
known as an axon, which splits into thousands of branches.
 At the end of each branch, a structure called a synapse converts the activity from
the axon into electrical effects that inhibit or excite activity in the connected
neurons.

7. A Neuron Model
 When a neuron receives excitatory input that is sufficiently large
compared with its inhibitory input, it sends a spike of electrical activity
down its axon. Learning occurs by changing the effectiveness of the
synapses so that the influence of one neuron on another changes.
 We conduct these neural networks by first trying to deduce the essential
features of neurons and their interconnections.
 We then typically program a computer to simulate these features.

8. A Simple Neuron
 An artificial neuron is a device with many inputs and one output.
 The neuron has two modes of operation;
 the training mode and
 the using mode.

9. The McCulloch-Pitts model
Neurons work by processing information. They receive and provide
information in form of spikes.
Inputs
Output
w2
w1
w3
wn
.
.
.
x1
x2
x3
…
xn-1
xn
y

10. Properties for Mc Culloch and Pitts Model
Input is 0 or 1
Weights are -1, 0 or +1
Threshold is an integer
Output is 0 or 1
Output is 1 if multiplication of weight and input is more than the threshold
else Outputs 0
Represent the gates NOT, OR and AND with the help of this model
Truth Table
L=0
-1
x
y
x y
0 1
1 0
x=0

11. The Perceptron Model

12. The Sigmoidal Function

13. Linear Separability

14. OR gate v/s XOR gate

15. Feed Forward Network

16. Summary of the simple networks
 Single layer nets have limited representation
power (linear separability problem)
 Error drive seems a good way to train a net
 Multi-layer nets (or nets with non-linear hidden
units) may overcome linear inseparability
problem, learning methods for such nets are
needed
 Threshold/step output functions hinders the
effort to develop learning methods for multi-
layered nets

17. Types of Problems
 Mathematical Modeling (Function Approximation)
 Classification
 Clustering
 Forecasting
 Vector Quantization
 Pattern Association
 Control
 Optimization

18. Training/ Learning
 Learning can be of one of the following forms:
 Supervised Learning
 Unsupervised Learning
 Reinforced Learning
 The patterns given to classifier may be on:
 Parametric Estimation
 Non- Parametric Estimation

19. Machine Learning in ANNs
 Supervised Learning − It involves a
teacher that is scholar than the ANN itself.
For example, the teacher feeds some
example data about which the teacher
already knows the answers.

20. Machine Learning in ANNs
 Unsupervised Learning − It is required
when there is no example data set with
known answers. For example, searching
for a hidden pattern. In this case, clustering
i.e. dividing a set of elements into groups
according to some unknown pattern is
carried out based on the existing data sets
present.

21. Machine Learning in ANNs
 Reinforcement Learning − This strategy
built on observation. The ANN makes a
decision by observing its environment. If
the observation is negative, the network
adjusts its weights to be able to make a
different required decision the next time.

22. Unsupervised Learning: why?
 Collecting and labeling a large set of sample patterns can
be costly.
 Train with large amounts of unlabeled data, and only then
use supervision to label the groupings found.
 In dynamic systems, the samples can change slowly.
 To find features that will then be useful for categorization.
To provide a form of data dependent smart processing or
smart feature extraction.
 To Perform exploratory data analysis, to find structure of
data, to form proper classes for supervised analysis.

23. Measure of Dissimilarity:
 Define a metric or distance function d on the vector space λ as
a real-valued function on the Cartesian product λX λ such that:
 Positive Definiteness:
0 < d(x,y) < ∞ for x,y ελ and d(x,y)=0 if and only if x=y
 Symmetry:
d(x,y) = d(y,x) for x,y ελ
 Triangular Inequality:
d(x,y) = d(x,z) + d(y,z) for x,y,z ελ
 Invariance or distance function: d(x+z,y+z) = d(x,y)

24. Error Computation
 Minkowski Matrix or Lk norm
 Manhattan Distance or L1 norm
 Euclidian Distance or L2 norm
 Ln norm

25.  No Free Lunch Theorem
 No classification method is inherently superior
to any other.Classifier to be decide on the
grounds of type of problem, prior distribution of
samples, training data, cost function.
 Ugly Duckling Theorem
 There is no privileged or ‘best’ feature
representation, and that even the notion of
similarity between patterns depends implicitly
on assumption that may or may not be correct.

26. Neural networks have performed
successfully where other methods have
not, predicting system behavior,
recognizing and matching complicated,
vague, or incomplete data patterns.
Apply ANNs to pattern recognition,
interpretation, prediction, diagnosis,
planning, monitoring, debugging, repair,
instruction, control
 Biomedical Signal Processing
 Biometric Identification
 Pattern Recognition
 System Reliability
 Business
 Target Tracking
Neural Network Applications

27. Pattern Recognition System
Sensing Segmentation
Classification (missing
features & context)
Post-processing (costs/
errors)
Feature Extraction
Input
Output (decision)