This document discusses different machine learning paradigms including supervised learning, unsupervised learning, and learning rules. It then describes techniques such as function approximation, system identification, and inverse modeling. Function approximation involves using a neural network to approximate an unknown function based on examples. System identification uses a neural network model to learn the input-output mapping of an unknown system. Inverse modeling constructs a neural network that produces the input given the output to learn the inverse of the unknown system.
3. LEARNING
PARADIGM
Training data
• A sample from the data source with the
correct classification/regression solution
already assigned.
Two Types of Learning
• SUPERVISED
• UNSUPERVISED
4. LEARNING
PARADIGM
Supervised learning : Learning
based on training data.
Example:- Perceptron, LDA, SVMs,
1. Training step: Learn classifier/regressor 2. Prediction step: Assign class
linear/ridge/kernel ridge regression are all
from training data. labels/functional values to test data.
supervised methods.
5. LEARNING
PARADIGM
Unsupervised learning: Learning
without training data.
Data clustering :
Dimension
Divide input
reduction
data into groups
techniques.
of similar points
6. Learning
Task
Pattern Pattern Function Beam
Approximation Controlling Filtering
Association Recognition forming
7. Function
Approximation
To design a neural network that
approximates the unknown function
f(.) such that the function F(.)
describing the input-output mapping
actually realized by the network, is
close enough to f(.) in a Euclidean sense
over all inputs.
8. Function Approximation
Consider a non linear input – output
mapping described by the functional
relationship
d f x
where
Vector x is input.
Vector d is output.
The vector valued function f(.) is assumed to
be unknown.
9. Function Approximation
To get the knowledge about the function
f(.), some set of examples are taken,
N
xi , di i 1
A neural network is designed to
approximate the unknown function in
Euclidean sense over all inputs, given
by the equation
F x f x
10. Function Approximation
Where
• Ε is a small positive number.
• Size N of training sample is large
enough and network is equipped with an
adequate number of free parameters,
• Thus approximation error ε can be
reduced.
• The approximation problem discussed
here would be example of supervised
learning.
11. FUNCTION
APPROXIMATION
SYSTEM INVERSE
IDENTIFICATION MODELING
12. SYSTEM
BLOCK DIAGRAM
IDENTIFICATION
di
UNKNOWN
SYSTEM
Input
Vector ei
xi
Σ
NEURAL
NETWORK
MODEL yi
13. System Identification
Let input-output relation of unknown memoryless MIMO
system i.e. time invariant system is
d f x
Set of examples are used to train a neural network as a model
of the system.
N
xi , di i 1
Where
Vector y i denote the actual output of the neural network.
14. System Identification
• x i denotes the input vector.
• d i denotes the desired response.
• ei denotes the error signal i.e. the difference between
d i and y i .
This error is used to adjust the free parameters of the
network to minimize the squared difference between the
outputsof the unknown system and neural network in a
statistical sense and computed over entire training samples.
15. INVERSE MODELING
BLOCK DIAGRAM
Error
ei
System
Output Model
Input UNKNOW
di Output xi
Vector INVERS
N
xi
SYSTEM
E
MODEL yi
Σ
f(.)
16. Inverse Modeling
In this we construct an inverse model that
produces the vector x in response to the vector d.
This can be given by the eqution :
x f 1 d
Where
f 1 denote inverse of f .
Again with the use of stated examples neural
network approximation of f 1 is constructed.
17. Inverse Modeling
Here d i is used as input and x i as desired response.
is the error signal between and produced
e
ini response to . xi yi
di
This error is used to adjust the free parameters of
the network to minimize the squared difference
between the outputsof the unknown system and
neural network in a statistical sense and computed
over entire training samples.
18. References
[1] Neural Network And Learning Machines, 3rd Edition, By : Simon
Haykins.
[2] Satish Kumar – Neural Network : A classroom approach.
[3] Jacek M.Zurada- Artificial Neural Networks.
[4] Rajasekaran & Pai – Neural networks, Fuzzy logic and genetic
algorithms.
[5] www.slideshare.net
[6] www.wikipedia.org
