In this paper we proposed a multiple classifiers system for handwritten Arabic alphabet recognition to investigate if it will really achieve a remarkable increase in the recognition accuracy compared to a single feature-based classifier system result

1. 1. Linear Combining Methods
a. Averaging
( ) ( )





= ∑=
= x
K
xQ
K
i
ij
N
j y
1
1
1
maxarg
N is the number of classes.
x is the input pattern.
K is the number of classifiers.
yij(x) is the output of the ith
classifier for the jth
class for the input
x.
Assign a number between zero and one for each candidate. Compare
the summation of the votes value, the higher is the winner.
Drawbacks: Sensitive towards skewed classifier values of voting. Back

2. 1. Linear Combining Methods
b. Weighted Averaging
( ) ( )





= ∑=
= xijijK
xQ
K
i
N
j yw
1
1
1
maxarg
N is the number of classes.
x is the input pattern.
K is the number of classifiers.
yij(x) is the output of the ith
classifier for the jth
class for the input
x.
The weights wi, i = 1, 2… K can be derived by minimizing the error of
the different classifiers on the training set.
The weights reflect the degree of confidence in each classifier,
with respect to any input pattern.
Back

3. 2. Non-Linear Combining Methods
Voting Methods
 Majority, Maximum, etc...
Rank Based Methods
 Borda Count
Probabilistic methods
 Bayesian Methods
Back

4. 2. Non-Linear Combining Methods
a. Voting Methods
Majority Vote
Bad if some classifiers (experts) are very good or very bad
Maximum Vote
Trust the most confident classifier/expert
Bad if some classifiers (experts) are badly trained
Sensitivity to over-confident base classifiers
Product Rule
Base Classifiers are never really independent
( ) ( )xxQ Yi
K
imaxarg 1==
( ) Yij
K
i
N
jxQ ∏=== 11maxarg
Back

5. 2. Non-Linear Combining Methods
b. Rank based Method
Borda Count
Bi,j(x) rank assigned by classifier i for class j given input x
Drawbacks: Does not consider information in the strengths
of the preferences
Back
( ) 





== ∑=
=
K
i
i
N
j jjBxQ B1
1 )()(maxarg

6. 2. Non-Linear Combining Methods
c. Probabilistic Methods
Bayesian Combination
 ci
is the confusion matrix estimated on a training set for the ith
classifier . Elements ci
jk denotes the number of data points that are
classified to be class k, whereas they are actually class j.
 The conditional probability that a sample x actually belongs to class
j, given that classifier i assigns it to class k, can be estimated as
 Assuming that the different classifiers are independent, a belief
value that the input x belongs to class j can be approximated by
Back
iccjxqxP jk
N
j
i
jkiij −∑==∈ =1/))(|( λ
iccjxqxP
iccjxqxP
jBel
jk
N
j
i
jkiij
K
i
N
j
jk
N
j
i
jkiij
K
i
−∑==∈∏∑
−∑==∈∏
=
===
==
111
11
/))(|(
/))(|(
)(
λ
λ

7. 2. Non-Linear Combining Methods
c. Probabilistic Methods
Bayesian Combination
 ci
is the confusion matrix estimated on a training set for the ith
classifier . Elements ci
jk denotes the number of data points that are
classified to be class k, whereas they are actually class j.
 The conditional probability that a sample x actually belongs to class
j, given that classifier i assigns it to class k, can be estimated as
 Assuming that the different classifiers are independent, a belief
value that the input x belongs to class j can be approximated by
Back
iccjxqxP jk
N
j
i
jkiij −∑==∈ =1/))(|( λ
iccjxqxP
iccjxqxP
jBel
jk
N
j
i
jkiij
K
i
N
j
jk
N
j
i
jkiij
K
i
−∑==∈∏∑
−∑==∈∏
=
===
==
111
11
/))(|(
/))(|(
)(
λ
λ