Q-filter Structures for Advancing Pattern Recognition Systems
Q-Aggregate Based Gene Expression Programming
1. Q-Aggregate Based
Gene Expression Programming
2006-03-16
Magdi Mohamed (PRR CoE, Labs)
Weimin Xiao (PRR CoE, Labs)
Chi Zhou (PRR CoE, Labs)
Method for Constructing Compact Infinite-Valued Logical Forms
Using Gene Expression Programming with Q-Aggregate Operators
2006:03:16 Magdi A. Mohamed 1/9
2. Q-Aggregate Operator Impacts
on soft computing and computational intelligence paradigms
1. Neural Networks
• avoids “sigmoid-like” function limitations
• simplifies implementations using per-unit values
2. Genetic Computing
• reduces complexity of Gene Expression Programming (GEP)
• provides natural interpretations of rules
3. Crisp and Soft Rule-Based Systems
• opens a new paradigm shift in rule-based systems (Beyond Fuzzy Logic)
• provides higher degrees of automation
2006:03:16 Magdi A. Mohamed 2/9
3. Closest Prior Art on GEP
IEEE-TEC 12/2003, “Evolving Accurate & Compact Classification Rules with GEP”, by Chi Zhou et al.
7−S
• A genetic-algorithm-based
/
symbolic regression algorithm
F ( P + 3)
(Ferreira 2001, Zhou 2003).
- *
• Simpler and faster than
7 S + F
conventional Genetic
Programming approaches.
3
• An expression tree is flattened
to form a chromosome.
P
/ - * 7 S + F 3 P
2006:03:16 Magdi A. Mohamed 3/9
4. The GEP Algorithm
Probabilistic Search
Create Chromosomes of Initial Population
Create Chromosomes of Initial Population
Express Chromosomes as ETs
Express Chromosomes as ETs
Evaluate Fitness of Expression Trees
Evaluate Fitness of Expression Trees
Select New Population Probabilistically
Select New Population Probabilistically
Crossover, Mutation, and Rotation
Crossover, Mutation, and Rotation
Chromosomes for New Generation
Chromosomes for New Generation
Yes
Termination Criterion Terminate
Satisfied?
No
2006:03:16 Magdi A. Mohamed 4/9
5. Innovation
Gene Expression Programming with Q-Aggregate Operators
• A binary operation expressed by a unique Q-Aggregate operator
characterizing disjunction, compensation, and conjunction operations:
A(x,y) = (x + y + lambda * x * y) / (2 + lambda)
• A unary operation expressed by a formula which generates value in the unit
interval characterizing Negation operations:
N(x) = (1 – x) / (1 + gamma * x)
• The training task is performed as an optimization process where flexible
model parameters (lambda/gamma values) are adjusted for each internal
node in a general non-fixed tree structure to minimize an overall risk criterion
using Gene Expression Programming and Differential Evolution techniques.
2006:03:16 Magdi A. Mohamed 5/9
6. Innovation
Gene Expression Programming with Q-Aggregate Operators
y1 y2
A A
A A A N
x1 x2 A 0.6 N x2 x3
0.3 N A
x3 0.7 N
x1
2006:03:16 Magdi A. Mohamed 6/9
7. Experimental Results
Nonlinear Regression Case
//XOR problem
a=0; b=0; Y = 0;
a=1; b=0; Y = 1;
a=0; b=1; Y = 1;
a=1; b=1; Y = 0;
//Best Tree
QA 0
|QA 0
||a 1.0
||N 0
|||b 1.0
|QA 0
||b 1.0
||N 0
|||a 1.0
;PGEP Formula:
;QA(QA(a, N(b, gamma = 0), lambda= 164015206785.06702), QA(b, N(a, gamma = 0),
lambda = 231379355057.6994 ), lambda = -0.99998 )
2006:03:16 Magdi A. Mohamed 7/9
8. Experimental Results
Nonlinear Classification Case
Test Accuracy on Monk's Problems
M1 M2 M3
C4.5 75.70% 65.00% 97.20%
C4.5Rules 100% 66.20% 96.30%
GEP 100% 99.07% 100%
QAB-GEP 100% …..% 100%
QAB-GEP is able to generate accurate, compact, and noise tolerant rules
Rule 1: QA(a5_1, N(a2_3, gamma= 0.050061), lambda= 0.548952)
Rule 2: N(QA(a2_3, a5_4, lambda= 0.221941), gamma= 0.535057)
Rule 3: QA(a4_1, a5_3, lambda= 0.602964)
The average size of Monks-3 rules is 4, while GEP has 14
The set of operations {QA, N} can replace the list of pre-selected functions
or operators, e.g., {IF, AND, OR, +, -, *, /, …}
2006:03:16 Magdi A. Mohamed 8/9
9. Advantages of QAB-GEP
Characteristics and Promises
1. compact set of operations { Aggregate (x,y), Negate (x) }
2. numerical stability
3. per unit calculations simplify implementations (software and hardware)
4. suitability for massive parallel implementations
5. automatic discovery of complex logical expressions
6. consistent handling of unary, binary, and n-ary operations
7. improvement over conventional GEP techniques in terms of average rule size
8. usability for both classification and regression applications
9. ease of use and interpretability
2006:03:16 Magdi A. Mohamed 9/9