1. A Further Look at UCS
Classifier System
Cl ifi S t
Albert Orriols-Puig
Ester Bernadó-Mansilla
Research Group in Intelligent Systems
Enginyeria i Arquitectura La Salle
Ramon Llull University
Barcelona, Spain
,p

2. Aim
Provide a deep insight into UCS
p g
Introduce a fitness sharing scheme in UCS
Highlight the differences between XCS and UCS
Enginyeria i Arquitectura la Salle Slide 2
GRSI

3. Outline
1. Description of XCS
2. Description of UCS
3. Differences b t
3 Diff between XCS and UCS
d
4.
4 Test-bed
5. Experimentation
6. Conclusions
Enginyeria i Arquitectura la Salle Slide 3
GRSI

4. 1. Description of XCS
2. Description of UCS
1. Description of XCS
p 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
In single-step tasks:
g p
Environment
Match Set [M]
Problem
instance
1C A PεF num as ts exp
Selected
3C A PεF num as ts exp
action
5C A PεF num as ts exp
Population [P] 6C A PεF num as ts exp
Match set
REWARD
…
generation
1C A PεF num as ts exp
Prediction Array
2C A PεF num as ts exp
3C A PεF num as ts exp
…
c1 c2 cn
4C A PεF num as ts exp
5C A PεF num as ts exp
6C A PεF num as ts exp Random Action
…
Action S t
A ti Set [A]
1C A PεF num as ts exp
Deletion
Classifier
3C A PεF num as ts exp
Selection, Reproduction,
Parameters
mutation
5C A PεF num as ts exp
Update
6C A PεF num as ts exp
…
Genetic Algorithm
Enginyeria i Arquitectura la Salle Slide 4
GRSI

5. 1. Description of XCS
2. Description of UCS
2. Description of UCS
p 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
Only for single-step tasks
y g p
Environment
Match Set [M]
M t hS t
Problem instance
P bl it
+
output class 1C A acc F num cs ts exp
3C A acc F num cs ts exp
Population [P] 5C A acc F num cs ts exp
6C A acc F num cs ts exp
…
1C A acc F num cs ts exp
2C A acc F num cs ts exp
3C A acc F num cs ts exp
4C A acc F num cs ts exp correct set Classifier
5C A acc F num cs ts exp generation
Parameters
6C A acc F num cs ts exp Match set
Update
… generation
Correct S t [C]
C t Set
3 C A acc F num cs ts exp # Correct
Deletion Selection, Reproduction,
acc =
6 C A acc F num cs ts exp
mutation
Experience
p
…
Fitness = accν
Genetic Algorithm
Enginyeria i Arquitectura la Salle Slide 5
GRSI

6. 1. Description of XCS
2. Description of UCS
3. Differences between XCS and UCS 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
Three main differences:
– Explore regime
– Parameter updates
– Fitness computation
Enginyeria i Arquitectura la Salle Slide 6
GRSI

7. 1. Description of XCS
2. Description of UCS
3. Differences between XCS and UCS 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
Explore Regime
Populations
XCS evolved
Prediction Maximal general classifiers predicting the correct class
…
c1 c2 cn
Array
Maximal general classifiers predicting the incorrect class
Random action So,
So XCS also explores low rewarded niches
[A] 1. 000 0#######:0 1000 0 …
Complete 2. 000 1#######:0 0 0…
action map …
UCS
Maximal general classifiers predicting the correct class
Environment
Always exploring the class of the input instance
Example + class
1. 000 0#######:0 1000 0 …
[C] Best 2. 000 1#######:1 0 0…
action map …
Enginyeria i Arquitectura la Salle Slide 7
GRSI

8. 1. Description of XCS
2. Description of UCS
3. Differences between XCS and UCS 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
Parameter Updates
rd
XCS
Influence of the rewar
pt +1 = pt + β (R − pt )
β=0.2
ε t +1 = ε t + β ( R − pt − ε t )
e t+2
t+1 t+3 t+4 t+5 t+6 t+7 t+8
UCS time
Influence of the reward
d
number correct
acc =
experience
time
Enginyeria i Arquitectura la Salle Slide 8
GRSI

9. 1. Description of XCS
2. Description of UCS
3. Differences between XCS and UCS 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
Fitness Sharing: XCS shares fitness but UCS does not
The advantages of fitness sharing are empirically
demonstrated (Bull & Hurst, 2002)
Scheme of fitness sharing in UCS:
if acc > acc0
⎧1
=⎨
kcl∈[C ]
α (acc / acc0 )ν otherwise
⎩ We share the accuracy
with all the classifiers
in [M]
kcl ·numcl
k 'cl =
∑ kcli ·numcli
cli ∈[ M ]
[M
F = F + β ·( k '− F )
(
Enginyeria i Arquitectura la Salle Slide 9
GRSI

10. 1. Description of XCS
2. Description of UCS
4. Test-bed 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
Problems
– Parity: two-class problem
Condition
length (l)
Number of 1 mod 2
01001010 :1
Complexity: It does not permit any generalization
– Decoder: multi-class problem
Condition
length (l)
Integer value of the input
000110 :5
Complexity: the number of classes increases with the condition length
Enginyeria i Arquitectura la Salle Slide 10
GRSI

11. 1. Description of XCS
2. Description of UCS
4. Test-bed 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
Problems
– Imbalanced Multiplexer: Imbalanced two-class problem
Condition
length (l)
Value of the position bit
000 10000100 :1 indicated by the selection bits
The class labeled as 1 is under-sampled
ir = proportion between majority
Complexity: For high imbalances there is a p
p y g poor and minority class examples
sampling of minority class examples i = log2ir
– Position: imbalanced multi-class problem
Condition
length (l)
Position of the left-most
000110 :2 one-valued
one valued bit
Complexity: the number of classes and the imbalance level
increase with the condition length
g
Enginyeria i Arquitectura la Salle Slide 11
GRSI

12. 1. Description of XCS
2. Description of UCS
4. Test-bed 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
Problems
– Multiplexer with Alternating noise
Value of the position bit
0000 1000010011100101 :1 indicated by the selection bits
The output is flipped with probability Px
Complexity: The system receives noisy instances
Enginyeria i Arquitectura la Salle Slide 12
GRSI

13. 1. Description of XCS
2. Description of UCS
5. Experimentation
p 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
We used the five binary-input problems to test:
– XCS
– UCS without fitness sharing: UCSns
– UCS with fitness sharing: UCSs
To permit comparison between XCS and UCS, we measured the
percentage of the best action map achieved
We configured XCS with the following parameters:
N=25 |[O]|, α=0.1, ν=5, Rmax = 1000, ε0=1, θGA=25, β=0.2,
χ=0.8, μ=0.4, θdel=20, δ=0.1, θsub=20, P#=0.6
selection=tournament, mutation=niched,
selection=tournament mutation=niched
GAsub=true, [A]sub=false
And for UCS, we added: acc0 = 0.999, ν=5
,
Enginyeria i Arquitectura la Salle Slide 13
GRSI

14. 1. Description of XCS
5. Experimentation 2. Description of UCS
3. Differences b. XCS and UCS
4. Test-bed
5 2 The Parity Problem
5.2. 5. Experimentation
6. Conclusions
Parity with l=3 to l 9
l 3 l=9
Complete Action Map Par3
000:0 100:1 000:1 100:0
001:1 101:0 001:0 101:1
010:1 110:0 010:0 110:1
011:0 111:1 p 011:1 111:0
When an optimal classifier is
- Correct optimalthe fitness of
discovered, classifiers
the other classifiers in the
- Incorrect optimal classifiers
population is not affected
Difficulty: Lack of fitness guidance
XCS: 00#001#:0 P = 500, ε=500
500
UCS: 00#001#:0 acc = 0.5
Enginyeria i Arquitectura la Salle Slide 14
GRSI

15. 1. Description of XCS
5. Experimentation 2. Description of UCS
3. Differences b. XCS and UCS
4. Test-bed
5 3 The Decoder Problem
5.3. 5. Experimentation
6. Conclusions
Decoder with l=3 to l 6
l 3 l=6
Complete Action Map Dec3
000:0 1##:0 #1#:0 ##1:0
XCS cannot solve Dec6 in 100,000
100 000
001:1 1##:1 #1#:1 ##0:1
learning iterations:
010:2 1##:2 #0#:2 ##1:2
UCSs slightly improves UCSns
011:3 1##:3 #0#:3 ##0:3
100:4 0##:4 #1#:4 ##1:4
101:5 0##:5 #1#:5 ##0:5
110:6 0##:6 #0#:6 ##1:6
111:7 0##:7 #0#:7 ##0:7
- Correct optimal classifiers
- Incorrect optimal classifiers
Difficulty: Multiple classes
Enginyeria i Arquitectura la Salle Slide 15
GRSI

16. 1. Description of XCS
5. Experimentation 2. Description of UCS
3. Differences b. XCS and UCS
4. Test-bed
5 3 The Decoder Problem
5.3. 5. Experimentation
6. Conclusions
Fitness Dilemma i XCS (B t et al 2003)
Fit Dil in (Butz t l
Condition Class Correct P Error
Ratio
R ti
Error increases
###1# 2 0.125 125 218.75
until P=500
##01# 2 0.250 250 375
#001# 2 0.500 500 500
0001# 2 1 1000 0
Enginyeria i Arquitectura la Salle Slide 16
GRSI

17. 1. Description of XCS
5. Experimentation 2. Description of UCS
3. Differences b. XCS and UCS
4. Test-bed
5 4 The Imbalanced Multiplexer Problem
5.4. 5. Experimentation
6. Conclusions
Imbalanced 11-Mux for i=0 to i=9
11 Mux
Example: for i=6
Complete Action Map for the Multiplexer Problem
000 0#######:0
0####### 0 000 1#######:1
1####### 1 000 0#######:1
0####### 1 000 1#######:0
1####### 0
Classifier acc F
001 #0######:0 001 #1######:1 001 #0######:1 001 #1######:0
### ########:0 0.9928 0.9302
010 ##0#####:0 010 ##1#####:1 010 ##0#####:1 010 ##1#####:0
UCSs can solve the multiplexer
000 0#######:0 1.00 1.00
011 ###0####:0 011 ###1####:1
up t 011 ###0####:1 011 ###1####:0
to i 9 and XCS up to i=8
i=9 d t i8
100 ####0###:0 100 ####1###:1 100 ####0###:1 100 ####1###:0
• Similar values of fitness
101 #####0##:0 101 #####1##:1 101 #####0##:1 101 #####1##:0
• The overgeneral has more genetic opportunities
110 ######0#:0 110 ######1#:1 110 ######0#:1 110 ######1#:0
111 #######0:0 111 #######1:1 111 #######0:1 111 #######1:0
- Correct optimal classifiers
- Incorrect optimal classifiers
The system were configured following the
guidelines in (Orriols and Bernadó, 2006)
Difficulty: As the imbalance level increases, the
sampling rate of minority class examples decreases.
That is, low search rate for promising rules
predicting the minority class
Enginyeria i Arquitectura la Salle Slide 17
GRSI

18. 1. Description of XCS
5. Experimentation 2. Description of UCS
3. Differences b. XCS and UCS
4. Test-bed
5 5 The Position Problem
5.5. 5. Experimentation
6. Conclusions
Position with l 3 to l 9
l=3 l=9
Complete Action Map for the Pos3
000:0 1##:0 #1#:0 ##1:0
XCS h to explore all the correct
has t l ll th t
001:1 1##:1 #1#:1 ##0:0
action map
01#:2 1##:2 #0#:2
UCS only0##:3
y explores the best action
p
1##:3
map - Correct optimal classifiers
- Incorrect optimal classifiers
Difficulty: Class imbalance and multiple classes.
Maximum imbalance ratio between classes:
irmax = 2l-1
Enginyeria i Arquitectura la Salle Slide 18
GRSI

19. 1. Description of XCS
5. Experimentation 2. Description of UCS
3. Differences b. XCS and UCS
4. Test-bed
5 6 The Multiplexer with Alternating Noise
5.6. 5. Experimentation
6. Conclusions
20-bit Multiplexer with alternating noise
g
Complete Action Map for the Multiplexer Problem
0000 0###############:0 0000 1###############:1 0000 0###############:1 0000 1###############:0
p
In all cases, optimal classifiers0001 #0##############:1
are
0001 #0##############:0 0001 #1##############:1 0001 #1##############:0
continuously created and removed ##0#############:1
0010 ##0#############:0 0010 ##1#############:1 0010 0010 ##1#############:0
Windowed0011 ###1############:1 0011 ###0############:1
averages make oscillate the
0011 ###0############:0 0011 ###1############:0
parameters of XCS’s classifiers 0100 ####0###########:1
0100 ####0###########:0 0100 ####1###########:1 0100 ####1###########:0
Optimal classifiers are considered #####0########## 1
as
0101 #####0########## 0
#####0##########:0 0101 #####1########## 1
#####1##########:1 0101 #####0##########:1 0101 #####1########## 0
#####1##########:0
inaccurate 0110 ######1#########:1 0110 ######0#########:1
0110 ######0#########:0 0110 ######1#########:0
A non-fitness sharing scheme presents
0111 #######0########:0 0111 #######1########:1 0111 #######0########:1 0111 #######1########:0
slightly better results
1000 ########0#######:0 0000 ########1#######:1 0000 ########0#######:1 0000 ########1#######:0
1001 #########0######:0 0001 #########1######:1 0001 #########0######:1 0001 #########1######:0
- Correct optimal classifiers
1010 ##########0#####:0 0010 ##########1#####:1 0010 ##########0#####:1 0010 ##########1#####:0
- Incorrect optimal classifiers
1011 ###########0####:0 0011 ###########1####:1 0011 ###########0####:1 0011 ###########1####:0
1100 ############0###:0 0100 ############1###:1 0100 ############0###:1 0100 ############1###:0
1101 #############0##:0 0101 #############1##:1 0101 #############0##:1 0101 #############1##:0
1110 ##############0#:0 0110 ##############1#:1 0110 ##############0#:1 0110 ##############1#:0
1111 ###############0:0 0111 ###############1:1 0111 ###############0:1 0111 ###############1:0
Difficulty: The system receive examples labeled wrongly
XCS: Optimal incorrect classifiers will receive Px positive rewards
UCS: The system will need to create classifiers
covering noisy examples. Lots of coverings.
Enginyeria i Arquitectura la Salle Slide 19
GRSI

20. 1. Description of XCS
2. Description of UCS
6. Conclusions 3. Differences b. XCS and UCS
4. Test-bed
5. Experimentation
6. Conclusions
We introduced UCS, and specialization of XCS
We improved UCS by introducing fitness sharing
– Fitness sharing is necessary in imbalanced datasets, avoiding
overgeneral classifiers when the optimal classifiers are discovered
g p
UCS presents some advantages in the tested domains:
– It does not suffer from fitness dilemma
– It only explores the correct class, decreasing the convergence time in
p
problems with large complete action maps
g p p
XCS is more general, and it can be applied to multi-step
problems
As further work, we want to analyze the differences of UCSs
and XCS with bilateral accuracy
Enginyeria i Arquitectura la Salle Slide 20
GRSI

21. A Further Look at UCS
Classifier System
Cl ifi S t
Albert Orriols-Puig
Ester Bernadó-Mansilla
Research Group in Intelligent Systems
Enginyeria i Arquitectura La Salle
Ramon Llull University
Barcelona, Spain
,p