Martin Butz presents the current state-of-the-union of XCS

1. XCS: Current capabilities and future
challenges
Martin V. Butz
Department of Cognitive Psychology (III)
University of Würzburg, Germany
http://www-illigal.ge.uiuc.edu/~butz
mbutz@psychologie.uni-wuerburg.de
02/23/2006

2. Overview
1. The XCS Classifier System
2. XCS - Capabilities
3. XCS - Future Challenges
4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 2

3. 1. The XCS Classifier System
1. The XCS Classifier System
1. Framework
2. Evolutionary Pressures
3. Computational Complexity
4. General Learning Intuition
2. XCS – Capabilities
3. XCS – Future Challenges
4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 3

4. 1 The XCS Classifier System
The XCS Classifier System
• Learning classifier system
– Rule-based representation of condition-action-predictions
– Steady-state GA for evolution of conditions
– Gradient-based techniques for estimation of predictions
• Major characteristics:
– Q-learning based reinforcement learning
– Relative accuracy-based fitness
– Action-set restricted selection, that is, niche selection
– Panmictic (population-wide) deletion
Goal:
Learn a complete maximally accurate,
maximally general predictive model.
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 4

5. 1 The XCS Classifier System
Rule Evaluation
• Gradient-based techniques for derivation of prediction
and error estimates
• Q-learning derived update
• Propagation of reward possible
• Rule quality depends on inverse of error estimate
• Accuracy of rule prediction determines fitness
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 5

6. 1 The XCS Classifier System
Evolutionary Algorithm
• Fixed population size
• Steady-state genetic algorithm
• Two reproductions and deletions per iteration
– Reproduction in action set based on fitness
– Deletion from whole population based on coverage
• Genetic operators:
– Mutation
– Recombination
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 6

7. 1 The XCS Classifier System
Learning Suitability
• XCS represents its solution by a collection of sub-solutions
(that is, the population of classifiers).
• XCS learns an effective problem space clustering
(subspaces) in its conditions.
• Clusters (subspaces) evolve to enable maximally accurate
predictions.
– Accuracy can be bounded (error threshold ε0 and population size
relation).
– Basically any form of prediction is possible (e.g. reward, next
sensory input, function value).
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 7

8. 1 The XCS Classifier System
XCS Power
• Combination of
– Gradient-based techniques (to generate predictions)
– Evolutionary techniques (to generate features / clusters)
• Advantage:
– Usage of gradient-based error-feedback learning where possible
– Usage of evolutionary techniques
• Where error-feedback is hard or impossible to propagate into
• Where error-feedback learning gets easily stuck in local optima
• Thus:
– Combine best approximation technique (error-feedback learning) with
best evolutionary technique (representation and operators)
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 8

9. 2. XCS – Current Capabilities
1. The XCS Classifier System
2. XCS – Current Capabilities
1. Binary and Real-world Classification Problems
2. Function Approximation Problems
3. (Multistep) Reinforcement Learning Problems
3. XCS – Future Challenges
4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 9

10. 2 XCS – Current Capabilities
The Multiplexer Problem
Optimal solution representation
C A R εF
000### 0 1000 0 1
Problem instance Class 000### 1 0 01
000000 0 001### 0 0 01
001000 1 001### 1 1000 0 1
000111 0 01#0## 0 1000 0 1
011011 0 01#0## 1 0 01
101101 0 01#1## 0 0 01
01#1## 1 1000 0 1
100010 1
10##0# 0 1000 0 1
100101 0
… … … ……
110000 0
… …
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 10

11. 2 XCS – Current Capabilities
XCS Performance in MP 70
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 11

12. 2 XCS – Current Capabilities
Hierarchical Classification Problem
• Hierarchical problems with low order dependencies
(“building blocks”) and further high-order dependencies
• BB structures are re-used on the higher level to derive
problem class.
• Example: Hierarchical 3-parity, 6-multiplexer problem:
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 12

13. 2 XCS – Capabilities
XCS/BOA Performance
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 13

14. 2 XCS – Capabilities
Classification of Real-World Datasets
• Conditions are coded with attributes dependent on type of
attribute in dataset (interval coding or Binary coding).
• Experiments in 42 datasets (from UCI and other sources)
• Comparisons with ten other ML systems (pairwise t-test)
• XCS learns competitively but it is a much more general
learning system.
XCS Maj. 1-R C4.5 Naïve PART IB1 IB3 SMO SMO SMO
Bayes (poly) (pol.3) (rad.)
99% 38/0 29/1 5/8 19/12 5/6 13/7 9/11 9/17 8/13 23/8
95% 38/0 30/1 5/9 19/12 7/6 14/7 9/15 9/18 9/14 24/9
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 14

15. 2 XCS – Capabilities
Piecewise Linear Function Approximation
f(x,y)
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
0.8 1
0 0.2 0.4 0.4 0.6
0.6 0.8 0 0.2
1 y
x
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 15

16. 2 XCS – Capabilities
Performance in 3D Sinusoidal Function
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 16

17. 2 XCS – Capabilities
Performance Evaluation in Maze6
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 17

18. 2 XCS – Capabilities
Performance Maze6 plus Irrelevant Bits
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 18

19. 3. XCS – Future Challenges
1. The XCS Classifier System
2. XCS – Performance Demonstration
3. XCS – Future Challenges
1. Representation & Operators
2. Niching
3. Dynamic Problems
4. Compactness of solution / population
5. Hierarchical classifier systems
4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 19

20. 4.1 XCS –Future Challenges
Representation & Operators
• Different representations of conditions
– Binary, real-valued, mixed
– Kernels, bases
– Combinations, Hybrids
• Different representations of predictions
– Constant, Linear, Polynomial
– State prediction, property prediction
– Control variable prediction
• Most suitable operators for representations
– Approximation operators (use best gradient method)
– Genetic operators (mind XCS problem bounds)
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 20

21. 4.1 XCS –Future Challenges
XCS and Niching
• Currently:
– Niching is done occurrence-based
– Number of classifiers in large problem niches unnecessary
large
– Number of classifiers in small but hard to approximate problem
spaces potentially too small -> niche loss
• Additional balancing mechanisms might be necessary!
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 21

22. 4.1 XCS –Future Challenges
Dynamic Problems
• Currently: XCS was applied mainly to static problems
– Makes iterative, adaptive approach not really necessary
• Dynamic problems
– Concept class changes
– Reward distribution changes
– Problem sampling changes
• Question:
– How quickly can XCS adapt to these changes?
– Can we improve adaptation?
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 22

23. Compactness of Solution / Population
• Problem in XCS:
– Population sizes get rather big.
– Final solution is not very compact
– Solutions indicate overfitting problems in dataminig problems
• Main generalization mechanism purely based on
occurrence frequency
• Borders between different classes are ill-defined.
• How can we efficiently compact the population online or
using post-processing (some approaches available)?
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 23

24. Hierarchical Classifier System
• Local structures are often used by many higher-order
structures (decomposability of environment, problems, etc.)
• Can we build higher-level classifier structures that build on
evolving lower-level structures….
• The hierarchical boolean function problems as a start?
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 24

25. 4. Summary and Conclusions
1. The XCS Classifier System
2. XCS – Performance Demonstration
3. XCS – Future Challenges
4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 25

26. 4 Summary and Conclusions
Conclusions
• XCS is designed to
– Cluster the problem space to enable
– Maximally accurate predictions
• XCS is a highly flexible learning system
– Conditions of various types possible
– Predictions of various types possible
• Major XCS challenges lie in the further development of
– Representation & Operators
– Niching
– Dynamic Problems
– Compactness of solution / population
– Hierarchical classifier systems
• XCS has a big potential due to the combination of
– Gradient-based update mechanisms
– Evolutionary-based feature extraction mechanisms
05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 26

27. Thank You for Your Attention