![Motivation
Michigan-style LCSs for supervised learning. Eg. UCS
– Evolve online highly accurate models
– Competitive to the most-used machine learning techniques
• (Bernadó et al, 03; Wilson, 02; Bacardit & Butz, 04; Butz, 06; Orriols & Bernadó, 07)
Main weakness: Intepretability of the rule sets
– Continuous attributes represented with intervals: [ i, ui] . Semantic-
p [l
free variables
– Number of rules or classifiers
• Reduction schemes
(Wilson, 02; Fu & Davis, 02; Dixon et al., 03, Orriols & Bernadó, 2005)
Enginyeria i Arquitectura la Salle Slide 2
GRSI](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (12)
Similaire à JAEM'2007: Aprendizaje Supervisado de Reglas Difusas mediante un Sistema Clasificador Evolutivo Estilo Michigan
Similaire à JAEM'2007: Aprendizaje Supervisado de Reglas Difusas mediante un Sistema Clasificador Evolutivo Estilo Michigan (20)
Plus de Albert Orriols-Puig
Plus de Albert Orriols-Puig (20)
Dernier
Dernier (20)
JAEM'2007: Aprendizaje Supervisado de Reglas Difusas mediante un Sistema Clasificador Evolutivo Estilo Michigan
- 1. Aprendizaje Supervisado de Reglas Difusas mediante un Sistema Clasificador Evolutivo Estilo Michigan Albert Orriols-Puig1,2 Orriols Puig Jorge Casillas2 Ester Bernadó-Mansilla1 1Grup de Recerca en Sistemes Intel·ligents Enginyeria i Arquitectura La Salle, Universitat Ramon Llull 2Dept. de Ciencias de la Computación e IA Universidad de Granada
- 2. Motivation Michigan-style LCSs for supervised learning. Eg. UCS – Evolve online highly accurate models – Competitive to the most-used machine learning techniques • (Bernadó et al, 03; Wilson, 02; Bacardit & Butz, 04; Butz, 06; Orriols & Bernadó, 07) Main weakness: Intepretability of the rule sets – Continuous attributes represented with intervals: [ i, ui] . Semantic- p [l free variables – Number of rules or classifiers • Reduction schemes (Wilson, 02; Fu & Davis, 02; Dixon et al., 03, Orriols & Bernadó, 2005) Enginyeria i Arquitectura la Salle Slide 2 GRSI
- 3. Motivation Jorge’s Proposal: – Let’s “fuzzify” UCS fuzzify” • Change the rule representation to fuzzy rules Framework on Michigan-style Learning Fuzzy-Classifier Systems (LFCS) – (Valenzuela-Radón, 91 & 98) – (Parodi & Bonelli, 93) – (Furuhashi, Nakaoka & Uchikawa, 94) – (Velasco, 98) – (Ishibuchi, Nakashima & Murata, 99 & 05): First LFCS for pattern classification – (Casillas, Carse & Bull, 07) Fuzzy-XCS Enginyeria i Arquitectura la Salle Slide 3 GRSI
- 4. Aim Propose Fuzzy-UCS – Accuracy based Michigan-style LFCS Accuracy-based Michigan style – Supervised learning scheme – Derived from UCS (Bernadó & Garrell, 2003) • Introduction of a linguistic fuzzy representation • Modification of all operators that deal with rules – We expect: • Achieve similar performance than UCS • Higher interpretability – Plus new opportunities: • Mine in uncertain environments Enginyeria i Arquitectura la Salle Slide 4 GRSI
- 5. Outline 1. Description of Fuzzy-UCS 1D ii fF UCS 2. 2 Experimental Methodology 3. Results 4. Conclusions Enginyeria i Arquitectura la Salle Slide 5 GRSI
- 6. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of UCS p 3. Results 4. Conclusions 4C li Michigan-style LCS’s (Holland, 1975): – Derived from XCS (Wilson 1995) a reinforcement learning (Wilson, 1995), method. – Designed specifically for supervised learning Rule representation: – C ti Continuous variables represented as i t intervals: [li, ui] i bl td l – Eg: IF x1 Є [l1, u1] ^ x2 Є [l2, u2] … ^ xn Є[ln, nn] THEN class1 – Matching instance e: for all ei: li ≤ ei ≤ ui – Set of parameters: Accuracy, Fitness, Numerosity, Experience, Correct set size Enginyeria i Arquitectura la Salle Slide 6 GRSI
- 7. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of UCS p 3. Results 4. Conclusions 4C li Stream of Environment examples Match Set M t h S t [M] Problem instance P bl it + output class acc F num cs ts exp 1C A acc F num cs ts exp 3C A Population [P] acc F num cs ts exp 5C A acc F num cs ts exp 6C A … acc F num cs ts exp 1C A acc F num cs ts exp 2C A acc F num cs ts exp 3C A correct set acc F num cs ts exp 4C A Classifier generation acc F num cs ts exp 5C A Parameters Match set acc F num cs ts exp 6C A Update generation … Correct Set [C] 3 C A acc F num cs ts exp Deletion # Correct Selection, Reproduction, acc = 6 C A acc F num cs ts exp mutation Experience p … If there are no classfiers in Genetic Algorithm Fitness = accν [C], covering is triggered Enginyeria i Arquitectura la Salle Slide 7 GRSI
- 8. Description of Fuzzy-UCS p y Describe the different components 1. Rule representation and matching 2. Learning interaction 3. Discovery component 3 Di t 4. Fuzzy-UCS in test mode Enginyeria i Arquitectura la Salle Slide 8 GRSI
- 9. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4C li 4. Conclusions Rule representation – Linguistic fuzzy rules – E.g.: IF x1 is A1 and x2 is A2 … and xn is An THEN class1 Disjunction of linguistic fuzzy terms – All variables share th same semantics i bl h the ti – Example: Ai = {small, medium, large} IF x1 is small and x2 is medium or large THEN class1 – Codification: IF [100 | 011] THEN class1 Enginyeria i Arquitectura la Salle Slide 9 GRSI
- 10. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4C li 4. Conclusions How do we know if a given input is small, medium or large? g p , g – Each linguistic term defined by a membership function Belongs to medium with a degree of 0 8 0.8 Belongs to small with a degree of 0 2 0.2 ei Attribute value Triangular-shaped membership functions Enginyeria i Arquitectura la Salle Slide 10 GRSI
- 11. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4C li 4. Conclusions Matching degree uAk(e) gg () [,] [0,1] k: IF x1 is small and x2 is medium or large THEN class1 Example: (e1, e2) 0.8 08 0.2 0.2 e1 e2 T-conorm: bounded sum max ( 1, 0.8 + 0.2) = 1 T-norm: product uAk(e) = 1 * 0.2 = 0.2 Enginyeria i Arquitectura la Salle Slide 11 GRSI
- 12. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li The role of matching changes: • UCS: A rule matches or not an example (binary function) • Fuzzy-UCS: A rule matches an example with a certain degree Enginyeria i Arquitectura la Salle Slide 12 GRSI
- 13. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Each classifier has the following parameters: 1. 1 Weight per class wj: • Soundness with which the rule predicts the class j. • The class value is dynamic and corresponds to the class j with higher wj 2. Fitness: • Quality of the rule 3. Other parameters directly inherited from UCS: • numerosity • Experience Enginyeria i Arquitectura la Salle Slide 13 GRSI
- 14. Description of Fuzzy-UCS p y Describe the different components 1. Rule representation and matching 2. Learning interaction 3. Discovery component 3 Di t 4. Fuzzy-UCS in test mode Enginyeria i Arquitectura la Salle Slide 14 GRSI
- 15. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4C li 4. Conclusions Learning interaction: – The environment provides an example e and its class c – Match set creation: all classifiers that match with uAk(x) > 0 – Correct set creation: all classifiers that advocate c – Covering: if there is not a classifier that maximally matches e • Create the classifier that match the input example with maximum degree. • Generalize the condition with probability P# For each variable: A1 A2 A3 Enginyeria i Arquitectura la Salle Slide 15 GRSI
- 16. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Parameters’ Update – Experience: – Sum of correct matching per class j cmj: Enginyeria i Arquitectura la Salle Slide 16 GRSI
- 17. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Parameters’ Update – Use cm to update of the weights per each class: • Rule that only matches instances of class c: • wc = 1 • For all the other classes j: wj = 0 • Rule that matches instances o a c asses u e t at atc es sta ces of all classes: • All weights wi ranging [0, 1] – Calculate the fitness Pressuring toward rules that correctly match instances of only one class Enginyeria i Arquitectura la Salle Slide 17 GRSI
- 18. Description of Fuzzy-UCS p y Describe the different components 1. Rule representation and matching 2. Learning interaction 3. Discovery component 3 Di t 4. Fuzzy-UCS in test mode Enginyeria i Arquitectura la Salle Slide 18 GRSI
- 19. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Discovery component – Steady state niched GA Steady-state – Roulette wheel selection Instances that have a higher g matching degree have more opportunities of being selected Enginyeria i Arquitectura la Salle Slide 19 GRSI
- 20. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Discovery component – Crossover and mutation applied on the antecedent • 2 point crossover IF [100 | 011] THEN class1 IF [101 | 100] THEN class1 • Mutation: – Expansion p IF [101 | 011] THEN class1 [ ] IF [100 | 011] THEN class1 [ ] – Contraction IF [100 | 001] THEN class1 IF [100 | 011] THEN class1 – Shift IF [010 | 011] THEN class1 IF [100 | 011] THEN class1 Enginyeria i Arquitectura la Salle Slide 20 GRSI
- 21. Description of Fuzzy-UCS p y Describe the different components 1. Rule representation and matching 2. Learning interaction 3. Discovery component 3 Di t 4. Fuzzy-UCS in test mode Enginyeria i Arquitectura la Salle Slide 21 GRSI
- 22. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Class inference of a test example e – Combining the information of all rules yields better results than taking a single rule for reasoning (Cordon et al. 1998) • Inference: – All experienced rules vote for the class they predict as: uAk(e) · Fk – The most voted class is returned. Enginyeria i Arquitectura la Salle Slide 22 GRSI
- 23. Outline 1. Description of Fuzzy-UCS 1D ii fF UCS 2. 2 Experimental Methodology 3. Results 4. Conclusions Enginyeria i Arquitectura la Salle Slide 23 GRSI
- 24. 1. Description of Fuzzy-UCS 2. Experimental Methodology Experimental Methodology p gy 3. Results 4. Conclusions 4C li Evaluating Fuzzy-UCS’ performance – Compare Fuzzy-UCS accuracy to: Fuzzy-UCS’ • Three non-fuzzy learners: UCS, SMO, and C4.5 • Two fuzzy learners: Fuzzy LogitBoost and Fuzzy GP – Default configuration for all methods –F Fuzzy-UCS configuration: UCS fi ti iter = 100,000, N = 6400, F0 = 0.99, v=10, {θGA, θdel, θsub} = 50, x =0.8, u 0.04, P#=0.6 0.8, u=0.04, 0.6 – Fuzzy learners: 5 linguistic labels per variable – 10 fold cross-validation 10-fold cross validation – Averages over 10 runs Enginyeria i Arquitectura la Salle Slide 24 GRSI
- 25. 1. Description of Fuzzy-UCS 2. Experimental Methodology Experimental Methodology p gy 3. Results 4. Conclusions 4C li Data domains #Inst #Fea #Re #In #No #Cl %Min %Max %MisAtt annealing 898 38 6 0 32 5 0.9 76.2 0 balance 625 4 4 0 0 3 7.8 46.1 0 bupa 345 6 6 0 0 2 42 58 0 glass 214 9 9 0 0 6 4.2 42 35.5 35 5 o heart-c 303 13 6 0 7 2 45,5 54.5 15,4 heart-s 270 13 13 0 0 2 44.4 56.6 0 iris 150 4 4 0 0 3 33.3 33.3 0 wbcd 699 9 0 9 0 2 34.5 65.5 11,1 wine 178 13 13 0 0 3 27 39.9 39 9 0 101 17 0 1 16 7 4 40.6 0 zoo Enginyeria i Arquitectura la Salle Slide 25 GRSI
- 26. Outline 1. Description of Fuzzy-UCS 1D ii fF UCS 2. 2 Experimental Methodology 3. Results 4. Conclusions Enginyeria i Arquitectura la Salle Slide 26 GRSI
- 27. 1. Description of Fuzzy-UCS 2. Experimental Methodology Results 3. Results 4. Conclusions 4C li • 1st objective: Competitive in terms of performance Enginyeria i Arquitectura la Salle Slide 27 GRSI
- 28. 1. Description of Fuzzy-UCS 2. Experimental Methodology Results 3. Results 4. Conclusions 4C li • 2nd objective: Improve the interpretability Example of rules evolved by UCS for iris Example of rules evolved by Fuzzy-UCS for iris – Linguistic terms: {XS, S, M, L, XL} Enginyeria i Arquitectura la Salle Slide 28 GRSI
- 29. 1. Description of Fuzzy-UCS 2. Experimental Methodology Further work 3. Results 4C li 4. Conclusions Still large rule-sets! Fuzzy-UCS Fuzzy UCS UCS 2769 4494 annealing 1212 2177 balance bl 1440 2961 bupa 2799 3359 glass 3574 2977 heart-c 2415 3735 heart-s 480 1039 iris 3130 2334 wbcd 3686 3685 wine 773 1291 zoo Solution: New inference schemes Enginyeria i Arquitectura la Salle Slide 29 GRSI
- 30. 1. Description of Fuzzy-UCS 2. Experimental Methodology Further work 3. Results 4C li 4. Conclusions Still large rule-sets! Fuzzy-UCS y Fuzzy-UCS Fuzzy UCS UCS best rule 36 2769 4494 annealing 75 1212 2177 balance bl 39 1440 2961 bupa 36 2799 3359 glass 46 3574 2977 heart-c 62 2415 3735 heart-s 7 480 1039 iris 28 3130 2334 wbcd 26 3686 3685 wine 10 773 1291 zoo Solution: New inference schemes Enginyeria i Arquitectura la Salle Slide 30 GRSI
- 31. Outline 1. Description of Fuzzy-UCS 1D ii fF UCS 2. 2 Experimental Methodology 3. Results 4. Conclusions Enginyeria i Arquitectura la Salle Slide 31 GRSI
- 32. 1. Description of Fuzzy-UCS 2. Experimental Methodology Conclusions and Further Work 3. Results 4. Conclusions 4C li Conclusions – We proposed a Michigan-style LFCS for supervised learning – Competitive with respect to: • Some of the most-used machine learners: UCS, SMO, and C4.5 • Recent proposals of Fuzzy-learners: Fuzzy LogitBoost and Fuzzy GP – Improvement in terms of interpretability with respect to UCS Further work – Evolve more reduced populations – Enhance the comparison with new real-world problems – Compare to other LFCS – Exploit the incremental learning approach to dig large datasets Enginyeria i Arquitectura la Salle Slide 32 GRSI
- 33. Aprendizaje Supervisado de Reglas Difusas mediante un Sistema Clasificador Evolutivo Estilo Michigan Albert Orriols-Puig1,2 Orriols Puig Jorge Casillas2 Ester Bernadó-Mansilla1 1Grup de Recerca en Sistemes Intel·ligents Enginyeria i Arquitectura La Salle, Universitat Ramon Llull 2Dept. de Ciencias de la Computación e IA Universidad of Granada