SlideShare une entreprise Scribd logo
1  sur  10
Télécharger pour lire hors ligne
Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla

                                 nmacia@salle.url.edu
                     jaume.bacardit@nottingham.ac.uk
                                  esterb@salle.url.edu
Methodology for learners’ assessment
The UCI repository. 134 classification problems



Source: GECCO’11 Proceedings
Algorithm refinement        Knowledge extraction




              Behind the experiments
Standard comparison



Source: Jaume Bacardit, Edmund K. Burke, and Natalio Krasnogor. Improving the scalability of rule-based evolutionary learning. (2009)
Taxonomy of problems
Complexity coverage
• How many data sets should we use in the experiments?
• Which ones? Synthetic data sets? Real-world problems?
  Both?
• Is the UCI our best sample?
• How should we select our referenced learners?
• Should we keep performing comparisons over an arbitrary
  set of problems?
• How can we characterise problems?
• More coverage? Benchmarks?
• Should we tackle one problem at a time?
• …
                      Questions. Answers?
Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla

                                 nmacia@salle.url.edu
                     jaume.bacardit@nottingham.ac.uk
                                  esterb@salle.url.edu

Contenu connexe

Plus de Daniele Loiacono

GPUs for GEC Competition @ GECCO-2013
GPUs for GEC Competition @ GECCO-2013GPUs for GEC Competition @ GECCO-2013
GPUs for GEC Competition @ GECCO-2013Daniele Loiacono
 
EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013Daniele Loiacono
 
2013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-20132013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-2013Daniele Loiacono
 
2012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-20122012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-2012Daniele Loiacono
 
2012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-20122012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-2012Daniele Loiacono
 
2012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-20122012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-2012Daniele Loiacono
 
Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012Daniele Loiacono
 
XCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental ForgettingXCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental ForgettingDaniele Loiacono
 
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...Daniele Loiacono
 
Introducing LCS to Digital Design Verification
Introducing LCS to Digital Design VerificationIntroducing LCS to Digital Design Verification
Introducing LCS to Digital Design VerificationDaniele Loiacono
 
A temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networksA temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networksDaniele Loiacono
 
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...Daniele Loiacono
 
Automatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier SystemsAutomatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier SystemsDaniele Loiacono
 
Voting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label ClassificationVoting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label ClassificationDaniele Loiacono
 
2011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-20112011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-2011Daniele Loiacono
 
2010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-20102010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-2010Daniele Loiacono
 
2010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-20102010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-2010Daniele Loiacono
 
2010 Simulated Car Racing Championship @ WCCI-2010
2010 Simulated Car Racing Championship @ WCCI-20102010 Simulated Car Racing Championship @ WCCI-2010
2010 Simulated Car Racing Championship @ WCCI-2010Daniele Loiacono
 
Car Setup Optimization Competition @ EvoStar 2010
Car Setup Optimization Competition @ EvoStar 2010Car Setup Optimization Competition @ EvoStar 2010
Car Setup Optimization Competition @ EvoStar 2010Daniele Loiacono
 

Plus de Daniele Loiacono (20)

GPUs for GEC Competition @ GECCO-2013
GPUs for GEC Competition @ GECCO-2013GPUs for GEC Competition @ GECCO-2013
GPUs for GEC Competition @ GECCO-2013
 
EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013
 
2013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-20132013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-2013
 
2012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-20122012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-2012
 
2012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-20122012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-2012
 
2012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-20122012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-2012
 
Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012
 
XCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental ForgettingXCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental Forgetting
 
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
 
One Step Fits All
One Step Fits AllOne Step Fits All
One Step Fits All
 
Introducing LCS to Digital Design Verification
Introducing LCS to Digital Design VerificationIntroducing LCS to Digital Design Verification
Introducing LCS to Digital Design Verification
 
A temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networksA temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networks
 
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
 
Automatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier SystemsAutomatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier Systems
 
Voting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label ClassificationVoting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label Classification
 
2011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-20112011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-2011
 
2010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-20102010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-2010
 
2010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-20102010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-2010
 
2010 Simulated Car Racing Championship @ WCCI-2010
2010 Simulated Car Racing Championship @ WCCI-20102010 Simulated Car Racing Championship @ WCCI-2010
2010 Simulated Car Racing Championship @ WCCI-2010
 
Car Setup Optimization Competition @ EvoStar 2010
Car Setup Optimization Competition @ EvoStar 2010Car Setup Optimization Competition @ EvoStar 2010
Car Setup Optimization Competition @ EvoStar 2010
 

Dernier

Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Sapana Sha
 
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions  for the students and aspirants of Chemistry12th.pptxOrganic Name Reactions  for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptxVS Mahajan Coaching Centre
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDThiyagu K
 
Mastering the Unannounced Regulatory Inspection
Mastering the Unannounced Regulatory InspectionMastering the Unannounced Regulatory Inspection
Mastering the Unannounced Regulatory InspectionSafetyChain Software
 
Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Disha Kariya
 
APM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAPM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAssociation for Project Management
 
Accessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactAccessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactdawncurless
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactPECB
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104misteraugie
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeThiyagu K
 
Interactive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationInteractive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationnomboosow
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docxPoojaSen20
 
The byproduct of sericulture in different industries.pptx
The byproduct of sericulture in different industries.pptxThe byproduct of sericulture in different industries.pptx
The byproduct of sericulture in different industries.pptxShobhayan Kirtania
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityGeoBlogs
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsTechSoup
 
Arihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfArihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfchloefrazer622
 
Disha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfDisha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfchloefrazer622
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13Steve Thomason
 

Dernier (20)

Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
 
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions  for the students and aspirants of Chemistry12th.pptxOrganic Name Reactions  for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SD
 
Mastering the Unannounced Regulatory Inspection
Mastering the Unannounced Regulatory InspectionMastering the Unannounced Regulatory Inspection
Mastering the Unannounced Regulatory Inspection
 
Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..Sports & Fitness Value Added Course FY..
Sports & Fitness Value Added Course FY..
 
APM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAPM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across Sectors
 
Accessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactAccessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impact
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global Impact
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and Mode
 
Interactive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationInteractive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communication
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docx
 
The byproduct of sericulture in different industries.pptx
The byproduct of sericulture in different industries.pptxThe byproduct of sericulture in different industries.pptx
The byproduct of sericulture in different industries.pptx
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activity
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
Arihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfArihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdf
 
Disha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfDisha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdf
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13
 

Testing learning classifier systems

  • 1. Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla nmacia@salle.url.edu jaume.bacardit@nottingham.ac.uk esterb@salle.url.edu
  • 3.
  • 4. The UCI repository. 134 classification problems Source: GECCO’11 Proceedings
  • 5. Algorithm refinement Knowledge extraction Behind the experiments
  • 6. Standard comparison Source: Jaume Bacardit, Edmund K. Burke, and Natalio Krasnogor. Improving the scalability of rule-based evolutionary learning. (2009)
  • 9. • How many data sets should we use in the experiments? • Which ones? Synthetic data sets? Real-world problems? Both? • Is the UCI our best sample? • How should we select our referenced learners? • Should we keep performing comparisons over an arbitrary set of problems? • How can we characterise problems? • More coverage? Benchmarks? • Should we tackle one problem at a time? • … Questions. Answers?
  • 10. Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla nmacia@salle.url.edu jaume.bacardit@nottingham.ac.uk esterb@salle.url.edu