Machine Learning: Decision Trees Chapter 18.1-18.3

B
Machine Learning: Decision Trees Chapter 18.1-18.3 Some material adopted from notes by  Chuck Dyer
What is learning? ,[object Object],[object Object],[object Object]
Why study learning? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
A general model of learning agents
Major paradigms of machine learning ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
The inductive learning problem ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Supervised concept learning ,[object Object],[object Object],[object Object]
Inductive learning framework ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Inductive learning as search ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Model spaces ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Model spaces + + - - Nearest neighbor Version space Decision tree I I + + - - I + + - -
Inductive learning and bias ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Preference bias: Ockham’s Razor ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Learning decision trees ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Color Shape Size + + - Size + - + big big small small round square red green blue
Decision tree-induced partition – example I Color Shape Size + + - Size + - + big big small small round square red green blue
Expressiveness ,[object Object],[object Object],[object Object],[object Object]
Hypothesis spaces ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
R&N’s restaurant domain ,[object Object],[object Object],[object Object],[object Object],[object Object]
A decision tree from introspection
Attribute-based representations ,[object Object],[object Object],[object Object],[object Object]
ID3 Algorithm ,[object Object],[object Object],[object Object],[object Object],[object Object]
Choosing the best attribute ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Choosing an attribute ,[object Object],[object Object]
ID3-induced  decision tree
Information theory 101 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Information theory 101 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Information theory II ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Huffman code ,[object Object],[object Object],[object Object],[object Object],[object Object]
Huffman code example ,[object Object],[object Object],[object Object],[object Object],[object Object],.5 .5 1 .125 .125 .25 A C B D .25 0 1 0 0 1 1 If we use this code to many messages (A,B,C or D) with this probability distribution, then, over time, the average bits/message should approach  1.75
Information for classification ,[object Object],[object Object],[object Object],[object Object],C 1 C 2 C 3 C 1 C 2 C 3 High information Low information
Information for classification II ,[object Object],[object Object],C 1 C 2 C 3 C 1 C 2 C 3 High information Low information
Information gain ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Computing information gain ,[object Object],[object Object],[object Object],Gain (Pat, T) = 1 - .47 = .53 Gain (Type, T) = 1 – 1 = 0 French Italian Thai Burger Empty Some Full Y Y Y Y Y Y N N N N N N
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
How well does it work? ,[object Object],[object Object],[object Object],[object Object]
Extensions of the decision tree learning algorithm ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Using gain ratios ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Computing gain ratio ,[object Object],[object Object],[object Object],Gain (Pat, T) =.53 Gain (Type, T) = 0 SplitInfo (Pat, T) = - (1/6 log 1/6 + 1/3 log 1/3 + 1/2 log 1/2) = 1/6*2.6 + 1/3*1.6 + 1/2*1   = 1.47 SplitInfo (Type, T) = 1/6 log 1/6 + 1/6 log 1/6 + 1/3 log 1/3 + 1/3 log 1/3   = 1/6*2.6 + 1/6*2.6 + 1/3*1.6 + 1/3*1.6 = 1.93 GainRatio (Pat, T) = Gain (Pat, T) / SplitInfo(Pat, T) = .53 / 1.47 = .36 GainRatio (Type, T) = Gain (Type, T) / SplitInfo (Type, T) = 0 / 1.93 = 0 French Italian Thai Burger Empty Some Full Y Y Y Y Y Y N N N N N N
Real-valued data ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Noisy data and overfitting ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Pruning decision trees ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],2 success 4 failure FAILURE Training Test Pruned Color 1 success 0 failure 0 success 2 failures red blue Color 1 success 3 failure 1 success 1 failure red blue
Converting decision trees to rules ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Evaluation methodology ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Performance measurement ,[object Object],[object Object],[object Object],[object Object]
Summary: Decision tree learning ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
1 sur 45

Recommandé

3_learning.ppt par
3_learning.ppt3_learning.ppt
3_learning.pptbutest
673 vues58 diapositives
Introduction to Machine Learning. par
Introduction to Machine Learning.Introduction to Machine Learning.
Introduction to Machine Learning.butest
501 vues35 diapositives
Lecture 7 par
Lecture 7Lecture 7
Lecture 7butest
337 vues45 diapositives
Machine Learning: Foundations Course Number 0368403401 par
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401butest
618 vues60 diapositives
Machine Learning 1 - Introduction par
Machine Learning 1 - IntroductionMachine Learning 1 - Introduction
Machine Learning 1 - Introductionbutest
662 vues34 diapositives
Machine Learning par
Machine LearningMachine Learning
Machine Learningbutest
551 vues24 diapositives

Contenu connexe

Tendances

Introduction to Machine Learning Classifiers par
Introduction to Machine Learning ClassifiersIntroduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersFunctional Imperative
12.6K vues38 diapositives
Machine learning Lecture 1 par
Machine learning Lecture 1Machine learning Lecture 1
Machine learning Lecture 1Srinivasan R
25.2K vues77 diapositives
.ppt par
.ppt.ppt
.pptbutest
326 vues15 diapositives
Classifiers par
ClassifiersClassifiers
ClassifiersAyurdata
112 vues3 diapositives
Machine learning by Dr. Vivek Vijay and Dr. Sandeep Yadav par
Machine learning by Dr. Vivek Vijay and Dr. Sandeep YadavMachine learning by Dr. Vivek Vijay and Dr. Sandeep Yadav
Machine learning by Dr. Vivek Vijay and Dr. Sandeep YadavAgile Testing Alliance
860 vues51 diapositives
MachineLearning.ppt par
MachineLearning.pptMachineLearning.ppt
MachineLearning.pptbutest
18.4K vues123 diapositives

Tendances(20)

Machine learning Lecture 1 par Srinivasan R
Machine learning Lecture 1Machine learning Lecture 1
Machine learning Lecture 1
Srinivasan R25.2K vues
.ppt par butest
.ppt.ppt
.ppt
butest326 vues
Classifiers par Ayurdata
ClassifiersClassifiers
Classifiers
Ayurdata112 vues
MachineLearning.ppt par butest
MachineLearning.pptMachineLearning.ppt
MachineLearning.ppt
butest18.4K vues
Machine Learning : why we should know and how it works par Kevin Lee
Machine Learning : why we should know and how it worksMachine Learning : why we should know and how it works
Machine Learning : why we should know and how it works
Kevin Lee138 vues
Decision Tree - C4.5&CART par Xueping Peng
Decision Tree - C4.5&CARTDecision Tree - C4.5&CART
Decision Tree - C4.5&CART
Xueping Peng1.5K vues
An Overview of Naïve Bayes Classifier par ananth
An Overview of Naïve Bayes Classifier An Overview of Naïve Bayes Classifier
An Overview of Naïve Bayes Classifier
ananth3.9K vues
Machine Learning and Data Mining: 16 Classifiers Ensembles par Pier Luca Lanzi
Machine Learning and Data Mining: 16 Classifiers EnsemblesMachine Learning and Data Mining: 16 Classifiers Ensembles
Machine Learning and Data Mining: 16 Classifiers Ensembles
Pier Luca Lanzi10.8K vues
Tree models with Scikit-Learn: Great models with little assumptions par Gilles Louppe
Tree models with Scikit-Learn: Great models with little assumptionsTree models with Scikit-Learn: Great models with little assumptions
Tree models with Scikit-Learn: Great models with little assumptions
Gilles Louppe7.5K vues
week9_Machine_Learning.ppt par butest
week9_Machine_Learning.pptweek9_Machine_Learning.ppt
week9_Machine_Learning.ppt
butest412 vues
ensemble learning par butest
ensemble learningensemble learning
ensemble learning
butest1.9K vues
original par butest
originaloriginal
original
butest290 vues

En vedette

Decision trees for machine learning par
Decision trees for machine learningDecision trees for machine learning
Decision trees for machine learningAmr BARAKAT
15.5K vues88 diapositives
Lecture 7 par
Lecture 7Lecture 7
Lecture 7butest
353 vues45 diapositives
Lecture notes par
Lecture notesLecture notes
Lecture notesbutest
322 vues21 diapositives
resume.doc par
resume.docresume.doc
resume.docbutest
830 vues4 diapositives
CC282 Decision trees Lecture 2 slides for CC282 Machine ... par
CC282 Decision trees Lecture 2 slides for CC282 Machine ...CC282 Decision trees Lecture 2 slides for CC282 Machine ...
CC282 Decision trees Lecture 2 slides for CC282 Machine ...butest
1.7K vues32 diapositives
Learning par
LearningLearning
Learningbutest
1.1K vues31 diapositives

En vedette(20)

Decision trees for machine learning par Amr BARAKAT
Decision trees for machine learningDecision trees for machine learning
Decision trees for machine learning
Amr BARAKAT15.5K vues
Lecture 7 par butest
Lecture 7Lecture 7
Lecture 7
butest353 vues
Lecture notes par butest
Lecture notesLecture notes
Lecture notes
butest322 vues
resume.doc par butest
resume.docresume.doc
resume.doc
butest830 vues
CC282 Decision trees Lecture 2 slides for CC282 Machine ... par butest
CC282 Decision trees Lecture 2 slides for CC282 Machine ...CC282 Decision trees Lecture 2 slides for CC282 Machine ...
CC282 Decision trees Lecture 2 slides for CC282 Machine ...
butest1.7K vues
Learning par butest
LearningLearning
Learning
butest1.1K vues
Chapter II.6 (Book Part VI) Learning par butest
Chapter II.6 (Book Part VI) LearningChapter II.6 (Book Part VI) Learning
Chapter II.6 (Book Part VI) Learning
butest642 vues
Robot Motor Skill Coordination with EM-based Reinforcement Learning par Petar Kormushev
Robot Motor Skill Coordination with EM-based Reinforcement LearningRobot Motor Skill Coordination with EM-based Reinforcement Learning
Robot Motor Skill Coordination with EM-based Reinforcement Learning
Petar Kormushev720 vues
Machine Learning CSCI 5622 par butest
Machine Learning CSCI 5622Machine Learning CSCI 5622
Machine Learning CSCI 5622
butest828 vues
Executive Summary Hare Chevrolet is a General Motors dealership ... par butest
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...
butest6.9K vues
EL MODELO DE NEGOCIO DE YOUTUBE par butest
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBE
butest9K vues
Lecture 18: Gaussian Mixture Models and Expectation Maximization par butest
Lecture 18: Gaussian Mixture Models and Expectation MaximizationLecture 18: Gaussian Mixture Models and Expectation Maximization
Lecture 18: Gaussian Mixture Models and Expectation Maximization
butest13.3K vues
Clustering:k-means, expect-maximization and gaussian mixture model par jins0618
Clustering:k-means, expect-maximization and gaussian mixture modelClustering:k-means, expect-maximization and gaussian mixture model
Clustering:k-means, expect-maximization and gaussian mixture model
jins06182.2K vues
Data Mining Techniques for CRM par Shyaamini Balu
Data Mining Techniques for CRMData Mining Techniques for CRM
Data Mining Techniques for CRM
Shyaamini Balu7.5K vues
Decision making environment par shubhamvaghela
Decision making environmentDecision making environment
Decision making environment
shubhamvaghela24.4K vues
Machine Learning Lecture 3 Decision Trees par ananth
Machine Learning Lecture 3 Decision TreesMachine Learning Lecture 3 Decision Trees
Machine Learning Lecture 3 Decision Trees
ananth5.8K vues
2013-1 Machine Learning Lecture 02 - Decision Trees par Dongseo University
2013-1 Machine Learning Lecture 02 - Decision Trees2013-1 Machine Learning Lecture 02 - Decision Trees
2013-1 Machine Learning Lecture 02 - Decision Trees
Learning coordination strategies using reinforcement learning myriam z abrams... par Chang Ching-Chao
Learning coordination strategies using reinforcement learning myriam z abrams...Learning coordination strategies using reinforcement learning myriam z abrams...
Learning coordination strategies using reinforcement learning myriam z abrams...
Chang Ching-Chao1.4K vues

Similaire à Machine Learning: Decision Trees Chapter 18.1-18.3

ppt par
pptppt
pptbutest
1.8K vues33 diapositives
c23_ml1.ppt par
c23_ml1.pptc23_ml1.ppt
c23_ml1.pptFaiz430036
5 vues51 diapositives
ML-DecisionTrees.ppt par
ML-DecisionTrees.pptML-DecisionTrees.ppt
ML-DecisionTrees.pptssuserec53e73
3 vues46 diapositives
ML-DecisionTrees.ppt par
ML-DecisionTrees.pptML-DecisionTrees.ppt
ML-DecisionTrees.pptssuserec53e73
3 vues46 diapositives
ML-DecisionTrees.ppt par
ML-DecisionTrees.pptML-DecisionTrees.ppt
ML-DecisionTrees.pptssuser71fb63
2 vues46 diapositives
Introduction par
IntroductionIntroduction
Introductionbutest
243 vues36 diapositives

Similaire à Machine Learning: Decision Trees Chapter 18.1-18.3(20)

ppt par butest
pptppt
ppt
butest1.8K vues
Introduction par butest
IntroductionIntroduction
Introduction
butest243 vues
Introduction par butest
IntroductionIntroduction
Introduction
butest189 vues
Introduction par butest
IntroductionIntroduction
Introduction
butest284 vues
Download presentation source par butest
Download presentation sourceDownload presentation source
Download presentation source
butest205 vues
Machine Learning presentation. par butest
Machine Learning presentation.Machine Learning presentation.
Machine Learning presentation.
butest28.6K vues
slides par butest
slidesslides
slides
butest973 vues
slides par butest
slidesslides
slides
butest352 vues
20070702 Text Categorization par midi
20070702 Text Categorization20070702 Text Categorization
20070702 Text Categorization
midi1.1K vues
Introduction to Machine Learning Aristotelis Tsirigos par butest
Introduction to Machine Learning Aristotelis Tsirigos Introduction to Machine Learning Aristotelis Tsirigos
Introduction to Machine Learning Aristotelis Tsirigos
butest400 vues
MS CS - Selecting Machine Learning Algorithm par Kaniska Mandal
MS CS - Selecting Machine Learning AlgorithmMS CS - Selecting Machine Learning Algorithm
MS CS - Selecting Machine Learning Algorithm
Kaniska Mandal590 vues
Data mining approaches and methods par sonangrai
Data mining approaches and methodsData mining approaches and methods
Data mining approaches and methods
sonangrai890 vues

Plus de butest

1. MPEG I.B.P frame之不同 par
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同butest
5K vues3 diapositives
LESSONS FROM THE MICHAEL JACKSON TRIAL par
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
2.4K vues3 diapositives
Timeline: The Life of Michael Jackson par
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jacksonbutest
3.7K vues10 diapositives
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ... par
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...butest
2.1K vues6 diapositives
LESSONS FROM THE MICHAEL JACKSON TRIAL par
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
2.4K vues6 diapositives
Com 380, Summer II par
Com 380, Summer IICom 380, Summer II
Com 380, Summer IIbutest
1.7K vues52 diapositives

Plus de butest(20)

1. MPEG I.B.P frame之不同 par butest
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同
butest5K vues
LESSONS FROM THE MICHAEL JACKSON TRIAL par butest
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
butest2.4K vues
Timeline: The Life of Michael Jackson par butest
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jackson
butest3.7K vues
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ... par butest
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
butest2.1K vues
LESSONS FROM THE MICHAEL JACKSON TRIAL par butest
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
butest2.4K vues
Com 380, Summer II par butest
Com 380, Summer IICom 380, Summer II
Com 380, Summer II
butest1.7K vues
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz par butest
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
butest1.4K vues
MICHAEL JACKSON.doc par butest
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.doc
butest1.5K vues
Social Networks: Twitter Facebook SL - Slide 1 par butest
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1
butest1.4K vues
Facebook par butest
Facebook Facebook
Facebook
butest1.1K vues
Welcome to the Dougherty County Public Library's Facebook and ... par butest
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...
butest1.1K vues
NEWS ANNOUNCEMENT par butest
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENT
butest910 vues
C-2100 Ultra Zoom.doc par butest
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.doc
butest1K vues
MAC Printing on ITS Printers.doc.doc par butest
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.doc
butest1.2K vues
Mac OS X Guide.doc par butest
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.doc
butest1.6K vues
hier par butest
hierhier
hier
butest2.4K vues
WEB DESIGN! par butest
WEB DESIGN!WEB DESIGN!
WEB DESIGN!
butest1.5K vues
Download par butest
DownloadDownload
Download
butest1.3K vues
Download.doc.doc par butest
Download.doc.docDownload.doc.doc
Download.doc.doc
butest691 vues

Machine Learning: Decision Trees Chapter 18.1-18.3