SlideShare a Scribd company logo

Decision Tree Learning

•Download as PPTX, PDF•
•
Md. Ariful Hoque
Md. Ariful Hoque

Basic of Decision Tree Learning. This slide includes definition of decision tree, basic example, basic construction of a decision tree, mathlab example

Science
1 of 21
Md. Ariful Hoque Roll: MSC 140202 Khulna University
 Introduction.  Decision Tree Terms.  Example.  Constructing A decision Tree.  Calculation Of Entropy.  Information Gain.  Gini Impurity.  Termination Criteria.  Mathlab Example.  Implementations.  Advantage  Limitation  Conclusion
 Decision tree learning is the construction of a decision tree from class- labeled training tuples.  A decision tree is model of decisions and their possible consequences.  It Includes chance event outcomes, resource costs, and utility.  Its follow top down approach.  Decision trees classify instances by sorting them down the tree from the root to some leaf node, which provides the classification of the instance
Root Node Condition Check Leaf Node(Decision Point) Leaf Node(Decision Point) Condition Check
Yes Color = Green ? Size = Big ? Color = Yellow ? Size =medium?Watermelon Apple Grape Shape = Round? Size = Small ? Taste = sweet? Apple Yes No Yes No Cherry Grape Size = Big? Grape Lemon Banana Yes YesYes Yes Yes No No No No No
There are many specific decision-tree algorithms-  ID3  C4.5  CART  CHAID  MARS
 Which attribute to choose? ◦ Information Gain  ENTROPY  Where to stop? ◦ Termination criteria
 Different algorithms use different metrics for measuring best.  These generally measure the homogeneity of the target variable within the subsets.  Some examples are given in the next slides.
◦ Entropy is a measure of uncertainty in the data Entropy(S) = ∑(i=1 to l)-|Si|/|S| * log2(|Si|/|S|)  S = set of examples  Si = subset of S with value vi under the target attribute  l = size of the range of the target attribute
 Used by the ID3, C4.5 and C5.0 tree-generation algorithms.  Information gain is based on the concept of entropy from information theory. Here , fi = fraction of items m = Items
 Used by the CART (classification and regression tree).  It measure incorrectly labeled of randomly chosen element.  Gini impurity can be computed by summing the probability of each item being chosen times the probability of a mistake in categorizing that item.  It reaches its minimum (zero) when all cases in the node fall into a single target category.  To compute Gini impurity for a set of items, suppose-  Let f be the fraction of items labeled with value i in the set.
 All the records at the node belong to one class  A significant majority fraction of records belong to a single class  The segment contains only one or very small number of records  The improvement is not substantial enough to warrant making the split.
 Create a classification decision tree for Fisher's iris data:  load fisheriris;  t = classregtree(meas,species,...  'names',{'SL' 'SW' 'PL' 'PW'})  view(t)
 t =  Decision tree for classification  if PL<2.45 then node 2 elseif PL>=2.45 then node 3 else setosa  class = setosa  if PW<1.75 then node 4 elseif PW>=1.75 then node 5 else versicolor  if PL<4.95 then node 6 elseif PL>=4.95 then node 7 else versicolor  class = virginica  if PW<1.65 then node 8 elseif PW>=1.65 then node 9 else versicolor  class = virginica  class = versicolor  class = virginica
 In data mining software.  Several examples include Salford Systems CART, IBM SPSS , KNIME, Microsoft SQL Server, and scikit-learn.
 Decision-tree learners can create over-complex trees.  There are concepts that are hard to learn because decision trees do not express them easily, such as XOR, parity or multiplexer problems.  When there are more records and very less number of attributes/features.
 Simple to understand and interpret.  Requires little data preparation.  Able to handle both numerical and categorical data.  Performs well with large datasets.
 Decision tree learning is one of the predictive modeling approaches used in statistics, data mining and machine learning.  In our example section we saw a classification tree.  Where the target variable can take a finite set of values.  In Mathlab example section we saw regression trees.  Where the target variable can take continuous values (typically real numbers).
1. Decision tree learning[Online]. Available:http://en.wikipedia.org/wiki/Decision_tree_learning 2. Classregtree[Online]. Available:http://www.mathworks.com/help/stats/classregtree.html 3. Richard O.Duda, Peter E. Hart, David G. Stok. Pattern Classification. Second Edition 4. Breiman, L., J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Boca Raton, FL: CRC Press, 1984.
Thanks

Recommended

Decision trees in Machine Learning
Decision trees in Machine Learning Decision trees in Machine Learning
Decision trees in Machine Learning Mohammad Junaid Khan
 
Decision tree
Decision treeDecision tree
Decision treeSoujanya V
 
Decision Trees
Decision TreesDecision Trees
Decision TreesStudent
 
Machine Learning with Decision trees
Machine Learning with Decision treesMachine Learning with Decision trees
Machine Learning with Decision treesKnoldus Inc.
 
Decision tree
Decision treeDecision tree
Decision treeAmi_Surati
 
Decision tree
Decision treeDecision tree
Decision treeshivani saluja
 
Random forest
Random forestRandom forest
Random forestMusa Hawamdah
 
Decision tree
Decision treeDecision tree
Decision treeSEMINARGROOT
 

Recommended

Decision trees in Machine Learning
Decision trees in Machine Learning Decision trees in Machine Learning
Decision trees in Machine Learning Mohammad Junaid Khan
 
Decision tree
Decision treeDecision tree
Decision treeSoujanya V
 
Decision Trees
Decision TreesDecision Trees
Decision TreesStudent
 
Machine Learning with Decision trees
Machine Learning with Decision treesMachine Learning with Decision trees
Machine Learning with Decision treesKnoldus Inc.
 
Decision tree
Decision treeDecision tree
Decision treeAmi_Surati
 
Decision tree
Decision treeDecision tree
Decision treeshivani saluja
 
Random forest
Random forestRandom forest
Random forestMusa Hawamdah
 
Decision tree
Decision treeDecision tree
Decision treeSEMINARGROOT
 
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Simplilearn
 
Decision Trees for Classification: A Machine Learning Algorithm
Decision Trees for Classification: A Machine Learning AlgorithmDecision Trees for Classification: A Machine Learning Algorithm
Decision Trees for Classification: A Machine Learning AlgorithmPalin analytics
 
Decision Tree - C4.5&CART
Decision Tree - C4.5&CARTDecision Tree - C4.5&CART
Decision Tree - C4.5&CARTXueping Peng
 
Random forest
Random forestRandom forest
Random forestUjjawal
 
2.2 decision tree
2.2 decision tree2.2 decision tree
2.2 decision treeKrish_ver2
 
Decision tree
Decision treeDecision tree
Decision treeShraddhaPandey45
 
Decision Tree Learning
Decision Tree LearningDecision Tree Learning
Decision Tree LearningMilind Gokhale
 
Decision tree
Decision treeDecision tree
Decision treeR A Akerkar
 
Random forest algorithm
Random forest algorithmRandom forest algorithm
Random forest algorithmRashid Ansari
 
Decision tree and random forest
Decision tree and random forestDecision tree and random forest
Decision tree and random forestLippo Group Digital
 
Classification Using Decision tree
Classification Using Decision treeClassification Using Decision tree
Classification Using Decision treeMohd. Noor Abdul Hamid
 
k medoid clustering.pptx
k medoid clustering.pptxk medoid clustering.pptx
k medoid clustering.pptxRoshan86572
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent methodProf. Neeta Awasthy
 
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic Concepts
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic ConceptsData Mining:Concepts and Techniques, Chapter 8. Classification: Basic Concepts
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic ConceptsSalah Amean
 
supervised learning
supervised learningsupervised learning
supervised learningAmar Tripathi
 
Classification in data mining
Classification in data mining Classification in data mining
Classification in data mining Sulman Ahmed
 
Decision tree in artificial intelligence
Decision tree in artificial intelligenceDecision tree in artificial intelligence
Decision tree in artificial intelligenceMdAlAmin187
 
Ensemble learning
Ensemble learningEnsemble learning
Ensemble learningHaris Jamil
 
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...Edureka!
 
Decision trees for machine learning
Decision trees for machine learningDecision trees for machine learning
Decision trees for machine learningAmr BARAKAT
 
Decision tree
Decision treeDecision tree
Decision treeVenkata Reddy Konasani
 
Decision Trees
Decision TreesDecision Trees
Decision TreesInternational School of Engineering
 

More Related Content

What's hot

Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Simplilearn
 
Decision Trees for Classification: A Machine Learning Algorithm
Decision Trees for Classification: A Machine Learning AlgorithmDecision Trees for Classification: A Machine Learning Algorithm
Decision Trees for Classification: A Machine Learning AlgorithmPalin analytics
 
Decision Tree - C4.5&CART
Decision Tree - C4.5&CARTDecision Tree - C4.5&CART
Decision Tree - C4.5&CARTXueping Peng
 
Random forest
Random forestRandom forest
Random forestUjjawal
 
2.2 decision tree
2.2 decision tree2.2 decision tree
2.2 decision treeKrish_ver2
 
Decision Tree Learning
Decision Tree LearningDecision Tree Learning
Decision Tree LearningMilind Gokhale
 
Random forest algorithm
Random forest algorithmRandom forest algorithm
Random forest algorithmRashid Ansari
 
k medoid clustering.pptx
k medoid clustering.pptxk medoid clustering.pptx
k medoid clustering.pptxRoshan86572
 
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic Concepts
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic ConceptsData Mining:Concepts and Techniques, Chapter 8. Classification: Basic Concepts
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic ConceptsSalah Amean
 
Classification in data mining
Classification in data mining Classification in data mining
Classification in data mining Sulman Ahmed
 
Decision tree in artificial intelligence
Decision tree in artificial intelligenceDecision tree in artificial intelligence
Decision tree in artificial intelligenceMdAlAmin187
 
Ensemble learning
Ensemble learningEnsemble learning
Ensemble learningHaris Jamil
 
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...Edureka!
 
Decision trees for machine learning
Decision trees for machine learningDecision trees for machine learning
Decision trees for machine learningAmr BARAKAT
 

What's hot (20)

Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
Random Forest Algorithm - Random Forest Explained | Random Forest In Machine ...
 
Decision Trees for Classification: A Machine Learning Algorithm
Decision Trees for Classification: A Machine Learning AlgorithmDecision Trees for Classification: A Machine Learning Algorithm
Decision Trees for Classification: A Machine Learning Algorithm
 
Decision Tree - C4.5&CART
Decision Tree - C4.5&CARTDecision Tree - C4.5&CART
Decision Tree - C4.5&CART
 
Random forest
Random forestRandom forest
Random forest
 
2.2 decision tree
2.2 decision tree2.2 decision tree
2.2 decision tree
 
Decision tree
Decision treeDecision tree
Decision tree
 
Decision Tree Learning
Decision Tree LearningDecision Tree Learning
Decision Tree Learning
 
Decision tree
Decision treeDecision tree
Decision tree
 
Random forest algorithm
Random forest algorithmRandom forest algorithm
Random forest algorithm
 
Decision tree and random forest
Decision tree and random forestDecision tree and random forest
Decision tree and random forest
 
Classification Using Decision tree
Classification Using Decision treeClassification Using Decision tree
Classification Using Decision tree
 
k medoid clustering.pptx
k medoid clustering.pptxk medoid clustering.pptx
k medoid clustering.pptx
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent method
 
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic Concepts
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic ConceptsData Mining:Concepts and Techniques, Chapter 8. Classification: Basic Concepts
Data Mining:Concepts and Techniques, Chapter 8. Classification: Basic Concepts
 
supervised learning
supervised learningsupervised learning
supervised learning
 
Classification in data mining
Classification in data mining Classification in data mining
Classification in data mining
 
Decision tree in artificial intelligence
Decision tree in artificial intelligenceDecision tree in artificial intelligence
Decision tree in artificial intelligence
 
Ensemble learning
Ensemble learningEnsemble learning
Ensemble learning
 
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...
Decision Tree Algorithm | Decision Tree in Python | Machine Learning Algorith...
 
Decision trees for machine learning
Decision trees for machine learningDecision trees for machine learning
Decision trees for machine learning
 

Viewers also liked

Decision tree powerpoint presentation templates
Decision tree powerpoint presentation templatesDecision tree powerpoint presentation templates
Decision tree powerpoint presentation templatesSlideTeam.net
 
Decision in Risk EVPI
Decision in Risk EVPIDecision in Risk EVPI
Decision in Risk EVPIJavaid Toosy
 
Decision tree for Predictive Modeling
Decision tree for Predictive ModelingDecision tree for Predictive Modeling
Decision tree for Predictive ModelingEdureka!
 
Decision Making and Risk Taking
Decision Making and Risk TakingDecision Making and Risk Taking
Decision Making and Risk TakingReina Mae Salido
 
Decision Tree- M.B.A -DecSci
Decision Tree- M.B.A -DecSciDecision Tree- M.B.A -DecSci
Decision Tree- M.B.A -DecSciLesly Lising
 
Data mining technique (decision tree)
Data mining technique (decision tree)Data mining technique (decision tree)
Data mining technique (decision tree)Shweta Ghate
 
Graduate problem solving
Graduate problem solvingGraduate problem solving
Graduate problem solvingPaul Walsh
 
Machine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree LearningMachine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree Learningbutest
 
Decision Making Process a Practical Example
Decision Making Process a Practical ExampleDecision Making Process a Practical Example
Decision Making Process a Practical ExampleAbid Siddiqui
 
Decision Tree Analysis
Decision Tree AnalysisDecision Tree Analysis
Decision Tree AnalysisAnand Arora
 

Viewers also liked (20)

Decision tree
Decision treeDecision tree
Decision tree
 
Decision Trees
Decision TreesDecision Trees
Decision Trees
 
Decision tree
Decision treeDecision tree
Decision tree
 
Decision tree Using c4.5 Algorithm
Decision tree Using c4.5 AlgorithmDecision tree Using c4.5 Algorithm
Decision tree Using c4.5 Algorithm
 
Decision tree powerpoint presentation templates
Decision tree powerpoint presentation templatesDecision tree powerpoint presentation templates
Decision tree powerpoint presentation templates
 
Decision tree example problem
Decision tree example problemDecision tree example problem
Decision tree example problem
 
Decision in Risk EVPI
Decision in Risk EVPIDecision in Risk EVPI
Decision in Risk EVPI
 
Decision tree for Predictive Modeling
Decision tree for Predictive ModelingDecision tree for Predictive Modeling
Decision tree for Predictive Modeling
 
Decision Making and Risk Taking
Decision Making and Risk TakingDecision Making and Risk Taking
Decision Making and Risk Taking
 
Decision Tree- M.B.A -DecSci
Decision Tree- M.B.A -DecSciDecision Tree- M.B.A -DecSci
Decision Tree- M.B.A -DecSci
 
About sourcing: Decision Tree
About sourcing: Decision TreeAbout sourcing: Decision Tree
About sourcing: Decision Tree
 
Data mining technique (decision tree)
Data mining technique (decision tree)Data mining technique (decision tree)
Data mining technique (decision tree)
 
Graduate problem solving
Graduate problem solvingGraduate problem solving
Graduate problem solving
 
Machine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree LearningMachine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree Learning
 
Decision tree
Decision treeDecision tree
Decision tree
 
Decision Making Process a Practical Example
Decision Making Process a Practical ExampleDecision Making Process a Practical Example
Decision Making Process a Practical Example
 
Decision Tree Analysis
Decision Tree AnalysisDecision Tree Analysis
Decision Tree Analysis
 
Decision tree
Decision treeDecision tree
Decision tree
 
Decision making process
Decision making processDecision making process
Decision making process
 
Decision theory
Decision theoryDecision theory
Decision theory
 

Similar to Decision Tree Learning

Introduction to Data Mining
Introduction to Data MiningIntroduction to Data Mining
Introduction to Data MiningKai Koenig
 
Perfomance Comparison of Decsion Tree Algorithms to Findout the Reason for St...
Perfomance Comparison of Decsion Tree Algorithms to Findout the Reason for St...Perfomance Comparison of Decsion Tree Algorithms to Findout the Reason for St...
Perfomance Comparison of Decsion Tree Algorithms to Findout the Reason for St...ijcnes
 
Machine learning session6(decision trees random forrest)
Machine learning session6(decision trees random forrest)Machine learning session6(decision trees random forrest)
Machine learning session6(decision trees random forrest)Abhimanyu Dwivedi
 
Data Science - Part V - Decision Trees & Random Forests
Data Science - Part V - Decision Trees & Random Forests Data Science - Part V - Decision Trees & Random Forests
Data Science - Part V - Decision Trees & Random Forests Derek Kane
 
Classfication Basic.ppt
Classfication Basic.pptClassfication Basic.ppt
Classfication Basic.ppthenonah
 
Data Mining Concepts and Techniques.ppt
Data Mining Concepts and Techniques.pptData Mining Concepts and Techniques.ppt
Data Mining Concepts and Techniques.pptRvishnupriya2
 
Data Mining Concepts and Techniques.ppt
Data Mining Concepts and Techniques.pptData Mining Concepts and Techniques.ppt
Data Mining Concepts and Techniques.pptRvishnupriya2
 
Classification (ML).ppt
Classification (ML).pptClassification (ML).ppt
Classification (ML).pptrajasamal1999
 
www1.cs.columbia.edu
www1.cs.columbia.eduwww1.cs.columbia.edu
www1.cs.columbia.edubutest
 
Efficient classification of big data using vfdt (very fast decision tree)
Efficient classification of big data using vfdt (very fast decision tree)Efficient classification of big data using vfdt (very fast decision tree)
Efficient classification of big data using vfdt (very fast decision tree)eSAT Journals
 
Data Mining In Market Research
Data Mining In Market ResearchData Mining In Market Research
Data Mining In Market Researchjim
 
Data Mining in Market Research
Data Mining in Market ResearchData Mining in Market Research
Data Mining in Market Researchbutest
 
Data Mining In Market Research
Data Mining In Market ResearchData Mining In Market Research
Data Mining In Market Researchkevinlan
 
Artificial intyelligence and machine learning introduction.pptx
Artificial intyelligence and machine learning introduction.pptxArtificial intyelligence and machine learning introduction.pptx
Artificial intyelligence and machine learning introduction.pptxChandrakalaV15
 
Slide3.ppt
Slide3.pptSlide3.ppt
Slide3.pptbutest
 

Similar to Decision Tree Learning (20)

Introduction to Data Mining
Introduction to Data MiningIntroduction to Data Mining
Introduction to Data Mining
 
Perfomance Comparison of Decsion Tree Algorithms to Findout the Reason for St...
Perfomance Comparison of Decsion Tree Algorithms to Findout the Reason for St...Perfomance Comparison of Decsion Tree Algorithms to Findout the Reason for St...
Perfomance Comparison of Decsion Tree Algorithms to Findout the Reason for St...
 
Machine learning session6(decision trees random forrest)
Machine learning session6(decision trees random forrest)Machine learning session6(decision trees random forrest)
Machine learning session6(decision trees random forrest)
 
Decision tree
Decision tree Decision tree
Decision tree
 
Dbm630 lecture06
Dbm630 lecture06Dbm630 lecture06
Dbm630 lecture06
 
Data Science - Part V - Decision Trees & Random Forests
Data Science - Part V - Decision Trees & Random Forests Data Science - Part V - Decision Trees & Random Forests
Data Science - Part V - Decision Trees & Random Forests
 
Classfication Basic.ppt
Classfication Basic.pptClassfication Basic.ppt
Classfication Basic.ppt
 
Classification
ClassificationClassification
Classification
 
Classification
ClassificationClassification
Classification
 
Data Mining Concepts and Techniques.ppt
Data Mining Concepts and Techniques.pptData Mining Concepts and Techniques.ppt
Data Mining Concepts and Techniques.ppt
 
Data Mining Concepts and Techniques.ppt
Data Mining Concepts and Techniques.pptData Mining Concepts and Techniques.ppt
Data Mining Concepts and Techniques.ppt
 
Classification (ML).ppt
Classification (ML).pptClassification (ML).ppt
Classification (ML).ppt
 
www1.cs.columbia.edu
www1.cs.columbia.eduwww1.cs.columbia.edu
www1.cs.columbia.edu
 
Efficient classification of big data using vfdt (very fast decision tree)
Efficient classification of big data using vfdt (very fast decision tree)Efficient classification of big data using vfdt (very fast decision tree)
Efficient classification of big data using vfdt (very fast decision tree)
 
Data Mining In Market Research
Data Mining In Market ResearchData Mining In Market Research
Data Mining In Market Research
 
Data Mining in Market Research
Data Mining in Market ResearchData Mining in Market Research
Data Mining in Market Research
 
Data Mining In Market Research
Data Mining In Market ResearchData Mining In Market Research
Data Mining In Market Research
 
Artificial intyelligence and machine learning introduction.pptx
Artificial intyelligence and machine learning introduction.pptxArtificial intyelligence and machine learning introduction.pptx
Artificial intyelligence and machine learning introduction.pptx
 
data mining.pptx
data mining.pptxdata mining.pptx
data mining.pptx
 
Slide3.ppt
Slide3.pptSlide3.ppt
Slide3.ppt
 

More from Md. Ariful Hoque

Automated Payment Gateway for small shops
Automated Payment Gateway for small shopsAutomated Payment Gateway for small shops
Automated Payment Gateway for small shopsMd. Ariful Hoque
 
Cost effective arena active protection system
Cost effective arena active protection systemCost effective arena active protection system
Cost effective arena active protection systemMd. Ariful Hoque
 
Implementation of RSA Algorithm for Speech Data Encryption and Decryption
Implementation of RSA Algorithm for Speech Data Encryption and DecryptionImplementation of RSA Algorithm for Speech Data Encryption and Decryption
Implementation of RSA Algorithm for Speech Data Encryption and DecryptionMd. Ariful Hoque
 

More from Md. Ariful Hoque (8)

Automated Payment Gateway for small shops
Automated Payment Gateway for small shopsAutomated Payment Gateway for small shops
Automated Payment Gateway for small shops
 
µIP (micro IP)
µIP (micro IP)µIP (micro IP)
µIP (micro IP)
 
Poz Software
Poz SoftwarePoz Software
Poz Software
 
Show me
Show meShow me
Show me
 
EIMS software for schools
EIMS software for schoolsEIMS software for schools
EIMS software for schools
 
Cost effective arena active protection system
Cost effective arena active protection systemCost effective arena active protection system
Cost effective arena active protection system
 
Amazon ec2cluster
Amazon ec2clusterAmazon ec2cluster
Amazon ec2cluster
 
Implementation of RSA Algorithm for Speech Data Encryption and Decryption
Implementation of RSA Algorithm for Speech Data Encryption and DecryptionImplementation of RSA Algorithm for Speech Data Encryption and Decryption
Implementation of RSA Algorithm for Speech Data Encryption and Decryption
 

Recently uploaded

Zoology 4th semester series (krishna).pdf
Zoology 4th semester series (krishna).pdfZoology 4th semester series (krishna).pdf
Zoology 4th semester series (krishna).pdfSumit Kumar yadav
 
GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)Areesha Ahmad
 
Presentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxPresentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxgindu3009
 
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 bAsymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 bSérgio Sacani
 
VIRUSES structure and classification ppt by Dr.Prince C P
VIRUSES structure and classification ppt by Dr.Prince C PVIRUSES structure and classification ppt by Dr.Prince C P
VIRUSES structure and classification ppt by Dr.Prince C PPRINCE C P
 
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 60009654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000Sapana Sha
 
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxPhysiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxAArockiyaNisha
 
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticsPulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticssakshisoni2385
 
Forensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfForensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfrohankumarsinghrore1
 
Disentangling the origin of chemical differences using GHOST
Disentangling the origin of chemical differences using GHOSTDisentangling the origin of chemical differences using GHOST
Disentangling the origin of chemical differences using GHOSTSérgio Sacani
 
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...ssifa0344
 
Pests of mustard_Identification_Management_Dr.UPR.pdf
Pests of mustard_Identification_Management_Dr.UPR.pdfPests of mustard_Identification_Management_Dr.UPR.pdf
Pests of mustard_Identification_Management_Dr.UPR.pdfPirithiRaju
 
DIFFERENCE IN BACK CROSS AND TEST CROSS
DIFFERENCE IN BACK CROSS AND TEST CROSSDIFFERENCE IN BACK CROSS AND TEST CROSS
DIFFERENCE IN BACK CROSS AND TEST CROSSLeenakshiTyagi
 
Biological Classification BioHack (3).pdf
Biological Classification BioHack (3).pdfBiological Classification BioHack (3).pdf
Biological Classification BioHack (3).pdfmuntazimhurra
 
Nanoparticles synthesis and characterization​ ​
Nanoparticles synthesis and characterization​ ​Nanoparticles synthesis and characterization​ ​
Nanoparticles synthesis and characterization​ ​kaibalyasahoo82800
 
fundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomologyfundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomologyDrAnita Sharma
 
Recombination DNA Technology (Nucleic Acid Hybridization )
Recombination DNA Technology (Nucleic Acid Hybridization )Recombination DNA Technology (Nucleic Acid Hybridization )
Recombination DNA Technology (Nucleic Acid Hybridization )aarthirajkumar25
 
Pests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPirithiRaju
 
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...anilsa9823
 

Recently uploaded (20)

Zoology 4th semester series (krishna).pdf
Zoology 4th semester series (krishna).pdfZoology 4th semester series (krishna).pdf
Zoology 4th semester series (krishna).pdf
 
GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)
 
Presentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxPresentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptx
 
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 bAsymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
 
VIRUSES structure and classification ppt by Dr.Prince C P
VIRUSES structure and classification ppt by Dr.Prince C PVIRUSES structure and classification ppt by Dr.Prince C P
VIRUSES structure and classification ppt by Dr.Prince C P
 
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 60009654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
9654467111 Call Girls In Raj Nagar Delhi Short 1500 Night 6000
 
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxPhysiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
 
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticsPulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
 
Forensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfForensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdf
 
Disentangling the origin of chemical differences using GHOST
Disentangling the origin of chemical differences using GHOSTDisentangling the origin of chemical differences using GHOST
Disentangling the origin of chemical differences using GHOST
 
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
 
Pests of mustard_Identification_Management_Dr.UPR.pdf
Pests of mustard_Identification_Management_Dr.UPR.pdfPests of mustard_Identification_Management_Dr.UPR.pdf
Pests of mustard_Identification_Management_Dr.UPR.pdf
 
DIFFERENCE IN BACK CROSS AND TEST CROSS
DIFFERENCE IN BACK CROSS AND TEST CROSSDIFFERENCE IN BACK CROSS AND TEST CROSS
DIFFERENCE IN BACK CROSS AND TEST CROSS
 
Biological Classification BioHack (3).pdf
Biological Classification BioHack (3).pdfBiological Classification BioHack (3).pdf
Biological Classification BioHack (3).pdf
 
Nanoparticles synthesis and characterization​ ​
Nanoparticles synthesis and characterization​ ​Nanoparticles synthesis and characterization​ ​
Nanoparticles synthesis and characterization​ ​
 
The Philosophy of Science
The Philosophy of ScienceThe Philosophy of Science
The Philosophy of Science
 
fundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomologyfundamental of entomology all in one topics of entomology
fundamental of entomology all in one topics of entomology
 
Recombination DNA Technology (Nucleic Acid Hybridization )
Recombination DNA Technology (Nucleic Acid Hybridization )Recombination DNA Technology (Nucleic Acid Hybridization )
Recombination DNA Technology (Nucleic Acid Hybridization )
 
Pests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdf
 
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
Lucknow 💋 Russian Call Girls Lucknow Finest Escorts Service 8923113531 Availa...
 

Decision Tree Learning

  • 1. Md. Ariful Hoque Roll: MSC 140202 Khulna University
  • 2.  Introduction.  Decision Tree Terms.  Example.  Constructing A decision Tree.  Calculation Of Entropy.  Information Gain.  Gini Impurity.  Termination Criteria.  Mathlab Example.  Implementations.  Advantage  Limitation  Conclusion
  • 3.  Decision tree learning is the construction of a decision tree from class- labeled training tuples.  A decision tree is model of decisions and their possible consequences.  It Includes chance event outcomes, resource costs, and utility.  Its follow top down approach.  Decision trees classify instances by sorting them down the tree from the root to some leaf node, which provides the classification of the instance
  • 4. Root Node Condition Check Leaf Node(Decision Point) Leaf Node(Decision Point) Condition Check
  • 5. Yes Color = Green ? Size = Big ? Color = Yellow ? Size =medium?Watermelon Apple Grape Shape = Round? Size = Small ? Taste = sweet? Apple Yes No Yes No Cherry Grape Size = Big? Grape Lemon Banana Yes YesYes Yes Yes No No No No No
  • 6. There are many specific decision-tree algorithms-  ID3  C4.5  CART  CHAID  MARS
  • 7.  Which attribute to choose? ◦ Information Gain  ENTROPY  Where to stop? ◦ Termination criteria
  • 8.  Different algorithms use different metrics for measuring best.  These generally measure the homogeneity of the target variable within the subsets.  Some examples are given in the next slides.
  • 9. ◦ Entropy is a measure of uncertainty in the data Entropy(S) = ∑(i=1 to l)-|Si|/|S| * log2(|Si|/|S|)  S = set of examples  Si = subset of S with value vi under the target attribute  l = size of the range of the target attribute
  • 10.  Used by the ID3, C4.5 and C5.0 tree-generation algorithms.  Information gain is based on the concept of entropy from information theory. Here , fi = fraction of items m = Items
  • 11.  Used by the CART (classification and regression tree).  It measure incorrectly labeled of randomly chosen element.  Gini impurity can be computed by summing the probability of each item being chosen times the probability of a mistake in categorizing that item.  It reaches its minimum (zero) when all cases in the node fall into a single target category.  To compute Gini impurity for a set of items, suppose-  Let f be the fraction of items labeled with value i in the set.
  • 12.  All the records at the node belong to one class  A significant majority fraction of records belong to a single class  The segment contains only one or very small number of records  The improvement is not substantial enough to warrant making the split.
  • 13.  Create a classification decision tree for Fisher's iris data:  load fisheriris;  t = classregtree(meas,species,...  'names',{'SL' 'SW' 'PL' 'PW'})  view(t)
  • 14.  t =  Decision tree for classification  if PL<2.45 then node 2 elseif PL>=2.45 then node 3 else setosa  class = setosa  if PW<1.75 then node 4 elseif PW>=1.75 then node 5 else versicolor  if PL<4.95 then node 6 elseif PL>=4.95 then node 7 else versicolor  class = virginica  if PW<1.65 then node 8 elseif PW>=1.65 then node 9 else versicolor  class = virginica  class = versicolor  class = virginica
  • 15.
  • 16.  In data mining software.  Several examples include Salford Systems CART, IBM SPSS , KNIME, Microsoft SQL Server, and scikit-learn.
  • 17.  Decision-tree learners can create over-complex trees.  There are concepts that are hard to learn because decision trees do not express them easily, such as XOR, parity or multiplexer problems.  When there are more records and very less number of attributes/features.
  • 18.  Simple to understand and interpret.  Requires little data preparation.  Able to handle both numerical and categorical data.  Performs well with large datasets.
  • 19.  Decision tree learning is one of the predictive modeling approaches used in statistics, data mining and machine learning.  In our example section we saw a classification tree.  Where the target variable can take a finite set of values.  In Mathlab example section we saw regression trees.  Where the target variable can take continuous values (typically real numbers).
  • 20. 1. Decision tree learning[Online]. Available:http://en.wikipedia.org/wiki/Decision_tree_learning 2. Classregtree[Online]. Available:http://www.mathworks.com/help/stats/classregtree.html 3. Richard O.Duda, Peter E. Hart, David G. Stok. Pattern Classification. Second Edition 4. Breiman, L., J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Boca Raton, FL: CRC Press, 1984.