SlideShare une entreprise Scribd logo
1  sur  60
Télécharger pour lire hors ligne
All images from wikimedia commons, a freely-licensed media repository
“ Classifiers” R & D project by Aditya M Joshi [email_address] IIT Bombay Under the guidance of   Prof. Pushpak Bhattacharyya [email_address] IIT Bombay
Overview
Introduction to Classification
What is classification? A machine learning task that deals with identifying the class to which an instance belongs  A classifier performs classification Classifier Test instance  Attributes (a1, a2,… an) Discrete-valued Class label ( Age, Marital status,  Health status, Salary ) Issue Loan? {Yes, No} ( Perceptive inputs ) Steer? { Left, Straight, Right } Category of document? {Politics, Movies, Biology} ( Textual features : Ngrams )
Classification learning Training  phase Testing  phase Learning the classifier from the available data  ‘Training set’ (Labeled) Testing how well the classifier performs ‘Testing set’
Generating datasets ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Evaluating classifiers ,[object Object],[object Object],[object Object],[object Object],[object Object]
Decision Trees
Diagram from Han-Kamber Example tree Intermediate nodes :  Attributes Leaf nodes :  Class predictions Edges :  Attribute value tests Example algorithms:  ID3, C4.5, SPRINT, CART
Decision Tree schematic Training data set a1 a2 a3 a4 a5   a6 a1 a2 a3 a4 a5   a6 X Y Z Pure node, Leaf node: Class  RED Impure node, Select best attribute and continue Impure node, Select best attribute and continue
Decision Tree Issues ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Lazy learners
Lazy learners ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
K-NN classifier schematic ,[object Object],[object Object],[object Object],[object Object]
K-NN classifier Issues ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Decision Lists
[object Object],[object Object],[object Object],[object Object],Decision Lists f ( y ) = cj, if j  = min { i | hi (y) = 1 } exists 0 otherwise
Decision List example Test instance ( h i , c i ) Unit Class label
Decision List learning R S’ = S Set of candidate feature functions For each hi, Qi = Pi U Ni ( hi  = 1 ) U i = max { | Pi| - pn * | Ni | , |Ni| - pp *|Pi| } Select hk, the feature with highest utility ( h k,   ) If  (| Pi| - pn * | Ni | > |Ni| - pp *|Pi| )  then 1 else 0 1 / 0 - Qk
Decision list Issues ,[object Object],[object Object],[object Object],[object Object],[object Object],Accuracy / Complexity tradeoff? Size of R : Complexity (Length of the list) S’ contains examples of both classes : Accuracy (Purity) ,[object Object],[object Object],[object Object],[object Object]
Probabilistic classifiers
[object Object],[object Object],Probabilistic classifiers : NB
Naïve Bayes Issues ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Problems due to sparsity of data? Problem : Probabilities for some values may be zero Solution : Laplace smoothing For each attribute value,  update probability m / n as : (m + 1) / (n + k)  where k = domain of values
[object Object],[object Object],Probabilistic classifiers : BBN Diagram from Han-Kamber An added term for conditional  probability between attributes:
[object Object],[object Object],[object Object],[object Object],[object Object],BBN learning
[object Object],[object Object],[object Object],Learning structure of BBN Loan Age Family status Marital status
Artificial Neural Networks
[object Object],[object Object],Artificial Neural Networks w0 w1 wn threshold output: : activation function p (v) where  p (v)  = sgn (w 0  + w 1 x 1  + … + w n x n  ) input
[object Object],[object Object],[object Object],[object Object],[object Object],Perceptron learning algorithm n : learning rate t : target output o : observed output
[object Object],Sigmoid perceptron
[object Object],Multilayer feedforward networks Input layer Output layer Hidden layer Diagram from Han-Kamber
[object Object],[object Object],Backpropagation Diagram from Han-Kamber
ANN Issues Addition of momentum But why? Choosing the learning factor A small learning factor means multiple iterations  required. A large learning factor means the learner may skip the global minimum What are the types of learning  approaches? Deterministic: Update weights after summing up Errors over all examples Stochastic: Update weights per example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Support vector machines
Support vector machines ,[object Object],Separating hyperplane : wx+b = 0 Margin Support vectors “ Maximum separating-margin classifier” +1 -1
SVM training ,[object Object],Minimize (1 / 2) || w || 2 w.r.t. (y i  ( w x i  + b ) – 1) >=  0 for all i  Lagrangian multipliers are zero for data instances other than support vectors Dot product of xk and xl
Focussing on dot product ,[object Object],[object Object],[object Object]
Kernel functions ,[object Object],[object Object]
Testing SVM SVM Test instance Class  label
SVM Issues SVMs are immune to the removal of  non-support-vector points What if n-classes are to be predicted? Problem : SVMs deal with two-class classification Solution : Have multiple SVMs each for one class
Combining classifiers
Combining Classifiers ,[object Object],[object Object],[object Object],[object Object],[object Object]
Bagging Total set Sample D 1 Classifier model M 1 At random. May use bootstrap sampling with replacement Training dataset D Classifier learning scheme Classifier model M n Test  set Majority vote Class Label
Boosting (AdaBoost) Total set Sample D 1 Classifier model M 1 Selection based on weight. May use bootstrap sampling with replacement Training dataset D Classifier learning scheme Classifier model M n Test  set Weighted vote Class Label Initialize weights of instances to 1/d Weights of  correctly classified instances multiplied by error / (1 – error) If error > 0.5? Error Error `
The last slice
Data preprocessing ,[object Object],[object Object],[object Object],[object Object]
Attribute subset selection ,[object Object],[object Object]
Dimensionality reduction ,[object Object],Number of attributes of  a data instance instance x in p-dimensions instance x in k-dimensions k < p s = Wx W is k x p transformation mtrx.
Principal Component Analysis ,[object Object],[object Object],Eigenvector matrix ( p X p ) First k are k PCs (  p X n ) ( p X n ) (k X n)  (p X n) (k X p)  Diagram from Han-Kamber
Weka &  Weka Demo
Weka &  Weka Demo ,[object Object],[object Object],[object Object],[object Object],[object Object]
ARFF file format ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Name of the relation Attribute definition Data instances : Comma separated, each on a new line
Parts of weka Explorer Basic interface to run ML  Algorithms Experimenter Comparing experiments on different algorithms Knowledge Flow Similar to Work Flow ‘ Customized’ to one’s needs
Weka demo
Key References ,[object Object],[object Object],[object Object]
end of  slideshow
Extra slides 1 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Learning structure of BBN
[object Object],[object Object],Delta rule

Contenu connexe

Tendances

Application of combined support vector machines in process fault diagnosis
Application of combined support vector machines in process fault diagnosisApplication of combined support vector machines in process fault diagnosis
Application of combined support vector machines in process fault diagnosisDr.Pooja Jain
 
Machine learning in science and industry — day 2
Machine learning in science and industry — day 2Machine learning in science and industry — day 2
Machine learning in science and industry — day 2arogozhnikov
 
Introduction to Boosted Trees by Tianqi Chen
Introduction to Boosted Trees by Tianqi ChenIntroduction to Boosted Trees by Tianqi Chen
Introduction to Boosted Trees by Tianqi ChenZhuyi Xue
 
MLHEP Lectures - day 2, basic track
MLHEP Lectures - day 2, basic trackMLHEP Lectures - day 2, basic track
MLHEP Lectures - day 2, basic trackarogozhnikov
 
Machine learning in science and industry — day 3
Machine learning in science and industry — day 3Machine learning in science and industry — day 3
Machine learning in science and industry — day 3arogozhnikov
 
MLHEP 2015: Introductory Lecture #4
MLHEP 2015: Introductory Lecture #4MLHEP 2015: Introductory Lecture #4
MLHEP 2015: Introductory Lecture #4arogozhnikov
 
Lecture 2
Lecture 2Lecture 2
Lecture 2butest
 
ECCV2010: distance function and metric learning part 2
ECCV2010: distance function and metric learning part 2ECCV2010: distance function and metric learning part 2
ECCV2010: distance function and metric learning part 2zukun
 
Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...
Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...
Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...Marina Santini
 
Reweighting and Boosting to uniforimty in HEP
Reweighting and Boosting to uniforimty in HEPReweighting and Boosting to uniforimty in HEP
Reweighting and Boosting to uniforimty in HEParogozhnikov
 
Neural Learning to Rank
Neural Learning to RankNeural Learning to Rank
Neural Learning to RankBhaskar Mitra
 
2.6 support vector machines and associative classifiers revised
2.6 support vector machines and associative classifiers revised2.6 support vector machines and associative classifiers revised
2.6 support vector machines and associative classifiers revisedKrish_ver2
 
MLHEP 2015: Introductory Lecture #3
MLHEP 2015: Introductory Lecture #3MLHEP 2015: Introductory Lecture #3
MLHEP 2015: Introductory Lecture #3arogozhnikov
 
Learning to Rank with Neural Networks
Learning to Rank with Neural NetworksLearning to Rank with Neural Networks
Learning to Rank with Neural NetworksBhaskar Mitra
 
[Paper reading] L-SHAPLEY AND C-SHAPLEY: EFFICIENT MODEL INTERPRETATION FOR S...
[Paper reading] L-SHAPLEY AND C-SHAPLEY: EFFICIENT MODEL INTERPRETATION FOR S...[Paper reading] L-SHAPLEY AND C-SHAPLEY: EFFICIENT MODEL INTERPRETATION FOR S...
[Paper reading] L-SHAPLEY AND C-SHAPLEY: EFFICIENT MODEL INTERPRETATION FOR S...Daiki Tanaka
 
MLHEP 2015: Introductory Lecture #1
MLHEP 2015: Introductory Lecture #1MLHEP 2015: Introductory Lecture #1
MLHEP 2015: Introductory Lecture #1arogozhnikov
 
Neural Learning to Rank
Neural Learning to RankNeural Learning to Rank
Neural Learning to RankBhaskar Mitra
 
MLHEP Lectures - day 3, basic track
MLHEP Lectures - day 3, basic trackMLHEP Lectures - day 3, basic track
MLHEP Lectures - day 3, basic trackarogozhnikov
 

Tendances (20)

Application of combined support vector machines in process fault diagnosis
Application of combined support vector machines in process fault diagnosisApplication of combined support vector machines in process fault diagnosis
Application of combined support vector machines in process fault diagnosis
 
Machine Learning
Machine LearningMachine Learning
Machine Learning
 
Machine learning in science and industry — day 2
Machine learning in science and industry — day 2Machine learning in science and industry — day 2
Machine learning in science and industry — day 2
 
Introduction to Boosted Trees by Tianqi Chen
Introduction to Boosted Trees by Tianqi ChenIntroduction to Boosted Trees by Tianqi Chen
Introduction to Boosted Trees by Tianqi Chen
 
MLHEP Lectures - day 2, basic track
MLHEP Lectures - day 2, basic trackMLHEP Lectures - day 2, basic track
MLHEP Lectures - day 2, basic track
 
Machine learning in science and industry — day 3
Machine learning in science and industry — day 3Machine learning in science and industry — day 3
Machine learning in science and industry — day 3
 
MLHEP 2015: Introductory Lecture #4
MLHEP 2015: Introductory Lecture #4MLHEP 2015: Introductory Lecture #4
MLHEP 2015: Introductory Lecture #4
 
Lecture 2
Lecture 2Lecture 2
Lecture 2
 
ECCV2010: distance function and metric learning part 2
ECCV2010: distance function and metric learning part 2ECCV2010: distance function and metric learning part 2
ECCV2010: distance function and metric learning part 2
 
Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...
Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...
Lecture 02: Machine Learning for Language Technology - Decision Trees and Nea...
 
Reweighting and Boosting to uniforimty in HEP
Reweighting and Boosting to uniforimty in HEPReweighting and Boosting to uniforimty in HEP
Reweighting and Boosting to uniforimty in HEP
 
Neural Learning to Rank
Neural Learning to RankNeural Learning to Rank
Neural Learning to Rank
 
2.6 support vector machines and associative classifiers revised
2.6 support vector machines and associative classifiers revised2.6 support vector machines and associative classifiers revised
2.6 support vector machines and associative classifiers revised
 
MLHEP 2015: Introductory Lecture #3
MLHEP 2015: Introductory Lecture #3MLHEP 2015: Introductory Lecture #3
MLHEP 2015: Introductory Lecture #3
 
Support Vector Machines ( SVM )
Support Vector Machines ( SVM ) Support Vector Machines ( SVM )
Support Vector Machines ( SVM )
 
Learning to Rank with Neural Networks
Learning to Rank with Neural NetworksLearning to Rank with Neural Networks
Learning to Rank with Neural Networks
 
[Paper reading] L-SHAPLEY AND C-SHAPLEY: EFFICIENT MODEL INTERPRETATION FOR S...
[Paper reading] L-SHAPLEY AND C-SHAPLEY: EFFICIENT MODEL INTERPRETATION FOR S...[Paper reading] L-SHAPLEY AND C-SHAPLEY: EFFICIENT MODEL INTERPRETATION FOR S...
[Paper reading] L-SHAPLEY AND C-SHAPLEY: EFFICIENT MODEL INTERPRETATION FOR S...
 
MLHEP 2015: Introductory Lecture #1
MLHEP 2015: Introductory Lecture #1MLHEP 2015: Introductory Lecture #1
MLHEP 2015: Introductory Lecture #1
 
Neural Learning to Rank
Neural Learning to RankNeural Learning to Rank
Neural Learning to Rank
 
MLHEP Lectures - day 3, basic track
MLHEP Lectures - day 3, basic trackMLHEP Lectures - day 3, basic track
MLHEP Lectures - day 3, basic track
 

En vedette

Hankintalain kokonaisuudistus, työryhmämietinnön esittely 13.5.2015
Hankintalain kokonaisuudistus, työryhmämietinnön esittely 13.5.2015Hankintalain kokonaisuudistus, työryhmämietinnön esittely 13.5.2015
Hankintalain kokonaisuudistus, työryhmämietinnön esittely 13.5.2015Työ- ja elinkeinoministeriö
 
ATP Blog06 - Review Of Humic Substances By - Pena Mendez
ATP Blog06 - Review Of  Humic  Substances By - Pena  MendezATP Blog06 - Review Of  Humic  Substances By - Pena  Mendez
ATP Blog06 - Review Of Humic Substances By - Pena Mendezatpcorporation
 
Best oral presentation was awarded to Dr Kostas Lathouras - BIARGSNewsletter2015
Best oral presentation was awarded to Dr Kostas Lathouras - BIARGSNewsletter2015Best oral presentation was awarded to Dr Kostas Lathouras - BIARGSNewsletter2015
Best oral presentation was awarded to Dr Kostas Lathouras - BIARGSNewsletter2015KOSTAS LATHOURAS
 
Vecer Na Temu25marec2010
Vecer Na Temu25marec2010Vecer Na Temu25marec2010
Vecer Na Temu25marec2010guestc27e91
 
Meegeven presentatie tool keukentafelspel-4
Meegeven presentatie tool keukentafelspel-4Meegeven presentatie tool keukentafelspel-4
Meegeven presentatie tool keukentafelspel-4Esther de Groot
 
ΠΑΡΟΥΣΙΑΣΗ YOUR NEXT HOME
ΠΑΡΟΥΣΙΑΣΗ YOUR NEXT HOMEΠΑΡΟΥΣΙΑΣΗ YOUR NEXT HOME
ΠΑΡΟΥΣΙΑΣΗ YOUR NEXT HOMEYOUR NEXT HOME
 
PowerPoint Presentation
PowerPoint PresentationPowerPoint Presentation
PowerPoint Presentationbutest
 
Sethia Grandeur Brochure - Zricks.com
Sethia Grandeur Brochure - Zricks.comSethia Grandeur Brochure - Zricks.com
Sethia Grandeur Brochure - Zricks.comZricks.com
 
Ativa Brochure (EN) - Scrollable
Ativa Brochure (EN) - ScrollableAtiva Brochure (EN) - Scrollable
Ativa Brochure (EN) - ScrollableRoberto Reyes
 

En vedette (13)

Hankintalain kokonaisuudistus, työryhmämietinnön esittely 13.5.2015
Hankintalain kokonaisuudistus, työryhmämietinnön esittely 13.5.2015Hankintalain kokonaisuudistus, työryhmämietinnön esittely 13.5.2015
Hankintalain kokonaisuudistus, työryhmämietinnön esittely 13.5.2015
 
ATP Blog06 - Review Of Humic Substances By - Pena Mendez
ATP Blog06 - Review Of  Humic  Substances By - Pena  MendezATP Blog06 - Review Of  Humic  Substances By - Pena  Mendez
ATP Blog06 - Review Of Humic Substances By - Pena Mendez
 
Describing others
Describing othersDescribing others
Describing others
 
HDsamplesLMC
HDsamplesLMCHDsamplesLMC
HDsamplesLMC
 
20161014091102871
2016101409110287120161014091102871
20161014091102871
 
Best oral presentation was awarded to Dr Kostas Lathouras - BIARGSNewsletter2015
Best oral presentation was awarded to Dr Kostas Lathouras - BIARGSNewsletter2015Best oral presentation was awarded to Dr Kostas Lathouras - BIARGSNewsletter2015
Best oral presentation was awarded to Dr Kostas Lathouras - BIARGSNewsletter2015
 
Vecer Na Temu25marec2010
Vecer Na Temu25marec2010Vecer Na Temu25marec2010
Vecer Na Temu25marec2010
 
Meegeven presentatie tool keukentafelspel-4
Meegeven presentatie tool keukentafelspel-4Meegeven presentatie tool keukentafelspel-4
Meegeven presentatie tool keukentafelspel-4
 
ΠΑΡΟΥΣΙΑΣΗ YOUR NEXT HOME
ΠΑΡΟΥΣΙΑΣΗ YOUR NEXT HOMEΠΑΡΟΥΣΙΑΣΗ YOUR NEXT HOME
ΠΑΡΟΥΣΙΑΣΗ YOUR NEXT HOME
 
PowerPoint Presentation
PowerPoint PresentationPowerPoint Presentation
PowerPoint Presentation
 
Sethia Grandeur Brochure - Zricks.com
Sethia Grandeur Brochure - Zricks.comSethia Grandeur Brochure - Zricks.com
Sethia Grandeur Brochure - Zricks.com
 
Tugas Pwer Poin
Tugas Pwer PoinTugas Pwer Poin
Tugas Pwer Poin
 
Ativa Brochure (EN) - Scrollable
Ativa Brochure (EN) - ScrollableAtiva Brochure (EN) - Scrollable
Ativa Brochure (EN) - Scrollable
 

Similaire à [ppt]

Data.Mining.C.6(II).classification and prediction
Data.Mining.C.6(II).classification and predictionData.Mining.C.6(II).classification and prediction
Data.Mining.C.6(II).classification and predictionMargaret Wang
 
20070702 Text Categorization
20070702 Text Categorization20070702 Text Categorization
20070702 Text Categorizationmidi
 
2.7 other classifiers
2.7 other classifiers2.7 other classifiers
2.7 other classifiersKrish_ver2
 
机器学习Adaboost
机器学习Adaboost机器学习Adaboost
机器学习AdaboostShocky1
 
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...Universitat Politècnica de Catalunya
 
MLHEP Lectures - day 1, basic track
MLHEP Lectures - day 1, basic trackMLHEP Lectures - day 1, basic track
MLHEP Lectures - day 1, basic trackarogozhnikov
 
Unit-1 Introduction and Mathematical Preliminaries.pptx
Unit-1 Introduction and Mathematical Preliminaries.pptxUnit-1 Introduction and Mathematical Preliminaries.pptx
Unit-1 Introduction and Mathematical Preliminaries.pptxavinashBajpayee1
 
3D Scene Analysis via Sequenced Predictions over Points and Regions
3D Scene Analysis via Sequenced Predictions over Points and Regions3D Scene Analysis via Sequenced Predictions over Points and Regions
3D Scene Analysis via Sequenced Predictions over Points and RegionsFlavia Grosan
 
Data Mining Lecture_10(b).pptx
Data Mining Lecture_10(b).pptxData Mining Lecture_10(b).pptx
Data Mining Lecture_10(b).pptxSubrata Kumer Paul
 
Machine Learning and Artificial Neural Networks.ppt
Machine Learning and Artificial Neural Networks.pptMachine Learning and Artificial Neural Networks.ppt
Machine Learning and Artificial Neural Networks.pptAnshika865276
 
Machine Learning: An Introduction Fu Chang
Machine Learning: An Introduction Fu ChangMachine Learning: An Introduction Fu Chang
Machine Learning: An Introduction Fu Changbutest
 
Textmining Predictive Models
Textmining Predictive ModelsTextmining Predictive Models
Textmining Predictive ModelsDatamining Tools
 
Textmining Predictive Models
Textmining Predictive ModelsTextmining Predictive Models
Textmining Predictive Modelsguest0edcaf
 
isabelle_webinar_jan..
isabelle_webinar_jan..isabelle_webinar_jan..
isabelle_webinar_jan..butest
 
Introduction to conventional machine learning techniques
Introduction to conventional machine learning techniquesIntroduction to conventional machine learning techniques
Introduction to conventional machine learning techniquesXavier Rafael Palou
 

Similaire à [ppt] (20)

Data.Mining.C.6(II).classification and prediction
Data.Mining.C.6(II).classification and predictionData.Mining.C.6(II).classification and prediction
Data.Mining.C.6(II).classification and prediction
 
Lect4
Lect4Lect4
Lect4
 
20070702 Text Categorization
20070702 Text Categorization20070702 Text Categorization
20070702 Text Categorization
 
2.7 other classifiers
2.7 other classifiers2.7 other classifiers
2.7 other classifiers
 
机器学习Adaboost
机器学习Adaboost机器学习Adaboost
机器学习Adaboost
 
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
 
MLHEP Lectures - day 1, basic track
MLHEP Lectures - day 1, basic trackMLHEP Lectures - day 1, basic track
MLHEP Lectures - day 1, basic track
 
Unit-1 Introduction and Mathematical Preliminaries.pptx
Unit-1 Introduction and Mathematical Preliminaries.pptxUnit-1 Introduction and Mathematical Preliminaries.pptx
Unit-1 Introduction and Mathematical Preliminaries.pptx
 
3D Scene Analysis via Sequenced Predictions over Points and Regions
3D Scene Analysis via Sequenced Predictions over Points and Regions3D Scene Analysis via Sequenced Predictions over Points and Regions
3D Scene Analysis via Sequenced Predictions over Points and Regions
 
Data Mining Lecture_10(b).pptx
Data Mining Lecture_10(b).pptxData Mining Lecture_10(b).pptx
Data Mining Lecture_10(b).pptx
 
Machine Learning and Artificial Neural Networks.ppt
Machine Learning and Artificial Neural Networks.pptMachine Learning and Artificial Neural Networks.ppt
Machine Learning and Artificial Neural Networks.ppt
 
Machine Learning: An Introduction Fu Chang
Machine Learning: An Introduction Fu ChangMachine Learning: An Introduction Fu Chang
Machine Learning: An Introduction Fu Chang
 
nnml.ppt
nnml.pptnnml.ppt
nnml.ppt
 
Textmining Predictive Models
Textmining Predictive ModelsTextmining Predictive Models
Textmining Predictive Models
 
Textmining Predictive Models
Textmining Predictive ModelsTextmining Predictive Models
Textmining Predictive Models
 
Textmining Predictive Models
Textmining Predictive ModelsTextmining Predictive Models
Textmining Predictive Models
 
Naive Bayes Presentation
Naive Bayes PresentationNaive Bayes Presentation
Naive Bayes Presentation
 
isabelle_webinar_jan..
isabelle_webinar_jan..isabelle_webinar_jan..
isabelle_webinar_jan..
 
Introduction to conventional machine learning techniques
Introduction to conventional machine learning techniquesIntroduction to conventional machine learning techniques
Introduction to conventional machine learning techniques
 
Winnow vs perceptron
Winnow vs perceptronWinnow vs perceptron
Winnow vs perceptron
 

Plus de butest

EL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEbutest
 
1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同butest
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
 
Timeline: The Life of Michael Jackson
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jacksonbutest
 
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...butest
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
 
Com 380, Summer II
Com 380, Summer IICom 380, Summer II
Com 380, Summer IIbutest
 
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazzbutest
 
MICHAEL JACKSON.doc
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.docbutest
 
Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1butest
 
Facebook
Facebook Facebook
Facebook butest
 
Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...butest
 
Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...butest
 
NEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTbutest
 
C-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docbutest
 
MAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docbutest
 
Mac OS X Guide.doc
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.docbutest
 
WEB DESIGN!
WEB DESIGN!WEB DESIGN!
WEB DESIGN!butest
 

Plus de butest (20)

EL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBE
 
1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
 
Timeline: The Life of Michael Jackson
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jackson
 
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
 
Com 380, Summer II
Com 380, Summer IICom 380, Summer II
Com 380, Summer II
 
PPT
PPTPPT
PPT
 
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
 
MICHAEL JACKSON.doc
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.doc
 
Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1
 
Facebook
Facebook Facebook
Facebook
 
Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...
 
Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...
 
NEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENT
 
C-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.doc
 
MAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.doc
 
Mac OS X Guide.doc
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.doc
 
hier
hierhier
hier
 
WEB DESIGN!
WEB DESIGN!WEB DESIGN!
WEB DESIGN!
 

[ppt]

  • 1. All images from wikimedia commons, a freely-licensed media repository
  • 2. “ Classifiers” R & D project by Aditya M Joshi [email_address] IIT Bombay Under the guidance of Prof. Pushpak Bhattacharyya [email_address] IIT Bombay
  • 5. What is classification? A machine learning task that deals with identifying the class to which an instance belongs A classifier performs classification Classifier Test instance Attributes (a1, a2,… an) Discrete-valued Class label ( Age, Marital status, Health status, Salary ) Issue Loan? {Yes, No} ( Perceptive inputs ) Steer? { Left, Straight, Right } Category of document? {Politics, Movies, Biology} ( Textual features : Ngrams )
  • 6. Classification learning Training phase Testing phase Learning the classifier from the available data ‘Training set’ (Labeled) Testing how well the classifier performs ‘Testing set’
  • 7.
  • 8.
  • 10. Diagram from Han-Kamber Example tree Intermediate nodes : Attributes Leaf nodes : Class predictions Edges : Attribute value tests Example algorithms: ID3, C4.5, SPRINT, CART
  • 11. Decision Tree schematic Training data set a1 a2 a3 a4 a5 a6 a1 a2 a3 a4 a5 a6 X Y Z Pure node, Leaf node: Class RED Impure node, Select best attribute and continue Impure node, Select best attribute and continue
  • 12.
  • 14.
  • 15.
  • 16.
  • 18.
  • 19. Decision List example Test instance ( h i , c i ) Unit Class label
  • 20. Decision List learning R S’ = S Set of candidate feature functions For each hi, Qi = Pi U Ni ( hi = 1 ) U i = max { | Pi| - pn * | Ni | , |Ni| - pp *|Pi| } Select hk, the feature with highest utility ( h k, ) If (| Pi| - pn * | Ni | > |Ni| - pp *|Pi| ) then 1 else 0 1 / 0 - Qk
  • 21.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 36.
  • 37.
  • 38.
  • 39.
  • 40. Testing SVM SVM Test instance Class label
  • 41. SVM Issues SVMs are immune to the removal of non-support-vector points What if n-classes are to be predicted? Problem : SVMs deal with two-class classification Solution : Have multiple SVMs each for one class
  • 43.
  • 44. Bagging Total set Sample D 1 Classifier model M 1 At random. May use bootstrap sampling with replacement Training dataset D Classifier learning scheme Classifier model M n Test set Majority vote Class Label
  • 45. Boosting (AdaBoost) Total set Sample D 1 Classifier model M 1 Selection based on weight. May use bootstrap sampling with replacement Training dataset D Classifier learning scheme Classifier model M n Test set Weighted vote Class Label Initialize weights of instances to 1/d Weights of correctly classified instances multiplied by error / (1 – error) If error > 0.5? Error Error `
  • 47.
  • 48.
  • 49.
  • 50.
  • 51. Weka & Weka Demo
  • 52.
  • 53.
  • 54. Parts of weka Explorer Basic interface to run ML Algorithms Experimenter Comparing experiments on different algorithms Knowledge Flow Similar to Work Flow ‘ Customized’ to one’s needs
  • 56.
  • 57. end of slideshow
  • 58.
  • 59.
  • 60.