SlideShare a Scribd company logo
1 of 38
Download to read offline
A	
  Prac'cal	
  Overview	
  of	
  Linear	
  
             Classifier
Discrimina've/Genera've	
  Pair
Ques'ons
•  Advantages/disadvantages	
  of	
  genera've/
   discrimina've	
  model?	
  
•  Can	
  we	
  combine	
  genera've	
  and	
  discrimina've	
  
   model?
Outline
•    Perceptron	
  
•    Margin	
  
•    Kernel	
  
•    Structure	
  
•    Online	
  
•    Ensemble
Generic	
  Perceptron
Margin
Max	
  Margin	
  Helps
Percptron	
  VS	
  SVM
Theory
•  Theore'cal	
  jus'fica'on:	
  
   –  Large	
  margin	
  implies	
  small	
  VC	
  dimension	
  
•  Sta's'cal	
  Learning	
  Theory	
  (SLT)	
  
   –  VC-­‐dimension	
  (Vapnik-­‐Chervonenkis)	
  
•  Probably	
  Approximately	
  Correct	
  (PAC)	
  Theory	
  
Parameter	
  Es'ma'on



	
  
•  SMO	
  algorithm	
  
•  Online	
  algorithm
Classifier	
  Comparison	
  on	
  
Sen'ment	
  Classifica'on




                              Accuracy
Ques'ons
•  What	
  if	
  not	
  linear	
  separable?	
  
•  Can	
  SVM	
  be	
  used	
  when	
  some	
  data	
  do	
  not	
  
   have	
  labels?
Kernel
Why	
  Linear	
  Classifier
•  Simple	
  
•  Applicable	
  
•  Fast
Kernel:	
  Nonlinear	
  -­‐>	
  Linear	
  
Kernel	
  SVM
Ques'ons
•  What	
  is	
  the	
  best	
  kernel?	
  
•  Can	
  we	
  use	
  mul'ple	
  kernels?	
  
•  kernels	
  for	
  NLP?
Structure
Feature	
  Func'ons
Applica'on1:	
  POS	
  Tagging
Applica'on2:	
  NP	
  Chunking
Ques'ons
•  Rela'onship	
  to	
  CRF?	
  
•  Structured	
  SVM?	
  
•  Maximum	
  Margin	
  Markov	
  Network?
Ensemble	
  Perceptron
•  Many	
  classifiers	
  are	
  beXer	
  than	
  one	
  
•  Vo'ng	
  perceptron	
  
•  Average	
  perceptron
Online	
  Learning
•  Batch	
  
•  Online	
  
•  MiniBatch
Implementa'ons
Algorithm                                  Implemena/on
Perceptron                                 Sklearn(Python),	
  Weka(Java)
SVM                                        LibSVM,	
  LibLinear(Linear	
  kernel)
Logis'c	
  regression/Maximum	
  Entropy   LibLinear
CRF                                        CRF++,	
  CRFSuite
Naïve	
  Bayes                             Sklean,	
  Weka
Perceptron	
  and	
  Beyond
            Kernel                                                      Online
                         Kernel	
  SVM         Online	
  SVM

             Kernel	
  perceptron                               Online	
  CRF




                                 Perceptron
Margin                                                     Vo'ng	
  perceptron
                 Linear	
  SVM                                                        Ensemble
                                                             Averaged	
  perceptron
  Margin	
  perceptron
                            Structured	
  Perceptron
                                                                            Boos'ng
                     Structured	
  SVM
                                                     CRF
                M3N
                                         Structure
Examples
•  hXp://www.mathworks.com/matlabcentral/
   fileexchange/28302-­‐svm-­‐demo	
  
•  hXp://www.eee.metu.edu.tr/~alatan/
   Courses/Demo/AppletSVM.html

More Related Content

Similar to Linear classifier

Using MXNet to Train and Deploy your Deep Learning Model
Using MXNet to Train and Deploy your Deep Learning ModelUsing MXNet to Train and Deploy your Deep Learning Model
Using MXNet to Train and Deploy your Deep Learning ModelQing Lan
 
Upgrading to SystemVerilog for FPGA Designs - FPGA Camp Bangalore, 2010
Upgrading to SystemVerilog for FPGA Designs - FPGA Camp Bangalore, 2010Upgrading to SystemVerilog for FPGA Designs - FPGA Camp Bangalore, 2010
Upgrading to SystemVerilog for FPGA Designs - FPGA Camp Bangalore, 2010FPGA Central
 
Convolutional Neural Network and RNN for OCR problem.
Convolutional Neural Network and RNN for OCR problem.Convolutional Neural Network and RNN for OCR problem.
Convolutional Neural Network and RNN for OCR problem.Vishal Mishra
 
Sparse Data Support in MLlib
Sparse Data Support in MLlibSparse Data Support in MLlib
Sparse Data Support in MLlibXiangrui Meng
 
ComputerVisionwithDeepLearning.pdf
ComputerVisionwithDeepLearning.pdfComputerVisionwithDeepLearning.pdf
ComputerVisionwithDeepLearning.pdfSyedMahmoodAliRoomi
 
Real-Time Non-Intrusive Speech Quality Estimation for VoIP
Real-Time Non-Intrusive Speech Quality Estimation for VoIPReal-Time Non-Intrusive Speech Quality Estimation for VoIP
Real-Time Non-Intrusive Speech Quality Estimation for VoIPadil raja
 
Support Vector machine(SVM) and Random Forest
Support Vector machine(SVM) and Random ForestSupport Vector machine(SVM) and Random Forest
Support Vector machine(SVM) and Random Forestumarcybermind
 
Towards Detecting Performance Anti-patterns Using Classification Techniques
Towards Detecting Performance Anti-patterns Using Classification TechniquesTowards Detecting Performance Anti-patterns Using Classification Techniques
Towards Detecting Performance Anti-patterns Using Classification TechniquesJames Hill
 
NLP and Deep Learning for non_experts
NLP and Deep Learning for non_expertsNLP and Deep Learning for non_experts
NLP and Deep Learning for non_expertsSanghamitra Deb
 
Complete solution for Recurrent neural network.pptx
Complete solution for Recurrent neural network.pptxComplete solution for Recurrent neural network.pptx
Complete solution for Recurrent neural network.pptxArunKumar674066
 
Under the Hood of Alignment Algorithms for NGS Researchers
Under the Hood of Alignment Algorithms for NGS ResearchersUnder the Hood of Alignment Algorithms for NGS Researchers
Under the Hood of Alignment Algorithms for NGS Researchers Golden Helix Inc
 
Knucth Morris and pratt_presentation.pptx
Knucth Morris and pratt_presentation.pptxKnucth Morris and pratt_presentation.pptx
Knucth Morris and pratt_presentation.pptxsiddharthyou29
 
Session 49 Practical Semantic Sticky Note
Session 49 Practical Semantic Sticky NoteSession 49 Practical Semantic Sticky Note
Session 49 Practical Semantic Sticky NoteISSGC Summer School
 
Icml2012 learning hierarchies of invariant features
Icml2012 learning hierarchies of invariant featuresIcml2012 learning hierarchies of invariant features
Icml2012 learning hierarchies of invariant featureszukun
 

Similar to Linear classifier (16)

Using MXNet to Train and Deploy your Deep Learning Model
Using MXNet to Train and Deploy your Deep Learning ModelUsing MXNet to Train and Deploy your Deep Learning Model
Using MXNet to Train and Deploy your Deep Learning Model
 
Upgrading to SystemVerilog for FPGA Designs - FPGA Camp Bangalore, 2010
Upgrading to SystemVerilog for FPGA Designs - FPGA Camp Bangalore, 2010Upgrading to SystemVerilog for FPGA Designs - FPGA Camp Bangalore, 2010
Upgrading to SystemVerilog for FPGA Designs - FPGA Camp Bangalore, 2010
 
Convolutional Neural Network and RNN for OCR problem.
Convolutional Neural Network and RNN for OCR problem.Convolutional Neural Network and RNN for OCR problem.
Convolutional Neural Network and RNN for OCR problem.
 
Sparse Data Support in MLlib
Sparse Data Support in MLlibSparse Data Support in MLlib
Sparse Data Support in MLlib
 
ComputerVisionwithDeepLearning.pdf
ComputerVisionwithDeepLearning.pdfComputerVisionwithDeepLearning.pdf
ComputerVisionwithDeepLearning.pdf
 
Real-Time Non-Intrusive Speech Quality Estimation for VoIP
Real-Time Non-Intrusive Speech Quality Estimation for VoIPReal-Time Non-Intrusive Speech Quality Estimation for VoIP
Real-Time Non-Intrusive Speech Quality Estimation for VoIP
 
Support Vector machine(SVM) and Random Forest
Support Vector machine(SVM) and Random ForestSupport Vector machine(SVM) and Random Forest
Support Vector machine(SVM) and Random Forest
 
Towards Detecting Performance Anti-patterns Using Classification Techniques
Towards Detecting Performance Anti-patterns Using Classification TechniquesTowards Detecting Performance Anti-patterns Using Classification Techniques
Towards Detecting Performance Anti-patterns Using Classification Techniques
 
NLP and Deep Learning for non_experts
NLP and Deep Learning for non_expertsNLP and Deep Learning for non_experts
NLP and Deep Learning for non_experts
 
Deep Learning
Deep LearningDeep Learning
Deep Learning
 
Complete solution for Recurrent neural network.pptx
Complete solution for Recurrent neural network.pptxComplete solution for Recurrent neural network.pptx
Complete solution for Recurrent neural network.pptx
 
Under the Hood of Alignment Algorithms for NGS Researchers
Under the Hood of Alignment Algorithms for NGS ResearchersUnder the Hood of Alignment Algorithms for NGS Researchers
Under the Hood of Alignment Algorithms for NGS Researchers
 
Knucth Morris and pratt_presentation.pptx
Knucth Morris and pratt_presentation.pptxKnucth Morris and pratt_presentation.pptx
Knucth Morris and pratt_presentation.pptx
 
Session 49 Practical Semantic Sticky Note
Session 49 Practical Semantic Sticky NoteSession 49 Practical Semantic Sticky Note
Session 49 Practical Semantic Sticky Note
 
Icml2012 learning hierarchies of invariant features
Icml2012 learning hierarchies of invariant featuresIcml2012 learning hierarchies of invariant features
Icml2012 learning hierarchies of invariant features
 
BriefPPT
BriefPPTBriefPPT
BriefPPT
 

Recently uploaded

Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embeddingZilliz
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 

Recently uploaded (20)

E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embedding
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 

Linear classifier