SlideShare une entreprise Scribd logo
1  sur  36
Introduction to Statistical Model Selection Huimin Chen Department of Electrical Engineering University of New Orleans New Orleans, LA 70148
Typical Problem ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Modeling and  Model Selection ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Outline of This Talk ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Regression Model ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Empirical Risk Functional ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
How to measure the model complexity with finite data? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Predictive Risk ,[object Object],[object Object],[object Object],[object Object],[object Object]
Predictive Risk (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Predictive Risk (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object]
Predictive Risk (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Bayesian Model Selection ,[object Object],[object Object],[object Object],[object Object],[object Object]
Bayesian Model Selection (Cont’d) ,[object Object],[object Object],[object Object]
Bayesian Model Selection (Cont’d) ,[object Object],[object Object],Define so  that Use quadratic approximation Log-likelihood ≈ −
Bayesian Model Selection (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Information Theoretic Measures ,[object Object],[object Object],[object Object],[object Object]
Information Theoretic Measures (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Information Theoretic Measures (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
MDL Principle ,[object Object],[object Object],[object Object]
MDL Principle (Cont’d) ,[object Object],[object Object]
MDL Principle (Cont’d) ,[object Object],[object Object],[object Object]
MDL Principle (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
MDL Principle (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
MDL Principle (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
MDL Principle (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object]
MDL Principle (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
MDL Principle (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Statistical Regularization ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Statistical Regularization (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object]
Adaptive Methods ,[object Object],[object Object],[object Object],[object Object],[object Object]
Adaptive Methods (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
From model selection to model evaluation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
From model selection to model evaluation (Cont’d) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Summary ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Future Research Directions ,[object Object],[object Object],[object Object],[object Object]
Further Readings ,[object Object],[object Object],[object Object],[object Object],[object Object]

Contenu connexe

Tendances

Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Simplilearn
 

Tendances (20)

KNN
KNN KNN
KNN
 
Machine Learning-Linear regression
Machine Learning-Linear regressionMachine Learning-Linear regression
Machine Learning-Linear regression
 
Introduction to Linear Discriminant Analysis
Introduction to Linear Discriminant AnalysisIntroduction to Linear Discriminant Analysis
Introduction to Linear Discriminant Analysis
 
Classification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsClassification Based Machine Learning Algorithms
Classification Based Machine Learning Algorithms
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent method
 
Sequential Pattern Mining and GSP
Sequential Pattern Mining and GSPSequential Pattern Mining and GSP
Sequential Pattern Mining and GSP
 
lazy learners and other classication methods
lazy learners and other classication methodslazy learners and other classication methods
lazy learners and other classication methods
 
Introduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersIntroduction to Machine Learning Classifiers
Introduction to Machine Learning Classifiers
 
Machine Learning With Logistic Regression
Machine Learning  With Logistic RegressionMachine Learning  With Logistic Regression
Machine Learning With Logistic Regression
 
Association rule mining
Association rule miningAssociation rule mining
Association rule mining
 
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature Engineering
 
Understanding Bagging and Boosting
Understanding Bagging and BoostingUnderstanding Bagging and Boosting
Understanding Bagging and Boosting
 
Logistic regression
Logistic regressionLogistic regression
Logistic regression
 
Performance Metrics for Machine Learning Algorithms
Performance Metrics for Machine Learning AlgorithmsPerformance Metrics for Machine Learning Algorithms
Performance Metrics for Machine Learning Algorithms
 
Classification and prediction
Classification and predictionClassification and prediction
Classification and prediction
 
Logistic regression
Logistic regressionLogistic regression
Logistic regression
 
Linear regression with gradient descent
Linear regression with gradient descentLinear regression with gradient descent
Linear regression with gradient descent
 
Principal Component Analysis (PCA) and LDA PPT Slides
Principal Component Analysis (PCA) and LDA PPT SlidesPrincipal Component Analysis (PCA) and LDA PPT Slides
Principal Component Analysis (PCA) and LDA PPT Slides
 
Decision trees in Machine Learning
Decision trees in Machine Learning Decision trees in Machine Learning
Decision trees in Machine Learning
 

En vedette

Presentation Of My Skills
Presentation  Of  My SkillsPresentation  Of  My Skills
Presentation Of My Skills
Mfoghama
 
Single Electron Spin Detection Slides For Uno Interview
Single Electron Spin Detection Slides For Uno InterviewSingle Electron Spin Detection Slides For Uno Interview
Single Electron Spin Detection Slides For Uno Interview
chenhm
 
Intro to Feature Selection
Intro to Feature SelectionIntro to Feature Selection
Intro to Feature Selection
chenhm
 
Intro to Multitarget Tracking for CURVE
Intro to Multitarget Tracking for CURVEIntro to Multitarget Tracking for CURVE
Intro to Multitarget Tracking for CURVE
chenhm
 

En vedette (15)

IMAG 4850 Library Presentation
IMAG 4850 Library PresentationIMAG 4850 Library Presentation
IMAG 4850 Library Presentation
 
Presentation Of My Skills
Presentation  Of  My SkillsPresentation  Of  My Skills
Presentation Of My Skills
 
Bilva
BilvaBilva
Bilva
 
Emerging Voices for Global Health
Emerging Voices for Global HealthEmerging Voices for Global Health
Emerging Voices for Global Health
 
Single Electron Spin Detection Slides For Uno Interview
Single Electron Spin Detection Slides For Uno InterviewSingle Electron Spin Detection Slides For Uno Interview
Single Electron Spin Detection Slides For Uno Interview
 
AfHEA 2011 a framework to study the process of removing user fee in LIC
AfHEA 2011 a framework to study the process of removing user fee in LICAfHEA 2011 a framework to study the process of removing user fee in LIC
AfHEA 2011 a framework to study the process of removing user fee in LIC
 
Source Text Re-Use in Engineering Master's Theses and Doctoral Dissertations ...
Source Text Re-Use in Engineering Master's Theses and Doctoral Dissertations ...Source Text Re-Use in Engineering Master's Theses and Doctoral Dissertations ...
Source Text Re-Use in Engineering Master's Theses and Doctoral Dissertations ...
 
Removing User Fees In SSA D Hercot
Removing User Fees In SSA D HercotRemoving User Fees In SSA D Hercot
Removing User Fees In SSA D Hercot
 
S I D A
S I D AS I D A
S I D A
 
Sports Programs
Sports ProgramsSports Programs
Sports Programs
 
Hercot How to do a policy delphi
Hercot How to do a policy delphiHercot How to do a policy delphi
Hercot How to do a policy delphi
 
Sinagoga Neologa din Cluj-Napoca
Sinagoga Neologa din Cluj-NapocaSinagoga Neologa din Cluj-Napoca
Sinagoga Neologa din Cluj-Napoca
 
Parcul Naţional Retezat
Parcul Naţional RetezatParcul Naţional Retezat
Parcul Naţional Retezat
 
Intro to Feature Selection
Intro to Feature SelectionIntro to Feature Selection
Intro to Feature Selection
 
Intro to Multitarget Tracking for CURVE
Intro to Multitarget Tracking for CURVEIntro to Multitarget Tracking for CURVE
Intro to Multitarget Tracking for CURVE
 

Similaire à Intro to Model Selection

Download It
Download ItDownload It
Download It
butest
 
Probability density estimation using Product of Conditional Experts
Probability density estimation using Product of Conditional ExpertsProbability density estimation using Product of Conditional Experts
Probability density estimation using Product of Conditional Experts
Chirag Gupta
 
ProbabilisticModeling20080411
ProbabilisticModeling20080411ProbabilisticModeling20080411
ProbabilisticModeling20080411
Clay Stanek
 
MachineLearning.ppt
MachineLearning.pptMachineLearning.ppt
MachineLearning.ppt
butest
 
MachineLearning.ppt
MachineLearning.pptMachineLearning.ppt
MachineLearning.ppt
butest
 
MachineLearning.ppt
MachineLearning.pptMachineLearning.ppt
MachineLearning.ppt
butest
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
butest
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
butest
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
butest
 
. An introduction to machine learning and probabilistic ...
. An introduction to machine learning and probabilistic .... An introduction to machine learning and probabilistic ...
. An introduction to machine learning and probabilistic ...
butest
 
November, 2006 CCKM'06 1
November, 2006 CCKM'06 1 November, 2006 CCKM'06 1
November, 2006 CCKM'06 1
butest
 
slides
slidesslides
slides
butest
 
original
originaloriginal
original
butest
 
Presentation on Machine Learning and Data Mining
Presentation on Machine Learning and Data MiningPresentation on Machine Learning and Data Mining
Presentation on Machine Learning and Data Mining
butest
 
A General Framework for Accurate and Fast Regression by Data Summarization in...
A General Framework for Accurate and Fast Regression by Data Summarization in...A General Framework for Accurate and Fast Regression by Data Summarization in...
A General Framework for Accurate and Fast Regression by Data Summarization in...
Yao Wu
 
Eswc2009
Eswc2009Eswc2009
Eswc2009
fanizzi
 
Jörg Stelzer
Jörg StelzerJörg Stelzer
Jörg Stelzer
butest
 

Similaire à Intro to Model Selection (20)

Download It
Download ItDownload It
Download It
 
Probability density estimation using Product of Conditional Experts
Probability density estimation using Product of Conditional ExpertsProbability density estimation using Product of Conditional Experts
Probability density estimation using Product of Conditional Experts
 
ProbabilisticModeling20080411
ProbabilisticModeling20080411ProbabilisticModeling20080411
ProbabilisticModeling20080411
 
Into to prob_prog_hari
Into to prob_prog_hariInto to prob_prog_hari
Into to prob_prog_hari
 
MachineLearning.ppt
MachineLearning.pptMachineLearning.ppt
MachineLearning.ppt
 
MachineLearning.ppt
MachineLearning.pptMachineLearning.ppt
MachineLearning.ppt
 
MachineLearning.ppt
MachineLearning.pptMachineLearning.ppt
MachineLearning.ppt
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
 
Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401Machine Learning: Foundations Course Number 0368403401
Machine Learning: Foundations Course Number 0368403401
 
. An introduction to machine learning and probabilistic ...
. An introduction to machine learning and probabilistic .... An introduction to machine learning and probabilistic ...
. An introduction to machine learning and probabilistic ...
 
Model-driven decision support for monitoring network design based on analysis...
Model-driven decision support for monitoring network design based on analysis...Model-driven decision support for monitoring network design based on analysis...
Model-driven decision support for monitoring network design based on analysis...
 
November, 2006 CCKM'06 1
November, 2006 CCKM'06 1 November, 2006 CCKM'06 1
November, 2006 CCKM'06 1
 
slides
slidesslides
slides
 
original
originaloriginal
original
 
Presentation on Machine Learning and Data Mining
Presentation on Machine Learning and Data MiningPresentation on Machine Learning and Data Mining
Presentation on Machine Learning and Data Mining
 
A General Framework for Accurate and Fast Regression by Data Summarization in...
A General Framework for Accurate and Fast Regression by Data Summarization in...A General Framework for Accurate and Fast Regression by Data Summarization in...
A General Framework for Accurate and Fast Regression by Data Summarization in...
 
CLIM Program: Remote Sensing Workshop, Foundations Session: A Discussion - Br...
CLIM Program: Remote Sensing Workshop, Foundations Session: A Discussion - Br...CLIM Program: Remote Sensing Workshop, Foundations Session: A Discussion - Br...
CLIM Program: Remote Sensing Workshop, Foundations Session: A Discussion - Br...
 
Eswc2009
Eswc2009Eswc2009
Eswc2009
 
Jörg Stelzer
Jörg StelzerJörg Stelzer
Jörg Stelzer
 

Dernier

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 

Dernier (20)

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 

Intro to Model Selection

  • 1. Introduction to Statistical Model Selection Huimin Chen Department of Electrical Engineering University of New Orleans New Orleans, LA 70148
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.