The Art and Power of Data-Driven Modeling: Statistical and Machine Learning Approaches - Nataliya Portman
•Download as PPTX, PDF•
1 like•563 views
This presentation illustrates distinct statistical and machine learning approaches to automated recognition of major brain tissues in 3D brain MRI. Nataliya Portman, Postdoctoral Fellow Faculty of Science, UOIT, Oshawa, ON Canada PhD in Applied Mathematics, University of Waterloo | Postdoctoral Research on Brain MRI Segmentation, Neuro | Current: Applied Machine Learning in Materials Science, University of Ontario Institute of Technology
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to The Art and Power of Data-Driven Modeling: Statistical and Machine Learning Approaches - Nataliya Portman
Similar to The Art and Power of Data-Driven Modeling: Statistical and Machine Learning Approaches - Nataliya Portman (20)
More from WithTheBest
More from WithTheBest (20)
Recently uploaded
Recently uploaded (20)
The Art and Power of Data-Driven Modeling: Statistical and Machine Learning Approaches - Nataliya Portman
- 1. The Art and Power of Data-Driven Modeling: Statistical and Machine Learning Approaches 1 PhD in Applied Mathematics Past: Postdoctoral research on brain MRI segmentation Current: Applied machine learning in materials science Nataliya Portman Postdoctoral Fellow Faculty of Science, UOIT, Oshawa, ON Canada “AI with the best” online conference September 24, 2016
- 2. • Statistical versus machine learning: - Principles - Goals - Applications in biomedical sciences • Automatic brain tissue classification of infant brain MRI (Montreal Neurological Institute) - Challenges of automated segmentation - Combined solution: Kernel-based classifier + perceptual image quality model • Conclusion Overview 2 Nataliya Portman
- 3. 3 Statistical Learning • Learning is a process of probabilistic inference • Instance space X (quantities of interest, e.g., wind) • Hypothesis space H (e.g., h1=strong, h2=weak) • Training samples D (observed data, N recordings of wind) Nataliya Portman The Posterior The probability that hypothesis h is true given the evidence D. The Evidence The probability of getting the evidence D if the hypothesis h were true. The Prior The probability of h being true, before gathering evidence. The marginal probability of the evidence (Probability of D over all possible hypotheses). Common statistical learning methods: • Bayesian • Maximum a posteriori (MAP) • Maximum likelihood 3
- 4. Bayesian Learning • An unknown quantity is a random variable • Requires the hypothesis prior P(hi) • Combines prior probabilities with observed data • Predictions are made by using all the hypotheses weighted by their probabilities Usually, a hypothesis determines a probability distribution over the unknown quantity of interest X (e.g., parameters of the Gaussian distribution). 4 Nataliya Portman The posterior The predictive probability
- 6. MAP Learning • For each hypothesis h in H, calculate the posterior probability • Output the hypothesis hMAP with the highest posterior probability 6 Nataliya Portman
- 7. Maximum Likelihood Learning • Assumes a prior P(h) is uniform over the space of hypotheses H • Chooses an h that maximizes P(D|h) • Reasonable approach when there is no reason to prefer one hypothesis over another a priori • A good approximation to MAP and Bayesian learning when the dataset is large 7 Nataliya Portman
- 8. MAP Learning implementation Distribution of grey level intensities of 3D adult brain MRI •Training dataset D: 3D brain MR images • Hypothesis space per voxel: {h1,h2,h3} with h1=WM, h2=GM, h3=CSF • Probability models of each tissue type: • Tissue class priors: P(WM), P(GM), P(CSF) Output: posterior probabilities (“soft” segmentation) Decision boundaries 8 Nataliya Portman
- 9. MAP Learning: Expectation-Maximization Algorithm We estimate initial tissue class priors • Interactively select representative voxels for each tissue type from each individual scan in the training dataset (and fit the Gaussians) • Compute the ratios of each tissue class voxels with respect to all the representative voxels in the training data. 9 Nataliya Portman
- 10. MAP Learning: Expectation-Maximization Algorithm Expectation step, mth iteration: Compute Maximization step: Update of the Gaussian parameters corresponding to the new posterior distribution obtained at the expectation step. If D is the training dataset then P(h | D) is a probabilistic brain atlas 10 Nataliya Portman
- 11. Clinical applications • Statistical learning is used in diagnostics classification. • Example: Diagnostics in oncology (e.g., the diagnosis of a tumor as being “benign” or “malignant”). • Relies on logistic regression model of the conditional probability • Regression coefficients are estimated from a sample of N individuals with known covariate values x(n)=(x1 (n), x2 (n),…,xp (n) ,) and known class h(n) in {0,1} via the minimization of a distance measure. 11 Nataliya Portman G. Schwarzer et al., Statistics in Medicine, 2000
- 12. Clinical applications (Machine Learning?) X1 X2 X3 X4 h P( h=1 | x )=f( x, w, W ) wij Wi Neural Networks is another approach to model the conditional probability with a logistic transfer function. G. Schwarzer et al., Statistics in Medicine, 2000. • Lacks an easy interpretation of NN model parameters • Generates implausible functions 12 Nataliya Portman
- 13. Given the training dataset of N observations of K-dimensional feature vector X and the corresponding outcomes Y, learn a mapping f(X) that minimizes the loss L(Y,f(X)). X Unknown Y 13Algorithm Machine learning Nataliya Portman
- 14. 14 Machine learning Modeling reduces to a problem of function optimization Machine learning = algorithmic modeling Target: find an algorithm that predicts the outcome for new samples outside of the training dataset Algorithms: • Support Vector Machines • Artificial Neural Networks • Convolutional Neural Networks • Random Forests • Boosting • Decision Trees Nataliya Portman
- 15. Brain tissue classification of infant brain MRI McConnell Brain Imaging Centre Montreal Neurological Institute McGill University Postdoctoral fellow 15 Nataliya Portman
- 16. The NIH (National Institutes of Health) pediatric “Objective-2” MRI database is the largest demographically diverse U.S. sample that consists of 69 subjects aged 10 days to 4.5 years of age. 16
- 17. Brain tissue classification of infant brain MRI 17 Nataliya Portman
- 18. Child = Greater intensity variation due to myelination of WM Adult: Noise Intensity non-uniformity + Partial Volume Effect Natural tissue intensity variation Brain tissue classification of infant brain MRI Challenges with existing software: • CIVET pipeline (developed at MNI) fails to perform automatic accurate automatic classification into GM, WM and the CSF • General anatomical image processing pipelines such as FSL (Smith et al., 2004) and SPM (Ashburner, 1997) poorly detect major tissue classes in NIH “Objective-2” dataset. 18 Nataliya Portman
- 19. Brain tissue classification of infant brain MRI Three major segmentation frameworks (supervised): Expectation-Maximization [VanLeemput et al., 1999], [Tohka et al., 2004], [Prastawa and Gerig, 2004], [Xue et al., 2007], [Murgasova et al., 2007] Registration-based [Collins et al., 1999], [Murgasova et al., 2007] Label Fusion [Weisenfeld and Warfield, 2009] 19 Nataliya Portman
- 20. Methodological limitations • Global estimation of tissue intensity distributions (EM, Label fusion). Due to biological intensity variation and Partial Volume Effect (PVE) tissue intensity distribution in infant MRI can differ from the Gaussian (EM). • Supervised (atlas-dependent) approach that assumes small deviations from average brain anatomy (EM, Registration-based). Brain tissue classification of infant brain MRI 20 Nataliya Portman
- 21. Imagine…. Human Visual System (HVS) Information extraction Computer Vision that we have built an intelligent machine (software) that effectively identifies brain structures with the same accuracy as our Human Visual System.21
- 22. Brain tissue classification of infant brain MRI Classification machine requirements: • Does not depend on a probabilistic brain atlas • Does not assume global models of tissue intensity distributions • Objectively evaluates the quality of classification as perceived by the Human Visual System • Multichannel • Flexible, can be extended to multiclass classification Impact: Alleviates an agonizing pain of • probabilistic atlas construction • manual segmentation • improves accuracy of segmentation of child brain MRI • accelerates research rate in the field of early brain development • revolutionizes the field of MRI segmentation 22 Nataliya Portman
- 23. Birth of a “Visionary” Brain tissue classification of infant brain MRI The “Visionary” is a MATLAB software that accomplishes a challenging task of brain tissue classification in child brain MRI. Perceptual image quality model: In absence of “ground truth” it tries to mimic human perception of the quality of classification Structural SIMilarity Index (SSIM). The philosophy underlying the SSIM approach: the Human Visual System is highly adapted to extract structural information from images. How is “Visionary” built? 23 Nataliya Portman
- 24. - the local means of the corresponding image patches x and y, - the local standard deviations (respectively), - the small positive constants to stabilize each term. Visionary classified image T1w template (08-11mon) MSSIM quantifies the degree of structural similarity between input and classified images. MSSIM=0.8614 Brain tissue classification of infant brain MRI 24 Nataliya Portman
- 25. Brain tissue classification of infant brain MRI The choice of the reference depends on the age of the subject. T1w serves as a reference for MR brain data for ages 8 months and later. Age: 02-05 months - = 25 Nataliya Portman
- 26. 26 Brain tissue classification of infant brain MRI Nataliya Portman KFDA-Kernel Fisher Discriminant Analysis
- 27. Modified KFDA criterion: - spatial regularization term in the feature space, K and H are the kernel and negative Laplacian matrices, M and N are between-class and within-class covariance matrices. • Feature selection method ( tissue intensities, morphological measurements, etc. ) in machine learning • KFDA separability criterion measures the discriminating ability of a feature or a subset of features to distinguish between different classes. • The power of KFDA lies in its generality (does not assume multivariate probability models of the classes) and closed form solution (algebraic). Input and KFDA-classified data in stereotaxic and intensity spaces Kernel Fisher Discriminant Analysis 27 Nataliya Portman
- 28. Brain tissue classification of infant brain MRI Results for the brain template 08 to 11 months 28 Nataliya Portman
- 29. MSSIM=0.8234 MSSIM=0.8537 Brain tissue classification of infant brain MRI WM, GM and CSF detection in brain MRI template for ages 08 to 11 months. T1w PVE Visionary Myelinated WM detection in the brain MRI template for ages 02 to 05 months. 29 Nataliya Portman
- 30. Brain tissue classification of infant brain MRI T2w PVE Visionary unmyelinated WM Objective-2 template, age range: 02-05 months 30 Nataliya Portman
- 31. Objective-2 template, age range: 44-60 months Reference Initialization Visionary (label transfer from an older brain) Brain tissue classification of infant brain MRI 31 Nataliya Portman
- 32. 32 Brain tissue classification of infant brain MRI Nataliya Portman …. still unpublished.
- 33. • Machine learning (ML) methods provide algorithmic models for an unknown mapping between predictor and outcome variables • ML techniques are differently motivated, the goal is to forecast the outcome with acceptable accuracy, to be transferrable to new datasets • Statistical learning methods are focused on estimation of the probability distribution over hypothesis space • In biomedical applications, models that explain the data are preferable as they allow to reveal statistically significant influences of some covariates on the outcome • In order to devise an appropriate method for data processing and analysis, one has to understand the data, namely, the source of noise and signal variation and mathematical assumptions of inference methodology 33 Conclusion Nataliya Portman
Editor's Notes
- Good afternoon, thank you for attending my talk. Let me introduce myself. I am a postdoctoral researcher at UOIT with PhD in Applied Mathematics and with my recent past and extensive experience conducting research in neuroscience. I have found a passion for ML when experimenting with kernel-based classifiers at NEURO lab and today I will illustrate the art and power of data-driven modelling on examples of typical recognition problems arising in medical computer vision.
- For the past two decades we have seen explosion of machine learning algorithms in science and technology including health care, financial modeling and marketing. Due to aggressive marketing, machine learning terminology has penetrated many fields of science, specifically biomedical field where traditional statistical methods for parameter estimation have been now called statistical learning methods. Both, machine learning and statistical learning are data-driven methods, but there is a distinction between statistical learning and machine learning that I will make clear in this talk. I will then illustrate a novel ml application developed by me at MNI for automatic brain tissue classification of infant brain MRI.
- I start with the principles of statistical learning. Learning is a process of probabilistic inference, that is, a sequence of updates of the probabilities for hypotheses as more evidence becomes available. We want to learn the posterior probability of a hypothesis given some examples. Common stat learning methods are based on Bayes rule comprised of the prior knowledge as a probability distribution over the hypothesis space and the conditional probability of observing evidence D if h were true. As a simple example for Statistical learning problem setup, we can consider wind as our object of interest and two hypotheses of wind being strong or weak and wind observations in the form of N recordings.
- Bayesian learning assumes that an unknown quantity denoted by X is a random variable and requires the hypothesis prior. This means placing probability distributions on the parameters about which the decisions are made using Bayes theorem. The beauty of this approach is that it allows you to weigh each piece of evidence and thus to compute the predictive probability as a weighted average over all possible hypotheses.
- Here is Calvin who is not convinced in the validity of Bayesian approach. In the 20th century, Bayesian methods received much criticism for the subjective nature of probability, basically, one builds his believes into the prior model of a probability distribution of a parameter to be estimated. However, prior knowledge is usually obtained from experimental studies, and incorporating it into our predictive model makes sense, as it decreases our uncertainty about model outcomes.
- In maximum a posteriori approach, we calculate the posterior probability for each hypothesis. In contrast to Bayesian approach we output the hypothesis with the highest posterior probability.
- Here is an 3D MRI scan of a human brain and the corresponding distribution of tissue grey level intensities across the brain which suggests that the Gaussian mixture is a proper model for the this kind of data. If we know means and variances of WM, GM and CSF intensity distributions then we can deduce the decision boundaries that will classify each voxel xijk to WM, GM or CSF class. Our goal is to estimate the posterior probabiltiies of belonging to brain tissue classes for each voxel of the interior brain. This is a typical setup for statistical learning of posterior probabilities. How do we start our learning process? This problem cannot be solved with closed-form expressions.
- We need to do manual work by interactively selecting representative voxels for each tissue type from each individual scan in the training dataset and fit the Gaussians. The Gaussian fit will yield the first approximation of Gaussian distribution parameters.
- Expectation –M algo provides an iterative parameter estimation scheme. We assume that the intensity of a voxel is conditionally independent from the intensity of other voxels. Assume that the intensity distribution of tissue type k is normally distributed with mean mu and variance sigma. We make a guess about the complete set of brain voxel labels h and solve for Phi that maximizes the expected log-likelihood of h. Once we have an estimate for Phi we make a better guess about the complete set of voxel labels and iterate.
- Statistical learning is widely used in oncology diagnostics such as classification of a tumour into benign or malignant classes. A usual modelling choice to predict class probabilities is logistic regression.
- Alternatively, we can also model the probability of malignant tumour using ANN with different architectures by means of varying the number of hidden neurons. The probability of class membership estimated with ANN does not appear meaningful here as we would expect that an increase in value of X2 covariate to result in an increase in probability to belong to class 1. A strict frontier between 0 and 1 is postulated. Lacks an easy explanation of the fitting process.
- Algorithm
- In this part of my presentation I am going to talk about a novel semi-supervised machine learning algorithm that I have developed for brain tissue classificaiotn of infant brain MRI.
- Here is the NIH pediatric MRI database that contains the largest demographically diverse U.S, sample of 69 subjects aged 10 days to 4.5 years of age. A movie shows how brain strucutures, specifically, white matter evolves during the first year of brain development
- Here is an illustration of a typical brain MRI processing pipeline called CIVET. For infant brain MRI CIVET fails to correctly identify basic brain tissue classes preventing further morphometric analysis.
- There are three major segmentation frameworks adapted to the specific properties of infant brain MRI populations.