Call Girls Ahmedabad +917728919243 call me Independent Escort Service
Thoughts on Machine Learning and Artificial Intelligence
1. Thoughts on
Machine Learning and Artificial Intelligence
Maarten van Smeden, PhD
Leiden University Medical Center, Netherlands
STRATOS Lorenz Meeting
21/09/2018
2. Interested reader perspective
• Statistician by training
• Limited experience applying machine learning techniques
• Three examples that I think are illustrative for ML/AI in medicine
as it is applied nowadays
• Focus: prediction
22. Example 2: lymph node metastases
• Researcher challenge competition
• Whole slide images of women diagnosed with breast cancer
• Training data: N = 270 (110 events); test data: N = 129 (49 events)
• 11 pathologists evaluating the test data
• 390 teams signed up for the competition
• 23 teams submitted 32 algorithms for evaluation
24. Example 2: lymph node metastases
• Unfair comparison between pathologists and DL
• Pathologists no access to regularly available diagnostics
• AUC comparison DL (continuous) vs pathologists (5-item
scale)
• Promising algorithms overrepresented (390 teams -> 32
algorithms submitted)
25. Example 2: lymph node metastases
• No attention to risk prediction / calibration
• ML: attention classification only without probability
• Hugh (often implicit) difference between the traditional (risk)
prediction modeling in medicine and (traditional ML)
• Probably fine for Netflix recommendations; not so much for
real life medical decision making
32. BS detection simulation
• Data generated from 2 independent MVN-distributions with .3 equal pairwise correlations
• “Sunday morning simulations”, code: https://github.com/MvanSmeden/DiabetesClusters
33. K-means clustering
“K-means finds a Voronoi partition, only if that partition coincides with a
"clustering" does it have a hope of actually doing clustering”
Max Little: https://twitter.com/MaxALittle/status/970277900871262213
35. What I observe is:
• Confusion and disagreement about what is and isn’t ML/AI
• Analyses labeled “ML/AI” have a tendency to concentrate on
classification (exceptions exist, e.g. high dimensional PS
approaches suggested that are called “ML”)
• Analyses labeled “ML/AI” in medicine are surprisingly often
done by people not thoroughly trained in statistics
• Basic statistical principles are often forgotten or ignored (e.g.
improper scoring rules)
36. Concluding remarks (1)
• Just because an algorithm is novel or flexible doesn’t mean it is
any good, obviously
• Dismissing the potential value of novel “ML/AI” algorithms out-
of-hand doesn’t make sense
• We need more realistic simulations and many applications to
compare the traditional vs more novel / flexible algorithms
• The primary issue in medical applications seems to be with the
modelers not so much with the models
37. Concluding remarks (2)
• Statisticians should be more involved in the application and
evaluation of novel / flexible algorithms, especially for risk
prediction
• Statisticians should be involved in studying performance of
novel / flexible algorithms (e.g. data hungriness) -> realistic
simulation studies
• Collaboration with computer scientists
• Computationally intensive -> may not be cheap
• Serious experimental design and reporting
38. Simulation is…
“…it is using simulation for multiplication that I find objectionable. Eight patients are
eight patients and so should remain.”
39. “All the impressive achievements of
deep learning amount to just curve
fitting”
Judea Pearl