SlideShare une entreprise Scribd logo
1  sur  28
Télécharger pour lire hors ligne
Learning
To Speak
Medicine
Xavier Amatriain
Part I. The Medical Conversation
Medical Diagnosis
● Diagnosis (R.A. Miller 1990):
○ Mapping from patient’s data
(history, examination, lab exams…)
to a possible condition.
○ It depends on ability to:
■ Evoke history
■ Surface symptoms and
findings
■ Generate hypotheses that
suggest how to refine or
pursue different hypothesis
○ In a compassionate,
cost-effective manner
● Doctor: “How can I help?”
● Patient: Chief complaint
● Doctor: “Anything else?”
● Patient: ….
● Start of a doctor-led Q&A:
○ E.g. “Do you have X?”
● Doctor communicates actionable
recommendation (diagnosis +
treatment, triage, referral…)
Medicine is hard(er)
● Doctors have ~15 minutes to capture
information* about a patient, diagnose,
and recommend treatment
● *Information
○ Patient’s history
○ Patient’s symptoms
○ Medical knowledge
■ Learned years ago
■ Latest research findings
■ Different demographics
● Data is growing over time, so is complexity
The future of healthcare
● Patient-centered, AI-driven, doctor-in-the-loop
● Increased value of patient/doctor interaction
● Data-enabled applications
● Individual/personalized recommendations
● Multimodal (images, voice, audio…)
● Pro-active Prognosis/Diagnosis (eg. when
sensor reading abnormal or new information
available)
● Identifying and warning about side effect of
medications
● Conversational systems are a core
component
Personalized AI agent
Sensors
Conversational
System
(eliciting and providing
information, recommending
outcomes, treatments, prevention)
Part II.
Why now?
Is it a new idea?
● Internist -1 (1971) led by Jack Myers
considered (one of) the best clinical
diagnostic experts in the US
● Process for adding a disease requires
2-4 weeks of full-time effort and doctors
reading 50 to 250 relevant publications
● Very structured heuristic-driven dialog
system
Is it still hot?
Research efforts
Data Availability
● Large-scale patient-level clinical data
○ Electronic health records
○ Electronic imaging: x-rays, scans
○ Genes
● Smart devices as health sensors
○ Wearables
○ FDA-approved phone apps
○ High-quality images
● Electronic access to medical research
● Much of this only available in the last few
years
The Language of Medicine
● ICD(10)
○ 10th revision of the International Statistical Classification of Diseases and
Related Health Problems (ICD)
○ Codes for diseases, symptoms, findings, complaints...
○ Evolved from Bertillon Classification of Causes of Death (1893)
○ UN gave WHO responsibility for the ICD in 1946
● Snomed Clinical Terms
○ Computer processable collection of medical terms used in clinical
documentation and reporting.
○ Clinical findings, symptoms, diagnoses, procedures, body structures,
organisms substances, pharmaceuticals, devices…
○ Started in 1965, has had multiple revisions and mutations
● UMLS
○ Compendium of many controlled vocabularies (1986)
○ Mapping structure among vocabularies
○ Allows to translate among the various terminology systems
Electronic health records
● Digital information about
patients encounters with
doctors or the health
system
● An encounter may include
○ Doctor notes, medications,
procedures, diagnosis
● Mix of structured data and
unstructured text
Open Data
● Hnlp
● I2b2
● Mimic-3
● ….
Part III. Building Medical
Dialogue Systems
Medical conversation as a task-oriented dialogue
Intent classification
Text representation
● Infer structured representation
An 18-year-old male student presents with severe headache and fever
that he has had for 3 days. Examination reveals fever, photophobia,
and neck stiffness
Demographics:
● Age: 18 years
● Gender: Male
Symptoms:
● headache
○ Severity: severe
● Fever
○ Duration: 3 days
● Photophobia
● Neck stiffness
Slot filling in frames
Text representation
● Infer latent semantic space through embeddings
A. L. Beam, B. Kompa, I. Fried, N. P. Palmer, X. Shi, T. Cai, and I. S. Kohane. 2018. Clinical Concept Embeddings Learned from Massive
Sources of Medical Data. ArXiv e-prints (April 2018).
108,477 medical
concepts using:
● insurance claims
database of 60 million
members
● 20 million clinical notes
● 1.7 million full text
biomedical journal
articles
Entity recognition
Entity recognition
The Deep Learning promise
Learning to infer structure in text
A. Jagannatha and H. Yu, Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records, 2016
A. Vani et.al. Grounded Recurrent Neural Networks, arXiv, 2017
bleeding due to warfarin
Adverse
drug effect
- - medication
End-to-end Deep Learning Dialog Systems SOTA
Language model input:
User: got home and when i 've been at work i 've been experiencing a
pain like a 4 on my left side by my rib and some days i feel where i had
my iv in my arm on march 2 when i went to the er for my constipation
Dr: ah i see . have you tried anything for relief ?
User: took an acetaminophen but that only last for about an hour my body
burns off medicine really fast i learned that at a young age
Language model output:
Dr: i see . how bad is the pain ? on a scale of 1 - 10 10 being the
worst how would you rate it ?
User: 8
Dr: sounds rough ! applying a warm compress 2 - 3 times a day warm baths
with epsom salt massaging...
End-to-end Deep Learning Dialog Systems
References
● “An interlingua for electronic interchange of medical information: using frames to map between clinical vocabularies.” Masarie. 1991
● “Health dialog systems for patients and consumers.” Bickmore. 2006
● “Designing a Chatbot for diabetic patients”, Lokman. 2007
● “Computer-assisted diagnostic decision support: history, challenges, and possible paths forward” Miller. 2009.
● “Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)” Murphy. 2010
● “Mining Biomedical Ontologies and Data Using RDF Hypergraphs” Liu et al. 2013.
● “From health search to healthcare: explorations of intention and utilization via query logs and user surveys” White. 2014
● “Pharmabot: A Pediatric Generic Medicine Consultant Chatbot”, Comendador. 2015
● “Identifiable Phenotyping using Constrained Non-Negative Matrix Factorization”. Joshi et al. 2016.
● “Clinical Tagging with Joint Probabilistic Models” . Halpern et al. 2016.
● “MIMIC-III, a freely accessible critical care database” Johnson. 2016
● “Disease named entity recognition by combining conditional random fields & bidirectional recurrent neural networks” Wei. 2016
● “Named Entity Recognition Over Electronic Health Records Through a Combined Dictionary-based Approach”, Pomares. 2016
● “Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records”, Jagannatha. 2016
● “Bringing semantic structures to user intent detection in online medical queries”, Zhang. 2017
● “A Conversational Chatbot Based on Kowledge-Graphs for Factoid Medical Questions”, A. Minutolo. 2017
● “MANDY: Towards a Smart Primary Care Chatbot Application”. Ni. 2017
● “Entity recognition from clinical texts via recurrent neural network”, Liu. 2017
● “Knowledge-driven Entity Recognition and Disambiguation in Biomedical Text”. Siu. 2017
● “Deep Learning for Dialogue Systems”. Vivian Chen. 2017
● “Universal Language Model Fine-tuning for Text Classification”, Howard. 2018
● “Clinical Concept Embeddings Learned from Massive Sources of Medical Data” A. L. Beam, et al. ArXiv e-prints (April 2018).
References
Yes, we’re hiring!

Contenu connexe

Tendances

Health Care Processes and Decision Making_lecture 1_slides
Health Care Processes and Decision Making_lecture 1_slidesHealth Care Processes and Decision Making_lecture 1_slides
Health Care Processes and Decision Making_lecture 1_slidesCMDLearning
 
How Real-time Analysis turns Big Medical Data into Precision Medicine
How Real-time Analysis turns Big Medical Data into Precision MedicineHow Real-time Analysis turns Big Medical Data into Precision Medicine
How Real-time Analysis turns Big Medical Data into Precision MedicineMatthieu Schapranow
 
GuoJian CV2014.6.26提交
GuoJian CV2014.6.26提交GuoJian CV2014.6.26提交
GuoJian CV2014.6.26提交JIan Guo
 
Introduction to Healthcare Analytics
Introduction to Healthcare Analytics Introduction to Healthcare Analytics
Introduction to Healthcare Analytics Experfy
 
Statistics For Health Science and Its Impacts
Statistics For Health Science and Its ImpactsStatistics For Health Science and Its Impacts
Statistics For Health Science and Its ImpactsCashews
 
Healthcare analytics
Healthcare analytics Healthcare analytics
Healthcare analytics Arun K
 
IRJET - An Effective Stroke Prediction System using Predictive Models
IRJET -  	  An Effective Stroke Prediction System using Predictive ModelsIRJET -  	  An Effective Stroke Prediction System using Predictive Models
IRJET - An Effective Stroke Prediction System using Predictive ModelsIRJET Journal
 
Analytics in healthcare
Analytics in healthcareAnalytics in healthcare
Analytics in healthcareAnushkaAlok
 
Valmeek Kudesia Professional Summary
Valmeek Kudesia Professional SummaryValmeek Kudesia Professional Summary
Valmeek Kudesia Professional SummaryValmeek Kudesia
 
Predictive Analytics in Healthcare
Predictive Analytics in HealthcarePredictive Analytics in Healthcare
Predictive Analytics in HealthcareICFAIEDGE
 
Data explosion in medicine: challenges and opportunities
Data explosion in medicine: challenges and opportunitiesData explosion in medicine: challenges and opportunities
Data explosion in medicine: challenges and opportunitiesOurlad Alzeus Tantengco
 
231109 rm-v.j.-review of literature
231109 rm-v.j.-review of literature231109 rm-v.j.-review of literature
231109 rm-v.j.-review of literatureVivek Vasan
 

Tendances (20)

Health Care Processes and Decision Making_lecture 1_slides
Health Care Processes and Decision Making_lecture 1_slidesHealth Care Processes and Decision Making_lecture 1_slides
Health Care Processes and Decision Making_lecture 1_slides
 
How Real-time Analysis turns Big Medical Data into Precision Medicine
How Real-time Analysis turns Big Medical Data into Precision MedicineHow Real-time Analysis turns Big Medical Data into Precision Medicine
How Real-time Analysis turns Big Medical Data into Precision Medicine
 
Overview of Health IT
Overview of Health ITOverview of Health IT
Overview of Health IT
 
GuoJian CV2014.6.26提交
GuoJian CV2014.6.26提交GuoJian CV2014.6.26提交
GuoJian CV2014.6.26提交
 
Introduction to Healthcare Analytics
Introduction to Healthcare Analytics Introduction to Healthcare Analytics
Introduction to Healthcare Analytics
 
Statistics For Health Science and Its Impacts
Statistics For Health Science and Its ImpactsStatistics For Health Science and Its Impacts
Statistics For Health Science and Its Impacts
 
Computer science for health
Computer science for healthComputer science for health
Computer science for health
 
Healthcare analytics
Healthcare analytics Healthcare analytics
Healthcare analytics
 
Venkatesh_CV
Venkatesh_CVVenkatesh_CV
Venkatesh_CV
 
IRJET - An Effective Stroke Prediction System using Predictive Models
IRJET -  	  An Effective Stroke Prediction System using Predictive ModelsIRJET -  	  An Effective Stroke Prediction System using Predictive Models
IRJET - An Effective Stroke Prediction System using Predictive Models
 
Stroke Prediction
Stroke PredictionStroke Prediction
Stroke Prediction
 
Analytics in healthcare
Analytics in healthcareAnalytics in healthcare
Analytics in healthcare
 
Overview of Health IT (October 2, 2016)
Overview of Health IT (October 2, 2016)Overview of Health IT (October 2, 2016)
Overview of Health IT (October 2, 2016)
 
Valmeek Kudesia Professional Summary
Valmeek Kudesia Professional SummaryValmeek Kudesia Professional Summary
Valmeek Kudesia Professional Summary
 
Data to help patients 101
Data to help patients 101Data to help patients 101
Data to help patients 101
 
resume
resumeresume
resume
 
Predictive Analytics in Healthcare
Predictive Analytics in HealthcarePredictive Analytics in Healthcare
Predictive Analytics in Healthcare
 
Data explosion in medicine: challenges and opportunities
Data explosion in medicine: challenges and opportunitiesData explosion in medicine: challenges and opportunities
Data explosion in medicine: challenges and opportunities
 
2015 CVDPG Trinity
2015 CVDPG Trinity2015 CVDPG Trinity
2015 CVDPG Trinity
 
231109 rm-v.j.-review of literature
231109 rm-v.j.-review of literature231109 rm-v.j.-review of literature
231109 rm-v.j.-review of literature
 

Similaire à Learning to speak medicine

Understanding medical concepts and codes through NLP methods
Understanding medical concepts and codes through NLP methodsUnderstanding medical concepts and codes through NLP methods
Understanding medical concepts and codes through NLP methodsAshis Chanda
 
Epidemiology versus Data Collection Bias - Studying the Needs of Platform Wor...
Epidemiology versus Data Collection Bias - Studying the Needs of Platform Wor...Epidemiology versus Data Collection Bias - Studying the Needs of Platform Wor...
Epidemiology versus Data Collection Bias - Studying the Needs of Platform Wor...Maria Wolters
 
What Is Medical Informatics?
What Is Medical Informatics?What Is Medical Informatics?
What Is Medical Informatics?Sanjoy Sanyal
 
Second-Generation HIT InformaticistsGreat discoveries can transfor.docx
Second-Generation HIT InformaticistsGreat discoveries can transfor.docxSecond-Generation HIT InformaticistsGreat discoveries can transfor.docx
Second-Generation HIT InformaticistsGreat discoveries can transfor.docxzenobiakeeney
 
Big Data and the Promise and Pitfalls when Applied to Disease Prevention and ...
Big Data and the Promise and Pitfalls when Applied to Disease Prevention and ...Big Data and the Promise and Pitfalls when Applied to Disease Prevention and ...
Big Data and the Promise and Pitfalls when Applied to Disease Prevention and ...Philip Bourne
 
Health Informatics for Clinical Research (November 25, 2021)
Health Informatics for Clinical Research (November 25, 2021)Health Informatics for Clinical Research (November 25, 2021)
Health Informatics for Clinical Research (November 25, 2021)Nawanan Theera-Ampornpunt
 
K Bobyk - %22A Primer on Personalized Medicine - The Imminent Systemic Shift%...
K Bobyk - %22A Primer on Personalized Medicine - The Imminent Systemic Shift%...K Bobyk - %22A Primer on Personalized Medicine - The Imminent Systemic Shift%...
K Bobyk - %22A Primer on Personalized Medicine - The Imminent Systemic Shift%...Kostyantyn Bobyk
 
The Dual Nature of InformaticsInformatics can be used for impr.docx
The Dual Nature of InformaticsInformatics can be used for impr.docxThe Dual Nature of InformaticsInformatics can be used for impr.docx
The Dual Nature of InformaticsInformatics can be used for impr.docxhe45mcurnow
 
The Nursing And Midwifery Council
The Nursing And Midwifery CouncilThe Nursing And Midwifery Council
The Nursing And Midwifery CouncilRachel Walters
 
NLM Georgia Biomedical Informatics
NLM Georgia Biomedical InformaticsNLM Georgia Biomedical Informatics
NLM Georgia Biomedical InformaticsAlison Aldrich
 
Population health informatics DME SoPH 20151006
Population health informatics DME SoPH 20151006Population health informatics DME SoPH 20151006
Population health informatics DME SoPH 20151006Kathleen Gray
 
The Prevalence Of Pressure Ulcers During Home Settings...
The Prevalence Of Pressure Ulcers During Home Settings...The Prevalence Of Pressure Ulcers During Home Settings...
The Prevalence Of Pressure Ulcers During Home Settings...Claudia Brown
 
Hilda Bastian - Selling Sickness 2010
Hilda Bastian - Selling Sickness 2010Hilda Bastian - Selling Sickness 2010
Hilda Bastian - Selling Sickness 2010Gezonde scepsis
 
iHT² Health IT Summit Denver 2013 - C.T. Lin, CMIO, University of Colorado H...
 iHT² Health IT Summit Denver 2013 - C.T. Lin, CMIO, University of Colorado H... iHT² Health IT Summit Denver 2013 - C.T. Lin, CMIO, University of Colorado H...
iHT² Health IT Summit Denver 2013 - C.T. Lin, CMIO, University of Colorado H...Health IT Conference – iHT2
 
Health care informatics evolutions and transitions to today
Health care informatics evolutions and transitions to todayHealth care informatics evolutions and transitions to today
Health care informatics evolutions and transitions to todayCashews
 
Submit20your20 powerpoint20file20here bernardp11_attempt_2012-12-05-21-24-27_...
Submit20your20 powerpoint20file20here bernardp11_attempt_2012-12-05-21-24-27_...Submit20your20 powerpoint20file20here bernardp11_attempt_2012-12-05-21-24-27_...
Submit20your20 powerpoint20file20here bernardp11_attempt_2012-12-05-21-24-27_...Xiaoming Zeng
 

Similaire à Learning to speak medicine (20)

Understanding medical concepts and codes through NLP methods
Understanding medical concepts and codes through NLP methodsUnderstanding medical concepts and codes through NLP methods
Understanding medical concepts and codes through NLP methods
 
Report from 2015 MedInfo Conference Brazil
Report from 2015 MedInfo Conference BrazilReport from 2015 MedInfo Conference Brazil
Report from 2015 MedInfo Conference Brazil
 
Epidemiology versus Data Collection Bias - Studying the Needs of Platform Wor...
Epidemiology versus Data Collection Bias - Studying the Needs of Platform Wor...Epidemiology versus Data Collection Bias - Studying the Needs of Platform Wor...
Epidemiology versus Data Collection Bias - Studying the Needs of Platform Wor...
 
Advancing-OSHMS High-Performance WS in OHM
Advancing-OSHMS High-Performance WS in OHMAdvancing-OSHMS High-Performance WS in OHM
Advancing-OSHMS High-Performance WS in OHM
 
What Is Medical Informatics?
What Is Medical Informatics?What Is Medical Informatics?
What Is Medical Informatics?
 
Second-Generation HIT InformaticistsGreat discoveries can transfor.docx
Second-Generation HIT InformaticistsGreat discoveries can transfor.docxSecond-Generation HIT InformaticistsGreat discoveries can transfor.docx
Second-Generation HIT InformaticistsGreat discoveries can transfor.docx
 
Big Data and the Promise and Pitfalls when Applied to Disease Prevention and ...
Big Data and the Promise and Pitfalls when Applied to Disease Prevention and ...Big Data and the Promise and Pitfalls when Applied to Disease Prevention and ...
Big Data and the Promise and Pitfalls when Applied to Disease Prevention and ...
 
Health Informatics for Clinical Research (November 25, 2021)
Health Informatics for Clinical Research (November 25, 2021)Health Informatics for Clinical Research (November 25, 2021)
Health Informatics for Clinical Research (November 25, 2021)
 
K Bobyk - %22A Primer on Personalized Medicine - The Imminent Systemic Shift%...
K Bobyk - %22A Primer on Personalized Medicine - The Imminent Systemic Shift%...K Bobyk - %22A Primer on Personalized Medicine - The Imminent Systemic Shift%...
K Bobyk - %22A Primer on Personalized Medicine - The Imminent Systemic Shift%...
 
The Dual Nature of InformaticsInformatics can be used for impr.docx
The Dual Nature of InformaticsInformatics can be used for impr.docxThe Dual Nature of InformaticsInformatics can be used for impr.docx
The Dual Nature of InformaticsInformatics can be used for impr.docx
 
New sources of big data for precision medicine: are we ready?
New sources of big data for precision medicine: are we ready?New sources of big data for precision medicine: are we ready?
New sources of big data for precision medicine: are we ready?
 
The Nursing And Midwifery Council
The Nursing And Midwifery CouncilThe Nursing And Midwifery Council
The Nursing And Midwifery Council
 
NLM Georgia Biomedical Informatics
NLM Georgia Biomedical InformaticsNLM Georgia Biomedical Informatics
NLM Georgia Biomedical Informatics
 
Population health informatics DME SoPH 20151006
Population health informatics DME SoPH 20151006Population health informatics DME SoPH 20151006
Population health informatics DME SoPH 20151006
 
The Prevalence Of Pressure Ulcers During Home Settings...
The Prevalence Of Pressure Ulcers During Home Settings...The Prevalence Of Pressure Ulcers During Home Settings...
The Prevalence Of Pressure Ulcers During Home Settings...
 
Hilda Bastian - Selling Sickness 2010
Hilda Bastian - Selling Sickness 2010Hilda Bastian - Selling Sickness 2010
Hilda Bastian - Selling Sickness 2010
 
iHT² Health IT Summit Denver 2013 - C.T. Lin, CMIO, University of Colorado H...
 iHT² Health IT Summit Denver 2013 - C.T. Lin, CMIO, University of Colorado H... iHT² Health IT Summit Denver 2013 - C.T. Lin, CMIO, University of Colorado H...
iHT² Health IT Summit Denver 2013 - C.T. Lin, CMIO, University of Colorado H...
 
Health care informatics evolutions and transitions to today
Health care informatics evolutions and transitions to todayHealth care informatics evolutions and transitions to today
Health care informatics evolutions and transitions to today
 
Integrated health monitoring
Integrated health monitoringIntegrated health monitoring
Integrated health monitoring
 
Submit20your20 powerpoint20file20here bernardp11_attempt_2012-12-05-21-24-27_...
Submit20your20 powerpoint20file20here bernardp11_attempt_2012-12-05-21-24-27_...Submit20your20 powerpoint20file20here bernardp11_attempt_2012-12-05-21-24-27_...
Submit20your20 powerpoint20file20here bernardp11_attempt_2012-12-05-21-24-27_...
 

Plus de Xavier Amatriain

Data/AI driven product development: from video streaming to telehealth
Data/AI driven product development: from video streaming to telehealthData/AI driven product development: from video streaming to telehealth
Data/AI driven product development: from video streaming to telehealthXavier Amatriain
 
AI-driven product innovation: from Recommender Systems to COVID-19
AI-driven product innovation: from Recommender Systems to COVID-19AI-driven product innovation: from Recommender Systems to COVID-19
AI-driven product innovation: from Recommender Systems to COVID-19Xavier Amatriain
 
Lessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systemsLessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systemsXavier Amatriain
 
From one to zero: Going smaller as a growth strategy
From one to zero: Going smaller as a growth strategyFrom one to zero: Going smaller as a growth strategy
From one to zero: Going smaller as a growth strategyXavier Amatriain
 
Recommender Systems In Industry
Recommender Systems In IndustryRecommender Systems In Industry
Recommender Systems In IndustryXavier Amatriain
 
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Xavier Amatriain
 
Past present and future of Recommender Systems: an Industry Perspective
Past present and future of Recommender Systems: an Industry PerspectivePast present and future of Recommender Systems: an Industry Perspective
Past present and future of Recommender Systems: an Industry PerspectiveXavier Amatriain
 
Staying Shallow & Lean in a Deep Learning World
Staying Shallow & Lean in a Deep Learning WorldStaying Shallow & Lean in a Deep Learning World
Staying Shallow & Lean in a Deep Learning WorldXavier Amatriain
 
Machine Learning for Q&A Sites: The Quora Example
Machine Learning for Q&A Sites: The Quora ExampleMachine Learning for Q&A Sites: The Quora Example
Machine Learning for Q&A Sites: The Quora ExampleXavier Amatriain
 
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systemsBIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systemsXavier Amatriain
 
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
Strata 2016 -  Lessons Learned from building real-life Machine Learning SystemsStrata 2016 -  Lessons Learned from building real-life Machine Learning Systems
Strata 2016 - Lessons Learned from building real-life Machine Learning SystemsXavier Amatriain
 
Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectiveXavier Amatriain
 
Barcelona ML Meetup - Lessons Learned
Barcelona ML Meetup - Lessons LearnedBarcelona ML Meetup - Lessons Learned
Barcelona ML Meetup - Lessons LearnedXavier Amatriain
 
10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConf10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConfXavier Amatriain
 
10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systems10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systemsXavier Amatriain
 
Machine Learning to Grow the World's Knowledge
Machine Learning to Grow  the World's KnowledgeMachine Learning to Grow  the World's Knowledge
Machine Learning to Grow the World's KnowledgeXavier Amatriain
 
MLConf Seattle 2015 - ML@Quora
MLConf Seattle 2015 - ML@QuoraMLConf Seattle 2015 - ML@Quora
MLConf Seattle 2015 - ML@QuoraXavier Amatriain
 
Lean DevOps - Lessons Learned from Innovation-driven Companies
Lean DevOps - Lessons Learned from Innovation-driven CompaniesLean DevOps - Lessons Learned from Innovation-driven Companies
Lean DevOps - Lessons Learned from Innovation-driven CompaniesXavier Amatriain
 
10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning Systems10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning SystemsXavier Amatriain
 
Recsys 2014 Tutorial - The Recommender Problem Revisited
Recsys 2014 Tutorial - The Recommender Problem RevisitedRecsys 2014 Tutorial - The Recommender Problem Revisited
Recsys 2014 Tutorial - The Recommender Problem RevisitedXavier Amatriain
 

Plus de Xavier Amatriain (20)

Data/AI driven product development: from video streaming to telehealth
Data/AI driven product development: from video streaming to telehealthData/AI driven product development: from video streaming to telehealth
Data/AI driven product development: from video streaming to telehealth
 
AI-driven product innovation: from Recommender Systems to COVID-19
AI-driven product innovation: from Recommender Systems to COVID-19AI-driven product innovation: from Recommender Systems to COVID-19
AI-driven product innovation: from Recommender Systems to COVID-19
 
Lessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systemsLessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systems
 
From one to zero: Going smaller as a growth strategy
From one to zero: Going smaller as a growth strategyFrom one to zero: Going smaller as a growth strategy
From one to zero: Going smaller as a growth strategy
 
Recommender Systems In Industry
Recommender Systems In IndustryRecommender Systems In Industry
Recommender Systems In Industry
 
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
 
Past present and future of Recommender Systems: an Industry Perspective
Past present and future of Recommender Systems: an Industry PerspectivePast present and future of Recommender Systems: an Industry Perspective
Past present and future of Recommender Systems: an Industry Perspective
 
Staying Shallow & Lean in a Deep Learning World
Staying Shallow & Lean in a Deep Learning WorldStaying Shallow & Lean in a Deep Learning World
Staying Shallow & Lean in a Deep Learning World
 
Machine Learning for Q&A Sites: The Quora Example
Machine Learning for Q&A Sites: The Quora ExampleMachine Learning for Q&A Sites: The Quora Example
Machine Learning for Q&A Sites: The Quora Example
 
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systemsBIG2016- Lessons Learned from building real-life user-focused Big Data systems
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
 
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
Strata 2016 -  Lessons Learned from building real-life Machine Learning SystemsStrata 2016 -  Lessons Learned from building real-life Machine Learning Systems
Strata 2016 - Lessons Learned from building real-life Machine Learning Systems
 
Past, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspectivePast, present, and future of Recommender Systems: an industry perspective
Past, present, and future of Recommender Systems: an industry perspective
 
Barcelona ML Meetup - Lessons Learned
Barcelona ML Meetup - Lessons LearnedBarcelona ML Meetup - Lessons Learned
Barcelona ML Meetup - Lessons Learned
 
10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConf10 more lessons learned from building Machine Learning systems - MLConf
10 more lessons learned from building Machine Learning systems - MLConf
 
10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systems10 more lessons learned from building Machine Learning systems
10 more lessons learned from building Machine Learning systems
 
Machine Learning to Grow the World's Knowledge
Machine Learning to Grow  the World's KnowledgeMachine Learning to Grow  the World's Knowledge
Machine Learning to Grow the World's Knowledge
 
MLConf Seattle 2015 - ML@Quora
MLConf Seattle 2015 - ML@QuoraMLConf Seattle 2015 - ML@Quora
MLConf Seattle 2015 - ML@Quora
 
Lean DevOps - Lessons Learned from Innovation-driven Companies
Lean DevOps - Lessons Learned from Innovation-driven CompaniesLean DevOps - Lessons Learned from Innovation-driven Companies
Lean DevOps - Lessons Learned from Innovation-driven Companies
 
10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning Systems10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning Systems
 
Recsys 2014 Tutorial - The Recommender Problem Revisited
Recsys 2014 Tutorial - The Recommender Problem RevisitedRecsys 2014 Tutorial - The Recommender Problem Revisited
Recsys 2014 Tutorial - The Recommender Problem Revisited
 

Dernier

Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 

Dernier (20)

Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 

Learning to speak medicine

  • 2. Part I. The Medical Conversation
  • 3. Medical Diagnosis ● Diagnosis (R.A. Miller 1990): ○ Mapping from patient’s data (history, examination, lab exams…) to a possible condition. ○ It depends on ability to: ■ Evoke history ■ Surface symptoms and findings ■ Generate hypotheses that suggest how to refine or pursue different hypothesis ○ In a compassionate, cost-effective manner ● Doctor: “How can I help?” ● Patient: Chief complaint ● Doctor: “Anything else?” ● Patient: …. ● Start of a doctor-led Q&A: ○ E.g. “Do you have X?” ● Doctor communicates actionable recommendation (diagnosis + treatment, triage, referral…)
  • 4. Medicine is hard(er) ● Doctors have ~15 minutes to capture information* about a patient, diagnose, and recommend treatment ● *Information ○ Patient’s history ○ Patient’s symptoms ○ Medical knowledge ■ Learned years ago ■ Latest research findings ■ Different demographics ● Data is growing over time, so is complexity
  • 5. The future of healthcare ● Patient-centered, AI-driven, doctor-in-the-loop ● Increased value of patient/doctor interaction ● Data-enabled applications ● Individual/personalized recommendations ● Multimodal (images, voice, audio…) ● Pro-active Prognosis/Diagnosis (eg. when sensor reading abnormal or new information available) ● Identifying and warning about side effect of medications ● Conversational systems are a core component Personalized AI agent Sensors Conversational System (eliciting and providing information, recommending outcomes, treatments, prevention)
  • 7. Is it a new idea? ● Internist -1 (1971) led by Jack Myers considered (one of) the best clinical diagnostic experts in the US ● Process for adding a disease requires 2-4 weeks of full-time effort and doctors reading 50 to 250 relevant publications ● Very structured heuristic-driven dialog system
  • 8. Is it still hot?
  • 10. Data Availability ● Large-scale patient-level clinical data ○ Electronic health records ○ Electronic imaging: x-rays, scans ○ Genes ● Smart devices as health sensors ○ Wearables ○ FDA-approved phone apps ○ High-quality images ● Electronic access to medical research ● Much of this only available in the last few years
  • 11. The Language of Medicine ● ICD(10) ○ 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD) ○ Codes for diseases, symptoms, findings, complaints... ○ Evolved from Bertillon Classification of Causes of Death (1893) ○ UN gave WHO responsibility for the ICD in 1946 ● Snomed Clinical Terms ○ Computer processable collection of medical terms used in clinical documentation and reporting. ○ Clinical findings, symptoms, diagnoses, procedures, body structures, organisms substances, pharmaceuticals, devices… ○ Started in 1965, has had multiple revisions and mutations ● UMLS ○ Compendium of many controlled vocabularies (1986) ○ Mapping structure among vocabularies ○ Allows to translate among the various terminology systems
  • 12. Electronic health records ● Digital information about patients encounters with doctors or the health system ● An encounter may include ○ Doctor notes, medications, procedures, diagnosis ● Mix of structured data and unstructured text
  • 13. Open Data ● Hnlp ● I2b2 ● Mimic-3 ● ….
  • 14. Part III. Building Medical Dialogue Systems
  • 15. Medical conversation as a task-oriented dialogue
  • 17. Text representation ● Infer structured representation An 18-year-old male student presents with severe headache and fever that he has had for 3 days. Examination reveals fever, photophobia, and neck stiffness Demographics: ● Age: 18 years ● Gender: Male Symptoms: ● headache ○ Severity: severe ● Fever ○ Duration: 3 days ● Photophobia ● Neck stiffness
  • 18. Slot filling in frames
  • 19. Text representation ● Infer latent semantic space through embeddings A. L. Beam, B. Kompa, I. Fried, N. P. Palmer, X. Shi, T. Cai, and I. S. Kohane. 2018. Clinical Concept Embeddings Learned from Massive Sources of Medical Data. ArXiv e-prints (April 2018). 108,477 medical concepts using: ● insurance claims database of 60 million members ● 20 million clinical notes ● 1.7 million full text biomedical journal articles
  • 22. The Deep Learning promise
  • 23. Learning to infer structure in text A. Jagannatha and H. Yu, Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records, 2016 A. Vani et.al. Grounded Recurrent Neural Networks, arXiv, 2017 bleeding due to warfarin Adverse drug effect - - medication
  • 24. End-to-end Deep Learning Dialog Systems SOTA Language model input: User: got home and when i 've been at work i 've been experiencing a pain like a 4 on my left side by my rib and some days i feel where i had my iv in my arm on march 2 when i went to the er for my constipation Dr: ah i see . have you tried anything for relief ? User: took an acetaminophen but that only last for about an hour my body burns off medicine really fast i learned that at a young age Language model output: Dr: i see . how bad is the pain ? on a scale of 1 - 10 10 being the worst how would you rate it ? User: 8 Dr: sounds rough ! applying a warm compress 2 - 3 times a day warm baths with epsom salt massaging...
  • 25. End-to-end Deep Learning Dialog Systems
  • 27. ● “An interlingua for electronic interchange of medical information: using frames to map between clinical vocabularies.” Masarie. 1991 ● “Health dialog systems for patients and consumers.” Bickmore. 2006 ● “Designing a Chatbot for diabetic patients”, Lokman. 2007 ● “Computer-assisted diagnostic decision support: history, challenges, and possible paths forward” Miller. 2009. ● “Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)” Murphy. 2010 ● “Mining Biomedical Ontologies and Data Using RDF Hypergraphs” Liu et al. 2013. ● “From health search to healthcare: explorations of intention and utilization via query logs and user surveys” White. 2014 ● “Pharmabot: A Pediatric Generic Medicine Consultant Chatbot”, Comendador. 2015 ● “Identifiable Phenotyping using Constrained Non-Negative Matrix Factorization”. Joshi et al. 2016. ● “Clinical Tagging with Joint Probabilistic Models” . Halpern et al. 2016. ● “MIMIC-III, a freely accessible critical care database” Johnson. 2016 ● “Disease named entity recognition by combining conditional random fields & bidirectional recurrent neural networks” Wei. 2016 ● “Named Entity Recognition Over Electronic Health Records Through a Combined Dictionary-based Approach”, Pomares. 2016 ● “Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records”, Jagannatha. 2016 ● “Bringing semantic structures to user intent detection in online medical queries”, Zhang. 2017 ● “A Conversational Chatbot Based on Kowledge-Graphs for Factoid Medical Questions”, A. Minutolo. 2017 ● “MANDY: Towards a Smart Primary Care Chatbot Application”. Ni. 2017 ● “Entity recognition from clinical texts via recurrent neural network”, Liu. 2017 ● “Knowledge-driven Entity Recognition and Disambiguation in Biomedical Text”. Siu. 2017 ● “Deep Learning for Dialogue Systems”. Vivian Chen. 2017 ● “Universal Language Model Fine-tuning for Text Classification”, Howard. 2018 ● “Clinical Concept Embeddings Learned from Massive Sources of Medical Data” A. L. Beam, et al. ArXiv e-prints (April 2018). References