SlideShare une entreprise Scribd logo
1  sur  39
Télécharger pour lire hors ligne
Generating Problem-Oriented Summary
from Electronic Medical Records
IBM Watson Health
Ching-Huei Tsou
April 14, 2016
Watson’s post-Jeopardy challenge: Healthcare
 Our first domain of exploration is medical decision support
because of its mature, complex and meaningful
problem solving nature
 After Watson’s win on Jeopardy!, people (outside of computer
science community) assumed that anything that could be
phrased as a question could be correctly answered by Watson:
Watson, “Given my medical record
<insert hundreds of pages of structured and unstructured
data here>
, what’s wrong with me?”
Filing System
Summarization Multi-Step ReasoningClinical Knowledge QA EMR Deep QA & Search
*Not to scale
From the generated
Problem oriented summary,
physician noticed the patient’s
“Creatinine level is high”
What has been done to treat
his diabetic nephropathy?
What else can I try?
Clinical Decision
Support System
EMR
What are the causes of
creatinine elevation?
What are the most
likely causes for
this patient?
Toward a Clinical Decision Support System
EMR Analysis Medical QA Reasoning EMR QA
Problem-Oriented Summarization
Electronic Medical Records
Unstructured Data
Clinical Notes
Semi-structured Data,
e.g.
Diagnoses,
medications,
lab test results
100s of notes for a
typical patient;
1,000s for older patients
with inpatient notes
Promises of Electronic Medical/Health Record
 Why a Dr. went into medicine
 Not why a Dr. went into medicine  Prevent medical errors
 Reduce health care costs
 Increase administrative efficiency
 Decrease paperwork
 Expand access to affordable care
Today’s EMR is Broken
 Digitizing medical records does not reduce physicians’ cognitive load
 Today’s EMR is largely billing-oriented
 Billing compliance regulations require that notes stand on their own, which
may promote duplication of text
 Detailed coding = bill = better quality?
General Coding Specific Coding
428.0 CHF NOS
428.9 HF NOS
428.21 Ac Systolic HF
428.23 Ac-Chr Systolic HF
428.31 Ac Diastolic HF
428.33 Ac-Chr Diastolic HF
428.41 Ac Comb S&D HF
428.43 Ac-Chr Comb S&D HF
$29,716 $53,670
How EMR Summarization Can Help a Physician?
 Consider a physician who is about to see a patient in an outpatient setting
 Perhaps this is the first encounter for the physician with the patient, or
 It has been a while since the physician has seen this patient
 Before seeing the patient, the physician may want to know
 What are the patient’s current problems?
 When was a problem last discussed / addressed?
 How a problem is being managed?
Current medications?
Related lab test results?
Most questions are problem-oriented
Problem-Oriented Medical Record (POMR) Summary
Problems List
Medications
Lab tests
“treated by”
“measured by”
“discussed in”
Procedures
Vitals
Clinical Notes & timeline
 POMR, as originally defined
by Dr. Lawrence Weed in the
1960s is the official record
keeping method in most US
hospitals
 Problem list is also a
mandatory section in the
CCD (continuity of care),
part of HL7’s CDA (clinical
document architecture)
standard
 The key to success?
An accurate problem list
The Problem List Challenge
 Unfortunately, manually maintained problem lists are not
accurate
Our assessment of existing problem lists based on a gold standard
indicates the challenge
Entered Problem List Accuracy:
Recall (Sensitivity) = 0.55
Precision (Positive Predictive Rate) = 0.28
Ground-Truth
Problem
Problems on
the entered list
Resolved Problem
Acute Problem
Problems added
for billing purpose
Patient’s pre-
existing problems
No time to
update the list
FN
FP
TP
Rule-out diagnoses
Automated Problem List Generation
Problem List: A list of current and active diagnoses as
well as past diagnoses relevant to the current care of
the patient
CMS (center of medicare and medicaid services)
Meaningful Use Stage 1
Problem List Definition
Problem-List Ground-Truth Annotation Guidelines
What to include:
1. Chronic disease like diabetes, hypertension, hyperlipidemia etc.
2. History of cardiovascular events such as CVA, MI, DVT, PE.
3. Non-injury related musculoskeletal conditions like degenerative disc disease, osteoarthritis, osteoporosis, and rickets
4. History of drug or alcohol ABUSE
5. All psychiatric diagnoses
6. Obesity and obesity related problems like sleep apnea, fatty liver disease etc.
7. Resolved problems of high importance such as recurrent PNA, anemia, etc.
8. Complications from other disease processes, such as diabetic neuropathy, CKD from hypertension etc
9. malignant Neoplasms (or history of) regardless of patient status and any benign neoplasms that need to be monitored
What should NOT be included:
1. all injuries
2. resolved problems of either low importance, or those which have been corrected by surgery(bronchitis, pneumonia, cholecystitis with cholecystectomy,
hernia that has been surgically corrected, appendicitis with appendectomy, etc).
3. Most dermatologic conditions including warts, transient skin rashes of low importance that are resolved. Only exception to this is Acne (regardless of severity)
is included.
4. Signs or symptoms of disease; chest pain, headache, abdominal pain, epistaxis, hematuria, etc. Usually these will have some corresponding diagnosis. If not
then it isn’t included. Only exception is Lumbago, which because of its usual chronicity IS included.
5. Severity of disease, as these tend to wax and wane in many chronic problems.
6. Cause of death or anything from an autopsy report
Where to take information from:
1. Any clinical note, operative report, telephone encounter, etc, where a specific diagnoses is discussed.
2. Do not make inferences. Ie, if a note says fasting glucose of 156, unless it explicitly says this patient has diabetes, leave the diagnosis off
3. Words like probable or suspected before a given diagnoses are situation dependent. Sometimes a later note will confirm or refute that diagnosis.
Tips:
1. Remember that notes have places for allergies, past surgeries, procedures, etc. so leaving things off of a problem list doesn’t mean the information isn’t
available.
2. Try to make the diagnosis as concise as possible, abbreviations are acceptable.
3. If you’re unsure then include it and it can always be removed during adjudication
Guidelines are
subject to
explanation and
extensive domain
knowledge is
required
EMRA Problem List Generation
Candidate Generation Scoring & Ranking
Find everything that looks
like a disorder from the
clinical notes
Look for contextual
information and
supporting evidences
Clinical Note
(Watson) Annotated Clinical Note / Entity Linking
 Parsing
 Sections
 Paragraphs
 Part-of-speech
 Entity-Linking
 Recognition
 Disambiguation
 Negation Detection
Context-aware Computing
 Given the context, we have no problem reading the
sentences above, even though the characters H and A
(and B and13) are identical
Context in EMR
Word
Sentence
Section
Note
Medication & Labs
Similar Pattern in Other EMRs
 Hypertension
 Hypertension: Yes
 Assessment and Plan
 Hypertension: Yes
 Mentioned in several other notes
 Taking HTN drugs, elevated BP
 Other patients with similar
pattern has been diagnosed
with hypertension
19
EMRA Problem List Accuracy:
Recall (Sensitivity) = 0.84
Precision (Positive Predictive Rate) = 0.52
Recall Oriented  F2 = 0.75
Entered Problem List Accuracy:
Recall (Sensitivity) = 0.55
Precision (Positive Predictive Rate) = 0.28
GroupingCandidate
Generation
Feature Generation
Information
Extraction
Text
Segmen-
tation
Scoring / WeightingEMR
Clinical
Factors
Extraction
CUI Confidence
Note Section
Notes
Structured
Data
(Medications,
Orders, Lab,
etc)
CUIs of unique
Disorders (100s)
Candidate Problems (10s)
CUIs of unique
Medications (10s),
Orders, Lab, etc.
Merging
and
Clustering
Closely
Related
Problems
Term Frequency
Relationship
LSA / DSRD
CUI Path
LSA / DSRD
CUI PathMedsLabs
Score
1.0
0 0.4
Confidence
Score
1.0
0 10
Term
Frequency
Score
1.0
0 0.3
LSA Score
Score
1.0
0 A may
treat B
Path
Pattern
Score
1.
0
0 PMH
Note
Section
Note Type
EMRA Problem List Generation
EMRA Problem List Generation
20
EMRA Problem List Accuracy:
Recall (Sensitivity) = 0.70
Precision (Positive Predictive Rate) = 0.73
Precision Oriented  F1 = 0.72
Entered Problem List Accuracy:
Recall (Sensitivity) = 0.55
Precision (Positive Predictive Rate) = 0.28
GroupingCandidate
Generation
Feature Generation
Information
Extraction
Text
Segmen-
tation
Scoring / WeightingEMR
Clinical
Factors
Extraction
CUI Confidence
Note Section
Notes
Structured
Data
(Medications,
Orders, Lab,
etc)
CUIs of unique
Disorders (100s)
Candidate Problems (10s)
CUIs of unique
Medications (10s),
Orders, Lab, etc.
Merging
and
Clustering
Closely
Related
Problems
Term Frequency
Relationship
LSA / DSRD
CUI Path
LSA / DSRD
CUI PathMedsLabs
Score
1.0
0 0.4
Confidence
Score
1.0
0 10
Term
Frequency
Score
1.0
0 0.3
LSA Score
Score
1.0
0 A may
treat B
Path
Pattern
Score
1.
0
0 PMH
Note
Section
Note Type
EMRA in Action
EMR Summarization
Watson generates
and groups Problems
by clinical relevance
Watson groups medications
by clinical relevance
Each panel contains answers to a pre-defined question
Context-aware User Interface
Labs show elevated
glucose and A1C among
the others…
When a problem is selected
Current and related meds
are highlighted
Relevant notes
are highlighted
Is the patient's diabetes well-controlled?
 What was patient's last HbA1c? When was it taken?
 Patient's hemoglobin A1c is red indicating it is not within normal range.
Patient’s HbA1c has been high except for a single reading in 2013, so
patient's diabetes has NOT been well-controlled.
A1C went down,
why?
A1C went up,
why?
A1C went down;
why?
A1C went up in most
recent test despite being
on Victoza (liraglutide);
why?
Endocrinology note on 03/06/2013
Endocrinology note on 07/17/2013
 EMRA makes it easy
to find and bring up
relevant notes
Is the patient's diabetes well-controlled?
Semantic Find
 Acute problems are normally not considered as problems, and don’t show
up in the Summarization UI
 Patient come in complaining of hearing problem
 has patient experienced this before?
 Was patient started on any treatment?
Quality Assessment
Quality Assessment
“I’d consider Watson extremely useful if it can
find one important problem that is missed by
physician”
Neil Mehta. M.D., Internist, Cleveland Clinic
Quality Assessment
 6 Cleveland Clinic physicians
reviewed 15 EMRs to
generated their own problem
lists, and then compared and
rated the problem lists
each physician reviewed 5
EMRs, and each EMR is
reviewed by 2 physicians
Watson generated lists were given after physicians completed their
own list. Physicians were asked to rate the Watson generated
problems one by one and as a whole
for each problem, is it correct? Is it on your list? If correct, how important is it?
as a whole, rate each list from 1-10 (Likert scale)
Very Important
Ground Truth
Physician
Watson
Important
Somewhat Important
Not at all Important
Quality Assessment
Manually
Maintained
Physician
Generated
Watson
Generated
 Average Rating
 Current System: 5.8
 Watson: 7.4
 Physician: 8.4 *The differences are statistical Significant (p=0.02)
Quality Assessment
 Simple linear regression indicates the most important factor to
higher Watson rating is “Percentage of very important
problems that are missed by physician and found by Watson”
 In average Watson found 1.2 very important or important
problem missed by physician per EMR (avg. 6 problems)
Specialty Specific and/or
Personalized
Type of False-Positive Problems
Transient
problem
51%
Correct
21%
Redundant
Problem
11%
Certainty
error
5%
System
error
4%
Noise
4%
Negation
error
3%
Human
error
1%
 Error analysis showed most of the
false-positives are “transient problems”
 Transient problems are true findings or
disorders of the patient that are less
important to the medical care
 Minor / self-limited problem
 waxing and waning, e.g. seasonal
 Resolved
 The definition is somewhat subjective
 a resolved problem to one physician
may be a significant past medical
history to another physician
 CMS (center of medicare and medicaid services) Meaningful Use
Stage 1
 Problem List: A list of current and active diagnoses as well as past
diagnoses relevant to the current care of the patient
Problem List Definition
Every known
findings / risk factors
/ disorders of the
patient
• “ideal” problem list
for a nephrologist
• The blue list contains
too many irrelevant
problems
• “ideal” problem list
from an internist
• The green list is too
specific and not
comprehensive
The Problem List Challenge
Cardiovascular
Digestive
BodySystem
Endocrine
Respiratory
Genitourinary
The Problem List Challenge
Cardiovascular
Digestive
BodySystem
Endocrine
Respiratory
Genitourinary
Active Learning (Personalized)
Active Learning (Sample Complexity)
0.5
0.6
0.7
0.8
0 50 100 150 200 250 300
F2Measure
Number of Training EMRs
Current Research Direction
Learning Supervised
(batch learning)
Supervised
(active learning)
Features
Knowledge-based features
O(100) selected using ADT
tree / boosting
Features O(1,000) extracted
and selected by DNN (e.g.
auto-encoder)
Temporal
Aspect
Modeled implicitly
Explicitly clustering
multivariate time series
Today Work in Progress

Contenu connexe

Tendances

Legal Implications of Nursing Documentation in Obstetrics
Legal Implications of Nursing Documentation in ObstetricsLegal Implications of Nursing Documentation in Obstetrics
Legal Implications of Nursing Documentation in ObstetricsMargaret Wood
 
Reports shift, transfer, incident, telephone
Reports shift, transfer, incident, telephoneReports shift, transfer, incident, telephone
Reports shift, transfer, incident, telephoneSiva Nanda Reddy
 
Documentation in nursing
Documentation in nursingDocumentation in nursing
Documentation in nursingMahmoud Shaqria
 
Documentation and reporting
Documentation and reportingDocumentation and reporting
Documentation and reportingSiva Nanda Reddy
 
Recording health assessment
Recording health assessmentRecording health assessment
Recording health assessmentANILKUMAR BR
 
Medical Records Role and its Maintenance.
Medical Records Role and its Maintenance.Medical Records Role and its Maintenance.
Medical Records Role and its Maintenance.Healthcare consultant
 
Documentation student outline
Documentation student outlineDocumentation student outline
Documentation student outlinesolracious
 
Documentation and Reporting
Documentation and ReportingDocumentation and Reporting
Documentation and Reportingsri nithi
 
Babitha's Note on documentation
Babitha's Note on documentationBabitha's Note on documentation
Babitha's Note on documentationBabitha Devu
 
Methods of nursing documentation final
Methods of nursing documentation finalMethods of nursing documentation final
Methods of nursing documentation finalTaghreed Hawsawi
 
Nursing Documentation (Sports Medicine Hospital) by: Nestor Salazar Jr
Nursing Documentation (Sports Medicine Hospital)  by: Nestor Salazar JrNursing Documentation (Sports Medicine Hospital)  by: Nestor Salazar Jr
Nursing Documentation (Sports Medicine Hospital) by: Nestor Salazar JrNestor Salazar
 
Medical Records
Medical RecordsMedical Records
Medical Recordswindleh
 

Tendances (20)

Medical documentation
Medical documentationMedical documentation
Medical documentation
 
Legal Implications of Nursing Documentation in Obstetrics
Legal Implications of Nursing Documentation in ObstetricsLegal Implications of Nursing Documentation in Obstetrics
Legal Implications of Nursing Documentation in Obstetrics
 
DOCUMENTATION IN NURSING
DOCUMENTATION IN NURSINGDOCUMENTATION IN NURSING
DOCUMENTATION IN NURSING
 
Unit 4 part 2
Unit 4 part 2Unit 4 part 2
Unit 4 part 2
 
Reports shift, transfer, incident, telephone
Reports shift, transfer, incident, telephoneReports shift, transfer, incident, telephone
Reports shift, transfer, incident, telephone
 
Recording & reporting
Recording & reportingRecording & reporting
Recording & reporting
 
Documentation
DocumentationDocumentation
Documentation
 
Documentation in nursing
Documentation in nursingDocumentation in nursing
Documentation in nursing
 
Documentation and reporting
Documentation and reportingDocumentation and reporting
Documentation and reporting
 
Recording health assessment
Recording health assessmentRecording health assessment
Recording health assessment
 
Medical Records Role and its Maintenance.
Medical Records Role and its Maintenance.Medical Records Role and its Maintenance.
Medical Records Role and its Maintenance.
 
Documentation student outline
Documentation student outlineDocumentation student outline
Documentation student outline
 
Accountable Care Organization Assessment Toolkit
Accountable Care Organization Assessment ToolkitAccountable Care Organization Assessment Toolkit
Accountable Care Organization Assessment Toolkit
 
Different Types of Nursing Documentation Methods
Different Types of Nursing Documentation MethodsDifferent Types of Nursing Documentation Methods
Different Types of Nursing Documentation Methods
 
Documentation and Reporting
Documentation and ReportingDocumentation and Reporting
Documentation and Reporting
 
Babitha's Note on documentation
Babitha's Note on documentationBabitha's Note on documentation
Babitha's Note on documentation
 
Methods of nursing documentation final
Methods of nursing documentation finalMethods of nursing documentation final
Methods of nursing documentation final
 
Documentation and Reporting
Documentation and ReportingDocumentation and Reporting
Documentation and Reporting
 
Nursing Documentation (Sports Medicine Hospital) by: Nestor Salazar Jr
Nursing Documentation (Sports Medicine Hospital)  by: Nestor Salazar JrNursing Documentation (Sports Medicine Hospital)  by: Nestor Salazar Jr
Nursing Documentation (Sports Medicine Hospital) by: Nestor Salazar Jr
 
Medical Records
Medical RecordsMedical Records
Medical Records
 

En vedette

Casualties referred to hospital by volunteer first aiders: a retrospective an...
Casualties referred to hospital by volunteer first aiders: a retrospective an...Casualties referred to hospital by volunteer first aiders: a retrospective an...
Casualties referred to hospital by volunteer first aiders: a retrospective an...Jamie Ranse
 
Modelling workflow processes for clinical information systems: impact on deci...
Modelling workflow processes for clinical information systems: impact on deci...Modelling workflow processes for clinical information systems: impact on deci...
Modelling workflow processes for clinical information systems: impact on deci...Phil Gooch
 
The Electronic Medical Record - David Beausang
The Electronic Medical Record - David BeausangThe Electronic Medical Record - David Beausang
The Electronic Medical Record - David Beausanghealthcareisi
 
Unified big data architecture
Unified big data architectureUnified big data architecture
Unified big data architectureDataWorks Summit
 
Attunity Solutions for Teradata
Attunity Solutions for TeradataAttunity Solutions for Teradata
Attunity Solutions for TeradataAttunity
 
Electronic health records
Electronic health recordsElectronic health records
Electronic health recordsJocelyn Garcia
 
Clinic Workflow Diagram
Clinic Workflow DiagramClinic Workflow Diagram
Clinic Workflow Diagramataveechai
 
Electronic health records
Electronic health recordsElectronic health records
Electronic health recordsSyed Ali Raza
 
EHR Implementation Plan Presentation
EHR Implementation Plan PresentationEHR Implementation Plan Presentation
EHR Implementation Plan PresentationDavid Montanez, PMP
 
Powerpoint on electronic health record lab 1
Powerpoint on electronic health record lab 1Powerpoint on electronic health record lab 1
Powerpoint on electronic health record lab 1nephrology193
 
Week 9 emr implementation final project
Week 9 emr implementation final projectWeek 9 emr implementation final project
Week 9 emr implementation final projectprdolfin
 

En vedette (14)

Casualties referred to hospital by volunteer first aiders: a retrospective an...
Casualties referred to hospital by volunteer first aiders: a retrospective an...Casualties referred to hospital by volunteer first aiders: a retrospective an...
Casualties referred to hospital by volunteer first aiders: a retrospective an...
 
Modelling workflow processes for clinical information systems: impact on deci...
Modelling workflow processes for clinical information systems: impact on deci...Modelling workflow processes for clinical information systems: impact on deci...
Modelling workflow processes for clinical information systems: impact on deci...
 
Healthcare copy
Healthcare   copyHealthcare   copy
Healthcare copy
 
The Electronic Medical Record - David Beausang
The Electronic Medical Record - David BeausangThe Electronic Medical Record - David Beausang
The Electronic Medical Record - David Beausang
 
Unified big data architecture
Unified big data architectureUnified big data architecture
Unified big data architecture
 
Attunity Solutions for Teradata
Attunity Solutions for TeradataAttunity Solutions for Teradata
Attunity Solutions for Teradata
 
Emr Facts N Figures
Emr Facts N FiguresEmr Facts N Figures
Emr Facts N Figures
 
Electronic health records
Electronic health recordsElectronic health records
Electronic health records
 
Clinic Workflow Diagram
Clinic Workflow DiagramClinic Workflow Diagram
Clinic Workflow Diagram
 
Electronic Health Records Implementation
Electronic Health Records ImplementationElectronic Health Records Implementation
Electronic Health Records Implementation
 
Electronic health records
Electronic health recordsElectronic health records
Electronic health records
 
EHR Implementation Plan Presentation
EHR Implementation Plan PresentationEHR Implementation Plan Presentation
EHR Implementation Plan Presentation
 
Powerpoint on electronic health record lab 1
Powerpoint on electronic health record lab 1Powerpoint on electronic health record lab 1
Powerpoint on electronic health record lab 1
 
Week 9 emr implementation final project
Week 9 emr implementation final projectWeek 9 emr implementation final project
Week 9 emr implementation final project
 

Similaire à "Toward Generating Domain-specific / Personalized Problem Lists from Electronic Medical Records"

Em score-medical-decision-making
Em score-medical-decision-makingEm score-medical-decision-making
Em score-medical-decision-makingSuperCoder LLC
 
Geriatric Population. Geriatric Clinician Practice. Your Medicare, Long-Term ...
Geriatric Population. Geriatric Clinician Practice. Your Medicare, Long-Term ...Geriatric Population. Geriatric Clinician Practice. Your Medicare, Long-Term ...
Geriatric Population. Geriatric Clinician Practice. Your Medicare, Long-Term ...Michelle Peck
 
Preoperative Evaluation
Preoperative EvaluationPreoperative Evaluation
Preoperative EvaluationKhalid
 
Developing a Quality Audit Report for General Practice Prescribing for Hypert...
Developing a Quality Audit Report for General Practice Prescribing for Hypert...Developing a Quality Audit Report for General Practice Prescribing for Hypert...
Developing a Quality Audit Report for General Practice Prescribing for Hypert...Health Informatics New Zealand
 
Clinical laboratory test: Which, Why and What do results mean?.full
Clinical laboratory test: Which, Why and What do results mean?.fullClinical laboratory test: Which, Why and What do results mean?.full
Clinical laboratory test: Which, Why and What do results mean?.fullPeter Bautista
 
Eblm pres final
Eblm pres finalEblm pres final
Eblm pres finalprasath172
 
Measurement and Modeling Issues with Adherence to Pharmacotherapy
Measurement and Modeling Issues with Adherence to PharmacotherapyMeasurement and Modeling Issues with Adherence to Pharmacotherapy
Measurement and Modeling Issues with Adherence to PharmacotherapyM. Christopher Roebuck
 
Analysis of Medication Possession Ratio for Improved Blood Pressure Control
Analysis of Medication Possession Ratio for Improved Blood Pressure ControlAnalysis of Medication Possession Ratio for Improved Blood Pressure Control
Analysis of Medication Possession Ratio for Improved Blood Pressure ControlHealth Informatics New Zealand
 
2 Best Practices to Improve Emergency Department Coding
2 Best Practices to Improve Emergency Department Coding2 Best Practices to Improve Emergency Department Coding
2 Best Practices to Improve Emergency Department CodingManish Jain
 
Ebm talk-general-mar99-ppt95
Ebm talk-general-mar99-ppt95Ebm talk-general-mar99-ppt95
Ebm talk-general-mar99-ppt95rubenroa
 
06 Am09 Presentations Gutman
06 Am09 Presentations   Gutman06 Am09 Presentations   Gutman
06 Am09 Presentations GutmanSimon Prince
 
see attachments I have complete a portion of the assignment but need
see attachments I have complete a portion of the assignment but needsee attachments I have complete a portion of the assignment but need
see attachments I have complete a portion of the assignment but needPazSilviapm
 
Nursing process
Nursing processNursing process
Nursing processSreeja S A
 
Va Decision Support Seminars 2009
Va Decision Support Seminars 2009Va Decision Support Seminars 2009
Va Decision Support Seminars 2009jbreeling
 
Pt assess clinical decision making
Pt assess   clinical decision makingPt assess   clinical decision making
Pt assess clinical decision makingBen Lesold
 

Similaire à "Toward Generating Domain-specific / Personalized Problem Lists from Electronic Medical Records" (20)

Em score-medical-decision-making
Em score-medical-decision-makingEm score-medical-decision-making
Em score-medical-decision-making
 
EMRs: Meaningful Use and Research
EMRs: Meaningful Use and ResearchEMRs: Meaningful Use and Research
EMRs: Meaningful Use and Research
 
Geriatric Population. Geriatric Clinician Practice. Your Medicare, Long-Term ...
Geriatric Population. Geriatric Clinician Practice. Your Medicare, Long-Term ...Geriatric Population. Geriatric Clinician Practice. Your Medicare, Long-Term ...
Geriatric Population. Geriatric Clinician Practice. Your Medicare, Long-Term ...
 
Preoperative Evaluation
Preoperative EvaluationPreoperative Evaluation
Preoperative Evaluation
 
Developing a Quality Audit Report for General Practice Prescribing for Hypert...
Developing a Quality Audit Report for General Practice Prescribing for Hypert...Developing a Quality Audit Report for General Practice Prescribing for Hypert...
Developing a Quality Audit Report for General Practice Prescribing for Hypert...
 
Clinical laboratory test: Which, Why and What do results mean?.full
Clinical laboratory test: Which, Why and What do results mean?.fullClinical laboratory test: Which, Why and What do results mean?.full
Clinical laboratory test: Which, Why and What do results mean?.full
 
Eblm pres final
Eblm pres finalEblm pres final
Eblm pres final
 
Measurement and Modeling Issues with Adherence to Pharmacotherapy
Measurement and Modeling Issues with Adherence to PharmacotherapyMeasurement and Modeling Issues with Adherence to Pharmacotherapy
Measurement and Modeling Issues with Adherence to Pharmacotherapy
 
Analysis of Medication Possession Ratio for Improved Blood Pressure Control
Analysis of Medication Possession Ratio for Improved Blood Pressure ControlAnalysis of Medication Possession Ratio for Improved Blood Pressure Control
Analysis of Medication Possession Ratio for Improved Blood Pressure Control
 
2 Best Practices to Improve Emergency Department Coding
2 Best Practices to Improve Emergency Department Coding2 Best Practices to Improve Emergency Department Coding
2 Best Practices to Improve Emergency Department Coding
 
Ebm talk-general-mar99-ppt95
Ebm talk-general-mar99-ppt95Ebm talk-general-mar99-ppt95
Ebm talk-general-mar99-ppt95
 
06 Am09 Presentations Gutman
06 Am09 Presentations   Gutman06 Am09 Presentations   Gutman
06 Am09 Presentations Gutman
 
The Nursing Process.ppt
The Nursing Process.pptThe Nursing Process.ppt
The Nursing Process.ppt
 
see attachments I have complete a portion of the assignment but need
see attachments I have complete a portion of the assignment but needsee attachments I have complete a portion of the assignment but need
see attachments I have complete a portion of the assignment but need
 
Gpt buchman
Gpt buchmanGpt buchman
Gpt buchman
 
Nursing process
Nursing processNursing process
Nursing process
 
Gpt buchman
Gpt buchmanGpt buchman
Gpt buchman
 
Problema orientado
Problema orientadoProblema orientado
Problema orientado
 
Va Decision Support Seminars 2009
Va Decision Support Seminars 2009Va Decision Support Seminars 2009
Va Decision Support Seminars 2009
 
Pt assess clinical decision making
Pt assess   clinical decision makingPt assess   clinical decision making
Pt assess clinical decision making
 

Plus de diannepatricia

Teaching cognitive computing with ibm watson
Teaching cognitive computing with ibm watsonTeaching cognitive computing with ibm watson
Teaching cognitive computing with ibm watsondiannepatricia
 
Cognitive systems institute talk 8 june 2017 - v.1.0
Cognitive systems institute talk   8 june 2017 - v.1.0Cognitive systems institute talk   8 june 2017 - v.1.0
Cognitive systems institute talk 8 june 2017 - v.1.0diannepatricia
 
Building Compassionate Conversational Systems
Building Compassionate Conversational SystemsBuilding Compassionate Conversational Systems
Building Compassionate Conversational Systemsdiannepatricia
 
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”diannepatricia
 
Cognitive Insights drive self-driving Accessibility
Cognitive Insights drive self-driving AccessibilityCognitive Insights drive self-driving Accessibility
Cognitive Insights drive self-driving Accessibilitydiannepatricia
 
Artificial Intellingence in the Car
Artificial Intellingence in the CarArtificial Intellingence in the Car
Artificial Intellingence in the Cardiannepatricia
 
“Semantic PDF Processing & Document Representation”
“Semantic PDF Processing & Document Representation”“Semantic PDF Processing & Document Representation”
“Semantic PDF Processing & Document Representation”diannepatricia
 
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...diannepatricia
 
170330 cognitive systems institute speaker series mark sherman - watson pr...
170330 cognitive systems institute speaker series    mark sherman - watson pr...170330 cognitive systems institute speaker series    mark sherman - watson pr...
170330 cognitive systems institute speaker series mark sherman - watson pr...diannepatricia
 
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”diannepatricia
 
Cognitive Assistance for the Aging
Cognitive Assistance for the AgingCognitive Assistance for the Aging
Cognitive Assistance for the Agingdiannepatricia
 
From complex Systems to Networks: Discovering and Modeling the Correct Network"
From complex Systems to Networks: Discovering and Modeling the Correct Network"From complex Systems to Networks: Discovering and Modeling the Correct Network"
From complex Systems to Networks: Discovering and Modeling the Correct Network"diannepatricia
 
The Role of Dialog in Augmented Intelligence
The Role of Dialog in Augmented IntelligenceThe Role of Dialog in Augmented Intelligence
The Role of Dialog in Augmented Intelligencediannepatricia
 
Developing Cognitive Systems to Support Team Cognition
Developing Cognitive Systems to Support Team CognitionDeveloping Cognitive Systems to Support Team Cognition
Developing Cognitive Systems to Support Team Cognitiondiannepatricia
 
Cyber-Social Learning Systems
Cyber-Social Learning SystemsCyber-Social Learning Systems
Cyber-Social Learning Systemsdiannepatricia
 
“IT Technology Trends in 2017… and Beyond”
“IT Technology Trends in 2017… and Beyond”“IT Technology Trends in 2017… and Beyond”
“IT Technology Trends in 2017… and Beyond”diannepatricia
 
"Curious Learning: using a mobile platform for early literacy education as a ...
"Curious Learning: using a mobile platform for early literacy education as a ..."Curious Learning: using a mobile platform for early literacy education as a ...
"Curious Learning: using a mobile platform for early literacy education as a ...diannepatricia
 
Embodied Cognition - Booch HICSS50
Embodied Cognition - Booch HICSS50Embodied Cognition - Booch HICSS50
Embodied Cognition - Booch HICSS50diannepatricia
 
KATE - a Platform for Machine Learning
KATE - a Platform for Machine LearningKATE - a Platform for Machine Learning
KATE - a Platform for Machine Learningdiannepatricia
 
Cognitive Computing for Aging Society
Cognitive Computing for Aging SocietyCognitive Computing for Aging Society
Cognitive Computing for Aging Societydiannepatricia
 

Plus de diannepatricia (20)

Teaching cognitive computing with ibm watson
Teaching cognitive computing with ibm watsonTeaching cognitive computing with ibm watson
Teaching cognitive computing with ibm watson
 
Cognitive systems institute talk 8 june 2017 - v.1.0
Cognitive systems institute talk   8 june 2017 - v.1.0Cognitive systems institute talk   8 june 2017 - v.1.0
Cognitive systems institute talk 8 june 2017 - v.1.0
 
Building Compassionate Conversational Systems
Building Compassionate Conversational SystemsBuilding Compassionate Conversational Systems
Building Compassionate Conversational Systems
 
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
 
Cognitive Insights drive self-driving Accessibility
Cognitive Insights drive self-driving AccessibilityCognitive Insights drive self-driving Accessibility
Cognitive Insights drive self-driving Accessibility
 
Artificial Intellingence in the Car
Artificial Intellingence in the CarArtificial Intellingence in the Car
Artificial Intellingence in the Car
 
“Semantic PDF Processing & Document Representation”
“Semantic PDF Processing & Document Representation”“Semantic PDF Processing & Document Representation”
“Semantic PDF Processing & Document Representation”
 
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
 
170330 cognitive systems institute speaker series mark sherman - watson pr...
170330 cognitive systems institute speaker series    mark sherman - watson pr...170330 cognitive systems institute speaker series    mark sherman - watson pr...
170330 cognitive systems institute speaker series mark sherman - watson pr...
 
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
 
Cognitive Assistance for the Aging
Cognitive Assistance for the AgingCognitive Assistance for the Aging
Cognitive Assistance for the Aging
 
From complex Systems to Networks: Discovering and Modeling the Correct Network"
From complex Systems to Networks: Discovering and Modeling the Correct Network"From complex Systems to Networks: Discovering and Modeling the Correct Network"
From complex Systems to Networks: Discovering and Modeling the Correct Network"
 
The Role of Dialog in Augmented Intelligence
The Role of Dialog in Augmented IntelligenceThe Role of Dialog in Augmented Intelligence
The Role of Dialog in Augmented Intelligence
 
Developing Cognitive Systems to Support Team Cognition
Developing Cognitive Systems to Support Team CognitionDeveloping Cognitive Systems to Support Team Cognition
Developing Cognitive Systems to Support Team Cognition
 
Cyber-Social Learning Systems
Cyber-Social Learning SystemsCyber-Social Learning Systems
Cyber-Social Learning Systems
 
“IT Technology Trends in 2017… and Beyond”
“IT Technology Trends in 2017… and Beyond”“IT Technology Trends in 2017… and Beyond”
“IT Technology Trends in 2017… and Beyond”
 
"Curious Learning: using a mobile platform for early literacy education as a ...
"Curious Learning: using a mobile platform for early literacy education as a ..."Curious Learning: using a mobile platform for early literacy education as a ...
"Curious Learning: using a mobile platform for early literacy education as a ...
 
Embodied Cognition - Booch HICSS50
Embodied Cognition - Booch HICSS50Embodied Cognition - Booch HICSS50
Embodied Cognition - Booch HICSS50
 
KATE - a Platform for Machine Learning
KATE - a Platform for Machine LearningKATE - a Platform for Machine Learning
KATE - a Platform for Machine Learning
 
Cognitive Computing for Aging Society
Cognitive Computing for Aging SocietyCognitive Computing for Aging Society
Cognitive Computing for Aging Society
 

Dernier

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxYounusS2
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 

Dernier (20)

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptx
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 

"Toward Generating Domain-specific / Personalized Problem Lists from Electronic Medical Records"

  • 1. Generating Problem-Oriented Summary from Electronic Medical Records IBM Watson Health Ching-Huei Tsou April 14, 2016
  • 2. Watson’s post-Jeopardy challenge: Healthcare  Our first domain of exploration is medical decision support because of its mature, complex and meaningful problem solving nature  After Watson’s win on Jeopardy!, people (outside of computer science community) assumed that anything that could be phrased as a question could be correctly answered by Watson: Watson, “Given my medical record <insert hundreds of pages of structured and unstructured data here> , what’s wrong with me?”
  • 3. Filing System Summarization Multi-Step ReasoningClinical Knowledge QA EMR Deep QA & Search *Not to scale From the generated Problem oriented summary, physician noticed the patient’s “Creatinine level is high” What has been done to treat his diabetic nephropathy? What else can I try? Clinical Decision Support System EMR What are the causes of creatinine elevation? What are the most likely causes for this patient? Toward a Clinical Decision Support System EMR Analysis Medical QA Reasoning EMR QA
  • 5. Electronic Medical Records Unstructured Data Clinical Notes Semi-structured Data, e.g. Diagnoses, medications, lab test results 100s of notes for a typical patient; 1,000s for older patients with inpatient notes
  • 6. Promises of Electronic Medical/Health Record  Why a Dr. went into medicine  Not why a Dr. went into medicine  Prevent medical errors  Reduce health care costs  Increase administrative efficiency  Decrease paperwork  Expand access to affordable care
  • 7. Today’s EMR is Broken  Digitizing medical records does not reduce physicians’ cognitive load  Today’s EMR is largely billing-oriented  Billing compliance regulations require that notes stand on their own, which may promote duplication of text  Detailed coding = bill = better quality? General Coding Specific Coding 428.0 CHF NOS 428.9 HF NOS 428.21 Ac Systolic HF 428.23 Ac-Chr Systolic HF 428.31 Ac Diastolic HF 428.33 Ac-Chr Diastolic HF 428.41 Ac Comb S&D HF 428.43 Ac-Chr Comb S&D HF $29,716 $53,670
  • 8. How EMR Summarization Can Help a Physician?  Consider a physician who is about to see a patient in an outpatient setting  Perhaps this is the first encounter for the physician with the patient, or  It has been a while since the physician has seen this patient  Before seeing the patient, the physician may want to know  What are the patient’s current problems?  When was a problem last discussed / addressed?  How a problem is being managed? Current medications? Related lab test results? Most questions are problem-oriented
  • 9. Problem-Oriented Medical Record (POMR) Summary Problems List Medications Lab tests “treated by” “measured by” “discussed in” Procedures Vitals Clinical Notes & timeline  POMR, as originally defined by Dr. Lawrence Weed in the 1960s is the official record keeping method in most US hospitals  Problem list is also a mandatory section in the CCD (continuity of care), part of HL7’s CDA (clinical document architecture) standard  The key to success? An accurate problem list
  • 10. The Problem List Challenge  Unfortunately, manually maintained problem lists are not accurate Our assessment of existing problem lists based on a gold standard indicates the challenge Entered Problem List Accuracy: Recall (Sensitivity) = 0.55 Precision (Positive Predictive Rate) = 0.28 Ground-Truth Problem Problems on the entered list Resolved Problem Acute Problem Problems added for billing purpose Patient’s pre- existing problems No time to update the list FN FP TP Rule-out diagnoses
  • 12. Problem List: A list of current and active diagnoses as well as past diagnoses relevant to the current care of the patient CMS (center of medicare and medicaid services) Meaningful Use Stage 1 Problem List Definition
  • 13. Problem-List Ground-Truth Annotation Guidelines What to include: 1. Chronic disease like diabetes, hypertension, hyperlipidemia etc. 2. History of cardiovascular events such as CVA, MI, DVT, PE. 3. Non-injury related musculoskeletal conditions like degenerative disc disease, osteoarthritis, osteoporosis, and rickets 4. History of drug or alcohol ABUSE 5. All psychiatric diagnoses 6. Obesity and obesity related problems like sleep apnea, fatty liver disease etc. 7. Resolved problems of high importance such as recurrent PNA, anemia, etc. 8. Complications from other disease processes, such as diabetic neuropathy, CKD from hypertension etc 9. malignant Neoplasms (or history of) regardless of patient status and any benign neoplasms that need to be monitored What should NOT be included: 1. all injuries 2. resolved problems of either low importance, or those which have been corrected by surgery(bronchitis, pneumonia, cholecystitis with cholecystectomy, hernia that has been surgically corrected, appendicitis with appendectomy, etc). 3. Most dermatologic conditions including warts, transient skin rashes of low importance that are resolved. Only exception to this is Acne (regardless of severity) is included. 4. Signs or symptoms of disease; chest pain, headache, abdominal pain, epistaxis, hematuria, etc. Usually these will have some corresponding diagnosis. If not then it isn’t included. Only exception is Lumbago, which because of its usual chronicity IS included. 5. Severity of disease, as these tend to wax and wane in many chronic problems. 6. Cause of death or anything from an autopsy report Where to take information from: 1. Any clinical note, operative report, telephone encounter, etc, where a specific diagnoses is discussed. 2. Do not make inferences. Ie, if a note says fasting glucose of 156, unless it explicitly says this patient has diabetes, leave the diagnosis off 3. Words like probable or suspected before a given diagnoses are situation dependent. Sometimes a later note will confirm or refute that diagnosis. Tips: 1. Remember that notes have places for allergies, past surgeries, procedures, etc. so leaving things off of a problem list doesn’t mean the information isn’t available. 2. Try to make the diagnosis as concise as possible, abbreviations are acceptable. 3. If you’re unsure then include it and it can always be removed during adjudication Guidelines are subject to explanation and extensive domain knowledge is required
  • 14. EMRA Problem List Generation Candidate Generation Scoring & Ranking Find everything that looks like a disorder from the clinical notes Look for contextual information and supporting evidences
  • 16. (Watson) Annotated Clinical Note / Entity Linking  Parsing  Sections  Paragraphs  Part-of-speech  Entity-Linking  Recognition  Disambiguation  Negation Detection
  • 17. Context-aware Computing  Given the context, we have no problem reading the sentences above, even though the characters H and A (and B and13) are identical
  • 18. Context in EMR Word Sentence Section Note Medication & Labs Similar Pattern in Other EMRs  Hypertension  Hypertension: Yes  Assessment and Plan  Hypertension: Yes  Mentioned in several other notes  Taking HTN drugs, elevated BP  Other patients with similar pattern has been diagnosed with hypertension
  • 19. 19 EMRA Problem List Accuracy: Recall (Sensitivity) = 0.84 Precision (Positive Predictive Rate) = 0.52 Recall Oriented  F2 = 0.75 Entered Problem List Accuracy: Recall (Sensitivity) = 0.55 Precision (Positive Predictive Rate) = 0.28 GroupingCandidate Generation Feature Generation Information Extraction Text Segmen- tation Scoring / WeightingEMR Clinical Factors Extraction CUI Confidence Note Section Notes Structured Data (Medications, Orders, Lab, etc) CUIs of unique Disorders (100s) Candidate Problems (10s) CUIs of unique Medications (10s), Orders, Lab, etc. Merging and Clustering Closely Related Problems Term Frequency Relationship LSA / DSRD CUI Path LSA / DSRD CUI PathMedsLabs Score 1.0 0 0.4 Confidence Score 1.0 0 10 Term Frequency Score 1.0 0 0.3 LSA Score Score 1.0 0 A may treat B Path Pattern Score 1. 0 0 PMH Note Section Note Type EMRA Problem List Generation
  • 20. EMRA Problem List Generation 20 EMRA Problem List Accuracy: Recall (Sensitivity) = 0.70 Precision (Positive Predictive Rate) = 0.73 Precision Oriented  F1 = 0.72 Entered Problem List Accuracy: Recall (Sensitivity) = 0.55 Precision (Positive Predictive Rate) = 0.28 GroupingCandidate Generation Feature Generation Information Extraction Text Segmen- tation Scoring / WeightingEMR Clinical Factors Extraction CUI Confidence Note Section Notes Structured Data (Medications, Orders, Lab, etc) CUIs of unique Disorders (100s) Candidate Problems (10s) CUIs of unique Medications (10s), Orders, Lab, etc. Merging and Clustering Closely Related Problems Term Frequency Relationship LSA / DSRD CUI Path LSA / DSRD CUI PathMedsLabs Score 1.0 0 0.4 Confidence Score 1.0 0 10 Term Frequency Score 1.0 0 0.3 LSA Score Score 1.0 0 A may treat B Path Pattern Score 1. 0 0 PMH Note Section Note Type
  • 22. EMR Summarization Watson generates and groups Problems by clinical relevance Watson groups medications by clinical relevance Each panel contains answers to a pre-defined question
  • 23. Context-aware User Interface Labs show elevated glucose and A1C among the others… When a problem is selected Current and related meds are highlighted Relevant notes are highlighted
  • 24. Is the patient's diabetes well-controlled?  What was patient's last HbA1c? When was it taken?  Patient's hemoglobin A1c is red indicating it is not within normal range. Patient’s HbA1c has been high except for a single reading in 2013, so patient's diabetes has NOT been well-controlled. A1C went down, why? A1C went up, why?
  • 25. A1C went down; why? A1C went up in most recent test despite being on Victoza (liraglutide); why? Endocrinology note on 03/06/2013 Endocrinology note on 07/17/2013  EMRA makes it easy to find and bring up relevant notes Is the patient's diabetes well-controlled?
  • 26. Semantic Find  Acute problems are normally not considered as problems, and don’t show up in the Summarization UI  Patient come in complaining of hearing problem  has patient experienced this before?  Was patient started on any treatment?
  • 28. Quality Assessment “I’d consider Watson extremely useful if it can find one important problem that is missed by physician” Neil Mehta. M.D., Internist, Cleveland Clinic
  • 29. Quality Assessment  6 Cleveland Clinic physicians reviewed 15 EMRs to generated their own problem lists, and then compared and rated the problem lists each physician reviewed 5 EMRs, and each EMR is reviewed by 2 physicians Watson generated lists were given after physicians completed their own list. Physicians were asked to rate the Watson generated problems one by one and as a whole for each problem, is it correct? Is it on your list? If correct, how important is it? as a whole, rate each list from 1-10 (Likert scale) Very Important Ground Truth Physician Watson Important Somewhat Important Not at all Important
  • 30. Quality Assessment Manually Maintained Physician Generated Watson Generated  Average Rating  Current System: 5.8  Watson: 7.4  Physician: 8.4 *The differences are statistical Significant (p=0.02)
  • 31. Quality Assessment  Simple linear regression indicates the most important factor to higher Watson rating is “Percentage of very important problems that are missed by physician and found by Watson”  In average Watson found 1.2 very important or important problem missed by physician per EMR (avg. 6 problems)
  • 33. Type of False-Positive Problems Transient problem 51% Correct 21% Redundant Problem 11% Certainty error 5% System error 4% Noise 4% Negation error 3% Human error 1%  Error analysis showed most of the false-positives are “transient problems”  Transient problems are true findings or disorders of the patient that are less important to the medical care  Minor / self-limited problem  waxing and waning, e.g. seasonal  Resolved  The definition is somewhat subjective  a resolved problem to one physician may be a significant past medical history to another physician
  • 34.  CMS (center of medicare and medicaid services) Meaningful Use Stage 1  Problem List: A list of current and active diagnoses as well as past diagnoses relevant to the current care of the patient Problem List Definition Every known findings / risk factors / disorders of the patient • “ideal” problem list for a nephrologist • The blue list contains too many irrelevant problems • “ideal” problem list from an internist • The green list is too specific and not comprehensive
  • 35. The Problem List Challenge Cardiovascular Digestive BodySystem Endocrine Respiratory Genitourinary
  • 36. The Problem List Challenge Cardiovascular Digestive BodySystem Endocrine Respiratory Genitourinary
  • 38. Active Learning (Sample Complexity) 0.5 0.6 0.7 0.8 0 50 100 150 200 250 300 F2Measure Number of Training EMRs
  • 39. Current Research Direction Learning Supervised (batch learning) Supervised (active learning) Features Knowledge-based features O(100) selected using ADT tree / boosting Features O(1,000) extracted and selected by DNN (e.g. auto-encoder) Temporal Aspect Modeled implicitly Explicitly clustering multivariate time series Today Work in Progress