1.
kHealth: Proactive Personalized Actionable Information for
Better Healthcare
Put Knoesis Banner
PDA@IoT, in conjunction with VLDB, September, 2014
Amit Sheth, Pramod Ananthram, T.K. Prasad
The Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis)
Wright State, USA

2.
2
A Historical Perspective on Collecting Health Observations
Imhotep
Laennec’s stethoscope
Image Credit: British Museum
2600 BC ~1815 Today
Diseases treated only
by external observations
First peek beyond just
external observations
Information overload!
Doctors relied only on
external observations
Stethoscope was the
first instrument to go
beyond just external
observations
Though the stethoscope
has survived, it is only one
among many observations
in modern medicine
http://en.wikipedia.org/wiki/Timeline_of_medicine_and_medical_technology

3.
“The next wave of dramatic Internet growth will come through the confluence of
people, process, data, and things — the Internet of Everything (IoE).”
- CISCO IBSG, 2013
Beyond the IoE based infrastructure, it is the possibility of developing applications that spans
Physical, Cyber and the Social Worlds that is very exciting.
3
http://www.cisco.com/web/about/ac79/docs/innov/IoE_Economy.pdf
What has changed now?

4.
‘OF human’ : Relevant Real-time Data Streams for Human Experience
Petabytes of Physical(sensory)-Cyber-Social Data everyday!
More on PCS Computing: http://wiki.knoesis.org/index.php/PCS
4

5.
6
MIT Technology Review, 2012
The Patient of the Future
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/

6.
‘FOR human’ : Improving Human Experience (Smart Health)
Weather Application
Asthma Healthcare
Application
Personal
Public Health
Detection of events, such as wheezing
sound, indoor temperature, humidity,
dust, and CO level
Close the window at home
during day to avoid CO in
gush, to avoid asthma attacks
at night
7
Population Level
Action in the Physical World
Luminosity
CO level
CO in gush
during day time

7.
‘FOR human’ : Improving Human Experience (Smart Energy)
Weather Application
Power Monitoring
Application
Personal Level Observations
Electricity usage over a day, device at
work, power consumption, cost/kWh,
heat index, relative humidity, and public
events from social stream
8
Population Level Observations
Action in the Physical World
Washing and drying has
resulted in significant cost
since it was done during peak
load period. Consider
changing this time to night.

8.
kHealth
Knowledge-enabled Healthcare
Four current applications:
To reduce preventable readmissions of patients with
ADHF and GI; Asthma in children; patients with Dementia
9

9.
Brief Introduction Video

10.
Empowering Individuals (who are not Larry Smarr!) for their own health
Through physical monitoring and
analysis, our cellphones could act as
an early warning system to detect
serious health conditions, and
provide actionable information
canary in a coal mine
kHealth: knowledge-enabled healthcare
11

11.
What?
• kHealth is a knowledge-based
approach/application for patient-centric
health-care that exploits:
(a) Web based tools and social media,
(b) Mobile phone technology and wireless sensors,
(c) For synthesizing personalized actions from
heterogeneous health data
(i) For disease prevention and treatment
(ii) For health, fitness and well-being
12

12.
kHealth Kit for the application for reducing ADHF readmission
Weight Scale
Heart Rate Monitor
Blood Pressure
Monitor
13
Sensors
Android Device
(w/ kHealth App)
Readmissions cost $17B/year: $50K/readmission;
Total kHealth kit cost: < $500
ADHF – Acute Decompensated Heart Failure

13.
kHealth Kit for the application for Asthma management
Sensordrone
(Carbon monoxide,
temperature, humidity)
Node Sensor
(exhaled Nitric
Oxide)
15
Sensors
Android Device
(w/ kHealth App)
Total cost: ~ $500
*Along with two sensors in the kit, the application uses a variety of population level signals from the web:
Pollen level Air Quality
Temperature & Humidity

14.
Why?
• “Unintelligible” health data deluge due to
– Continuous monitoring of patients using passive and
active sensors
– Continuous monitoring of environment using sensors
– Public health reports
– Population level information
– Social media conversations
– Personal Electronic Medical Records (EMRs)
– Wide use of affordable mobile/wireless technologies
19

15.
Why?
• Empowering patients to improve health by
– Abstracting and integrating low-level sensor data
to more meaningful health signals
– Recommending personalized actions
• Ubiquitous, timely and effective health
management and telemedicine
– Involve patient and health-care team without
causing “interaction fatigue”
20

16.
kHealth: Health Signal Processing Architecture
Personal level
Signals
Public level
Signals
Population level
Signals
Domain
Knowledge
Risk Model
Events from
Social Streams
Take Medication before
going to work
Contact doctor
Avoid going out in the
evening due to high pollen
levels
Analysis
Personalized
Actionable
Information
Data Acquisition &
aggregation
21

17.
How?
• Data collection from various sources
– Active and passive sensing devices
– Social media crawling
– EMR
• Syntactic and semantic integration
– Qualitative/imprecise citizen observations
– Quantitative/precise sensor observations
• Provide complementary and collaborative information
• Using Semantic Web technologies, e.g., SemSOS
22

18.
How?
• Semantic Perception: Reasoning for decision
making and action generation
– Perception cycle
– Personalized action recommendation using
• Patient health score (linear scale, RYG-abstraction)
• Patient vulnerability score (personalization)
– Qualify vs quantify
• Domain (e.g. disease) specific knowledge
23

19.
24
Asthma Domain Knowledge
Asthma Control
and Actionable Information
Domain
Knowledge
Asthma Control
à
Daily Medication
Choices for starting
therapy
Not Well Controlled Poor Controlled
Severity Level
of Asthma
(Recommended Action) (Recommended Action) (Recommended Action)
Intermittent Asthma SABA prn - -
Mild Persistent Asthma Low dose ICS Medium ICS Medium ICS
Moderate Persistent
Asthma
Medium dose ICS alone
Or with
LABA/montelukast
Medium ICS +
LABA/Montelukast
Or High dose ICS
Medium ICS +
LABA/Montelukast
Or High dose ICS*
Severe Persistent Asthma High dose ICS with
LABA/montelukast
Needs specialist care Needs specialist care
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist ;
*consider referral to specialist

20.
25
Patient Health Score (diagnostic)
How controlled is my asthma?
Risk assessment
model
Semantic
Perception
Personal level
Signals
Public level
Signals
Domain
Knowledge
Population level
Signals
GREEN -- Well Controlled
YELLOW – Not well controlled
Red -- poor controlled

21.
26
Patient Vulnerability Score (prognostic)
How vulnerable* is my control level today?
Risk assessment
model
Semantic
Perception
Personal level
Signals
Public level
Signals
Domain
Knowledge
Population level
Signals
Patient health
Score
*considering changing environmental conditions and current control level

22.
Background
Knowledge
31
Health Signal Extraction to Understanding
Physical-Cyber-Social System Observations Health Signal Extraction Health Signal Understanding
Personal
Population Level
Acceleration readings from
on-phone sensors
Wheeze – Yes
Do you have tightness of chest? –Yes
Risk Category assigned by
doctors
<Wheezing=Yes, time, location>
<ChectTightness=Yes, time, location>
<PollenLevel=Medium, time, location>
<Pollution=Yes, time, location>
<Activity=High, time, location>
PollenLevel
Wheezing
ChectTightness
Pollution
Activity
PollenLevel
Wheezing
ChectTightness
Pollution
Activity
RiskCategory
<PollenLevel, ChectTightness, Pollution,
Activity, Wheezing, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
.
.
.
Expert
Knowledge
Sensor and personal
observations
tweet reporting pollution level
and asthma attacks
Signals from personal, personal
spaces, and community spaces
Qualify
Quantify
Enrich
Outdoor pollen and pollution
Public Health
Well Controlled - continue
Not Well Controlled – contact nurse
Poor Controlled – contact doctor

23.
36
How are machines supposed to integrate and interpret sensor data?
RDF OWL
Semantic Sensor Networks (SSN)

24.
39
W3C Semantic Sensor Network Ontology
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K.,
Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).

25.
41
What if we could automate this sense making ability?
… and do it efficiently and at scale

26.
SSN
Ontology
2 Interpreted data
(deductive)
[in OWL]
e.g., threshold
1 Annotated Data
[in RDF]
e.g., label
0 Raw Data
[in TEXT]
e.g., number
Levels of Abstraction
3 Interpreted data
(abductive)
[in OWL]
e.g., diagnosis
Intellego
Hyperthyroidism
… …
Elevated
Blood
Pressure
Systolic blood pressure of 150 mmHg
“150”
42

27.
43
Making sense of sensor data with

28.
People are good at making sense of sensory input
What can we learn from cognitive models of perception?
• The key ingredient is prior knowledge
44

29.
Semantic Perception : Perception Cycle
Semantic perception in kHealth involves:
• Abductive reasoning to derive candidate
explanations for sensor data, and
• Deductive reasoning to disambiguate among
multiple explanations with patient inputs and
additional targeted sensor observations.
Intellego
45

30.
Observe
Property
* based on Neisser’s cognitive model of perception
Perceive
Feature
Explanation
Discrimination
1
2
Perception Cycle*
Translating low-level signals
into high-level knowledge
Focusing attention on those
aspects of the environment that
provide useful information
Prior Knowledge
46

31.
To enable machine perception,
Semantic Web technology is used to integrate
sensor data with prior knowledge on the Web
47

32.
Prior knowledge on the Web
W3C Semantic Sensor
Network (SSN) Ontology Bi-partite Graph
48

33.
Prior knowledge on the Web
W3C Semantic Sensor
Network (SSN) Ontology Bi-partite Graph
49

34.
Explanation is the act of choosing the objects or events that best account for a
set of observations; often referred to as hypothesis building
Observe
Property
Perceive
Feature
Explanation
1
Explanation
Translating low-level signals
into high-level knowledge
50

35.
Discrimination is the act of finding those properties that, if observed, would help distinguish
between multiple explanatory features
Observe
Property
Perceive
Feature
Explanation
Discrimination
2
Focusing attention on those
aspects of the environment that
provide useful information
Discrimination
51

36.
Discrimination
Discriminating Property: is neither expected nor not-applicable
DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty
Discriminating Property Explanatory Feature
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
52

37.
Semantic Perception : Abstraction
• Mapping low-level sensor values to coarse-grain
abstract values
– E.g., Blood pressure: 150/100 => High bp
• Extracting signatures for high-level human
comprehensible features from low-level
sensor data stream.
– E.g., Parkinson disease : unsteady walk, fall,
slurred speech, etc.
53

38.
How do we implement machine perception efficiently on a
resource-constrained device?
Use of OWL reasoner is resource intensive
(especially on resource-constrained devices),
in terms of both memory and time
• Runs out of resources with prior knowledge >> 15 nodes
• Asymptotic complexity: O(n3)
57

39.
Approach 1: Send all sensor observations
to the cloud for processing
Approach 2: downscale semantic
processing so that each device is capable
of machine perception
intelligence at the edge
Henson et al. 'An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices,
ISWC 2012.
58

40.
Efficient execution of machine perception
Use bit vector encodings and their operations to encode prior knowledge and
execute semantic reasoning
010110001101
0011110010101
1000110110110
101100011010
0111100101011
000110101100
0110100111
59

41.
Evaluation on a mobile device
Efficiency Improvement
• Problem size increased from 10’s to 1000’s of nodes
• Time reduced from minutes to milliseconds
• Complexity growth reduced from polynomial to linear
O(n3) < x < O(n4) O(n)
60

42.
Semantic Perception for smarter analytics: 3 ideas to takeaway
1 Translate low-level data to high-level knowledge
Machine perception can be used to convert low-level sensory
signals into high-level knowledge useful for decision making
2 Prior knowledge is the key to perception
Using SW technologies, machine perception can be formalized and
integrated with prior knowledge on the Web
3 Intelligence at the edge
By downscaling semantic inference, machine perception can
execute efficiently on resource-constrained devices
61

43.
62
thank you, and please visit us at
http://knoesis.org