An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices
1. 0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
An Efficient Bit Vector Approach to Semantics-based
Machine Perception in Resource-constrained Devices
0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University, Dayton, OH, USA
1
2. The Patient of the Future
MIT Technology Review, 2012
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/ 2
3. What if we could automate this
sense making ability?
… and do it efficiently and at scale
3
4. Sensing is a key enabler of the Internet of Things
50 Billion Things by 2020 (Cisco)
BUT, how do we make sense of the resulting avalanche
of sensor data?
4
5. People are good at making sense of sensory input
What can we learn from cognitive models of perception?
• The key ingredient is prior knowledge
5
6. Perception Cycle*
Translating low-level signals Explanation
into high-level knowledge 1
Observe Perceive
Property Feature
Prior Knowledge
Focusing attention on those
2 aspects of the environment that
Discrimination provide useful information
* based on Neisser’s cognitive model of perception 6
7. To enable machine perception,
Semantic Web technology is used to integrate
sensor data with prior knowledge on the Web
7
9. Prior knowledge on the Web
W3C Semantic Sensor
Network (SSN) Ontology Bi-partite Graph
9
10. Prior knowledge on the Web
W3C Semantic Sensor
Network (SSN) Ontology Bi-partite Graph
10
11. Explanation
Explanation is the act of choosing the objects or events that best account for a set of
observations; often referred to as hypothesis building
Translating low-level signals Explanation
into high-level knowledge 1
Observe Perceive
Property Feature
11
12. Explanation
Explanation is the act of choosing the objects or events that best account for a set of
observations; often referred to as hypothesis building
Inference to the best explanation
• In general, explanation is an abductive problem;
and hard to compute
Finding the sweet spot between abduction and OWL
• Single-feature assumption* enables use of
OWL-DL deductive reasoner
* An explanation must be a single feature which accounts for
all observed properties
12
13. Explanation
Explanatory Feature: a feature that explains the set of observed properties
ExplanatoryFeature ≡ ∃ssn:isPropertyOf—.{p1} ⊓ … ⊓ ∃ssn:isPropertyOf—.{pn}
Observed Property Explanatory Feature
elevated blood pressure Hypertension
clammy skin Hyperthyroidism
palpitations Pulmonary Edema
13
14. Discrimination
Discrimination is the act of finding those properties that, if observed, would help
distinguish between multiple explanatory features
Explanation
Observe Perceive
Property Feature
Focusing attention on those
2 aspects of the environment that
Discrimination provide useful information
14
15. Discrimination
Expected Property: would be explained by every explanatory feature
ExpectedProperty ≡ ∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ∃ssn:isPropertyOf.{fn}
Expected Property Explanatory Feature
elevated blood pressure Hypertension
clammy skin Hyperthyroidism
palpitations Pulmonary Edema
15
16. Discrimination
Not Applicable Property: would not be explained by any explanatory feature
NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ¬∃ssn:isPropertyOf.{fn}
Not Applicable Property Explanatory Feature
elevated blood pressure Hypertension
clammy skin Hyperthyroidism
palpitations Pulmonary Edema
16
18. Our Motivation
kHealth: knowledge-enabled healthcare
Through physical monitoring and
analysis, our cellphones could act
as an early warning system to
detect serious health conditions
canary in a coal mine
18
19. How do we implement machine perception efficiently on a
resource-constrained device?
Use of OWL reasoner is resource intensive
(especially on resource-constrained devices),
in terms of both memory and time
• Runs out of resources with prior knowledge >> 15 nodes
• Asymptotic complexity: O(n3)
19
20. Approach 1: Send all sensor
observations to the cloud for
processing
Approach 2: downscale semantic
processing so that each device is
capable of machine perception
intelligence at the edge
20
21. Efficient execution of machine perception
Use bit vector encodings and their operations to encode prior knowledge
and execute semantic reasoning
010110001101
0011110010101
1000110110110
101100011010
0111100101011
000110101100
0110100111
21
22. Lifting and lowering knowledge
Translate prior knowledge, observations, and explanations between SW and
bit vector representation
lower
lift
22
23. Explanation: efficient algorithm
INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and dismiss
those features that cannot explain the set of observed properties.
Observed Previous Current
Property Prior Knowledge Explanatory Feature Explanatory Feature
HN HM PE HN HM PE HN HM PE
bp 1 bp 1 1 1 AND 1 1 1 => 1 1 1
cs 0 cs 0 1 0
pa 1 pa 1 1 0 AND => 1 1 0
23
24. Discrimination: efficient algorithm
INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and assemble
those features that discriminate between the explanatory features
Observed Previous Current Discriminatin
Property Prior Knowledge Explanatory Feature Explanatory Feature g
Property
HN HM PE
bp 1 bp 0 1 1 HN HM PE HN HM PE
bp 0
cs 0 cs 0 1 0 AND 1 1 0 => 0 1 0 cs 1
0
pa 1 pa 1 1 0 pa 0
… expected? = 0 1 0 => FALSE
Is the property
discriminating? ZERO Bit Vector
… not-applicable? = => FALSE
0 0 0
24
25. Evaluation on a mobile device
Efficiency Improvement
• Problem size increased from 10’s to 1000’s of nodes
• Time reduced from minutes to milliseconds
• Complexity growth reduced from polynomial to linear
O(n3) < x < O(n4) O(n)
25
26. 3 ideas to takeaway
1 Translate low-level data to high-level knowledge
Machine perception can be used to convert low-level sensory signals
into high-level knowledge useful for decision making
2 Prior knowledge is the key to perception
Using SW technologies, machine perception can be formalized and
integrated with prior knowledge on the Web
3 Intelligence at the edge
By downscaling semantic inference, machine perception can execute
efficiently on resource-constrained devices
26
27. Thank you.
0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
An Efficient Bit Vector Approach to Semantics-based
Machine Perception in Resource-constrained Devices
0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
27
Notes de l'éditeur
This is one example of an application being build from the research I will be discussingTitled: An efficient bit vector approach to semantics-based machine perception in resource-constrained devices
- Larry Smarr is a professor at the University of California, San DiegoAnd he was diagnosed with Chrones DiseaseWhat’s interesting about this case is that Larry diagnosed himselfHe is a pioneer in the area of Quantified-Self, which uses sensors to monitor physiological symptomsThrough this process he discovered inflamation, which led him to discovery of Chrones DiseaseThis type of self-tracking is becoming more and more common
- what if we could automate this sense making ability?- and what if we could do this at scale?
sense making based on human cognitive models
perception cycle contains two primary phasesexplanationtranslating low-level signals into high-level abstractions inference to the best explanationdiscriminationfocusing attention on those properties that will help distinguish between multiple possible explanationsused to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)
perception cycle contains two primary phasesexplanationtranslating low-level signals into high-level abstractions inference to the best explanationdiscriminationfocusing attention on those properties that will help distinguish between multiple possible explanationsused to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)
perception cycle contains two primary phasesexplanationtranslating low-level signals into high-level abstractions inference to the best explanationdiscriminationfocusing attention on those properties that will help distinguish between multiple possible explanationsused to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)
- With this ability,many problems could be solved- For example: we could help solve health problems (before they become serious health problems) through monitoring symptoms and real-time sense making, acting as an early warning system to detect problematic health conditions
Intelligence distributed at the edge of the networkRequires resource-constrained devices (mobile phones, gateway notes, etc.) to be able to utilize SW technologies
Intelligence distributed at the edge of the networkRequires resource-constrained devices (mobile phones, gateway notes, etc.) to be able to utilize SW technologies
- compute machine perception inferences -- i.e., explanation and discrimination -- of high-complexity on a resource-constrained devices in miliseconds