Robust solutions for ambient assisted living are numerous, yet predominantly specific in their scope of usability. In this paper, we de- scribe the potential contribution of semantic web technologies to building more versatile solutions — a step towards adaptable context-aware en- gines and simplified deployments. Our conception and deployment work in hindsight, we highlight some implementation challenges and require- ments for semantic web tools that would help to ease the development of context-aware services and thus generalize real-life deployment of se- mantically driven assistive technologies. We also compare available tools with regard to these requirements and validate our choices by providing some results from a real-life deployment.

1. ISWC 2012
Semantic Reasoning in Context-Aware Assistive Environments
to Support Ageing with Dementia
Thibaut Tiberghien, Mounir Mokhtari, Hamdi Aloulou, Jit Biswas
Boston, MA, 15 Nov. 2012
www.ipal.cnrs.fr

2. OUR COMMUNITY
‣ Ambient Assisted Living (AAL)
‣ Deployment in Peacehaven nursing home (Singapore)
iPhone/Android
for nurse (3G)
IPTV
(Wi-Fi)
Shower
Bed
Cupboard Nursing console
(Wi-Fi)
Shake sensor
Passive infrared Speaker (Bluetooth) RFID bracelet on resident
Pressure sensor Tiny debian machine
ZigBee gateway
RFID reader Wi-Fi router
over ZigBee
2

3. OUR COMMUNITY
CHALLENGE
‣ Limited scope of usability of existing solutions
‣ Generic service framework to drive adoption
- interoperable services
- reduce cost by sharing hardware & software resources
- independent modules on top of Linked Data
- atomic end-user services
3

4. OUTLINE
‣ Semantic technologies in the AAL use-case
‣ First steps
‣ Finding the appropriate reasoner
- requirements for AAL
- pragmatic comparison
‣ UbiSmart service platform, an integrated solution
‣ Deployment results
‣ Future work
4

5. SEMTECH CONTRIBUTION FOR AAL
‣ Yet another Internet of Things + Semantic Web mix...
‣ Some peculiarities
1. Modeling assistance in smart space
2. Entities integration: discovery & plug’n’play
3. Modules collaboration: shared model, linked data
4. Intelligence: semantic inference
5

6. SEMTECH CONTRIBUTION FOR AAL
1. MODELING ASSISTANCE IN SMART SPACE
[KB skeleton]
runningFor
usedBy
Service
Person helpsWith name
repeat onDevice
name id
snoozeTime hasContext timeSent
Context stage Device
name
needHands name
hasAckService handheld
Caregiver Resident hasSolvingContext
busy stageForAlert Reminder
Notification
deployedIn
watchesAfter Activity ackHandled
Deviance escalateTo acknowledgement
Location solved
Legend
Class
data property
object property
subClassOf
6

7. SEMTECH CONTRIBUTION FOR AAL
1. MODELING ASSISTANCE IN SMART SPACE
[KB skeleton]
runningFor
usedBy
Service
Person helpsWith name
repeat onDevice
name id
snoozeTime hasContext timeSent
Context stage Device
name
needHands name
hasAckService handheld
Caregiver Resident hasSolvingContext
busy stageForAlert Reminder
Notification
deployedIn
watchesAfter Activity ackHandled
Deviance escalateTo acknowledgement
Location solved
Legend
Class
data property
8 Service s, Resident r, Location l, Device dc, Activity a, Deviance da
object property (r hasContext da) ^ (s helpsWith da) ) (s runningFor r)
(s runningFor r) ^ (r locatedIn l) ^ (dc deployedIn l) ) (s onDevice dc)
subClassOf (r hasContext a) ^ (a needHands true) ^ (dc handheld true) ) (dc fitted false)
6

8. SEMTECH CONTRIBUTION FOR AAL
‣ Yet another Internet of Things + Semantic Web mix...
‣ Some peculiarities
1. Modeling assistance in smart space
2. Entities integration: discovery & plug’n’play
3. Modules collaboration: shared model, linked data
4. Intelligence: semantic inference
7

9. SEMTECH CONTRIBUTION FOR AAL
2. ENTITIES INTEGRATION: DISCOVERY & PLUG’N’PLAY
End-user SOA software representation
Services of plug&play agents
Reasoning Engine
(Context Awareness)
OSGi container
plug & play
Sensors Devices Actuators
8

10. SEMTECH CONTRIBUTION FOR AAL
‣ Yet another Internet of Things + Semantic Web mix...
‣ Some peculiarities
1. Modeling assistance in smart space
2. Entities integration: discovery & plug’n’play
3. Modules collaboration: shared model, linked data
4. Intelligence: semantic inference
9

11. SEMTECH CONTRIBUTION FOR AAL
3. MODULES COLLABORATION: SHARED MODEL, LINKED DATA
Context Producer
Sensor
Knowledge Base [KB] Service
Module
based on Euler Delivery
UI Parser
User Interface
Plasticity
Context Module
Acquisition Stream
Module
Interaction
Handler
Context Synthetiser
Context
Understanding Context
ss
Module Awarene Service
Selection
Module
Service
Service
Service
Service
ment Service
Manage
Context Consumer
10

12. SEMTECH CONTRIBUTION FOR AAL
3. MODULES COLLABORATION: SHARED MODEL, LINKED DATA
Context Producer
Sensor
Knowledge Base [KB] Service
Module
based on Euler Delivery
UI Parser
User Interface
Plasticity
Context Module
Acquisition Stream
Module
Interaction
Handler
Context Synthetiser
Context
Understanding Context
ss
Module Awarene Service
Selection
Module
Service
Service
Service
Service
ment Service
Manage
Context Consumer
10

13. SEMTECH CONTRIBUTION FOR AAL
3. MODULES COLLABORATION: SHARED MODEL, LINKED DATA
Context Producer
Sensor
Knowledge Base [KB] Service
Module
based on Euler Delivery
UI Parser
User Interface
Plasticity
Context Module
Acquisition Stream
Module
Interaction
Handler
Context Synthetiser
Context
Understanding Context
ss
Module Awarene Service
Selection
Module
Service
Service
Service
Service
ment Service
Manage
Context Consumer
10

14. SEMTECH CONTRIBUTION FOR AAL
3. MODULES COLLABORATION: SHARED MODEL, LINKED DATA
Context Producer
Sensor
Knowledge Base [KB] Service
Module
based on Euler Delivery
UI Parser
User Interface
Plasticity
Context Module
Acquisition Stream
Module
Interaction
Handler
Context Synthetiser
Context
Understanding Context
ss
Module Awarene Service
Selection
Module
Service
Service
Service
Service
ment Service
Manage
Context Consumer
10

15. SEMTECH CONTRIBUTION FOR AAL
‣ Yet another Internet of Things + Semantic Web mix...
‣ Some peculiarities
1. Modeling assistance in smart space
2. Entities integration: discovery & plug’n’play
3. Modules collaboration: shared model, linked data
4. Intelligence: semantic inference
11

16. SEMTECH CONTRIBUTION FOR AAL
4. INTELLIGENCE: SEMANTIC INFERENCE
‣ Imperative approach
- Proof of concept in 2010
- stable but not scalable
“decision support rules
- no iteration possible are not easily accessible to knowledge engineers
for maintenance; new rules require programming resources to
‣ Declarative approach implement rules and custom data fetches;
deployments are not reusable across
multiple systems.” 1
1 Goldberg, H. S., Vashevko, M., Postilnik, A., Smith, K., Plaks, N., & Blumenfeld, B. M. (2006). Evaluation of a commercial rule engine as
a basis for a clinical decision support service. In AMIA Annual Symposium Proceedings (Vol. 2006, p. 294). American Medical
Informatics Association.
12

17. SEMTECH CONTRIBUTION FOR AAL
4. INTELLIGENCE: SEMANTIC INFERENCE
‣ Imperative approach
‣ Declarative approach
- separate knowledge & logic
- learning curve & design phase
- scalable, iteration possible
- easier to maintain or reuse
- better fitted for scenario-based deployments
13

18. FIRST STEPS
‣ OSGi - Java - Jena framework
- fully featured APIs
- poor integrated inference engine
- what alternatives?
‣ Marko Luther et al.
“choosing the appropriate combination of a reasoning engine, a
communication interface and expressivity of the utilized ontology is
an underestimated complex and time consuming task.” 2
2Luther, M., Liebig, T., Böhm, S., Noppens, O.: Who the Heck Is the Father of Bob? In: Aroyo, L., Traverso, P., Ciravegna, F., Cimiano, P.,
Heath, T., Hyvönen, E., Mizoguchi, R., Oren, E., Sabou, M., Simperl, E. (eds.) ESWC 2009. LNCS, vol. 5554, pp. 66–80. Springer,
Heidelberg (2009)
14

19. THE APPROPRIATE REASONER?
REQUIREMENTS GATHERING
‣ Retractability of knowledge
- contextual information is dynamic & temporal
- ease to retract information: asserted & inferred
15

20. THE APPROPRIATE REASONER?
REQUIREMENTS GATHERING
‣ Retractability of knowledge
- contextual information is dynamic & temporal
- ease to retract information: asserted & inferred
- opposition to RDF monotonicity assumption
- Solutions:
- higher complexity in reasoner’s language
- no live state reasoner
15

21. THE APPROPRIATE REASONER?
REQUIREMENTS GATHERING
‣ Processing efficiency
- pseudo-real-time constraints
- highly dynamic data
16

22. THE APPROPRIATE REASONER?
REQUIREMENTS GATHERING
‣ Processing efficiency
- pseudo-real-time constraints
- highly dynamic data
‣ Scalability
- size of monitored space? room, building, smart city...
- keep linked data
- beware of the reasoner’s complexity
16

23. THE APPROPRIATE REASONER?
REQUIREMENTS GATHERING
‣ Processing efficiency
- pseudo-real-time constraints
- highly dynamic data
‣ Scalability
- size of monitored space? room, building, smart city...
- keep linked data
- beware of the reasoner’s complexity
‣ Quality of Information
16

24. THE APPROPRIATE REASONER?
A PRAGMATIC COMPARISON
Jena Pellet RacerPro Euler
Consistency incomplete ✓ ✓ ✓
Rule format own SWRL own, powerful N3Logic
Retract info ✓ ✕ ✓ ✕
Ease of use average (manual) easy complex easy
100 783ms 442ms ~ 310ms 4ms
Speed
1000 29,330ms 38,836ms ~ 44,166ms 40ms
10 000 out of memory 436ms
Scalability ✕ ~ ✕ ✓
Size (download) 22.3Mb 24.3Mb 60.3Mb 12.9Mb
Licensing free, open-source free, open-source $, closed-source free, open-source
17

25. THE APPROPRIATE REASONER?
A PRAGMATIC COMPARISON
Jena Pellet RacerPro Euler
Consistency incomplete ✓ ✓ ✓
Rule format own SWRL own, powerful N3Logic
Retract info ✓ ✕ ✓ ✕
Ease of use average (manual) easy complex easy
100 783ms 442ms ~ 310ms 4ms
Speed
1000 29,330ms 38,836ms ~ 44,166ms 40ms
10 000 out of memory 436ms
Scalability ✕ ~ ✕ ✓
Size (download) 22.3Mb 24.3Mb 60.3Mb 12.9Mb
Licensing free, open-source free, open-source $, closed-source free, open-source
17

26. THE APPROPRIATE REASONER?
A PRAGMATIC COMPARISON
Jena Pellet RacerPro Euler
Consistency incomplete ✓ ✓ ✓
Rule format own SWRL own, powerful N3Logic
Retract info ✓ ✕ ✓ ✕
Ease of use average (manual) easy complex easy
100 783ms 442ms ~ 310ms 4ms
Speed
1000 29,330ms 38,836ms ~ 44,166ms 40ms
10 000 out of memory 436ms
Scalability ✕ ~ ✕ ✓
Size (download) 22.3Mb 24.3Mb 60.3Mb 12.9Mb
Licensing free, open-source free, open-source $, closed-source free, open-source
17

27. THE APPROPRIATE REASONER?
A PRAGMATIC COMPARISON
Jena Pellet RacerPro Euler
Consistency incomplete ✓ ✓ ✓
Rule format own SWRL own, powerful N3Logic
Retract info ✓ ✕ ✓ ✕
Ease of use average (manual) easy complex easy
100 783ms 442ms ~ 310ms 4ms
Speed
1000 29,330ms 38,836ms ~ 44,166ms 40ms
10 000 out of memory 436ms
Scalability ✕ ~ ✕ ✓
Size (download) 22.3Mb 24.3Mb 60.3Mb 12.9Mb
Licensing free, open-source free, open-source $, closed-source free, open-source
17

28. THE APPROPRIATE REASONER?
A PRAGMATIC COMPARISON
Jena Pellet RacerPro Euler
Consistency incomplete ✓ ✓ ✓
Rule format own SWRL own, powerful N3Logic
Retract info ✓ ✓
✕ ✓ ✓
✕
Ease of use average (manual) easy complex easy
100 783ms 442ms ~ 310ms 4ms
Speed
1000 29,330ms 38,836ms ~ 44,166ms 40ms
10 000 out of memory 436ms
Scalability ✕ ~ ✕ ✓
Size (download) 22.3Mb 24.3Mb 60.3Mb 12.9Mb
Licensing free, open-source free, open-source $, closed-source free, open-source
17

29. THE APPROPRIATE REASONER?
A PRAGMATIC COMPARISON
Jena Pellet RacerPro Euler
Consistency incomplete ✓ ✓ ✓
Rule format own SWRL own, powerful N3Logic
Retract info ✓ ✓
✕ ✓ ✓
✕
Ease of use average (manual) easy complex easy
100 783ms 442ms ~ 310ms 4ms
Speed
1000 29,330ms 38,836ms ~ 44,166ms 40ms
10 000 out of memory 436ms
Scalability ✕ ~ ✕ ✓
Size (download) 22.3Mb 24.3Mb 60.3Mb 12.9Mb
Licensing free, open-source free, open-source $, closed-source free, open-source
17

30. THE APPROPRIATE REASONER?
A PRAGMATIC COMPARISON
Jena Pellet RacerPro Euler
Consistency incomplete ✓ ✓ ✓
Rule format own SWRL own, powerful N3Logic
Retract info ✓ ✓
✕ ✓ ✓
✕
Ease of use average (manual) easy complex easy
100 783ms 442ms ~ 310ms 4ms
Speed
1000 29,330ms 38,836ms ~ 44,166ms 40ms
10 000 out of memory 436ms
Scalability ✕ ~ ✕ ✓
Size (download) 22.3Mb 24.3Mb 60.3Mb 12.9Mb
Licensing free, open-source free, open-source $, closed-source free, open-source
17

31. UBISMART SERVICE PLATFORM
INTEGRATION OF A REASONER IN OUR ARCHITECTURE
Sensors Devices
ZigBee Wi-Fi/3G/BT
HomeControlService
Euler (API)
DeviceManager
ServiceControl
ReminderService
WSMS
Apache Felix container
18

32. UBISMART SERVICE PLATFORM
INTEGRATION OF A REASONER IN OUR ARCHITECTURE
bundle auto-generation
Sensors Devices
ZigBee Wi-Fi/3G/BT
HomeControlService
Euler (API)
DeviceManager
ServiceControl
ReminderService
WSMS
ors
ns
Se
Apache Felix container
18

33. UBISMART SERVICE PLATFORM
INTEGRATION OF A REASONER IN OUR ARCHITECTURE
bundle auto-generation
Sensors Devices
ZigBee DPWS Wi-Fi/3G/BT
EnvironmentDiscovery
HomeControlService
Euler (API)
DeviceManager
ServiceControl
ReminderService
WSMS
ors
ns
Se
Apache Felix container
18

34. UBISMART SERVICE PLATFORM
INTEGRATION OF A REASONER IN OUR ARCHITECTURE
bundle auto-generation
Sensors Devices
ZigBee DPWS Wi-Fi/3G/BT
EnvironmentDiscovery
HomeControlService
Euler (API)
DeviceManager
ServiceControl
ReminderService
WSMS
ors es
ns vic
Se De
Apache Felix container
18

35. UBISMART SERVICE PLATFORM
INTEGRATION OF A REASONER IN OUR ARCHITECTURE
bundle auto-generation
Sensors Devices
ZigBee DPWS Wi-Fi/3G/BT
EnvironmentDiscovery
HomeControlService
Euler (API)
DeviceManager
ServiceControl
ReminderService
WSMS
rules.n3 & query.n3 ors es
ns vic
environment.n3 Se De
skeleton.n3
input.n3
Apache Felix container
18

36. UBISMART SERVICE PLATFORM
INTEGRATION OF A REASONER IN OUR ARCHITECTURE
bundle auto-generation
1. profiles of users, devices, sensors
2. context information, devices status
3. selected service and device
4. start/stop service on a device
5. services status Sensors Devices
6. service instantiation on device
ZigBee DPWS Wi-Fi/3G/BT
EnvironmentDiscovery
HomeControlService
Euler (API)
DeviceManager
ServiceControl
ReminderService
WSMS
rules.n3 & query.n3 ors es
ns vic
environment.n3 Se De
3 skeleton.n3 1
input.n3
5 2
6 4
Apache Felix container
18

37. DEPLOYMENT RESULTS
ACCURACY & TIMELINESS
‣ Accuracy of event recognition
Atomic events 34 times/day 71%
Complex events 7 times/day 70%
‣ Timeliness
Euler module Networking Misc. modules Total
1.226s 0.735s 0.752s 2.713s
19

38. DEPLOYMENT RESULTS
ADAPTABILITY
‣ Time to adapt the reasoning to a new deployment specificities
imperative platform semantic platform
first implementation ~5 days a few months
new deployment ~3 days a few hours
20

39. FUTURE WORK
‣ Smart space composer
- on site configuration by linked data generation
‣ More powerful matching of context with services
- needs parametric context description
‣ Handling of quality of information
- reasoning with uncertainty of contexts
‣ Cohabitation of rule-base inference and AI algorithms
- context classification from linked data
21

40. DISCUSSION
Thibaut Tiberghien Image & Pervasive Access Lab
Doctoral Student International joint research unit - UMI CNRS 2955
thibaut.tiberghien@ipal.cnrs.fr www.ipal.cnrs.fr