This is the presentation supporting the invited keynote I gave at the IEEE ComSoc 5th Global Information Infrastructure and Networking Symposium GIIS 2013
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
20131031 giis 2013 keynote r.giaffreda
1. Cognitive Management of Objects and
Applications for the Internet of Things
Raffaele Giaffreda (CREATE-NET)
Keynote at GIIS conference
Trento, 31 Oct 2013
2. Outline
• Introduction, IoT vs. the Internet
• object virtualisation – separation between object data and
object mgmt concerns
• overview of IoT standardisation activities
• interoperability and objects as services – IoT reliability and
resilience
• a top-down perspective on the IoT – user friendliness, wide
adoption
• Real World Knowledge modelling and use of cognitive
technologies in IoT
• examples of ongoing trials
• conclusions
3. transistor density / space efficiency
Turing’s Pilot ACE: Automatic
Computing Engine
12. the Internet parallel
• imagine the Internet with no browser, no
plugins
• collection of bespoke, non interoperable
content specific applications enabling access
and visualisation of connected files
13. The Internet parallel
HTTP/WWW
search engines
HTML
represent info / aggregate info
connect your info
TCP/IP
WWW
find info
personalised knowledge
collections, blogs...
VALUE!
The Semantic Web
14. The Internet parallel
early stages for the IoT...
HTTP/WWW
search engines
HTML
object
connect your info
represent info / aggregate info
find info
WWW
personalised knowledge
collections, blogs...
TCP/IP
VALUE!
today
The Semantic Web
15. Internet vs. Internet of Things
• files vs. objects
• static memory cells vs. energy standalone
units
• need to separate data source from data mgmt
and operations
• objects virtualisation
17. the VO concept
Exposed APIs
•
VO exposes several APIs to the upper
layers
VO SW agent host
– Features, functionalities and resources
can be re-used
•
•
VO APIs
Cognitive control enabled by exposing
APIs which can be used to optimize the
behaviour of the ICT object
VO SW agent may or may not be
installed on the ICT object
VO SW agent
– Depends on ICT object capabilities
•
•
Association management between ICT
and non-ICT is a real challenge!
RESILIENCE ASPECTS
(remote) proprietary
API calls
ICT object
ICT APIs
ICT
object processes
Association
17
A simple example:
VO SW agent host = laptop with Zigbee dongle
ICT object = Zigbee temperature meter
non-ICT object = room
non-ICT object
18. what do VOs achieve: logical level
Application: pure function
VO Front
End
VO Front
End
VO Back
End:
Net Driver
VO Back
End:
Net Driver
iCore FW
VO Front
End
VO Back
End:
Net Driver
VO Back
End:
RWO Driver
Gateway
RWO1
18
19. fostering automation - discovery
• description associated with an IoT Object, it better be
machine readable
• i.e. semantic enrichment based on info model for semanticbased selection
• what is this good for?
– selection “by relevance”: performance and “selection quality” is
dependent upon combination of enrichment + algorithm that exploits
it...
– assessment of “proximity” is a prerequisite in achieving more
automatic and scalable solutions
21. Examples (energy efficiency for
sensors)
• besides discovering and selecting
• virtual representative “takes the heat off” real
sensors becoming their actual “manager”
–
–
–
–
energy efficiency
reuse
resilience
self-x for constrained resource devices
• conflict resolution (actuators)
• Examples
– compression algorithms, data caching, pub/sub
schemes, rules for self-x management
22. added value besides sensing efficiency
HUMAN
MACHINE
cars increasingly more
complex
OBD
increasing competition On Board Diagnostics
for owner’s attention
what happens when it becomes
easier and easier to tap into
object produced data?
23. added value besides sensing
efficiency – Innovation potential
we make “machines” step-in, assisting us!
HUMAN
MACHINE
“Innovation”: one
can focus on apps!!!
MACHINE
HUMAN
OBD
On Board Diagnostics
24. the story so far...
• increasing number of objects
• discovery and self-management of objects
• connect and virtualise your objects, unlock
value
• no mention of application domains...
25. DATA / INFORMATION OVERLOAD, BUT...
siloed and bespoke IoT applications
SENSORS
SENSORS
SENSORS
SENSORS
APPS
APPS
APPS
APPS
APPS
APPS
APPS
PATIENT
PATIENT
PATIENT
PATIENT
PATIENT
PATIENT
TRUCK
PATIENT
APPS
FRIDGE
APPS
HOUSE
APPS
CAR
APPS
SENSORS
SENSORS
SENSORS
SENSORS
SENSORS
SENSORS
SENSORS
26. IF A WELL-DEFINED INTERFACE INTO CAR
SENSORS BRINGS SUCH POTENTIAL...
SENSORS
SENSORS
SENSORS
SENSORS
APPS
APPS
APPS
APPS
APPS
APPS
APPS
PATIENT
PATIENT
PATIENT
PATIENT
PATIENT
PATIENT
TRUCK
PATIENT
APPS
FRIDGE
APPS
HOUSE
APPS
CAR
APPS
SENSORS
SENSORS
SENSORS
SENSORS
SENSORS
SENSORS
SENSORS
27. of course that’s a dream far from
becoming true...
http://readwrite.com/2013/06/14/whats-holding-up-the-internet-of-things
28. the IoT standardisation jungle
M2M
Real-World Knowledge Model (RDF Concepts & Facts)
Service Templates
Repository
SES
API
User Characterisation
Situation Projection
Service Request Analysis
Learning
Mechanisms
Situation Recognition
ITU-T
FG Distraction
Situation Detection
ISO/IEC
JTC1 WG7
M2M
RDF Rules
Inference
Engine
Intent Recognition
M2M
API
EPCGlobal
IoT-GSI
Authentication
W3C PROV
PROV-DM / PROV-O /
PROV-AQ / PROV-LINK
M2M
ITU-T
CVO Registry
Access
Control
CLOUD
W3C PROV
PROV-DM /PROV-O /
PROV-AQ / PROV-XML
Orchestration /
Workflow
Management
Approximation &
Reuse
Opportunity
Detection
Authentication
M2M
LWM2M
Access
Control
VO Registry
3GPP
SPS
SES
W3C PROV
PROV-DM /PROV-O /
PROV-AQ / PROV-XML
SAS
ITU-T
IoT-GSI
M2M
CVO Execution Request
SOS
LWM2M
VO Factory
WNS
3GPP
ISO/IEC
JTC1 WG7 VO VO
VO VO VO
SensorML
LWM2M
CoAP
ISO/IEC
JTC1 WG7
VO Templates Repository
Device
manufactu
rer
Actuator
VO Management Unit
SPS
3GPP
MQTT
SPS
VO Lifecycle
Manager
Resource
Optimisation
M2M
LWM2M
VO
ISO/IEC Back End: RWO Driver ITU-T
JTC1 WG7
FG M2M
EPCGlobal
3GPP
GTW/Controller
…………..
Sensor
LWM2M
VO Front End
VO VO VO
VO VO
ITU-T
IoT-GSI
MQTT
GTW/Controller
Resource
VO Container (WS host)
Resource
Actuator
Quality
Assurance
ITU-T
FG Distraction
SAS
ITU-T
FG M2M
Coordination
Performance
Management
SOS
CVO
CVO
CVO
CVO
CVO
AQ / PROV-CONSTRAINT
CVO Lifecycle
Manager
CLOUD
CVO Container (Execution)
SOS
LWM2M
CVO Management Unit
CVO
CVO
Situation Observer
CVO
W3C PROV
Situation Observer
CVO
PROV-DM / PROV-O / PROV- Situation Observer
Situation Observer
…..
API
SSN-XG
Installer/
User
O&M
Installs
Sensor/Ac
EPCGlobal
tuator
Devices
M2M
ITU-T
IoT-GSI
SPS
SOS
Learning
Mechanisms
System Knowledge Model
SIR
CoAP
CSW
CLOUD
Service Execution Request
Learning
Statistics
Real-World Information DB
W3C PROV
PROV-DM /PROV-O /
PROV-AQ / PROV-XML
CVO Factory
CVO
LWM2M Composition
Engine
CVO Templates Repository
SOR
Semantic
Query
Matcher
Queried Fact Collector
W3C PROV
PROV-DM /PROV-O /
PROV-AQ / PROV-XML
Data
Processing
Domain
Expert /
Developer
Authentication
Service Analysis
W3C PROV
PROV-DM / PROV-O / PROVAQ / PROV-CONSTRAINT
Sensor
courtesy of Panagiotis Vlacheas and Vera Stavroulaki (Piraeus University )
Coordination
Data Manipulation
/ Reconciliation
ITU-T
IoT-GSI
Authentication
Situation Classification
Administration & Management I/F
Domain
Expert /
Knowledge
Engineer
Authentication
Situation Awareness
ITU-T
FG M2M
Service Request
(SPARQL)
Natural Language Processing
(NLP)
…..
API
P1723
GUI
Service Requester (Technology Agnostic)
Authentication
29. some (good) candidates
• imagine the Internet with no browser, no plugins
• collection of bespoke, non interoperable content
specific applications enabling access and
visualisation of connected files
an IP based web services view from Sensinode
Courtesy of Zach Shelby (Sensinode)
http://www.iot-week.eu/presentations/thursday/02_Shelby-IoT-Smart-Cities.pdf
30. fostering interoperability
• at service level (ESBs)
• at communication level (PUB/SUB MQTT bus)
• at device level (GSN)
• no silver bullet...a lot of it will depend on
application context...
31. useful ingredients?
• common interfaces to interact with
objects (i.e. REST)
• + extra containers for metadata
• let the systems know what the object
is good for, its location (“I am a Temp
sensor in Room A”), its accuracy, its
energy levels etc.
“I am a webpage and I talk about Paris (city of France) history”
take inspiration from HTML and the Semantic Web
Integration at “application level” with all pros and cons associated with it
32. the story so far...
• increasing number of objects
• discovery and self-management of objects
• connect and virtualise your objects, unlock
value
• interoperability across application domains
and reliability still big issues...
33. once achieved the means to access
an objects as a service...
• object redundancy would allow me to cope with resource
constraint nature of objects as well as with the diversity of
interfaces
– if I had a bunch of VO temp objects to chose from I would be much
more likely to tell you what the temperature is...
• semantic enrichment allows me to find alternatives, to foster
object reuse and achieve service approximation concepts
• here we start entering more the “cognitive-inside” IoT object
management territory
• having a logic for choosing the appropriate Virtual Objects
according to the application expectations
• having the means to easily connect objects together in a more or
less complex graph (CEPs, PUB/SUB channels)
• features of Composite Virtual Objects and associated “CVO
Templates”
cognitive mash-ups of semantically interoperable VOs (and their offered
services) which render services matching the application requirements
35. CVO concept allows for approximate services...
PATIENT
APPS
FRIDGE
APPS
HOUSE
APPS
CAR
APPS
SENSORS
SENSORS
SENSORS
SENSORS
PATIENT is driving the CAR
CAR is near the HOUSE
PATIENT is near the FRIDGE
objects reuse
across domains
KitchenPresDetect
PatientStatusDetect
36. CVOs allow Automatic Composition
CVOType 1
CVO 1
FIND
VOType :: Temp sensor
getTemp()
Subject to constraints:
- Dist (Pos, myPos) < 10m
- Not already allocated
VOType :: Press sensor
getPressure()
VOx
CP Solver to find VO
allocations that satisfies
all constraints and
minimizes network traffic
USE
Logic:
If getTemp() > 20° and getPressure() > 2bar
then NiceWeather
leveraging on System Knowledge (i.e. VOx is good and fully
charged) to maintain IoT-based services...
VOy
37. CVO templates
• factoring “smart logic algorithms” out of users /
developers concerns
– IF “crash” THEN “alertRSA”
– “crash” (IF VO_x = TRUE THEN crash := TRUE)
– (IF VO_x = TRUE AND VO_y = TRUE THEN crash := TRUE)
• “ready meals” for IoT apps
VO_x
TAG:
crash
detect
VO_y
TAG:
crash
detect
factor out cognitive technologies
IF VO_x = TRUE
THEN crash := TRUE
IF (VO_x = TRUE) AND (VO_y = TRUE)
THEN crash := TRUE
IF (VO_x > TH_x) AND (VO_y > TH_y)
THEN crash := TRUE
38. workflow-based SEP for CVOs
Car’s sensors/actuators
courtesy of Michele Stecca (M3S)
more info: http://www.slideshare.net/steccami/ieee-icin-2011
Open Data (Web)
39. Event based CVO execution
CVO Container
Observer
Observer
CVO
CVO
CVO
CVO
Machine Learning
extensions
CEP engine
Event / (C)VO Bus
(pub/sub based on MQTT)
VO Container
Sensor
VO
Sensor
VO
Sensor
VO
Actuator
VO
Actuator
VO
courtesy of Walter Waterfeld (Software AG)
more info: http://terracotta.org/downloads/universal-messaging
40. Internet vs. IoT
• a page + a page + a page...connect info
• represent info – HTML
• aggregate info – hyperlink
• a (sensor) feed + a feed + a feed...
• represent feeds – VO
• aggregate feeds – CVO
41. the story so far...bottom-up
what’s in here?
user friendliness and
wide adoption...
42. the story so far...
•
•
•
•
increasing number of objects
discovery and self-management of objects
connect and virtualise your objects, unlock value
exploit redundancy pick the most suitable /
interoperable / reliable objects
• VO / CVO services like Lego bricks fostering innovation
from IoT makers
• cognitive inside? so far only application-driven matchmaking
• ultimate goal: user-friendly IoT services fostering wide
adoption
43. a ‘top-down’ view
• routine jobs: water the plants, feed the fish,
take my pills, track sent items etc.
• there are objects, sensors, actuators
• there are people (busy lives, forgetful
patients, green fingers vs. fingers that “kill
every plant they look after”)
• objects can be connected
• objects can be mashed-up
• create your own IoT apps (this is what IoT
makers do) vs. provide some input and have
this interpreted so the right actions are set
to achieve your goals
• make the IoT easy to use and rely upon...
44. unlocking a huge potential
patterns exist ...
CVOs
data
data
data
H/W
data
VOs
data
data
data data
data
data
data data
data data
data data
SENSING
Real World Objects
(RWO)
data goldmine
and lots of
siloed
applications
interpret
data
presence
derive patterns of ...
presence
46. the need for cognitive technologies
• rather than for the selection of appropriate templates,
here focus is on refinement of selected one according
to observed system-reality matching
• Real-World-Knowledge “growing”
• Learning and adaptation to the users preferences
VO_x
TAG:
crash
detect
VO_y
TAG:
crash
detect
REFINE
TH_x, and TH_y
IF (VO_x > TH_x) AND (VO_y > TH_y)
THEN crash := TRUE
assess
QUALITY of
PREDICTION
47. Real World and System Knowledge
models
interpret
data
Real World Knowledge
(RWK)
Models
derive patterns of ...
presence
What are these
good for?
(SK)
Models
System Knowledge
48. Cognitive Inside where and why...
• Service Level: gather data relate to actions /
situations
• support users (OBSERVE – LEARN – REPLACE)
– routine jobs (watering plants, feeding the fish, taking
pills, switch on/off lights)
– non-expert alerts (a fire, a leak, a fault)
• provide feedback
– improvement of system performance
49. Some examples please?
• tracking cars in a smart city
• medical equipment tracking and asset
management
50. tracking cars in a smart city
Best demo
award at
FuNeMS 2013
courtesy of Marc Roelands (Bell Labs – Alcatel Lucent)
more info: http://www.iot-icore.eu/attachments/article/66/iCore_FuNeMS%2713_ALU.pdf
51. tracking medical equipment
5
Execute
3
Validate
Database of location
information(spatial &
temporal) of objects
2a
In the demo implementation,
location data of objects is
simulated
RWO parameter reconfiguration
recommendations to improve energy
efficiency of location sensors
4
Train
4a
6
2
1
7
53. IoT, Cloud and Big Data
the challenges ahead...
• Big data: “big” relates to the huge number of data
sources
– have data, patterns exist
– Need to purposefully aggregate data
– scaling-up use of machine learning is a challenge...
• Cloud: constrained devices and limited scope for data
processing
– dynamic deployment of data-processing resources on the
data-source data-consumer path is a challenge...
• IoT Networking: delivery of “object-produced” data
– M2M traffic and dynamic deployment of connectivity
resources is a challenge...
54. Conclusions
•
•
•
•
•
•
•
•
•
increasing number of objects
discovery and self-management of objects
connect and virtualise your objects, unlock value
exploit redundancy pick the most suitable / interoperable /
reliable objects
VO / CVO services like Lego bricks fostering innovation from
IoT makers
cognitive inside: the importance of modelling the Real World
Cognitive IoT: user-friendly services fostering wide adoption
implementations exploiting iCore project results in real trial
settings
challenging times ahead!
55. Further info / links
[REF1] IERC April 2013 Newsletter – Foreword (see THIS LINK)
[REF2] P. Vlacheas, R. Giaffreda et al. "Enabling Smart Cities Through
a Cognitive Management Framework for the Internet of Things“,
IEEE Communications Magazine - Special Issue on Smart Cities (June
2013)
[REF3] iCore website (www.iot-icore.eu/latest-news)
Best Demo Award at FuNeMS 2013
56. Thank you!
Raffaele Giaffreda
Smart IoT (RIoT) Research Area Head
(CREATE-NET)
EU FP7 iCore Project Coordinator
raffaele.giaffreda@create-net.org
Websites:
www.create-net.org/research/research-areas/riot
www.iot-icore.eu
58. the iCore Architecture
iCore User
User Profiling
Real World Knowledge/Model
Natural Language
Processing
iCore User
Preferences
API
Domain Expert
/ Knowledge
Engineer
Service Request
Service Execution Request
RWK Update
CVO Template
API Design & Store
System Knowledge/Model
CVO
Registry
CVO Container (Execution)
CVO Templates
Repository
VO Execution Request
API
VO Template
Repository
CVO
CVO
CVO
CVO
VO Data Session
VO Management Unit
VO
Registry
Device
Install
er
Device
manufac
turer
CVO
CVO
Situation Observer
CVO Factory
API
Data
Processing
Domain
Expert /
Developer
CVO Management Unit
VO Factory
VO
VO
Container VO VO VO O
Real World Objects
(RWO)
VO Front End
VO Back End: RWO Driver
API
Service Request Analysis
Situation
Modelling
System Administration &
Management
People
modelling
System Administration &
Management
Service Templates
Repository
API
& Store
RWK
API Design
API
iCore
System
Operator
59. Cognitive Inside – take-away messages
more dependable IoT
(RWK)
Models
support users of future
Smart Cities applications
(routine + alerts)
(SK)
Models
more resilient IoT
IoT resilience and fault
tolerance
more reliable and
interoperable IoT
IoT reliability and durability
through VOs
60. Dublinked initiative
IBM Research Ireland
mash-up data across domains
build models and predict!
personalised journey tips
throughout the execution
61. iCore ID
ID Card
3 yrs EU FP7 Integrated Project
(started 1st Oct 2011)
20 Partners with strong industrial
representation
8.7mEur EU Funding
EU + China and Japan
Japan