“IoT 360″ Summer School, October 2015– Rome, Italy

1. 1
Semantic technologies for
the Internet of Things
Payam Barnaghi
Institute for Communication Systems (ICS)
University of Surrey
Guildford, United Kingdom
International “IoT 360 Summer School″
October 2015– Rome, Italy

2. Real world data
2image credits: Smarter Data - I.03_C by Gwen Vanhee

3. Data in the IoT
− Data is collected by sensory devices and also crowd sensing
sources.
− It is time and location dependent.
− It can be noisy and the quality can vary.
− It is often continuous - streaming data.
− There are other important issues such as:
− Device/network management
− Actuation and feedback (command and control)
− Service and entity descriptions

4. Device/Data interoperability
4
The slide adapted from the IoT talk given by Jan Holler of Ericsson at IoT Week 2015 in Lisbon.

5. Heterogeneity, multi-modality and volume are
among the key issues.
We need interoperable and machine-interpretable
solutions…
5

6. Semantics and Data
− Data with semantic annotations
− Provenance, quality of information
− Interpretable formats
− Links and interconnections
− Background knowledge, domain information
− Hypotheses, expert knowledge
− Adaptable and context-aware solutions
6

7. Interoperable and semantically described
data is the starting point to create an
efficient set of actions.
The goal is often to create actionable
information.

8. Wireless Sensor (and Actuator)
Networks
Sink
node Gateway
Core network
e.g. Internet
Gateway
End-user
Computer services
- The networks typically run Low Power Devices
- Consist of one or more sensors, could be different type of sensors (or actuators)
Operating
Systems?
Services?
Protocols?
Protocols?
In-node Data
Processing
Data
Aggregation/
Fusion
Inference/
Processing of
IoT data
Interoperable/
Machine-
interpretable
representations
Interoperable/
Machine-
interpretable
Representations?
“Web of Things”
Interoperable/
Machine-
interpretable
representations

9. What we are going to study
− The sensors (and in general “Things”) are increasingly being connected
with Web infrastructure.
− This can be supported by embedded devices that directly support IP and
web-based connection (e.g. 6LowPAN and CoAp) or devices that are
connected via gateway components.
− Broadening the IoT to the concept of “Web of Things”
− There are already standards such as Sensor Web Enablement (SWE) set
developed by the Open Geospatial Consortium (OGC) that are widely
being adopted in industry, government and academia.
− While such frameworks provide some interoperability, semantic
technologies are increasingly seen as key enabler for integration of IoT
data and broader Web information systems.
9

10. Observation and measurement data- annotation
10
Tags
Data formats
Location
Source: Cosm.com

11. Observation and measurement data
11
value
Unit of
measurement
Time
Longitude
Latitude
How to make the data representations more machine-readable
and machine-interpretable;
15, C, 08:15, 51.243057, -0.589444

12. Observation and measurement data
12
<value>
<unit>
<Time>
<Longitude>
<Latitude>
And this?
<value>15</value>
<unit>C</unit>
<time>08:15</time>
<longitude>51.243057</longitude>
<latitude>-0.58944</latitude>
15, C, 08:15, 51.243057, -0.589444

13. XML Representation
<?xml version="1.0“ encoding="ISO-8859-1"?>
<measurement>
<value type=“Decimal”>15</value>
<unit>C</unit>
<time>08:15</time>
<longitude>51.243057</longitude>
<latitude>-0.58944</latitude>
</measurement>
13

14. Well Formed XML Documents
− A "Well Formed" XML document has correct XML syntax.
− XML documents must have a root element
− XML elements must have a closing tag
− XML tags are case sensitive
− XML elements must be properly nested
− XML attribute values must be quoted
14Source: W3C Schools, http://www.w3schools.com/

15. XML Documents– revisiting the example
<?xml version="1.0"?>
<measurement>
<value>15</value>
<unit>C</unit>
<time>08:15</time>
<longitude>51.243057</longitude>
<latitude>-0.58944</latitude>
</measurement>
15
<?xml version="1.0"?>
<sensor_data>
<reading>15</reading>
<u>C</u>
<timestamp>08:15</timestamp>
<long>51.243057</long>
<lat>-0.58944</lat>
</sensor_data>

16. XML
− Meaning of XML-Documents is intuitively clear
− due to "semantic" Mark-Up
− tags are domain-terms
− But, computers do not have intuition
− tag-names do not provide semantics for machines.
− DTDs or XML Schema specify the structure of documents, not
the meaning of the document contents
− XML lacks a semantic model
− has only a "surface model”, i.e. tree
16Source: Semantic Web, John Davies, BT, 2003.

17. Semantic Web technologies
− XML provide a metadata format.
− It defines the elements but does not provide any modelling
primitive nor describes the meaningful relations between
different elements.
− Using semantic technologies to solve these issues.
17

18. A bit of history
− “The Semantic Web is an extension of the current web in
which information is given well-defined meaning, better
enabling computers and people to work in co-operation.“
(Tim Berners-Lee et al, 2001)
18
Image source: Miller 2004

19. Semantics & the IoT
−The Semantic Sensor (&Actuator) Web is an
extension of the current Web/Internet in which
information is given well-defined meaning, better
enabling objects, devices and people to work in co-
operation and to also enable autonomous
interactions between devices and/or objects.
19

20. Resource Description Framework (RDF)
− A W3C standard
− Relationships between documents
− Consisting of triples or sentences:
− <subject, property, object>
− <“Sensor”, hasType, “Temperature”>
− <“Node01”, hasLocation, “Room_BA_01” >
− RDFS extends RDF with standard “ontology vocabulary”:
− Class, Property
− Type, subClassOf
− domain, range
20

21. RDF for semantic annotation
− RDF provides metadata about resources
− Object -> Attribute-> Value triples or
− Object -> Property-> Subject
− It can be represented in XML
− The RDF triples form a graph
21

22. RDF Graph
22
xsd:decimal
Measurement
hasValue
hasTime
xsd:double
xsd:time
xsd:double
xsd:string
hasLongitude hasLatitude
hasUnit

23. RDF Graph- an instance
23
15
Measurement#0
001
hasValue
hasTime
-0.589444
08:15
51.243057
C
hasLongitude hasLatitude
hasUnit

24. RDF/XML (simplified)
<rdf:RDF>
<rdf:Description rdf:about=“Measurment#0001">
<hasValue>15</hasValue>
<hasUnit>C</hasUnit>
<hasTime>08:15</hasTime>
<hasLongitude>51.243057</hasLongitude>
<hasLatitude>-0.589444</hasLatitude>
</rdf:Description>
</rdf:RDF>
24

25. Let’s add a bit more structure
(complexity?)
25
xsd:decimal
Location
hasValue
hasTime
xsd:double
xsd:time
xsd:double
xsd:string
hasLongitude
hasLatitude
hasUnit
Measurement
hasLocation

26. An instance of our model
26
15
Location
#0126
hasValue
hasTime
51.243057
08:15
-0.589444
C
hasLongitude
hasLatitude
hasUnit
Measurement#0
001
hasLocation

27. RDF: Basic Ideas
−Resources
−Every resource has a URI (Universal Resource Identifier)
−A URI can be a URL (a web address) or a some other kind
of identifier;
−An identifier does not necessarily enable access to a
resources
−We can think of a resources as an object that we want to
describe it.
−Car
−Person
−Places, etc.
27

28. RDF: Basic Ideas
− Properties
− Properties are special kind of resources;
− Properties describe relations between resources.
− For example: “hasLocation”, “hasType”, “hasID”, “sratTime”,
“deviceID”,.
− Properties in RDF are also identified by URIs.
− This provides a global, unique naming scheme.
− For example:
− “hasLocation” can be defined as:
− URI: http://www.loanr.it/ontologies/DUL.owl#hasLocation
− SPARQL is a query language for the RDF data.
− SPARQL provide capabilities to query RDF graph patterns along with
their conjunctions and disjunctions.
28

29. Ontologies
−The term ontology is originated from philosophy. In
that context it is used as the name of a subfield of
philosophy, namely, the study of the nature of
existence.
−In the Semantic Web:
−An ontology is a formal specification of a domain; concepts
in a domain and relationships between the concepts (and
some logical restrictions).
29

30. Ontologies and Semantic Web
− In general, an ontology describes a set of concepts in a
domain.
− An ontology consists of a finite list of terms and the
relationships between the terms.
− The terms denote important concepts (classes of objects) of
the domain.
− For example, in a university setting, staff members, students,
courses, modules, lecture theatres, and schools are some
important concepts.
30

31. Web Ontology Language (OWL)
− RDF(S) is useful to describe the concepts and their
relationships, but does not solve all possible requirements
− Complex applications may want more possibilities:
− similarity and/or differences of terms (properties or classes)
− construct classes, not just name them
− can a program reason about some terms? e.g.:
− each «Sensor» resource «A» has at least one «hasLocation»
− each «Sensor» resource «A» has maximum one ID
− This lead to the development of Web Ontology Language or
OWL.
31

32. Ontology engineering
− An ontology: classes and properties (also referred to as
schema ontology)
− Knowledge base: a set of individual instances of classes and
their relationships
− Steps for developing an ontology:
− defining classes in the ontology and arranging the classes in a
taxonomic (subclass–superclass) hierarchy
− defining properties and describing allowed values and restriction for
these properties
− Adding instances and individuals

33. Basic rules for designing ontologies
− There is no one correct way to model a domain; there are
always possible alternatives.
− The best solution almost always depends on the application that you
have in mind and the required scope and details.
− Ontology development is an iterative process.
− The ontologies provide a sharable and extensible form to represent a
domain model.
− Concepts that you choose in an ontology should be close to
physical or logical objects and relationships in your domain of
interest (using meaningful nouns and verbs).

34. A simple methodology
1. Determine the domain and scope of the model that you want to design
your ontology.
2. Consider reusing existing concepts/ontologies; this will help to increase
the interoperability of your ontology.
3. Enumerate important terms in the ontology; this will determine what are
the key concepts that need to be defined in an ontology.
4. Define the classes and the class hierarchy; decide on the classes and the
parent/child relationships
5. Define the properties of classes; define the properties that relate the
classes;
6. Define features of the properties; if you are going to add restriction or
other OWL type restrictions/logical expressions.
7. Define/add instances
8. Document your ontology
34

35. Semantic technologies in the IoT
−Applying semantic technologies to IoT can support:
−Interoperability
−effective data access and integration
−resource discovery
−reasoning and processing of data
−knowledge extraction (for automated decision making and
management)
35

36. Sensor Markup Language (SensorML)
36
Source: http://www.mitre.org/
The Sensor Model Language
Encoding (SensorML) defines
models and XML encoding to
represent the geometric,
dynamic, and observational
characteristics of sensors and
sensor systems.

37. Semantic modelling
− Lightweight: experiences show that a lightweight ontology
model that well balances expressiveness and inference
complexity is more likely to be widely adopted and reused;
also large number of IoT resources and huge amount of data
need efficient processing
− Compatibility: an ontology needs to be consistent with those
well designed, existing ontologies to ensure compatibility
wherever possible.
− Modularity: modular approach to facilitate ontology evolution,
extension and integration with external ontologies.
37

38. Existing models- SSN Ontology
− W3C Semantic Sensor Network Incubator Group’s SSN
ontology (mainly for sensors and sensor networks, platforms
and systems).
http://www.w3.org/2005/Incubator/ssn/

39. SSN Ontology Modules
39

40. SSN Ontology
40
Ontology Link: http://www.w3.org/2005/Incubator/ssn/ssnx/ssn
M. Compton et al, "The SSN Ontology of the W3C Semantic Sensor Network Incubator Group", Journal of Web Semantics, 2012.

41. 41
W3C SSN Ontology
41
makes observations of
this type
Where it is
What it
measures
units
SSN-XG ontologies
SSN-XG annotations
SSN-XG Ontology Scope

42. What SSN does not model
− Sensor types and models
− Networks: communication, topology
− Representation of data and units of measurement
− Location, mobility or other dynamic behaviours
− Control and actuation
− ….
42

43. 43
IoT and Semantics: Challenges and
issues

44. Several ontologies and description models
44

45. 45
We have good models and description
frameworks;
The problem is that having good models and
developing ontologies is not enough.

46. 46
Semantic descriptions are intermediary
solutions, not the end product.
They should be transparent to the end-user and
probably to the data producer as well.

47. 47
A WoT/IoT Framework
WSN
WSN
WSN
WSN
WSN
Network-enabled
Devices
Semantically
annotate data
Gateway
CoAP
HTTP
CoAP
CoAP
HTTP
6LowPAN
Semantically
annotate data
http://mynet1/snodeA23/readTemp?
WSN
MQTT
MQTT
Gateway
And several other
protocols and solutions…

48. Publishing Semantic annotations
− We need a model (ontology) – this is often the easy part for
a single application.
− Interoperability between the models is a big issue.
− Express-ability vs Complexity is a challenge
− How and where to add the semantics
− Where to publish and store them
− Semantic descriptions for data, streams, devices (resources)
and entities that are represented by the devices, and
description of the services.
48

49. 49
Simplicity can be very useful…

50. Hyper/CAT
50
Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html
- Servers provide catalogues of resources to
clients.
- A catalogue is an array of URIs.
- Each resource in the catalogue is annotated
with metadata (RDF-like triples).

51. Hyper/CAT model
51
Source: Toby Jaffey, HyperCat Consortium, http://www.hypercat.io/standard.html

52. 52
Complex models are (sometimes) good for
publishing research papers….
But they are often difficult to implement and use
in real world products.

53. What happens afterwards is more important
− How to index and query the annotated data
− How to make the publication suitable for constrained
environments and/or allow them to scale
− How to query them (considering the fact that here we are
dealing with live data and often reducing the processing time
and latency is crucial)
− Linking to other sources
53

54. IoT is a dynamic, online and rapidly
changing world
54
isPartOf
Annotation for the (Semantic) Web
Annotation for the IoT
Image sources: ABC Australia and 2dolphins.com

55. Make your model fairly simple and modular
55
SSNO model

56. 56
Creating common vocabularies and taxonomies
are also equally important e.g. event
taxonomies.

57. 57
We should accept the fact that sometimes we
do not need (full) semantic descriptions.
Think of the applications and use-cases before
starting to annotate the data.

58. 58
Semantic descriptions can be fairly static on the
Web;
In the IoT, the meaning of data and the
annotations can change over time/space…

59. Dynamic annotations for data in the process chain
59S. Kolozali et al, A Knowledge-based Approach for Real-Time IoT Data Stream Annotation and Processing", iThings 2014, 2014.

60. Dynamic annotations for provenance data
60S. Kolozali et al, A Knowledge-based Approach for Real-Time IoT Data Stream Annotation and Processing", iThings 2014, 2014.

61. 61
Semantic descriptions can also be learned and
created automatically.

62. Overall, we need semantic technologies in
the IoT and these play a key role in providing
interoperability.

63. However, we should design and use the
semantics carefully and consider the
constraints and dynamicity of the IoT
environments.

64. #1: Design for large-scale and provide tools and APIs.
#2: Think of who will use the semantics and how when
you design your models.
#3: Provide means to update and change the semantic
annotations.
64

65. #4: Create tools for validation and interoperability
testing.
#5: Create taxonomies and vocabularies.
#6: Of course you can always create a better model,
but try to re-use existing ones as much as you can.
65

66. #7: Link your data and descriptions to other existing
resources.
#8: Define rules and/or best practices for providing the
values for each attribute.
#9: Remember the widely used semantic descriptions
on the Web are simple ones like FOAF.
66

67. #10: Semantics are only one part of the solution and
often not the end-product so the focus of the design
should be on creating effective methods, tools and APIs
to handle and process the semantics.
Query methods, machine learning, reasoning and data
analysis techniques and methods should be able to
effectively use these semantics.
67

68. Some examples

69. IoTLite
69
http://iot.ee.surrey.ac.uk/fiware/ontologies/iot-lite

70. IoTLite ontology
70

71. Stream Annotation Ontology (SAO)
71
http://iot.ee.surrey.ac.uk/citypulse/ontologies/sao/sao

72. W3C/OGC Working Group on Spatial Data on the Web
72
http://www.w3.org/2015/spatial/wiki/Main_Page
- Use cases
- Best Practices

73. Q&A
− Thank you.
http://personal.ee.surrey.ac.uk/Personal/P.Barnaghi/
@pbarnaghi
p.barnaghi@surrey.ac.uk