The presentation I gave at DeSemWeb about stream exchange and processing on the Web. Article available at: http://w3id.org/wesp

1. On a web of data streams
http://w3id.org/wesp
Daniele Dell’Aglio, Danh Le Phuoc, Anh Lê Tuán,
Muhammad Intizar Ali and Jean-Paul Calbimonte
dellaglio@ifi.uzh.ch http://dellaglio.org @dandellaglio
Vienna, Austria, 22.10.2017

2. RDF Stream Processing
Stream
Processing
RDF
&
SPARQL
RDF
Stream
Processing
(RSP)
Real-time
processing of
highly dynamic
data
SemanticWeb
technologies for
data exchange
through theWeb
DeSemWeb, 22.10.2017On a web of data streams2/16

3. Finding agreements
DeSemWeb, 22.10.2017On a web of data streams3/16
 Many topics
 RDF streams
 Stream reasoning
 Complex event processing
 Stream query processing
 Internet/web of things
 Many studies
 Data models
 Query models
 Prototypes
 Benchmarks
 Datasets
 W3C RSP community group (2013 – 2016)
 Effort to (discuss | formalise | standardise | combine | evangelise) the
existing studies on RSP
 Outcomes
 Abstract model for RDF streams
 Requirements document for query languages of RDF streams
 More at: https://www.w3.org/community/rsp/

4. But...
DeSemWeb, 22.10.2017On a web of data streams4/16
 W3C RSP sets some foundations and requirements, but:
 Standard protocols and exchanging mechanisms for RDF
stream are still missing
 We need generic and flexible solutions for making RDF
streams available and exchangeable on theWeb.

5. The goal: a decentralized web of RSPs
Morph
Streams
CSPARQL
TrOWL Stream
Rule
CQELS
On a web of data streams DeSemWeb, 22.10.20175/16
What is a suitable data exchange infrastructure
to perform stream processing on theWeb?
CSPARQL

6. How far are we?
DeSemWeb, 22.10.2017On a web of data streams6/16
 Documents from RSP
 Abstract model of RDF Stream
 Requirements for query languages for RDF Stream
 Protocols to exchange data streams on the web and
internet
 WebSocket, MQTT
 Description of the stream
 SSN
 Interfaces to control RSP engines

7. Requirements
DeSemWeb, 22.10.2017On a web of data streams7/16
A framework for RDF stream exchange should
1. prioritize active paradigms for data stream exchange
2. enable the combination of streaming and stored data
3. enable the possibility to build reliable, distributed and
scalable streaming applications
4. guarantee a wide range of operations over the streams
5. support the publication of information about the stream
6. support the exchange of a wide variety of streams
7. exploit as much as possible existing protocols and
standards

8. WeSP
DeSemWeb, 22.10.2017On a web of data streams8/16
 A framework to publish and exchange RDF streams on
the Web
 A model to serialise RDF streams
 A model to describe RDF streams
 A communication protocol

9. A model to serialise RDF streams
DeSemWeb, 22.10.2017On a web of data streams9/16
 An RDF stream can be represented as an (infinite) ordered sequence of
time-annotated data items (RDF graphs)…
 ... serialized in (compliant) JSON-LD
[{ "@graph": {
"@id": "http://.../G1",
{ "@id": "http://.../a",
"http://.../isIn": {"@id":"http://.../rRoom"}}
},{ "@id": "http://.../G1",
"prov:generatedAt":"2016-16-12T00:01:00"
}
},{ "@graph": {
"@id": "http://.../G2",
{ "@id": "http://.../b",
"http://.../isIn": {"@id":"http://.../rRoom"}}
},{ "@id": "http://.../G2",
"prov:generatedAt":" 2016-16-12T00:03:00"
}
},…
 Compliant withW3C RSP abstract data model
G1
G2
G3
{:a :isIn :rRoom}
{:b :isIn :bRoom}
{:c :talksIn :rRoom,
:d :talksIn :bRoom}
S
3
5
1
t

10. A model to describe RDF streams
 A description of the RDF stream should be provided
 The identifier of the stream
 A description of the schema of the stream items
 Data item samples
 The location of the stream endpoint (e.g. WebSocket URL)
 This description is provided through the RDF Stream
Descriptor
 Serialised in RDF
 An extension of DCAT and SPARQL Service Descriptor
 Published according to the linked data principles
DeSemWeb, 22.10.2017On a web of data streams10/16

11. A communication protocol
DeSemWeb, 22.10.2017On a web of data streams11/16
 Two interfaces
 Producer
 Consumer
 We distinguish three types of actors (depending on the
implemented interfaces)
Producer Consumer
Stream source
Stream transformer
Stream sink

12. A communication protocol
DeSemWeb, 22.10.2017On a web of data streams12/16
Producer
Consumer
Stream Descriptor
endpoint
RDF stream
endpoint
Get stream descriptor (SD)
SD
Process
SD
Subscribe to stream
Stream item
Stream item
Stream item
…
Process
stream

13. Protocols
DeSemWeb, 22.10.2017On a web of data streams13/16
 The RDF Stream Descriptor is accessible through HTTP
 The transmission of the stream can happen through
different protocols
 HTTP chunked encoding
 WebSocket
 Message QueingTelemetryTransport (MQTT)
 Server-Sent Events (SSE)
 ...

14. Proof of concepts
DeSemWeb, 22.10.2017On a web of data streams14/16
 TripleWave
 Stream source
 Native implementation of WeSP
 http://streamreasoning.github.io/TripleWave
 C-SPARQL
 Stream transformer
 WeSP implemented as a wrapper
 https://github.com/dellaglio/csparql-wesp
 CQELS
 Stream transformer
 Native implementation of WeSP
 https://github.com/cqels/CQELS-1.x/

15. Conclusions
DeSemWeb, 22.10.2017On a web of data streams15/16
 WeSP: framework to exchange RDF streams on the web
 RDF to serialise the stream items
 RDF to describe the stream
 Application and communication protocols: HTTP, WebSocket,
MQTT, etc.
 Interfaces to produce and consume RDF streams
 What’s next?
 Relation with other technologies: LDN,Activity Streams, etc.
 Adoption
 Federated stream processing over theWeb

16. Thank you! Questions?
On a web of data streams
http://w3id.org/wesp
Daniele Dell’Aglio, Danh Le Phuoc, Anh LêTuán,
Muhammad Intizar Ali and Jean-Paul Calbimonte
dellaglio@ifi.uzh.ch
http://dellaglio.org
@dandellaglio
DeSemWeb, 22.10.2017On a web of data streams16/16