SlideShare une entreprise Scribd logo

RDF Stream Processing: Let's React

RDF Stream processing and reactive systems

1  sur  30
Télécharger pour lire hors ligne
RDF Stream Processing: 
Let’s React! 
@jpcik 
Jean-Paul Calbimonte 
LSIR EPFL 
OrdRing 2014. International Semantic Web Conference ISWC 2014 
Riva del Garda, 20.10.2014
The RSP Community 
Research work 
Many Papers 
PhD Thesis 
Datasets 
Prototypes 
Benchmarks 
RDF Streams 
Stream Reasoning 
Complex Event Processing 
Stream Query Processing 
Stream Compression 
Semantic Sensor Web 
Many topics 
Tons of work 
W3C RSP Community Group 
http://www.w3.org/community/rsp 
discuss 
standardize 
combine 
formalize 
evangelize 
Effort to our work on RDF stream processing 
2
The RSP Community 
3
Why Streams? 
Internet of Things 
Sensor Networks 
Mobile Networks 
Smart Devices 
Participatory Sensing 
Transportation 
Financial Data 
Social Media 
Urban Planning 
Health Monitoring 
Marketing 
“It’s a 
streaming 
world!”[1] 
4 Della Valle, et al : It's a Streaming World! Reasoning upon Rapidly Changing Information. IEEE Intelligent Systems
Why Streams? 
Web standards 
Data discovery 
Data sharing 
Web queries 
Go Web 
Semantics 
Vocabularies 
Data Harvesting 
Data linking 
Matching 
Integration 
Ontologies 
Expressivity 
Inference 
Rule processing 
Knowledge bases 
Reasoning 
Query languages 
Query answering 
Efficient processing 
Query Federation 
Processing 
Is this what 
we require? 
5
Looking back 10 years ago… 
Stonebreaker et al. The 8 requirement of Real-TimeStream Processing. SIGMOD Record. 2005. 
“8 requirements of 
real-time stream 
processing”[2] 
Do we address them? 
Do we have more requirements? 
Do we need to do more? 
Keep data moving 
Query with stream SQL 
Handle imperfections 
Predictable outcomes 
Integrate stored data 
Data safety & availability 
Partition & scale 
Respond Instantaneously 
8 Requirements 
6

Recommandé

RDF Stream Processing Tutorial: RSP implementations
RDF Stream Processing Tutorial: RSP implementationsRDF Stream Processing Tutorial: RSP implementations
RDF Stream Processing Tutorial: RSP implementationsJean-Paul Calbimonte
 
OrdRing 2013 keynote - On the need for a W3C community group on RDF Stream Pr...
OrdRing 2013 keynote - On the need for a W3C community group on RDF Stream Pr...OrdRing 2013 keynote - On the need for a W3C community group on RDF Stream Pr...
OrdRing 2013 keynote - On the need for a W3C community group on RDF Stream Pr...Oscar Corcho
 
RDF Stream Processing and the role of Semantics
RDF Stream Processing and the role of SemanticsRDF Stream Processing and the role of Semantics
RDF Stream Processing and the role of SemanticsJean-Paul Calbimonte
 
Connecting Stream Reasoners on the Web
Connecting Stream Reasoners on the WebConnecting Stream Reasoners on the Web
Connecting Stream Reasoners on the WebJean-Paul Calbimonte
 
Triplewave: a step towards RDF Stream Processing on the Web
Triplewave: a step towards RDF Stream Processing on the WebTriplewave: a step towards RDF Stream Processing on the Web
Triplewave: a step towards RDF Stream Processing on the WebDaniele Dell'Aglio
 
TripleWave: Spreading RDF Streams on the Web
TripleWave: Spreading RDF Streams on the WebTripleWave: Spreading RDF Streams on the Web
TripleWave: Spreading RDF Streams on the WebAndrea Mauri
 
Query Rewriting in RDF Stream Processing
Query Rewriting in RDF Stream ProcessingQuery Rewriting in RDF Stream Processing
Query Rewriting in RDF Stream ProcessingJean-Paul Calbimonte
 
Summary of the Stream Reasoning workshop at ISWC 2016
Summary of the Stream Reasoning workshop at ISWC 2016Summary of the Stream Reasoning workshop at ISWC 2016
Summary of the Stream Reasoning workshop at ISWC 2016Daniele Dell'Aglio
 

Contenu connexe

Tendances

On the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingOn the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingPlanetData Network of Excellence
 
RSP4J: An API for RDF Stream Processing
RSP4J: An API for RDF Stream ProcessingRSP4J: An API for RDF Stream Processing
RSP4J: An API for RDF Stream ProcessingRiccardo Tommasini
 
Heaven: A Framework for Systematic Comparative Research Approach for RSP Engines
Heaven: A Framework for Systematic Comparative Research Approach for RSP EnginesHeaven: A Framework for Systematic Comparative Research Approach for RSP Engines
Heaven: A Framework for Systematic Comparative Research Approach for RSP EnginesRiccardo Tommasini
 
A Hierarchical approach towards Efficient and Expressive Stream Reasoning
A Hierarchical approach towards Efficient and Expressive Stream ReasoningA Hierarchical approach towards Efficient and Expressive Stream Reasoning
A Hierarchical approach towards Efficient and Expressive Stream ReasoningRiccardo Tommasini
 
RDF-Gen: Generating RDF from streaming and archival data
RDF-Gen: Generating RDF from streaming and archival dataRDF-Gen: Generating RDF from streaming and archival data
RDF-Gen: Generating RDF from streaming and archival dataGiorgos Santipantakis
 
Streaming Day - an overview of Stream Reasoning
Streaming Day - an overview of Stream ReasoningStreaming Day - an overview of Stream Reasoning
Streaming Day - an overview of Stream ReasoningRiccardo Tommasini
 
Towards efficient processing of RDF data streams
Towards efficient processing of RDF data streamsTowards efficient processing of RDF data streams
Towards efficient processing of RDF data streamsAlejandro Llaves
 
Information-Rich Programming in F# with Semantic Data
Information-Rich Programming in F# with Semantic DataInformation-Rich Programming in F# with Semantic Data
Information-Rich Programming in F# with Semantic DataSteffen Staab
 
Introduction to SparkR
Introduction to SparkRIntroduction to SparkR
Introduction to SparkRKien Dang
 
A Little SPARQL in your Analytics
A Little SPARQL in your AnalyticsA Little SPARQL in your Analytics
A Little SPARQL in your AnalyticsDr. Neil Brittliff
 
Introduction to Spark R with R studio - Mr. Pragith
Introduction to Spark R with R studio - Mr. Pragith Introduction to Spark R with R studio - Mr. Pragith
Introduction to Spark R with R studio - Mr. Pragith Sigmoid
 
LarKC Tutorial at ISWC 2009 - Data Model
LarKC Tutorial at ISWC 2009 - Data ModelLarKC Tutorial at ISWC 2009 - Data Model
LarKC Tutorial at ISWC 2009 - Data ModelLarKC
 
ParlBench: a SPARQL-benchmark for electronic publishing applications.
ParlBench: a SPARQL-benchmark for electronic publishing applications.ParlBench: a SPARQL-benchmark for electronic publishing applications.
ParlBench: a SPARQL-benchmark for electronic publishing applications.Tatiana Tarasova
 
First impressions of SparkR: our own machine learning algorithm
First impressions of SparkR: our own machine learning algorithmFirst impressions of SparkR: our own machine learning algorithm
First impressions of SparkR: our own machine learning algorithmInfoFarm
 
Python and R for quantitative finance
Python and R for quantitative financePython and R for quantitative finance
Python and R for quantitative financeLuca Sbardella
 
Enabling ontology based streaming data access final
Enabling ontology based streaming data access finalEnabling ontology based streaming data access final
Enabling ontology based streaming data access finalJean-Paul Calbimonte
 
On unifying query languages for RDF streams
On unifying query languages for RDF streamsOn unifying query languages for RDF streams
On unifying query languages for RDF streamsDaniele Dell'Aglio
 

Tendances (20)

On the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingOn the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream Processing
 
RSP4J: An API for RDF Stream Processing
RSP4J: An API for RDF Stream ProcessingRSP4J: An API for RDF Stream Processing
RSP4J: An API for RDF Stream Processing
 
Heaven: A Framework for Systematic Comparative Research Approach for RSP Engines
Heaven: A Framework for Systematic Comparative Research Approach for RSP EnginesHeaven: A Framework for Systematic Comparative Research Approach for RSP Engines
Heaven: A Framework for Systematic Comparative Research Approach for RSP Engines
 
A Hierarchical approach towards Efficient and Expressive Stream Reasoning
A Hierarchical approach towards Efficient and Expressive Stream ReasoningA Hierarchical approach towards Efficient and Expressive Stream Reasoning
A Hierarchical approach towards Efficient and Expressive Stream Reasoning
 
RDF-Gen: Generating RDF from streaming and archival data
RDF-Gen: Generating RDF from streaming and archival dataRDF-Gen: Generating RDF from streaming and archival data
RDF-Gen: Generating RDF from streaming and archival data
 
Streaming Day - an overview of Stream Reasoning
Streaming Day - an overview of Stream ReasoningStreaming Day - an overview of Stream Reasoning
Streaming Day - an overview of Stream Reasoning
 
Towards efficient processing of RDF data streams
Towards efficient processing of RDF data streamsTowards efficient processing of RDF data streams
Towards efficient processing of RDF data streams
 
Information-Rich Programming in F# with Semantic Data
Information-Rich Programming in F# with Semantic DataInformation-Rich Programming in F# with Semantic Data
Information-Rich Programming in F# with Semantic Data
 
Shebanq gniezno
Shebanq gnieznoShebanq gniezno
Shebanq gniezno
 
Introduction to SparkR
Introduction to SparkRIntroduction to SparkR
Introduction to SparkR
 
A Little SPARQL in your Analytics
A Little SPARQL in your AnalyticsA Little SPARQL in your Analytics
A Little SPARQL in your Analytics
 
Introduction to Spark R with R studio - Mr. Pragith
Introduction to Spark R with R studio - Mr. Pragith Introduction to Spark R with R studio - Mr. Pragith
Introduction to Spark R with R studio - Mr. Pragith
 
LD4KD 2015 - Demos and tools
LD4KD 2015 - Demos and toolsLD4KD 2015 - Demos and tools
LD4KD 2015 - Demos and tools
 
LarKC Tutorial at ISWC 2009 - Data Model
LarKC Tutorial at ISWC 2009 - Data ModelLarKC Tutorial at ISWC 2009 - Data Model
LarKC Tutorial at ISWC 2009 - Data Model
 
ParlBench: a SPARQL-benchmark for electronic publishing applications.
ParlBench: a SPARQL-benchmark for electronic publishing applications.ParlBench: a SPARQL-benchmark for electronic publishing applications.
ParlBench: a SPARQL-benchmark for electronic publishing applications.
 
Efficient RDF Interchange (ERI) Format for RDF Data Streams
Efficient RDF Interchange (ERI) Format for RDF Data StreamsEfficient RDF Interchange (ERI) Format for RDF Data Streams
Efficient RDF Interchange (ERI) Format for RDF Data Streams
 
First impressions of SparkR: our own machine learning algorithm
First impressions of SparkR: our own machine learning algorithmFirst impressions of SparkR: our own machine learning algorithm
First impressions of SparkR: our own machine learning algorithm
 
Python and R for quantitative finance
Python and R for quantitative financePython and R for quantitative finance
Python and R for quantitative finance
 
Enabling ontology based streaming data access final
Enabling ontology based streaming data access finalEnabling ontology based streaming data access final
Enabling ontology based streaming data access final
 
On unifying query languages for RDF streams
On unifying query languages for RDF streamsOn unifying query languages for RDF streams
On unifying query languages for RDF streams
 

En vedette

RSEP-QL: A Query Model to Capture Event Pattern Matching in RDF Stream Proces...
RSEP-QL: A Query Model to Capture Event Pattern Matching in RDF Stream Proces...RSEP-QL: A Query Model to Capture Event Pattern Matching in RDF Stream Proces...
RSEP-QL: A Query Model to Capture Event Pattern Matching in RDF Stream Proces...Daniele Dell'Aglio
 
RDF Stream Processing Models (RSP2014)
RDF Stream Processing Models (RSP2014)RDF Stream Processing Models (RSP2014)
RDF Stream Processing Models (RSP2014)Daniele Dell'Aglio
 
Optimizing RDF Data Cubes for Efficient Processing of Analytical Queries
Optimizing RDF Data Cubes for Efficient Processing of Analytical QueriesOptimizing RDF Data Cubes for Efficient Processing of Analytical Queries
Optimizing RDF Data Cubes for Efficient Processing of Analytical QueriesKim Ahlstrøm
 
Introduction to Semantic Web for GIS Practitioners
Introduction to Semantic Web for GIS PractitionersIntroduction to Semantic Web for GIS Practitioners
Introduction to Semantic Web for GIS PractitionersEmanuele Della Valle
 
Stream Reasoning: State of the Art and Beyond
Stream Reasoning: State of the Art and BeyondStream Reasoning: State of the Art and Beyond
Stream Reasoning: State of the Art and BeyondEmanuele Della Valle
 
Cep 소개 - for developers
Cep 소개 - for developersCep 소개 - for developers
Cep 소개 - for developersJuhyeon Lee
 
Rdf(Refused Derived Fuel)기술
Rdf(Refused Derived Fuel)기술Rdf(Refused Derived Fuel)기술
Rdf(Refused Derived Fuel)기술Alex Park
 
Parallel Datalog Reasoning in RDFox Presentation
Parallel Datalog Reasoning in RDFox PresentationParallel Datalog Reasoning in RDFox Presentation
Parallel Datalog Reasoning in RDFox PresentationDBOnto
 
온톨로지 추론 개요
온톨로지 추론 개요온톨로지 추론 개요
온톨로지 추론 개요Sang-Kyun Kim
 
Tomas Knap | RDF Data Processing and Integration Tasks in UnifiedViews: Use C...
Tomas Knap | RDF Data Processing and Integration Tasks in UnifiedViews: Use C...Tomas Knap | RDF Data Processing and Integration Tasks in UnifiedViews: Use C...
Tomas Knap | RDF Data Processing and Integration Tasks in UnifiedViews: Use C...semanticsconference
 

En vedette (11)

RSEP-QL: A Query Model to Capture Event Pattern Matching in RDF Stream Proces...
RSEP-QL: A Query Model to Capture Event Pattern Matching in RDF Stream Proces...RSEP-QL: A Query Model to Capture Event Pattern Matching in RDF Stream Proces...
RSEP-QL: A Query Model to Capture Event Pattern Matching in RDF Stream Proces...
 
RDF Stream Processing Models (RSP2014)
RDF Stream Processing Models (RSP2014)RDF Stream Processing Models (RSP2014)
RDF Stream Processing Models (RSP2014)
 
Optimizing RDF Data Cubes for Efficient Processing of Analytical Queries
Optimizing RDF Data Cubes for Efficient Processing of Analytical QueriesOptimizing RDF Data Cubes for Efficient Processing of Analytical Queries
Optimizing RDF Data Cubes for Efficient Processing of Analytical Queries
 
Introduction to Semantic Web for GIS Practitioners
Introduction to Semantic Web for GIS PractitionersIntroduction to Semantic Web for GIS Practitioners
Introduction to Semantic Web for GIS Practitioners
 
Stream Reasoning: State of the Art and Beyond
Stream Reasoning: State of the Art and BeyondStream Reasoning: State of the Art and Beyond
Stream Reasoning: State of the Art and Beyond
 
Cep 소개 - for developers
Cep 소개 - for developersCep 소개 - for developers
Cep 소개 - for developers
 
Rdf(Refused Derived Fuel)기술
Rdf(Refused Derived Fuel)기술Rdf(Refused Derived Fuel)기술
Rdf(Refused Derived Fuel)기술
 
Parallel Datalog Reasoning in RDFox Presentation
Parallel Datalog Reasoning in RDFox PresentationParallel Datalog Reasoning in RDFox Presentation
Parallel Datalog Reasoning in RDFox Presentation
 
온톨로지 추론 개요
온톨로지 추론 개요온톨로지 추론 개요
온톨로지 추론 개요
 
Tomas Knap | RDF Data Processing and Integration Tasks in UnifiedViews: Use C...
Tomas Knap | RDF Data Processing and Integration Tasks in UnifiedViews: Use C...Tomas Knap | RDF Data Processing and Integration Tasks in UnifiedViews: Use C...
Tomas Knap | RDF Data Processing and Integration Tasks in UnifiedViews: Use C...
 
Interviewing PPT
Interviewing PPTInterviewing PPT
Interviewing PPT
 

Similaire à RDF Stream Processing: Let's React

Towards efficient processing of RDF data streams
Towards efficient processing of RDF data streamsTowards efficient processing of RDF data streams
Towards efficient processing of RDF data streamsAlejandro Llaves
 
Transient and persistent RDF views over relational databases in the context o...
Transient and persistent RDF views over relational databases in the context o...Transient and persistent RDF views over relational databases in the context o...
Transient and persistent RDF views over relational databases in the context o...Nikolaos Konstantinou
 
On the need for applications aware adaptive middleware in real-time RDF data ...
On the need for applications aware adaptive middleware in real-time RDF data ...On the need for applications aware adaptive middleware in real-time RDF data ...
On the need for applications aware adaptive middleware in real-time RDF data ...Zia Ush Shamszaman
 
Unified Big Data Processing with Apache Spark
Unified Big Data Processing with Apache SparkUnified Big Data Processing with Apache Spark
Unified Big Data Processing with Apache SparkC4Media
 
TWDI Accelerate Seattle, Oct 16, 2017: Distributed and In-Database Analytics ...
TWDI Accelerate Seattle, Oct 16, 2017: Distributed and In-Database Analytics ...TWDI Accelerate Seattle, Oct 16, 2017: Distributed and In-Database Analytics ...
TWDI Accelerate Seattle, Oct 16, 2017: Distributed and In-Database Analytics ...Debraj GuhaThakurta
 
TDWI Accelerate, Seattle, Oct 16, 2017: Distributed and In-Database Analytics...
TDWI Accelerate, Seattle, Oct 16, 2017: Distributed and In-Database Analytics...TDWI Accelerate, Seattle, Oct 16, 2017: Distributed and In-Database Analytics...
TDWI Accelerate, Seattle, Oct 16, 2017: Distributed and In-Database Analytics...Debraj GuhaThakurta
 
Sustainable queryable access to Linked Data
Sustainable queryable access to Linked DataSustainable queryable access to Linked Data
Sustainable queryable access to Linked DataRuben Verborgh
 
Stanford CS347 Guest Lecture: Apache Spark
Stanford CS347 Guest Lecture: Apache SparkStanford CS347 Guest Lecture: Apache Spark
Stanford CS347 Guest Lecture: Apache SparkReynold Xin
 
Swift Parallel Scripting for High-Performance Workflow
Swift Parallel Scripting for High-Performance WorkflowSwift Parallel Scripting for High-Performance Workflow
Swift Parallel Scripting for High-Performance WorkflowDaniel S. Katz
 
Unified Big Data Processing with Apache Spark (QCON 2014)
Unified Big Data Processing with Apache Spark (QCON 2014)Unified Big Data Processing with Apache Spark (QCON 2014)
Unified Big Data Processing with Apache Spark (QCON 2014)Databricks
 
Model Building with RevoScaleR: Using R and Hadoop for Statistical Computation
Model Building with RevoScaleR: Using R and Hadoop for Statistical ComputationModel Building with RevoScaleR: Using R and Hadoop for Statistical Computation
Model Building with RevoScaleR: Using R and Hadoop for Statistical ComputationRevolution Analytics
 
Building Scalable Data Pipelines - 2016 DataPalooza Seattle
Building Scalable Data Pipelines - 2016 DataPalooza SeattleBuilding Scalable Data Pipelines - 2016 DataPalooza Seattle
Building Scalable Data Pipelines - 2016 DataPalooza SeattleEvan Chan
 
2014 sept 26_thug_lambda_part1
2014 sept 26_thug_lambda_part12014 sept 26_thug_lambda_part1
2014 sept 26_thug_lambda_part1Adam Muise
 
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...Databricks
 
Big Data Analytics-Open Source Toolkits
Big Data Analytics-Open Source ToolkitsBig Data Analytics-Open Source Toolkits
Big Data Analytics-Open Source ToolkitsDataWorks Summit
 
2009 0807 Lod Gmod
2009 0807 Lod Gmod2009 0807 Lod Gmod
2009 0807 Lod GmodJun Zhao
 
Apache Flink Overview at SF Spark and Friends
Apache Flink Overview at SF Spark and FriendsApache Flink Overview at SF Spark and Friends
Apache Flink Overview at SF Spark and FriendsStephan Ewen
 
6° Sessione - Ambiti applicativi nella ricerca di tecnologie statistiche avan...
6° Sessione - Ambiti applicativi nella ricerca di tecnologie statistiche avan...6° Sessione - Ambiti applicativi nella ricerca di tecnologie statistiche avan...
6° Sessione - Ambiti applicativi nella ricerca di tecnologie statistiche avan...Jürgen Ambrosi
 

Similaire à RDF Stream Processing: Let's React (20)

Towards efficient processing of RDF data streams
Towards efficient processing of RDF data streamsTowards efficient processing of RDF data streams
Towards efficient processing of RDF data streams
 
Transient and persistent RDF views over relational databases in the context o...
Transient and persistent RDF views over relational databases in the context o...Transient and persistent RDF views over relational databases in the context o...
Transient and persistent RDF views over relational databases in the context o...
 
On the need for applications aware adaptive middleware in real-time RDF data ...
On the need for applications aware adaptive middleware in real-time RDF data ...On the need for applications aware adaptive middleware in real-time RDF data ...
On the need for applications aware adaptive middleware in real-time RDF data ...
 
Unified Big Data Processing with Apache Spark
Unified Big Data Processing with Apache SparkUnified Big Data Processing with Apache Spark
Unified Big Data Processing with Apache Spark
 
Streams of RDF Events Derive2015
Streams of RDF Events Derive2015Streams of RDF Events Derive2015
Streams of RDF Events Derive2015
 
TWDI Accelerate Seattle, Oct 16, 2017: Distributed and In-Database Analytics ...
TWDI Accelerate Seattle, Oct 16, 2017: Distributed and In-Database Analytics ...TWDI Accelerate Seattle, Oct 16, 2017: Distributed and In-Database Analytics ...
TWDI Accelerate Seattle, Oct 16, 2017: Distributed and In-Database Analytics ...
 
TDWI Accelerate, Seattle, Oct 16, 2017: Distributed and In-Database Analytics...
TDWI Accelerate, Seattle, Oct 16, 2017: Distributed and In-Database Analytics...TDWI Accelerate, Seattle, Oct 16, 2017: Distributed and In-Database Analytics...
TDWI Accelerate, Seattle, Oct 16, 2017: Distributed and In-Database Analytics...
 
Sustainable queryable access to Linked Data
Sustainable queryable access to Linked DataSustainable queryable access to Linked Data
Sustainable queryable access to Linked Data
 
Stanford CS347 Guest Lecture: Apache Spark
Stanford CS347 Guest Lecture: Apache SparkStanford CS347 Guest Lecture: Apache Spark
Stanford CS347 Guest Lecture: Apache Spark
 
Swift Parallel Scripting for High-Performance Workflow
Swift Parallel Scripting for High-Performance WorkflowSwift Parallel Scripting for High-Performance Workflow
Swift Parallel Scripting for High-Performance Workflow
 
Unified Big Data Processing with Apache Spark (QCON 2014)
Unified Big Data Processing with Apache Spark (QCON 2014)Unified Big Data Processing with Apache Spark (QCON 2014)
Unified Big Data Processing with Apache Spark (QCON 2014)
 
Model Building with RevoScaleR: Using R and Hadoop for Statistical Computation
Model Building with RevoScaleR: Using R and Hadoop for Statistical ComputationModel Building with RevoScaleR: Using R and Hadoop for Statistical Computation
Model Building with RevoScaleR: Using R and Hadoop for Statistical Computation
 
Building Scalable Data Pipelines - 2016 DataPalooza Seattle
Building Scalable Data Pipelines - 2016 DataPalooza SeattleBuilding Scalable Data Pipelines - 2016 DataPalooza Seattle
Building Scalable Data Pipelines - 2016 DataPalooza Seattle
 
2014 sept 26_thug_lambda_part1
2014 sept 26_thug_lambda_part12014 sept 26_thug_lambda_part1
2014 sept 26_thug_lambda_part1
 
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
Extending the R API for Spark with sparklyr and Microsoft R Server with Ali Z...
 
Big Data Analytics-Open Source Toolkits
Big Data Analytics-Open Source ToolkitsBig Data Analytics-Open Source Toolkits
Big Data Analytics-Open Source Toolkits
 
2009 0807 Lod Gmod
2009 0807 Lod Gmod2009 0807 Lod Gmod
2009 0807 Lod Gmod
 
Apache Flink Overview at SF Spark and Friends
Apache Flink Overview at SF Spark and FriendsApache Flink Overview at SF Spark and Friends
Apache Flink Overview at SF Spark and Friends
 
6° Sessione - Ambiti applicativi nella ricerca di tecnologie statistiche avan...
6° Sessione - Ambiti applicativi nella ricerca di tecnologie statistiche avan...6° Sessione - Ambiti applicativi nella ricerca di tecnologie statistiche avan...
6° Sessione - Ambiti applicativi nella ricerca di tecnologie statistiche avan...
 
Spark meetup TCHUG
Spark meetup TCHUGSpark meetup TCHUG
Spark meetup TCHUG
 

Plus de Jean-Paul Calbimonte

Towards Collaborative Creativity in Persuasive Multi-agent Systems
Towards Collaborative Creativity in Persuasive Multi-agent SystemsTowards Collaborative Creativity in Persuasive Multi-agent Systems
Towards Collaborative Creativity in Persuasive Multi-agent SystemsJean-Paul Calbimonte
 
A Platform for Difficulty Assessment and Recommendation of Hiking Trails
A Platform for Difficulty Assessment andRecommendation of Hiking TrailsA Platform for Difficulty Assessment andRecommendation of Hiking Trails
A Platform for Difficulty Assessment and Recommendation of Hiking TrailsJean-Paul Calbimonte
 
Decentralized Management of Patient Profiles and Trajectories through Semanti...
Decentralized Management of Patient Profiles and Trajectories through Semanti...Decentralized Management of Patient Profiles and Trajectories through Semanti...
Decentralized Management of Patient Profiles and Trajectories through Semanti...Jean-Paul Calbimonte
 
Personal Data Privacy Semantics in Multi-Agent Systems Interactions
Personal Data Privacy Semantics in Multi-Agent Systems InteractionsPersonal Data Privacy Semantics in Multi-Agent Systems Interactions
Personal Data Privacy Semantics in Multi-Agent Systems InteractionsJean-Paul Calbimonte
 
SanTour: Personalized Recommendation of Hiking Trails to Health Pro files
SanTour: Personalized Recommendation of Hiking Trails to Health ProfilesSanTour: Personalized Recommendation of Hiking Trails to Health Profiles
SanTour: Personalized Recommendation of Hiking Trails to Health Pro filesJean-Paul Calbimonte
 
Multi-agent interactions on the Web through Linked Data Notifications
Multi-agent interactions on the Web through Linked Data NotificationsMulti-agent interactions on the Web through Linked Data Notifications
Multi-agent interactions on the Web through Linked Data NotificationsJean-Paul Calbimonte
 
The MedRed Ontology for Representing Clinical Data Acquisition Metadata
The MedRed Ontology for Representing Clinical Data Acquisition MetadataThe MedRed Ontology for Representing Clinical Data Acquisition Metadata
The MedRed Ontology for Representing Clinical Data Acquisition MetadataJean-Paul Calbimonte
 
Linked Data Notifications for RDF Streams
Linked Data Notifications for RDF StreamsLinked Data Notifications for RDF Streams
Linked Data Notifications for RDF StreamsJean-Paul Calbimonte
 
Fundamentos de Scala (Scala Basics) (español) Catecbol
Fundamentos de Scala (Scala Basics) (español) CatecbolFundamentos de Scala (Scala Basics) (español) Catecbol
Fundamentos de Scala (Scala Basics) (español) CatecbolJean-Paul Calbimonte
 
Toward Semantic Sensor Data Archives on the Web
Toward Semantic Sensor Data Archives on the WebToward Semantic Sensor Data Archives on the Web
Toward Semantic Sensor Data Archives on the WebJean-Paul Calbimonte
 
Detection of hypoglycemic events through wearable sensors
Detection of hypoglycemic events through wearable sensorsDetection of hypoglycemic events through wearable sensors
Detection of hypoglycemic events through wearable sensorsJean-Paul Calbimonte
 
The Schema Editor of OpenIoT for Semantic Sensor Networks
The Schema Editor of OpenIoT for Semantic Sensor NetworksThe Schema Editor of OpenIoT for Semantic Sensor Networks
The Schema Editor of OpenIoT for Semantic Sensor NetworksJean-Paul Calbimonte
 
Scala Programming for Semantic Web Developers ESWC Semdev2015
Scala Programming for Semantic Web Developers ESWC Semdev2015Scala Programming for Semantic Web Developers ESWC Semdev2015
Scala Programming for Semantic Web Developers ESWC Semdev2015Jean-Paul Calbimonte
 
XGSN: An Open-source Semantic Sensing Middleware for the Web of Things
XGSN: An Open-source Semantic Sensing Middleware for the Web of ThingsXGSN: An Open-source Semantic Sensing Middleware for the Web of Things
XGSN: An Open-source Semantic Sensing Middleware for the Web of ThingsJean-Paul Calbimonte
 
GSN Global Sensor Networks for Environmental Data Management
GSN Global Sensor Networks for Environmental Data ManagementGSN Global Sensor Networks for Environmental Data Management
GSN Global Sensor Networks for Environmental Data ManagementJean-Paul Calbimonte
 
SSN2013 Demo: tablet based visualization of transport data with SPARQLStream
SSN2013 Demo: tablet based visualization of transport data with SPARQLStreamSSN2013 Demo: tablet based visualization of transport data with SPARQLStream
SSN2013 Demo: tablet based visualization of transport data with SPARQLStreamJean-Paul Calbimonte
 
Tutorial Stream Reasoning SPARQLstream and Morph-streams
Tutorial Stream Reasoning SPARQLstream and Morph-streamsTutorial Stream Reasoning SPARQLstream and Morph-streams
Tutorial Stream Reasoning SPARQLstream and Morph-streamsJean-Paul Calbimonte
 

Plus de Jean-Paul Calbimonte (20)

Towards Collaborative Creativity in Persuasive Multi-agent Systems
Towards Collaborative Creativity in Persuasive Multi-agent SystemsTowards Collaborative Creativity in Persuasive Multi-agent Systems
Towards Collaborative Creativity in Persuasive Multi-agent Systems
 
A Platform for Difficulty Assessment and Recommendation of Hiking Trails
A Platform for Difficulty Assessment andRecommendation of Hiking TrailsA Platform for Difficulty Assessment andRecommendation of Hiking Trails
A Platform for Difficulty Assessment and Recommendation of Hiking Trails
 
Stream reasoning agents
Stream reasoning agentsStream reasoning agents
Stream reasoning agents
 
Decentralized Management of Patient Profiles and Trajectories through Semanti...
Decentralized Management of Patient Profiles and Trajectories through Semanti...Decentralized Management of Patient Profiles and Trajectories through Semanti...
Decentralized Management of Patient Profiles and Trajectories through Semanti...
 
Personal Data Privacy Semantics in Multi-Agent Systems Interactions
Personal Data Privacy Semantics in Multi-Agent Systems InteractionsPersonal Data Privacy Semantics in Multi-Agent Systems Interactions
Personal Data Privacy Semantics in Multi-Agent Systems Interactions
 
RDF data validation 2017 SHACL
RDF data validation 2017 SHACLRDF data validation 2017 SHACL
RDF data validation 2017 SHACL
 
SanTour: Personalized Recommendation of Hiking Trails to Health Pro files
SanTour: Personalized Recommendation of Hiking Trails to Health ProfilesSanTour: Personalized Recommendation of Hiking Trails to Health Profiles
SanTour: Personalized Recommendation of Hiking Trails to Health Pro files
 
Multi-agent interactions on the Web through Linked Data Notifications
Multi-agent interactions on the Web through Linked Data NotificationsMulti-agent interactions on the Web through Linked Data Notifications
Multi-agent interactions on the Web through Linked Data Notifications
 
The MedRed Ontology for Representing Clinical Data Acquisition Metadata
The MedRed Ontology for Representing Clinical Data Acquisition MetadataThe MedRed Ontology for Representing Clinical Data Acquisition Metadata
The MedRed Ontology for Representing Clinical Data Acquisition Metadata
 
Linked Data Notifications for RDF Streams
Linked Data Notifications for RDF StreamsLinked Data Notifications for RDF Streams
Linked Data Notifications for RDF Streams
 
Fundamentos de Scala (Scala Basics) (español) Catecbol
Fundamentos de Scala (Scala Basics) (español) CatecbolFundamentos de Scala (Scala Basics) (español) Catecbol
Fundamentos de Scala (Scala Basics) (español) Catecbol
 
Toward Semantic Sensor Data Archives on the Web
Toward Semantic Sensor Data Archives on the WebToward Semantic Sensor Data Archives on the Web
Toward Semantic Sensor Data Archives on the Web
 
Detection of hypoglycemic events through wearable sensors
Detection of hypoglycemic events through wearable sensorsDetection of hypoglycemic events through wearable sensors
Detection of hypoglycemic events through wearable sensors
 
The Schema Editor of OpenIoT for Semantic Sensor Networks
The Schema Editor of OpenIoT for Semantic Sensor NetworksThe Schema Editor of OpenIoT for Semantic Sensor Networks
The Schema Editor of OpenIoT for Semantic Sensor Networks
 
Scala Programming for Semantic Web Developers ESWC Semdev2015
Scala Programming for Semantic Web Developers ESWC Semdev2015Scala Programming for Semantic Web Developers ESWC Semdev2015
Scala Programming for Semantic Web Developers ESWC Semdev2015
 
XGSN: An Open-source Semantic Sensing Middleware for the Web of Things
XGSN: An Open-source Semantic Sensing Middleware for the Web of ThingsXGSN: An Open-source Semantic Sensing Middleware for the Web of Things
XGSN: An Open-source Semantic Sensing Middleware for the Web of Things
 
X-GSN in OpenIoT SummerSchool
X-GSN in OpenIoT SummerSchoolX-GSN in OpenIoT SummerSchool
X-GSN in OpenIoT SummerSchool
 
GSN Global Sensor Networks for Environmental Data Management
GSN Global Sensor Networks for Environmental Data ManagementGSN Global Sensor Networks for Environmental Data Management
GSN Global Sensor Networks for Environmental Data Management
 
SSN2013 Demo: tablet based visualization of transport data with SPARQLStream
SSN2013 Demo: tablet based visualization of transport data with SPARQLStreamSSN2013 Demo: tablet based visualization of transport data with SPARQLStream
SSN2013 Demo: tablet based visualization of transport data with SPARQLStream
 
Tutorial Stream Reasoning SPARQLstream and Morph-streams
Tutorial Stream Reasoning SPARQLstream and Morph-streamsTutorial Stream Reasoning SPARQLstream and Morph-streams
Tutorial Stream Reasoning SPARQLstream and Morph-streams
 

Dernier

maximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsmaximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsssuser82c38d
 
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A..."Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...ISPMAIndia
 
AI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriAI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriISPMAIndia
 
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...emili denli
 
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)GDSCNiT
 
SPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementSPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementISPMAIndia
 
No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!Anthony Dahanne
 
killing camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfkilling camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfssuser82c38d
 
P1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetP1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetMatthewTHawley
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkTimothy Spann
 
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...ISPMAIndia
 
killingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfkillingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfssuser82c38d
 
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAutokey
 
OpenChain AI Study Group - North America and Europe - 2024-02-20
OpenChain AI Study Group - North America and Europe - 2024-02-20OpenChain AI Study Group - North America and Europe - 2024-02-20
OpenChain AI Study Group - North America and Europe - 2024-02-20Shane Coughlan
 
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...ISPMAIndia
 
LLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowLLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowNaoki (Neo) SATO
 
sql ppt for students who preparing for sql
sql ppt for students who preparing for sqlsql ppt for students who preparing for sql
sql ppt for students who preparing for sqlbharatjanadharwarud
 
Getting Started with Trello for Beginners.pptx
Getting Started with Trello for Beginners.pptxGetting Started with Trello for Beginners.pptx
Getting Started with Trello for Beginners.pptxmavinoikein
 
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이ssuser82c38d
 
The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!ISPMAIndia
 

Dernier (20)

maximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsmaximum subarray ppt for killing camp students
maximum subarray ppt for killing camp students
 
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A..."Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
 
AI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit BendigiriAI Product Management by Abhijit Bendigiri
AI Product Management by Abhijit Bendigiri
 
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
 
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
 
SPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementSPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product Management
 
No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!No more Dockerfiles? Buildpacks to help you ship your image!
No more Dockerfiles? Buildpacks to help you ship your image!
 
killing camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdfkilling camp week 6 problem - maximal matrix.pdf
killing camp week 6 problem - maximal matrix.pdf
 
P1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 SmartsheetP1 Inspection Types in Municity 5 Smartsheet
P1 Inspection Types in Municity 5 Smartsheet
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
 
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
 
killingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfkillingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdf
 
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
 
OpenChain AI Study Group - North America and Europe - 2024-02-20
OpenChain AI Study Group - North America and Europe - 2024-02-20OpenChain AI Study Group - North America and Europe - 2024-02-20
OpenChain AI Study Group - North America and Europe - 2024-02-20
 
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
 
LLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowLLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flow
 
sql ppt for students who preparing for sql
sql ppt for students who preparing for sqlsql ppt for students who preparing for sql
sql ppt for students who preparing for sql
 
Getting Started with Trello for Beginners.pptx
Getting Started with Trello for Beginners.pptxGetting Started with Trello for Beginners.pptx
Getting Started with Trello for Beginners.pptx
 
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
killingcamp 광고삽입문제 풀이, killingcamp 광고삽입문제 풀이
 
The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!
 

RDF Stream Processing: Let's React

  • 1. RDF Stream Processing: Let’s React! @jpcik Jean-Paul Calbimonte LSIR EPFL OrdRing 2014. International Semantic Web Conference ISWC 2014 Riva del Garda, 20.10.2014
  • 2. The RSP Community Research work Many Papers PhD Thesis Datasets Prototypes Benchmarks RDF Streams Stream Reasoning Complex Event Processing Stream Query Processing Stream Compression Semantic Sensor Web Many topics Tons of work W3C RSP Community Group http://www.w3.org/community/rsp discuss standardize combine formalize evangelize Effort to our work on RDF stream processing 2
  • 4. Why Streams? Internet of Things Sensor Networks Mobile Networks Smart Devices Participatory Sensing Transportation Financial Data Social Media Urban Planning Health Monitoring Marketing “It’s a streaming world!”[1] 4 Della Valle, et al : It's a Streaming World! Reasoning upon Rapidly Changing Information. IEEE Intelligent Systems
  • 5. Why Streams? Web standards Data discovery Data sharing Web queries Go Web Semantics Vocabularies Data Harvesting Data linking Matching Integration Ontologies Expressivity Inference Rule processing Knowledge bases Reasoning Query languages Query answering Efficient processing Query Federation Processing Is this what we require? 5
  • 6. Looking back 10 years ago… Stonebreaker et al. The 8 requirement of Real-TimeStream Processing. SIGMOD Record. 2005. “8 requirements of real-time stream processing”[2] Do we address them? Do we have more requirements? Do we need to do more? Keep data moving Query with stream SQL Handle imperfections Predictable outcomes Integrate stored data Data safety & availability Partition & scale Respond Instantaneously 8 Requirements 6
  • 7. Reactive Systems Keep data moving Query with stream SQL Handle imperfections Predictable outcomes Integrate stored data Data safety & availability Partition & scale Respond Instantaneously 8 Requirements Event-Driven Jonas Boner. Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems. 2013. Events: react to Load: Scalable Failure: Resilient Users: Responsive Do we address them? Do we have more requirements? Do we need to do more? 7
  • 8. Warning: You may see code 8
  • 9. ① Keep the data moving Process data in-stream Not required to store Active processing model input streams RSP queries/ rules output streams/events RDF Streams 9
  • 10. … RDF Stream Gi Gi+1 Gi+2 … Gi+n … unbounded sequence Gi {(s1,p1,o1), (s2,p2,o2),…} [ti] 1+ triples implicit/explicit timestamp/interval public class SensorsStreamer extends RdfStream implements Runnable { public void run() { .. while(true){ the stream is “observable” ... RdfQuadruple q=new RdfQuadruple(subject,predicate,object, System.currentTimeMillis()); this.put(q); } } } How do I code this? something to run on a thread timestamped triple Data structure, execution and callbacks are mixed 10 Observer pattern Tightly coupled listener
  • 11. Actor Model Actor 1 11 send: fire-forget m No shared mutable state Actor 2 Avoid blocking operators Lightweight objects Loose coupling communicate through messages mailbox state non-blocking response behavior More goodies later… Implementations: e.g. Akka for Java/Scala
  • 12. RDF Stream Data structure object DemoStreams { ... def streamTriples={ Ideas using akka actors Iterator.from(1) map{i=> ... new Triple(subject,predicate,object) } } Infinite triple iterator Execution val f=Future(DemoStreams.streamTriples) f.map{a=>a.foreach{triple=> //do something }} Asynchronous iteration Message passing f.map{a=>a.foreach{triple=> someSink ! triple }} send triple to actor Immutable RDF stream  avoid shared mutable state  avoid concurrent writes  unbounded sequence Futures  non blocking composition  concurrent computations  work with not-yet-computed results Actors  message-based  share-nothing async  distributable 12
  • 13. RDF Stream  Loose coupling  Immutable data streams  Asynchronous message passing  Well defined input/output 13 … other issues: Graph implementation? Timestamps: application vs system? Serialization?
  • 14. ② Query using SQL on Streams Model Continuous execution Union, Join, Optional, Filter Aggregates Time window Triple window R2S operator Sequence, Co-ocurrence Time function TA-SPARQL TA-RDF ✗ ✔ Limited ✗ ✗ ✗ ✗ ✗ tSPARQL tRDF ✗ ✔ ✗ ✗ ✗ ✗ ✗ ✗ Streaming RDF Stream ✔ ✔ ✗ ✔ ✔ ✗ ✗ ✗ SPARQL C-SPARQL RDF Stream ✔ ✔ ✔ ✔ ✔ ✗ ✗ ✔ CQELS RDF Stream ✔ ✔ ✔ ✔ ✔ ✗ ✗ ✗ SPARQLStream (Virtual) RDF Stream ✔ ✔ ✔ ✔ ✗ ✔ ✗ ✗ EP-SPARQL RDF Stream ✔ ✔ ✔ ✗ ✗ ✗ ✔ ✗ Instans RDF ✔ ✔ ✔ ✗ ✗ ✗ ✗ ✗ W3C RSP  review features in existing systems  agree on fundamental operators  discuss on possible semantics https://www.w3.org/community/rsp/wiki/RSP_Query_Features RSP is not always/only SPARQL-like querying SPARQL protocol is not enough RSP RESTful interfaces? 14 Powerful languages for continuous query processing
  • 15. SELECT ?person ?loc WHERE { STREAM <http://deri.org/streams/rfid> [RANGE 3s] {?person :detectedAt ?loc} } ExecContext context=new ExecContext(HOME, false); register query String queryString =" SELECT ?person ?loc … ContinuousSelect selQuery=context.registerSelect(queryString); selQuery.register(new ContinuousListener() { public void update(Mapping mapping){ adding listener get result updates String result=""; for(Iterator<Var> vars=mapping.vars();vars.hasNext();) result+=" "+ context.engine().decode(mapping.get(vars.next())); System.out.println(result); } }); RSP Querying Example with CQELS (code.google.com/p/cqels) CQELS continuous query: “Queries evaluated continuously against the changing dataset”[4] Le Phuoc et al. An Native and adaptive approach for unifyied processing of Linked Streams and Linked Data. ISWC2011. 15 Tightly coupled listeners Results delivery: push & pull?
  • 16. Dynamic Push-Pull Producer 16 Consumer demand flow m data flow Push when consumer is faster Pull when producer is faster Dynamically switch modes Communication is dynamic depending on demand vs supply
  • 17. ③ Handle stream imperfections Delayed, missing, out-of-order … Setting timeouts in message passing val future = myActor.ask("hello")(5 seconds) context.setReceiveTimeout(100 milliseconds) def receive = { case "Hello" => //do something case ReceiveTimeout => throw new RuntimeException("Timed out") 17 timeout after 5 sec Timeout receiving messages in case of timeout “order matters!”[1] Async message passing helps Still to be studied at protocol level Different guarantee requirements In W3C RSP: usually ‘out-of-scope’ 
  • 18. ④ Generate predictable outcomes Correctness in RSP 18 “RSP engines produce different but correct results”[1] RSP queries: operational semantics Common query execution model Comparable / predictable results Dell’Aglio Calbimonte, Balduini, Corcho, Della Valle. On Correctness on RDF Stream Processing Benchmarking. ISWC 2013
  • 19. ⑤ Integrate stored and streaming data Integration in RSP languages SELECT ?person1 ?person2 FROM NAMED <http://deri.org/floorplan/> WHERE { GRAPH <http://deri.org/floorplan/> {?loc1 lv:connected ?loc2} STREAM <http://deri.org/streams/rfid> [NOW] {?person1 lv:detectedAt ?loc1} SELECT ?road ?speed WHERE { { ?road :slowTrafficDue ?observ } { ?observ rdfs:subClassOf :SlowTraffic { ?observ :speed ?speed } FILTER (getDURATION() < ”P1H”^^xsd:duration) Loading background knowledge 19 } stored RDF graph RDF stream Clean query model for stream+stored data Web-ready inter-dataset queries Shared vocabularies/ontologies RDF stream + stored “EP-SPARQL uses background ontologies to enable stream reasoning”[1] More than just joins with stored data Stream reasoning / inferences context.loadDataset("{URI OF GRAPH}", "{DIRECTORY TO LOAD}"); Anicic et al. EP-SPARQL: a unified language for event processing and stream reasoning. WWW2011.
  • 20. ⑥ Guarantee data safety and availability Restart, Suspend, Stop, Escalate, etc 20 Parent Actor 1 Automatic supervision Isolate failures Manage local failures Supervision strategies: All-for-One One-for-one Death watch handling Supervision hierarchy Supervision Actor 2 Actor 3 4X Actor
  • 21. Data availability High load: input streams 21 RSP queries/ rules under stress “eviction: delete from cache of operators”[1] binding1 binding2 .. bindingX binding3 binding5 .. bindingY X Join operator X Evict cache items based on: recency LRU Likeliness of matching Gao et al. The CLOCK Data-Aware Eviction Approach: Towards Processing Linked Data Streams with Limited Resources. ESWC 2014
  • 22. ⑦ Partition and scale automatically 22 Dynamite: Parallel Materialization Maintaining a dynamic ontology Parallel threads
  • 23. Actors everywhere Actor 1 23 m No difference in Actor 2 one core many cores many servers Actor 3 Actor 4 Transparent Remoting Locality optimization Define Routing policies Define actor clusters m m Existing ‘map reduce’ for streams: Storm, S4, Spark, Akka Streams Create workflows of Stream Processors?
  • 24. ⑧ Process and respond instantaneously Blocking queries Sync communication Bottlenecks No end to end reactivity  24 SPARQLStream Virtual RDF Stream DSMS CEP Sensor middleware … rewritten queries users, applications query processing Morph-streams data layer “Query virtual RDF streams”[1] Push: Using Websockets!  Two way communication Responsive answers
  • 25. RSP Stream Reasoning Example with TrOWL (trowl.eu) Ontologies evolve over time! ONTO + axiom1 + axiom2 ONTO’ - axiom3 ONTO’’ Adding and removing axioms over time “reasoning with changing knowledge”[3] add 10 axioms get instances of ‘System’ class Ren, Pan, Zhao. Towards Scalable Reasoning on Ontology Streams via Syntactic Approximation. IWOD2010. val onto=mgr.createOntology val reasoner = new RELReasonerFactory().createReasoner(onto) (1 to 10).foreach{i=> reasoner += (Indiv(rspEx+"sys"+i) ofClass system) reasoner.reclassify reclassify } println (reasoner.getInstances(SystemClass, true)) 25 Powerful Dynamic Ontology Maintenance Streaming query results? Notify new inferences?
  • 26. A lot to do… 26
  • 27. Reactive RSPs Keep data moving Query with stream SQL Handle imperfections Predictable outcomes Integrate stored data Data safety & availability Partition & scale Respond Instantaneously 8 Requirements We go beyond only these 27 Data Heterogeneity Data Modeling Stream Reasoning Data discovery Stream data linking Query optimization … more Reactive Principles Needed if we want to build relevant systems
  • 28. Reactive RSP workflows 28 Morph Streams CSPARQL s Etalis TrOWL s Dyna mite s Event-driven message passing Async communication Immutable streams Transparent Remoting Parallel and distributed Supervised Failure Handling Responsive processing s CQELS Minimal agreements: standards, serialization, interfaces Formal models for RSPs and reasoning Working prototypes/systems! Reactive RSPs
  • 29. We like to do things 29
  • 30. Muchas gracias! @jpcik Jean-Paul Calbimonte LSIR EPFL