SlideShare une entreprise Scribd logo
1  sur  53
Télécharger pour lire hors ligne
Getting Started
with
OpenSplice and Esper
 Angelo Corsaro, Ph.D.          Thomas Bernhardt
 Chief Technology Officer        Chief Technology Officer / Funder
 OMG DDS Sig Co-Chair           EsperTech
 PrismTech                      tom@espertech.com
 angelo.corsaro@prismtech.com
Agenda
    Welcome & Intro




                          !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
!


!   DDS Overview
!   CEP Overview
!   DDS + CEP in Action
!   Q&A
DDS Overview
...from a Stream Processing Perspective




Angelo CORSARO, Ph.D.
Chief Technology Officer
OMG DDS Sig Co-Chair
PrismTech
angelo.corsaro@prismtech.com
Stream Processing                          [1/3]

!   Stream Processing is an architectural style for building




                                                               !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
    systems that operate over continuous (theoretically
    infinite) streams of data
!   Stream Processing is often reified under one of its many
    declinations, such as:
    !   Reactive Systems
    !   Signal Processing Systems
    !   Functional Stream Programming
    !   Data Flow Systems
Stream Processing                                 [2/3]

!   The Stream Processing Architecture very




                                                              !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
    naturally models systems reacting to streams of
    data and events produced by the external
    world, such as the data produced by sensors, a
    camera or even the data produced by the
    stock exchange.

!   Stream Processing Systems usually operate in
    real-time over streams and generate in turns
    other streams of data providing information on
    what is happening or suggesting actions to
    perform, such as by stock X, raise alarm Y, or
    detected spatial violation, etc.
Stream Processing                                        [3/3]

!   Stream Processing Systems are typically




                                                                                              !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
    modeled as collection of modules
    communicating via typed data
    channels called usually streams
                                                           Filter
!   Modules usually play one of the                                           Filter
    following roles:
                                                           Filter
    !   Sources: Injecting data into the System
    !   Filters/Actors: Performing some
        computation over sources                                                       Sink
    !   Sinks: Consuming the data produced by                        Stream
        the system                                source
More on Streams
Defining Streams
!   In abstract terms, a stream is an infinite sequence of




                                                             !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
    data samples of a given type T
!   Streams can be further classified in continuous and
    discrete streams. Sometimes referred as Behaviors/
    Signals and Events
!   In this presentation we’ll refer to Continuous Streams
    as Data Streams and to Discrete Streams as Event
    Streams
Data Streams
Data Streams                     Temp




                                               !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
! The value of a Data Stream

  is always defined, i.e.
  continuous.
! Good examples of a Data

  Stream are the value
  assumed by a real-world
  entity, such as temperature,          time
  pressure, a price, etc.
Event Streams
Event Streams                    OverheatAlarm




                                                        !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
! The value of the stream is

  defined at precise point in
  time, i.e. it is discrete
! Good examples of Event

  Streams are events in the real
  world, such a violation of a
  regulatory compliance, the
                                                 time
  temperature higher than a
  given value, etc.
[A Stream Perspective]

What is DDS?
!   DDS is a high-performance, real-time, highly-




                                                         !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
    available, fully-distributed, messaging technology
    that allows you to define data/event streams and
    make them dynamically discoverable
!   DDS is equipped with a rich set of QoS providing
    control on the key temporal and availability
    properties of data
Defining Streams in DDS
DDS Topics                     [1/2]
                                                “org.opensplice.demo.TTempSensor”

!   A Topic defines a stream class/




                                                                                        !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
    category
!   A Topic has associated a user
    defined type and QoS
!   The Topic name, type and QoS
    defines the key functional and
    non-functional invariants
!   Topics can be discovered or
    locally defined                   struct TempSensor {
                                        long   Id;                        DURABILITY,
                                        float temp;                       DEADLINE,
                                        float hum;
                                      }                                   PRIORITY,
                                      #pragma keylist TempSensor id       …
DDS Topics                   [2/2]
                                            “org.opensplice.demo.TTempSensor”




                                                                                    !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
!   DDS Topic types can have
    associated keys
!   Each unique key-value
    identifies a unique sub-
    stream of values -- called
    Topic Instance
                                 struct TempSensor {
                                   long   id;                         DURABILITY,
                                   float temp;                        DEADLINE,
                                   float hum;
                                 }                                    PRIORITY,
                                 #pragma keylist TempSensor id        …
“Seeing” Streams




                                                       !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
   id =701

                     struct TempSensor {
id =809                 @key long  id;
             Topic      float temp;
                        float hum;
                     };
id =977



 Instances                                 Instances
QoS Policies




!"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
Anatomy of a DDS Application




                                                                                                          !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
Domain    val dp = DomainParticipant(0)
                                                                               Domain
                                                                               Participant


          // Create a Publisher / Subscriber
          Publisher p = dp.create_publisher();
                                                               Publisher       Topic         Subscriber
Session   Subscriber s = dp.create_subscriber();
          // Create a Topic
          Topic<TempSensor> t =                                  Gives access to a
             dp.create_topic<TempSensor>(“com.myco.TSTopic”)     DDS Domain
                                                               DataWrter                     DataReader

Reader/Writers
User Defined for Types
      // Create a DataWriter/DataWriter
      DataWriter<TempSensor> dw = pub.create_datawriter(t);
      DataReader<TempSensor> dr = sub.create_datareader(t);
Anatomy of a DDS Application




                                                                                                             !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
Domain    val dp = DomainParticipant(0)
                                                                                  Domain
                                                                                  Participant


          // Create   a Publisher / Subscriber
          val pub =   Publisher(dp)
                                                                Publisher         Topic         Subscriber
Session   val sub =   Subscriber(dp)
          // Create   a Topic
          val topic   = Topic[TempSensor](dp,
                           “org.opensplice.demo.TTempSensor”)

                                                                DataWrter                       DataReader

Reader/Writers
User Defined for Types                                             Pub/Sub
      // Create a DataWriter/DataWriter
                                                                   Abstractions
      DataWriter<TempSensor> dw = pub.create_datawriter(t);
      DataReader<TempSensor> dr = sub.create_datareader(t);
Anatomy of a DDS Application




                                                                                                             !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
Domain    val dp = DomainParticipant(0)
                                                                               Domain
                                                                               Participant


          // Create   a Publisher / Subscriber
          val pub =   Publisher(dp)
                                                                Publisher      Topic            Subscriber
Session   val sub =   Subscriber(dp)
          // Create   a Topic
          val topic   = Topic[TempSensor](dp,
                           “org.opensplice.demo.TTempSensor”)

                                                                DataWrter                       DataReader

Reader/Writers for User Defined for Types
      // Create a DataWriter/DataWriter
      val writer = DataWriter[TempSensor](pub, topic)
      val reader = DataReader[TempSensor](sub, topic)                       Reader/Writer for
                                                                            application
      // Write data                                                         defined Topic
      val t = new TempSensor ts(101, 25, 40)
      writer write ts;                                                      Types
Data & Event Streams
!   DDS does not provide different types for Data/Event Streams.




                                                                                       !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
    The difference between the two can be made through the
    DataReader API by using properly using read/take operations
!   DataReader::read
    !   Reads the value of the stream w/o removing it from the stream. As a result
        multiple read can see the last known value of the stream

!   DataReader::take
    !   Takes the value available on the stream (if any yet) and removes it from the
        stream
Stream Processing
Content Filtered Topics
!   Content Filtered Topics




                                                                                          !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
    provide a way of defining a     Example:
    filters over an incoming       // Create a Topic (on default domain)
    stream associated with a       val topic = Topic[TempSensor](“TTempSensor”)
                                   val ftopic =
    given topic                      ContentFilteredTopic[TempSensor](“CFTempSensor”,
                                                                      topic,
                                                                      filter,
!   Filters are expressed as the                                      params)

    “WHERE” clause of an SQL       // - filter is a WHERE-like clause, such as:
                                   //     “temp > 20 AND hum > 50”
    statement                      //     “temp > %0”
                                   //     “temp > hum”
                                   //     “temp BETWEEN (%0 AND %1)
!   Filters can operate on any     //
                                   // - params is the list of parameters to pass to the
    attribute of the type          //   filter expression – if any

    associated with the topic
Filter Expression Syntax




                                     !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
!   DDS Filters are condition over
    a topic type attributes

!   Temporal properties or
    causality cannot be
    captured via DDS filter
    expression
History




                                                                           !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
                                     Data older than “n samples ago”
                                     get’s out the window
!   DDS provides a way of
    controlling data        future                                  past
    windows through the
    History QoS                           now
                                                 The window keeps
                                                 the last n data samples
[Putting it All Together]

TempSensor Moving Average
   object MovingAverageFilter {
      def main(args: Array[String]) {
       if (args.length < 2) {




                                                                                        !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
         println("USAGE:ntMovingAverageFilter <window> <filter-expression>")
       }

         val topic = Topic[TempSensor]("TTempSensor")
         val ftopic = ContentFilteredTopic[TempSensor]("CFTempSensor",topic, args(1))

         val rqos = DataReaderQos() <= KeepLastHistory(args(0).toInt)
         val reader = DataReader[TempSensor](ftopic, rqos)

         reader.reactions += {
           case e: DataAvailable[_] => {
             var average: Float = 0
             val window = e[TempSensor].reader.history
             window foreach (average += _.temp)
             average = average / window.length
             println("+--------------------------------------------------------")
             println("Moving Average: " + average)
           }
         }
     }
Product Organization
Product Organization
                      Commercial Edition
                      !   No Cost Runtime Licenses




                                                               !"#$%&'()*+,--.*/%&01234(*5**677*8&'()0*8303%93:
                      !   Your choice of licensing
                            !   LGPL or Commercial
                            !   Subscription or Perpetual
                      !   Complete DDS Implementation
    Commercial
      Edition
                      !   Comprehensive Developer and
                          Deployment Support Options with a
                          range of Service Level Agreements
                      Commercial Add-Ons
                      ! Individually licensable technologies


 Commercial Add-ons
                      !   Rich ecosystem covering tools,
                          integration, testing, etc.
Key Points So Far




                                                                   !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:;
!   DDS key abstractions for building stream processing
    architectures
!   DDS provides some event processing capabilities that
    facilitate the development of Stream Processing Filters
!   What else can you use for “Stream Processing” in combination
    with DDS?
    ! Let’s have Tom introduce us to the world of CEP and Esper!
!

	 	
	

! 


!	

#$
	

%!#

	

!!
'(
Getting Started with OpenSplice and Esper
! 



 
   *!
+!!
	#


   	,	
! 
-      !-#
	
!
+!!
	#

   .

/
#
   0#	
	!
12
#!


+
2
,
!	

1
3

3
   		455 !
 

   #	2
	!
6
	
' #	
!	2
7$82
1$9(
   !#
#

 	-	
 
/
   !#
#

	
!	
         :
!#!-#	2
	
!!!-#	
         0#	
##2
!#;!	
                                                    )
#$
	


4
#$
	

 !	
	!

	
 !	!



 #
,	 
!!

/#
	=

	
#!	
'(
)

 !
	!	

	


#!	
'(
0		
!		
!
	

 
/#

! 

	!
8
	,
	
 








##, 
-
?!@

 ! 

	!


	A
 
8


 	
! 
B

8
#!
!-
8
#
	A




                                           1
! 
	
0CD%

 0CD%                    
   %	
 !	!
'!
#	(        %	
#
   :! #
E            :! #
 !	!
   %F
'
!-	!	 (      G 4
	

!!
 E	5          %--53	
 	

           
6
!!#	
 	
	              H IJ	
EJK
8
!#
 #!	
!#4
	

	
		
	

 


  	
#$
	2
  
#!	2
! 
	

-	L

  
	
   	
!#

	

#!	

	


	
   	
!#

	

#!	

	

 

!	!
5
!

   .		
!#

	
-	

 		
! 
$#	
#$
	

 #
!#4
	

!		
! 
 #
!
+
5

 0C
5
C
	


',	
!
#
-!=
	(
    
!#

;
%F
!		
! 
+!!
!		

 #!!	

   #
'
	
-

	
!
	 (
   	!	
-	


 
	
=
+
0C
! 
C

	!	


   -
#
   ,
	

 
M!#	
#
5
:
!#!-#	
,	
!
#

    	-	 
!#!-#
#!	
	!	
# 
-N	
	!
%#	
#!
 %#	
#!
 [insert into insert_into_def]
 select select_list
 from stream_def [as name] [, stream_def [as name]] [,...]
 [where search_conditions]
 [group by grouping_expression_list]
 [having grouping_search_conditions]
 [output output_specification]
 [order by order_by_expression_list]
 

 %!#
 select acctId, sum(amount)
 from Withdrawal.win:time(1 minute)
 group by acctId
 having sum(amount)  1000
 order by acctId asc




                                        9
!		
$
O
	
P
.!	
 !		
$
	

!		
!	
! 
!		

 !	
 	4
C!
-# 
-#=
   M#	
$
   .-

	-! 
	
   	
#
-

$	!#
	2

!
.%
	

	,=

   	
 .!	
   !#
.!	4
.!	
	!	
	#
##4
Getting Started with OpenSplice and Esper
Getting Started with OpenSplice and Esper
/! 4
#
! 

	!	
$
-! 

$	!#

       	Q
M##,
R!!4!S
	!$
   !#
!	4

                                               7
!#
.!	
 .!	
$!#

   M##, 
-
 select *
 from pattern [w=Withdrawal - Deposit(acctId = w.acctId)]

   !	

	
$
   !#

 w=Withdrawal -
   (pin=PinChange(acctId = w.acctId) and
     pwd=PasswordChange(acctId = w.acctId)
   )


 	##
		
,
Getting Started with OpenSplice and Esper
every (
    (w=Withdrawal - Deposit(acctId = w.acctId))
    or
    (d=Deposit - Withdrawal(acctId = d.acctId))
  )

                                         T
!#
.!	
	
 .!	
$!#

   	
!	
!	
	
		
!#
   .-

Contenu connexe

Tendances

Rails Model Basics
Rails Model BasicsRails Model Basics
Rails Model BasicsJames Gray
 
Swift 3 Programming for iOS : Protocol
Swift 3 Programming for iOS : ProtocolSwift 3 Programming for iOS : Protocol
Swift 3 Programming for iOS : ProtocolKwang Woo NAM
 
Python client api
Python client apiPython client api
Python client apidreampuf
 
DashProfiler 200807
DashProfiler 200807DashProfiler 200807
DashProfiler 200807Tim Bunce
 
Class 31: Deanonymizing
Class 31: DeanonymizingClass 31: Deanonymizing
Class 31: DeanonymizingDavid Evans
 
Gpu programming with java
Gpu programming with javaGpu programming with java
Gpu programming with javaGary Sieling
 

Tendances (6)

Rails Model Basics
Rails Model BasicsRails Model Basics
Rails Model Basics
 
Swift 3 Programming for iOS : Protocol
Swift 3 Programming for iOS : ProtocolSwift 3 Programming for iOS : Protocol
Swift 3 Programming for iOS : Protocol
 
Python client api
Python client apiPython client api
Python client api
 
DashProfiler 200807
DashProfiler 200807DashProfiler 200807
DashProfiler 200807
 
Class 31: Deanonymizing
Class 31: DeanonymizingClass 31: Deanonymizing
Class 31: Deanonymizing
 
Gpu programming with java
Gpu programming with javaGpu programming with java
Gpu programming with java
 

En vedette

JUDCon India 2012 Drools Fusion
JUDCon  India 2012 Drools FusionJUDCon  India 2012 Drools Fusion
JUDCon India 2012 Drools FusionMark Proctor
 
Aggregate Sharing for User-Define Data Stream Windows
Aggregate Sharing for User-Define Data Stream WindowsAggregate Sharing for User-Define Data Stream Windows
Aggregate Sharing for User-Define Data Stream WindowsParis Carbone
 
Graphs as Streams: Rethinking Graph Processing in the Streaming Era
Graphs as Streams: Rethinking Graph Processing in the Streaming EraGraphs as Streams: Rethinking Graph Processing in the Streaming Era
Graphs as Streams: Rethinking Graph Processing in the Streaming EraVasia Kalavri
 
Graph Stream Processing : spinning fast, large scale, complex analytics
Graph Stream Processing : spinning fast, large scale, complex analyticsGraph Stream Processing : spinning fast, large scale, complex analytics
Graph Stream Processing : spinning fast, large scale, complex analyticsParis Carbone
 
Gelly-Stream: Single-Pass Graph Streaming Analytics with Apache Flink
Gelly-Stream: Single-Pass Graph Streaming Analytics with Apache FlinkGelly-Stream: Single-Pass Graph Streaming Analytics with Apache Flink
Gelly-Stream: Single-Pass Graph Streaming Analytics with Apache FlinkVasia Kalavri
 
Internet of Things and Complex event processing (CEP)/Data fusion
Internet of Things and Complex event processing (CEP)/Data fusionInternet of Things and Complex event processing (CEP)/Data fusion
Internet of Things and Complex event processing (CEP)/Data fusionBAINIDA
 
Android Chromium Rendering Pipeline
Android Chromium Rendering PipelineAndroid Chromium Rendering Pipeline
Android Chromium Rendering PipelineHyungwook Lee
 
Elvis Presley
Elvis PresleyElvis Presley
Elvis Presleyhelenavf1
 
Stzuminus First Nation Afl
Stzuminus First Nation   AflStzuminus First Nation   Afl
Stzuminus First Nation AflFaye Brownlie
 
Africa 6B
Africa 6BAfrica 6B
Africa 6BC FM
 
Homeselling Proposal
Homeselling ProposalHomeselling Proposal
Homeselling Proposalguestd23b1
 
Focus 2 - Principii de psihologie aplicata in software
Focus 2 - Principii de psihologie aplicata in softwareFocus 2 - Principii de psihologie aplicata in software
Focus 2 - Principii de psihologie aplicata in softwareValentin Bora
 
Arg corrugate cooler unit sd
Arg corrugate cooler unit sdArg corrugate cooler unit sd
Arg corrugate cooler unit sdSteve Danford
 

En vedette (20)

JUDCon India 2012 Drools Fusion
JUDCon  India 2012 Drools FusionJUDCon  India 2012 Drools Fusion
JUDCon India 2012 Drools Fusion
 
Aggregate Sharing for User-Define Data Stream Windows
Aggregate Sharing for User-Define Data Stream WindowsAggregate Sharing for User-Define Data Stream Windows
Aggregate Sharing for User-Define Data Stream Windows
 
Graphs as Streams: Rethinking Graph Processing in the Streaming Era
Graphs as Streams: Rethinking Graph Processing in the Streaming EraGraphs as Streams: Rethinking Graph Processing in the Streaming Era
Graphs as Streams: Rethinking Graph Processing in the Streaming Era
 
Graph Stream Processing : spinning fast, large scale, complex analytics
Graph Stream Processing : spinning fast, large scale, complex analyticsGraph Stream Processing : spinning fast, large scale, complex analytics
Graph Stream Processing : spinning fast, large scale, complex analytics
 
Gelly-Stream: Single-Pass Graph Streaming Analytics with Apache Flink
Gelly-Stream: Single-Pass Graph Streaming Analytics with Apache FlinkGelly-Stream: Single-Pass Graph Streaming Analytics with Apache Flink
Gelly-Stream: Single-Pass Graph Streaming Analytics with Apache Flink
 
Internet of Things and Complex event processing (CEP)/Data fusion
Internet of Things and Complex event processing (CEP)/Data fusionInternet of Things and Complex event processing (CEP)/Data fusion
Internet of Things and Complex event processing (CEP)/Data fusion
 
Android Chromium Rendering Pipeline
Android Chromium Rendering PipelineAndroid Chromium Rendering Pipeline
Android Chromium Rendering Pipeline
 
Elvis Presley
Elvis PresleyElvis Presley
Elvis Presley
 
Kerstwensen
KerstwensenKerstwensen
Kerstwensen
 
Excellent Roth IRA Alternative
Excellent Roth IRA  AlternativeExcellent Roth IRA  Alternative
Excellent Roth IRA Alternative
 
Bain
BainBain
Bain
 
MRLC Feb Reading
MRLC Feb ReadingMRLC Feb Reading
MRLC Feb Reading
 
2009takvimi
2009takvimi2009takvimi
2009takvimi
 
Stzuminus First Nation Afl
Stzuminus First Nation   AflStzuminus First Nation   Afl
Stzuminus First Nation Afl
 
Africa 6B
Africa 6BAfrica 6B
Africa 6B
 
Homeselling Proposal
Homeselling ProposalHomeselling Proposal
Homeselling Proposal
 
Egipto
EgiptoEgipto
Egipto
 
Focus 2 - Principii de psihologie aplicata in software
Focus 2 - Principii de psihologie aplicata in softwareFocus 2 - Principii de psihologie aplicata in software
Focus 2 - Principii de psihologie aplicata in software
 
Arg corrugate cooler unit sd
Arg corrugate cooler unit sdArg corrugate cooler unit sd
Arg corrugate cooler unit sd
 
ikd312-02-three-schema
ikd312-02-three-schemaikd312-02-three-schema
ikd312-02-three-schema
 

Similaire à Getting Started with OpenSplice and Esper

Open splicedds espercep-webinar
Open splicedds espercep-webinarOpen splicedds espercep-webinar
Open splicedds espercep-webinarTomasz Waszczyk
 
Complex Event Processing with Esper
Complex Event Processing with EsperComplex Event Processing with Esper
Complex Event Processing with EsperAntónio Alegria
 
The Data Distribution Service Tutorial
The Data Distribution Service TutorialThe Data Distribution Service Tutorial
The Data Distribution Service TutorialAngelo Corsaro
 
Open source Tools and Frameworks for M2M - Sierra Wireless Developer Days
Open source Tools and Frameworks for M2M - Sierra Wireless Developer DaysOpen source Tools and Frameworks for M2M - Sierra Wireless Developer Days
Open source Tools and Frameworks for M2M - Sierra Wireless Developer DaysBenjamin Cabé
 
The DDS Tutorial Part II
The DDS Tutorial Part IIThe DDS Tutorial Part II
The DDS Tutorial Part IIAngelo Corsaro
 
Cassandra Day Chicago 2015: Apache Cassandra Data Modeling 101
Cassandra Day Chicago 2015: Apache Cassandra Data Modeling 101Cassandra Day Chicago 2015: Apache Cassandra Data Modeling 101
Cassandra Day Chicago 2015: Apache Cassandra Data Modeling 101DataStax Academy
 
Cassandra Day London 2015: Data Modeling 101
Cassandra Day London 2015: Data Modeling 101Cassandra Day London 2015: Data Modeling 101
Cassandra Day London 2015: Data Modeling 101DataStax Academy
 
Cassandra Day Atlanta 2015: Data Modeling 101
Cassandra Day Atlanta 2015: Data Modeling 101Cassandra Day Atlanta 2015: Data Modeling 101
Cassandra Day Atlanta 2015: Data Modeling 101DataStax Academy
 
Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...
Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...
Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...InfluxData
 
Introduction to data modeling with apache cassandra
Introduction to data modeling with apache cassandraIntroduction to data modeling with apache cassandra
Introduction to data modeling with apache cassandraPatrick McFadin
 
CQRS / ES & DDD Demystified
CQRS / ES & DDD DemystifiedCQRS / ES & DDD Demystified
CQRS / ES & DDD DemystifiedVic Metcalfe
 
Solaris DTrace, An Introduction
Solaris DTrace, An IntroductionSolaris DTrace, An Introduction
Solaris DTrace, An Introductionsatyajit_t
 
Distributed Systems: How to connect your real-time applications
Distributed Systems: How to connect your real-time applicationsDistributed Systems: How to connect your real-time applications
Distributed Systems: How to connect your real-time applicationsJaime Martin Losa
 
DDS Tutorial -- Part I
DDS Tutorial -- Part IDDS Tutorial -- Part I
DDS Tutorial -- Part IAngelo Corsaro
 
DDS-PSM-Cxx and simd-cxx
DDS-PSM-Cxx and simd-cxxDDS-PSM-Cxx and simd-cxx
DDS-PSM-Cxx and simd-cxxAngelo Corsaro
 
Application Security
Application SecurityApplication Security
Application Securityflorinc
 
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...Hakka Labs
 

Similaire à Getting Started with OpenSplice and Esper (20)

Open splicedds espercep-webinar
Open splicedds espercep-webinarOpen splicedds espercep-webinar
Open splicedds espercep-webinar
 
Complex Event Processing with Esper
Complex Event Processing with EsperComplex Event Processing with Esper
Complex Event Processing with Esper
 
A22 Introduction to DTrace by Kyle Hailey
A22 Introduction to DTrace by Kyle HaileyA22 Introduction to DTrace by Kyle Hailey
A22 Introduction to DTrace by Kyle Hailey
 
The Data Distribution Service Tutorial
The Data Distribution Service TutorialThe Data Distribution Service Tutorial
The Data Distribution Service Tutorial
 
Open source Tools and Frameworks for M2M - Sierra Wireless Developer Days
Open source Tools and Frameworks for M2M - Sierra Wireless Developer DaysOpen source Tools and Frameworks for M2M - Sierra Wireless Developer Days
Open source Tools and Frameworks for M2M - Sierra Wireless Developer Days
 
The DDS Tutorial Part II
The DDS Tutorial Part IIThe DDS Tutorial Part II
The DDS Tutorial Part II
 
Cassandra Day Chicago 2015: Apache Cassandra Data Modeling 101
Cassandra Day Chicago 2015: Apache Cassandra Data Modeling 101Cassandra Day Chicago 2015: Apache Cassandra Data Modeling 101
Cassandra Day Chicago 2015: Apache Cassandra Data Modeling 101
 
Cassandra Day London 2015: Data Modeling 101
Cassandra Day London 2015: Data Modeling 101Cassandra Day London 2015: Data Modeling 101
Cassandra Day London 2015: Data Modeling 101
 
Cassandra Day Atlanta 2015: Data Modeling 101
Cassandra Day Atlanta 2015: Data Modeling 101Cassandra Day Atlanta 2015: Data Modeling 101
Cassandra Day Atlanta 2015: Data Modeling 101
 
Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...
Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...
Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...
 
Introduction to data modeling with apache cassandra
Introduction to data modeling with apache cassandraIntroduction to data modeling with apache cassandra
Introduction to data modeling with apache cassandra
 
CQRS / ES & DDD Demystified
CQRS / ES & DDD DemystifiedCQRS / ES & DDD Demystified
CQRS / ES & DDD Demystified
 
Solaris DTrace, An Introduction
Solaris DTrace, An IntroductionSolaris DTrace, An Introduction
Solaris DTrace, An Introduction
 
Distributed Systems: How to connect your real-time applications
Distributed Systems: How to connect your real-time applicationsDistributed Systems: How to connect your real-time applications
Distributed Systems: How to connect your real-time applications
 
DDS Tutorial -- Part I
DDS Tutorial -- Part IDDS Tutorial -- Part I
DDS Tutorial -- Part I
 
DDS In Action Part II
DDS In Action Part IIDDS In Action Part II
DDS In Action Part II
 
DDS-PSM-Cxx and simd-cxx
DDS-PSM-Cxx and simd-cxxDDS-PSM-Cxx and simd-cxx
DDS-PSM-Cxx and simd-cxx
 
Application Security
Application SecurityApplication Security
Application Security
 
DDS ISO C++ PSM
DDS ISO C++ PSMDDS ISO C++ PSM
DDS ISO C++ PSM
 
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
 

Plus de Angelo Corsaro

zenoh: The Edge Data Fabric
zenoh: The Edge Data Fabriczenoh: The Edge Data Fabric
zenoh: The Edge Data FabricAngelo Corsaro
 
Data Decentralisation: Efficiency, Privacy and Fair Monetisation
Data Decentralisation: Efficiency, Privacy and Fair MonetisationData Decentralisation: Efficiency, Privacy and Fair Monetisation
Data Decentralisation: Efficiency, Privacy and Fair MonetisationAngelo Corsaro
 
zenoh: zero overhead pub/sub store/query compute
zenoh: zero overhead pub/sub store/query computezenoh: zero overhead pub/sub store/query compute
zenoh: zero overhead pub/sub store/query computeAngelo Corsaro
 
zenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocolzenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocolAngelo Corsaro
 
zenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocolzenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocolAngelo Corsaro
 
Breaking the Edge -- A Journey Through Cloud, Edge and Fog Computing
Breaking the Edge -- A Journey Through Cloud, Edge and Fog ComputingBreaking the Edge -- A Journey Through Cloud, Edge and Fog Computing
Breaking the Edge -- A Journey Through Cloud, Edge and Fog ComputingAngelo Corsaro
 
fog05: The Fog Computing Infrastructure
fog05: The Fog Computing Infrastructurefog05: The Fog Computing Infrastructure
fog05: The Fog Computing InfrastructureAngelo Corsaro
 
Cyclone DDS: Sharing Data in the IoT Age
Cyclone DDS: Sharing Data in the IoT AgeCyclone DDS: Sharing Data in the IoT Age
Cyclone DDS: Sharing Data in the IoT AgeAngelo Corsaro
 
fog05: The Fog Computing Platform
fog05: The Fog Computing Platformfog05: The Fog Computing Platform
fog05: The Fog Computing PlatformAngelo Corsaro
 
Programming in Scala - Lecture Four
Programming in Scala - Lecture FourProgramming in Scala - Lecture Four
Programming in Scala - Lecture FourAngelo Corsaro
 
Programming in Scala - Lecture Three
Programming in Scala - Lecture ThreeProgramming in Scala - Lecture Three
Programming in Scala - Lecture ThreeAngelo Corsaro
 
Programming in Scala - Lecture Two
Programming in Scala - Lecture TwoProgramming in Scala - Lecture Two
Programming in Scala - Lecture TwoAngelo Corsaro
 
Programming in Scala - Lecture One
Programming in Scala - Lecture OneProgramming in Scala - Lecture One
Programming in Scala - Lecture OneAngelo Corsaro
 
Data Sharing in Extremely Resource Constrained Envionrments
Data Sharing in Extremely Resource Constrained EnvionrmentsData Sharing in Extremely Resource Constrained Envionrments
Data Sharing in Extremely Resource Constrained EnvionrmentsAngelo Corsaro
 
The DDS Security Standard
The DDS Security StandardThe DDS Security Standard
The DDS Security StandardAngelo Corsaro
 
The Data Distribution Service
The Data Distribution ServiceThe Data Distribution Service
The Data Distribution ServiceAngelo Corsaro
 
RUSTing -- Partially Ordered Rust Programming Ruminations
RUSTing -- Partially Ordered Rust Programming RuminationsRUSTing -- Partially Ordered Rust Programming Ruminations
RUSTing -- Partially Ordered Rust Programming RuminationsAngelo Corsaro
 

Plus de Angelo Corsaro (20)

Zenoh: The Genesis
Zenoh: The GenesisZenoh: The Genesis
Zenoh: The Genesis
 
zenoh: The Edge Data Fabric
zenoh: The Edge Data Fabriczenoh: The Edge Data Fabric
zenoh: The Edge Data Fabric
 
Zenoh Tutorial
Zenoh TutorialZenoh Tutorial
Zenoh Tutorial
 
Data Decentralisation: Efficiency, Privacy and Fair Monetisation
Data Decentralisation: Efficiency, Privacy and Fair MonetisationData Decentralisation: Efficiency, Privacy and Fair Monetisation
Data Decentralisation: Efficiency, Privacy and Fair Monetisation
 
zenoh: zero overhead pub/sub store/query compute
zenoh: zero overhead pub/sub store/query computezenoh: zero overhead pub/sub store/query compute
zenoh: zero overhead pub/sub store/query compute
 
zenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocolzenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocol
 
zenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocolzenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocol
 
Breaking the Edge -- A Journey Through Cloud, Edge and Fog Computing
Breaking the Edge -- A Journey Through Cloud, Edge and Fog ComputingBreaking the Edge -- A Journey Through Cloud, Edge and Fog Computing
Breaking the Edge -- A Journey Through Cloud, Edge and Fog Computing
 
Eastern Sicily
Eastern SicilyEastern Sicily
Eastern Sicily
 
fog05: The Fog Computing Infrastructure
fog05: The Fog Computing Infrastructurefog05: The Fog Computing Infrastructure
fog05: The Fog Computing Infrastructure
 
Cyclone DDS: Sharing Data in the IoT Age
Cyclone DDS: Sharing Data in the IoT AgeCyclone DDS: Sharing Data in the IoT Age
Cyclone DDS: Sharing Data in the IoT Age
 
fog05: The Fog Computing Platform
fog05: The Fog Computing Platformfog05: The Fog Computing Platform
fog05: The Fog Computing Platform
 
Programming in Scala - Lecture Four
Programming in Scala - Lecture FourProgramming in Scala - Lecture Four
Programming in Scala - Lecture Four
 
Programming in Scala - Lecture Three
Programming in Scala - Lecture ThreeProgramming in Scala - Lecture Three
Programming in Scala - Lecture Three
 
Programming in Scala - Lecture Two
Programming in Scala - Lecture TwoProgramming in Scala - Lecture Two
Programming in Scala - Lecture Two
 
Programming in Scala - Lecture One
Programming in Scala - Lecture OneProgramming in Scala - Lecture One
Programming in Scala - Lecture One
 
Data Sharing in Extremely Resource Constrained Envionrments
Data Sharing in Extremely Resource Constrained EnvionrmentsData Sharing in Extremely Resource Constrained Envionrments
Data Sharing in Extremely Resource Constrained Envionrments
 
The DDS Security Standard
The DDS Security StandardThe DDS Security Standard
The DDS Security Standard
 
The Data Distribution Service
The Data Distribution ServiceThe Data Distribution Service
The Data Distribution Service
 
RUSTing -- Partially Ordered Rust Programming Ruminations
RUSTing -- Partially Ordered Rust Programming RuminationsRUSTing -- Partially Ordered Rust Programming Ruminations
RUSTing -- Partially Ordered Rust Programming Ruminations
 

Dernier

Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 

Dernier (20)

Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 

Getting Started with OpenSplice and Esper

  • 1. Getting Started with OpenSplice and Esper Angelo Corsaro, Ph.D. Thomas Bernhardt Chief Technology Officer Chief Technology Officer / Funder OMG DDS Sig Co-Chair EsperTech PrismTech tom@espertech.com angelo.corsaro@prismtech.com
  • 2. Agenda Welcome & Intro !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; ! ! DDS Overview ! CEP Overview ! DDS + CEP in Action ! Q&A
  • 3. DDS Overview ...from a Stream Processing Perspective Angelo CORSARO, Ph.D. Chief Technology Officer OMG DDS Sig Co-Chair PrismTech angelo.corsaro@prismtech.com
  • 4. Stream Processing [1/3] ! Stream Processing is an architectural style for building !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; systems that operate over continuous (theoretically infinite) streams of data ! Stream Processing is often reified under one of its many declinations, such as: ! Reactive Systems ! Signal Processing Systems ! Functional Stream Programming ! Data Flow Systems
  • 5. Stream Processing [2/3] ! The Stream Processing Architecture very !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; naturally models systems reacting to streams of data and events produced by the external world, such as the data produced by sensors, a camera or even the data produced by the stock exchange. ! Stream Processing Systems usually operate in real-time over streams and generate in turns other streams of data providing information on what is happening or suggesting actions to perform, such as by stock X, raise alarm Y, or detected spatial violation, etc.
  • 6. Stream Processing [3/3] ! Stream Processing Systems are typically !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; modeled as collection of modules communicating via typed data channels called usually streams Filter ! Modules usually play one of the Filter following roles: Filter ! Sources: Injecting data into the System ! Filters/Actors: Performing some computation over sources Sink ! Sinks: Consuming the data produced by Stream the system source
  • 8. Defining Streams ! In abstract terms, a stream is an infinite sequence of !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; data samples of a given type T ! Streams can be further classified in continuous and discrete streams. Sometimes referred as Behaviors/ Signals and Events ! In this presentation we’ll refer to Continuous Streams as Data Streams and to Discrete Streams as Event Streams
  • 9. Data Streams Data Streams Temp !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; ! The value of a Data Stream is always defined, i.e. continuous. ! Good examples of a Data Stream are the value assumed by a real-world entity, such as temperature, time pressure, a price, etc.
  • 10. Event Streams Event Streams OverheatAlarm !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; ! The value of the stream is defined at precise point in time, i.e. it is discrete ! Good examples of Event Streams are events in the real world, such a violation of a regulatory compliance, the time temperature higher than a given value, etc.
  • 11. [A Stream Perspective] What is DDS? ! DDS is a high-performance, real-time, highly- !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; available, fully-distributed, messaging technology that allows you to define data/event streams and make them dynamically discoverable ! DDS is equipped with a rich set of QoS providing control on the key temporal and availability properties of data
  • 13. DDS Topics [1/2] “org.opensplice.demo.TTempSensor” ! A Topic defines a stream class/ !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; category ! A Topic has associated a user defined type and QoS ! The Topic name, type and QoS defines the key functional and non-functional invariants ! Topics can be discovered or locally defined struct TempSensor { long Id; DURABILITY, float temp; DEADLINE, float hum; } PRIORITY, #pragma keylist TempSensor id …
  • 14. DDS Topics [2/2] “org.opensplice.demo.TTempSensor” !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; ! DDS Topic types can have associated keys ! Each unique key-value identifies a unique sub- stream of values -- called Topic Instance struct TempSensor { long id; DURABILITY, float temp; DEADLINE, float hum; } PRIORITY, #pragma keylist TempSensor id …
  • 15. “Seeing” Streams !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; id =701 struct TempSensor { id =809 @key long id; Topic float temp; float hum; }; id =977 Instances Instances
  • 17. Anatomy of a DDS Application !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; Domain val dp = DomainParticipant(0) Domain Participant // Create a Publisher / Subscriber Publisher p = dp.create_publisher(); Publisher Topic Subscriber Session Subscriber s = dp.create_subscriber(); // Create a Topic Topic<TempSensor> t = Gives access to a dp.create_topic<TempSensor>(“com.myco.TSTopic”) DDS Domain DataWrter DataReader Reader/Writers User Defined for Types // Create a DataWriter/DataWriter DataWriter<TempSensor> dw = pub.create_datawriter(t); DataReader<TempSensor> dr = sub.create_datareader(t);
  • 18. Anatomy of a DDS Application !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; Domain val dp = DomainParticipant(0) Domain Participant // Create a Publisher / Subscriber val pub = Publisher(dp) Publisher Topic Subscriber Session val sub = Subscriber(dp) // Create a Topic val topic = Topic[TempSensor](dp, “org.opensplice.demo.TTempSensor”) DataWrter DataReader Reader/Writers User Defined for Types Pub/Sub // Create a DataWriter/DataWriter Abstractions DataWriter<TempSensor> dw = pub.create_datawriter(t); DataReader<TempSensor> dr = sub.create_datareader(t);
  • 19. Anatomy of a DDS Application !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; Domain val dp = DomainParticipant(0) Domain Participant // Create a Publisher / Subscriber val pub = Publisher(dp) Publisher Topic Subscriber Session val sub = Subscriber(dp) // Create a Topic val topic = Topic[TempSensor](dp, “org.opensplice.demo.TTempSensor”) DataWrter DataReader Reader/Writers for User Defined for Types // Create a DataWriter/DataWriter val writer = DataWriter[TempSensor](pub, topic) val reader = DataReader[TempSensor](sub, topic) Reader/Writer for application // Write data defined Topic val t = new TempSensor ts(101, 25, 40) writer write ts; Types
  • 20. Data & Event Streams ! DDS does not provide different types for Data/Event Streams. !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; The difference between the two can be made through the DataReader API by using properly using read/take operations ! DataReader::read ! Reads the value of the stream w/o removing it from the stream. As a result multiple read can see the last known value of the stream ! DataReader::take ! Takes the value available on the stream (if any yet) and removes it from the stream
  • 22. Content Filtered Topics ! Content Filtered Topics !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; provide a way of defining a Example: filters over an incoming // Create a Topic (on default domain) stream associated with a val topic = Topic[TempSensor](“TTempSensor”) val ftopic = given topic ContentFilteredTopic[TempSensor](“CFTempSensor”, topic, filter, ! Filters are expressed as the params) “WHERE” clause of an SQL // - filter is a WHERE-like clause, such as: // “temp > 20 AND hum > 50” statement // “temp > %0” // “temp > hum” // “temp BETWEEN (%0 AND %1) ! Filters can operate on any // // - params is the list of parameters to pass to the attribute of the type // filter expression – if any associated with the topic
  • 23. Filter Expression Syntax !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; ! DDS Filters are condition over a topic type attributes ! Temporal properties or causality cannot be captured via DDS filter expression
  • 24. History !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; Data older than “n samples ago” get’s out the window ! DDS provides a way of controlling data future past windows through the History QoS now The window keeps the last n data samples
  • 25. [Putting it All Together] TempSensor Moving Average object MovingAverageFilter { def main(args: Array[String]) { if (args.length < 2) { !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; println("USAGE:ntMovingAverageFilter <window> <filter-expression>") } val topic = Topic[TempSensor]("TTempSensor") val ftopic = ContentFilteredTopic[TempSensor]("CFTempSensor",topic, args(1)) val rqos = DataReaderQos() <= KeepLastHistory(args(0).toInt) val reader = DataReader[TempSensor](ftopic, rqos) reader.reactions += { case e: DataAvailable[_] => { var average: Float = 0 val window = e[TempSensor].reader.history window foreach (average += _.temp) average = average / window.length println("+--------------------------------------------------------") println("Moving Average: " + average) } } }
  • 27. Product Organization Commercial Edition ! No Cost Runtime Licenses !"#$%&'()*+,--.*/%&01234(*5**677*8&'()0*8303%93: ! Your choice of licensing ! LGPL or Commercial ! Subscription or Perpetual ! Complete DDS Implementation Commercial Edition ! Comprehensive Developer and Deployment Support Options with a range of Service Level Agreements Commercial Add-Ons ! Individually licensable technologies Commercial Add-ons ! Rich ecosystem covering tools, integration, testing, etc.
  • 28. Key Points So Far !"#$%&'()*+,-,.*/%&01234(*5**677*8&'()0*8303%93:; ! DDS key abstractions for building stream processing architectures ! DDS provides some event processing capabilities that facilitate the development of Stream Processing Filters ! What else can you use for “Stream Processing” in combination with DDS? ! Let’s have Tom introduce us to the world of CEP and Esper!
  • 31. ! *! +!! # , ! - !-# ! +!! # . / # 0# !
  • 34. 3 3 455 ! # 2 ! 6 ' # ! 2 7$82
  • 35. 1$9( !# # - / !# # ! : !#!-# 2 !!!-# 0# ##2 !#;! )
  • 36. #$ 4 #$ ! ! ! ! # , !! /# = #! '( ) ! ! #! '(
  • 37. 0 ! ! /# ! ! 8 , ##, - ?!@ ! ! A 8 ! B 8 #! !- 8 # A 1
  • 38. ! 0CD% 0CD% % ! ! '! # ( % # :! # E :! # ! ! %F ' !- ! ( G 4 !! E 5 %--53 6 !!# H IJ EJK
  • 39. 8
  • 40. !# #! !#4 #$ 2 #! 2 ! - L !# #! !# #! ! ! 5 ! . !# - ! $# #$ # !#4 ! ! # ! +
  • 41. 5 0C 5 C ', ! # -!= ( !# ; %F ! ! +!! ! #!! # ' - ! ( ! - = +
  • 42. 0C ! C ! - # , M!# # 5 : !#!-# , ! # - !#!-# #! ! # -N !
  • 43. %# #! %# #! [insert into insert_into_def] select select_list from stream_def [as name] [, stream_def [as name]] [,...] [where search_conditions] [group by grouping_expression_list] [having grouping_search_conditions] [output output_specification] [order by order_by_expression_list] %!# select acctId, sum(amount) from Withdrawal.win:time(1 minute) group by acctId having sum(amount) 1000 order by acctId asc 9
  • 44. ! $ O P .! ! $ ! ! ! ! ! 4 C! -# -#= M# $ .- -! # - $ !# 2 ! .% ,= .! !# .! 4
  • 48. /! 4
  • 49. # ! ! $ -! $ !# Q M##, R!!4!S !$ !# ! 4 7
  • 50. !# .! .! $!# M##, - select * from pattern [w=Withdrawal - Deposit(acctId = w.acctId)] ! $ !# w=Withdrawal - (pin=PinChange(acctId = w.acctId) and pwd=PasswordChange(acctId = w.acctId) ) ## ,
  • 52. every ( (w=Withdrawal - Deposit(acctId = w.acctId)) or (d=Deposit - Withdrawal(acctId = d.acctId)) ) T
  • 53. !# .! .! $!# ! ! !# .-
  • 54. ! ! !# every login=LoginEvent - (timer:interval(20 sec) and not LogoutEvent(userId=login.userId)) /!
  • 55. -$ ! ! !# every login=LoginEvent - logout=LogOutEvent where timer:within(2 sec) ! every [2] login=LoginEvent ! every [2..] login=LoginEvent until logout=LogoutEvent
  • 56. .
  • 57. #
  • 58. H
  • 61. !
  • 62. # $% H , $ !! H E2 !! 2 N2 !!# ! 2 N !# ! !2 #, #2 H ! # ! # H : 2 #, #! H % ! !# - 2 2 +!! ! !
  • 63. H . ! !2 # # =2 #!# ! H # ! -!= !!-# ! ! ! H 04 3 !Q % ! ## F% ! H %# ! !# E Q $ -#2
  • 64. !#
  • 65. '
  • 66. ,# *-./ '# ( 0 1 ) 2
  • 67. '
  • 68. : # '% ( (
  • 69. ) *+ !# ! ! ! % ! .+. 6 % ! .+. . ! ! %--
  • 70. ! ! 3! , : !# ! !
  • 71. : !# ! ! F 6 !!# ! !! ! #! !
  • 72. # ! 2 #!# !2 ! !2 # ! -!=
  • 76. DDS + CEP in Action
  • 77. OpenSplice + Esper !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; ! Esper Provides an EsperIO framework for plugging-in new stream transports ! Plugging OpenSplice into Esper is trivial even w/o relying on the EsperIO framework ! Let’s have a look…
  • 79. iShapes Application Spotted shapes represent subscriptions ! To explore play with OpenSplice and Pierced shapes represent publications Esper, we’ll use the simd-cxx ishapes !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; application ! Three Topics ! Circle, Square, Triangle ! One Type: struct ShapeType { string color; long x; long y; long shapesize; }; #pragma keylist Shapetype color
  • 80. Esper Setup !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; Step 1: Register Topic Types val config = new Configuration val ddsConf = new ConfigurationEventTypeLegacy ddsConf.setAccessorStyle(ConfigurationEventTypeLegacy.AccessorStyle.PUBLIC) config.addEventType(ShapeType, classOf[org.opensplice.demo.ShapeType].getName, ddsConf) val cep: EPServiceProvider = EPServiceProviderManager.getDefaultProvider(config)
  • 81. Esper Setup !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; Step 2: Register a Listener for receiving Esper Events val listener = new UpdateListener { def update(ne: Array[EventBean], oe: Array[EventBean]) { ne foreach(e = { ! // Handle the event }) } }
  • 82. Esper Setup !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; Step 3: Hook-up DDS to Esper reader.reactions += { case e: DataAvailable[ShapeType] = { (e.reader read) foreach(runtime sendEvent _) } }
  • 83. iShapes FrameRate !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; ! Let’s suppose that we wanted to keep under control the iShapes Frame rate for ech given color ! In Esper this can be achieved with the following expression: insert into ShapesxSec select color, count(*) as cnt from ShapeType.win:time_batch(1 second) group by color
  • 84. iShapes Center of Mass !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; ! Suppose that we wanted to compute the center of mass of all the shapes currently displayed over the last second ! The Esper expression for this would be: select ShapeFactory.createShape(color, cast(avg(x),int), cast(avg (y),int), shapesize) as NewShape from ShapeType.win:time(10 sec)
  • 85. References !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; #1 OMG DDS Implementation #1 Java-Based CEP Engine Open Source Open Source www.opensplice.org www.espertech.com Fastest growing JVM Language Scala API for OpenSplice DDS Open Source Open Source www.scala-lang.org code.google.com/p/escalier
  • 86. :: Connect with Us :: !#$%'()*+,-,.*/%01234(*5**677*8'()0*8303%93:; ! opensplice.com ! forums.opensplice.org ! @acorsaro ! opensplice.org ! opensplicedds@prismtech.com ! @prismtech ! crc@prismtech.com ! sales@prismtech.com ! youtube.com/opensplicetube ! slideshare.net/angelo.corsaro