SlideShare une entreprise Scribd logo
1  sur  82
Télécharger pour lire hors ligne
epts
event processing technical society




                                                                                  Tutorial v1.1




                                                        epts
                                                        event processing technical society
                                                        Content by: members of the
                                                        EPTS Reference Architecture Working Group



                            Event Processing Architectures
                            leading to an EPTS Reference Architecture
                            Courtesy of:   Adrian Paschke (Freie Universitaet Berlin)
                                           Paul Vincent (TIBCO Software)
                                           Catherine Moxey (IBM)
                                           Alex Alves (Oracle)
                                           Themis Palpanas (University of Trento)
epts
    event processing technical society




Tutorial

       • Event Processing is an increasingly important area in
            the field of IT
       • Event Processing Architectures have evolved to
            handle the needs of low-latency / high-throughput
            event processing
       • Event Processing Architecture diagrams are used to
            describe the functions and component layouts of
            event processing systems
       • Various providers and suppliers use their own
            architectural descriptions, and EPTS has collated and
            refined these into a candidate “reference” architecture
2
epts
    event processing technical society




Agenda

       • Introduction to architectures, architecture
            methodologies, and event processing
       • Member architectures and salient features
       • Skeleton reference architecture from EPTS
            Reference Architecture Working Group
       • Summary and future work of the EPTS Reference
            Architecture Working Group




3
epts
    event processing technical society




Agenda

       • Introduction to architectures, architecture
            methodologies, and event processing
       • Member architectures and salient features
       • Skeleton reference architecture from EPTS
            Reference Architecture Working Group
       • Summary and future work of the EPTS Reference
            Architecture Working Group




4
epts
    event processing technical society




Introduction to Event Processing

       • Event-centric view of IT
       • Events are
            – Sent and Received
            – Aggregated, Transformed into Data, Deleted
            – Processed in queries, rules etc
            – Cause actions like processes, service invocations, etc




5
epts
      event processing technical society



Introduction to Event Processing

                                                                  Event Processing,
                                                              Complex Event Processing,
                                                               Event Stream Processing
                           Event Processing Rules, etc.
 Event Type Definitions,




                                                             Event Abstraction, Event Pattern
                                                              Detection, Event Composition




                                                                                                             Event Management
                                                                           etc. etc.



                                                                Events                 Derived
                                                                                       Events



                                                          Event Producer           Event Consumer
                                                          (Event Source,           (Event sink, event
                                                          Event Emitter)             handler, event
                                                                                       listener)

Design time                                                                Run time                     Administration
epts
     event processing technical society



    About the EPTS Reference Architecture Working Group
•   Started March, 2009
    – 18 members, co-chairs are Adrian Paschke (RuleML) and Paul Vincent (TIBCO)
    – July 09 added responsibilities from Metamodel Working Group
•   Scope
    – Define architecture patterns that are compatible with EPTS members’ Event
      Processing solutions and products.
    – Define terminology and components regarding Event Processing in
      accordance with EPTS
    – Identify and utilize best practices and methods for Technical Architecture
      descriptions and interchange
    – Liaise with relevant standards bodies for EP metamodels and reference
      architectures
•   Current work is focused on
    – Discovery of existing Event Processing Architectures
       • collected RAs from e.g. IBM, Oracle, Tibco, Streambase, Aleri, Microsoft…
    – Definition of Terminology and Methodology for comparing and describing
      Event Processing Reference Architectures
7
epts
      event processing technical society




    Relationship with the other EPTS Groups / Areas
                   •    Application and Analysis
                        – EPTS Use Case WG
                                                   •   Value and ROI
                                                       – EPTS Business Value WG

•   Terminology
    – EPTS Glossary WG


           •    Architecture & Metamodels
                – EPTS Reference Architecture WG


                                                        •   Features and Components
                                                            – EPTS Language WG
epts
    event processing technical society




Reference Architecture and Reference Model

       • Reference Architecture
            A reference architecture models the abstract
            architectural elements in the domain independent of
            the technologies, protocols, and products that are used
            to implement the domain.


       • Reference Model
            A reference model describes the important concepts
            and relationships in the domain focusing on what
            distinguishes the elements of the domain.

9
epts
  event processing technical society



Motivation and Benefits
• Motivation
  – Event Processing is evolving from many existing technologies
    and creating or adapting different tools
  – Potential adopters (stakeholders) may have problems
    understanding and adequately defining EP-based
    architectures and solutions.
• Benefits
  – a Reference Architecture aids efficient Event Processing
    solution development, by predefining
    customizable abstract frames of reference
    for specific stakeholder concerns and application domains.
     • aids in reusability of successful EP architectures for frequently
       occurring EP design problems
     • enables easier comparison of proposed EP solutions
  – Underlying Reference Model defines / explains the
    terminology and components in Event Processing architectures
epts
     event processing technical society




ANSI/IEEE Std 1471 :: ISO/IEC 42010 Methodology

 •     Recommended Practice for Architectural
       Description of Software-intensive Systems
     – Now an ISO/IEC 42010:2007 standard
     – Includes 6 elements
             1. Architectural description
             2. System stakeholders and their concerns
             3. One or more architectural views
             4. Viewpoints
             5. A record of all known inconsistencies among the architectural
                description’s required constituents
             6. A rationale for selection of the architecture

11
epts
     event processing technical society




ISO/IEC 42010:2007 Terminology (1)
        •    Architecture
             – The fundamental organization of a system embodied in its
               components, their relationships to each other, and to the
               environment, and the principles guiding ist design and
               evolution.
        •    Architectural Description
             – A collection of products that document the architecture.
        •    System
             – A collection of components organized to accomplish a
               specific function or set of functions.
        •    System Stakeholder
             – A system stakeholder is an individual, team, or
               organization (or classes thereof) with interests in, or
               concerns relative to, a system
12
epts
     event processing technical society




ISO/IEC 42010:2007 Terminology (2)
     •   View
         – A representation of the whole system from the perspective
           of a related set of concerns.
     •   Viewpoint
         – A specification of the conventions for constructing and
           using a view - a pattern or template which to develop
           individual views by establishing the purposes and audience
           for a view and the techniques for its creation and analysis.

     • Model
         – A view may consist of one or more models and a model
           may participate in one or more views.
         – Each model is defined according to the methods
           established in the corresponding viewpoint definition.

13
epts
     event processing technical society



Conceptual model of architectural description
from IEEE Std 1471- 2000 (ISO/IEC 42010:2007 )




14
epts
     event processing technical society



 Declaring a Viewpoint
 •   Each viewpoint is specified by:
     – Viewpoint name
     – The stakeholders addressed by the viewpoint
     – The stakeholder concerns to be addressed by the viewpoint
     – The viewpoint language, modeling techniques, or analytical methods
       used
     – The source, if any, of the viewpoint (e.g., author, literature citation)
 •   A viewpoint may also include:
     – Any consistency or completeness checks associated with the
       underlying method to be applied to models within the view
     – Any evaluation or analysis techniques to be applied to models within
       the view
     – Any heuristics, patterns, or other guidelines which aid in the synthesis
       of an associated view or its models
15
epts
         event processing technical society




Architecture Views - Examples

     •    Domain Architecture
          domain models represent domain requirement and logic
               • e.g. process representation, use case analysis, free text
     •    Application Architecture
          structural, logic architecture
          – Functional Layering, e.g. functional multi-tier structure
          – Functional Decomposition, e.g. Component Architecture, UML
            (class/activity/sequence)
     •    System Architecture
          – Concrete implementation of an architecture in a target platform


16
epts
  event processing technical society




Example UML Architectural Views and Diagrams
 UML defines 13 diagram models that describe 4+1 architectural views




                                       4+1 architectural views model by Philippe Kruchten, IBM
epts
      event processing technical society




Event Processing Reference Architecture Viewpoints
                                                                 Viewpoint
     Viewpoint                             Engineering EP   Managing EP         Business with EP
     Element                               Architecture     Architecture        Architecture
     Concepts                              How to           How to apply?       How to utilize / sell /
                                           implement?                           own?
     Stakeholders Architects /                              Project Manager Decision Maker,
                  Engineers                                                 Customer, Provider
     Concerns                              Effective        Operational         Strategic and
                                           construction and Management          tactical
                                           deployment                           management
     Techniques /                          Modeling,        IT (service/appl)   Monitoring,
     Languages                             Engineering      management,         Enterprise Decision
                                                            project             Management,
                                                            management          Governance
18
epts
     event processing technical society




Agenda

        • Introduction to architectures, architecture
             methodologies, and event processing
        • Member architectures and salient features
        • Skeleton reference architecture from EPTS
             Reference Architecture Working Group
        • Summary and future work of the EPTS Reference
             Architecture Working Group




19
epts
  event processing technical society




Overview – Sample Member Architectures

     • University of Trento - Themis Palpanas
     • TIBCO – Paul Vincent
     • Oracle - Alex Alves
     • IBM - Catherine Moxey
epts
     event processing technical society




Example Use Case




                                          Courtesy of Opher Etzion
                                            & Peter Niblett, IBM
21
epts
       event processing technical society




Use Case Detail: FFD delivery and bid requests
                                                               GPS
                                        Delivery             Location
                                        Request                         Van
     Florist                                       System

                                                              Bid       Driver
                                                            Request




22
epts
     event processing technical society




FDD Use Case: Stakeholders, Events, and Details




23
epts
  event processing technical society




Overview – Sample Member Architectures

     • University of Trento - Themis Palpanas
     • TIBCO – Paul Vincent
     • Oracle - Alex Alves
     • IBM - Catherine Moxey
epts
  event processing technical society



Entity Name System Service for CEP
Entity Identification

        •    what is the problem?
             – different pieces of data may refer to the same real world entity
                  • entity: person, organization, location, conference, piece of hardware, etc
             – references to entities are often in an event’s payload




                                                                                                 25
epts
  event processing technical society




Entity Identification

        •    what is the problem?
             – different pieces of data may refer to the same real world entity
                  • entity: person, organization, location, conference, piece of hardware, etc
             – references to entities are often in an event’s payload
        •    why is this problem important?
             – incorrect identification of entities may lead to
                  • erroneous decisions in subsequent processing steps
                  • reduced overall performance quality




                                                                                                 26
epts
  event processing technical society




Entity Identification

        •    what is the problem?
             – different pieces of data may refer to the same real world entity
                  • entity: person, organization, location, conference, piece of hardware, etc
             – references to entities are often in an event’s payload
        •    why is this problem important?
             – incorrect identification of entities may lead to
                  • erroneous decisions in subsequent processing steps
                  • reduced overall performance quality
        •    why is this problem relevant?
             – events originate from several different sources
             – sources may refer to same real-world entity in different ways
             – if entity identification problem not tackled, event
               filtering/correlation/etc. will produce wrong results

                                                                                                 27
epts
  event processing technical society




Our solution

        •    the Entity Name System (see References [1][2])
             – scalable infrastructure for assigning and managing unique global
               identifiers for named entities


        •    basic ideas
             – any description of an entity is “resolved” into its global ID
             – not a universal knowledge base about entities
                  • stores minimal amount of information about an entity for
                    – distinguishing entities one from another
                    – finding entities and their identifiers as a result of a query
             – similar to lightweight master data management system




                                                                                      28
epts
  event processing technical society




Entity Name System (ENS)




                                       entity
                                       repository




                                                    29
epts
  event processing technical society




Entity Name System (ENS)




                                              entity     e1 id
                                              repository


                    event                              event
                 source A
                                       …     • e1                 …    event
                                                       source B   e1   source C
                                       e1    •…
                                             •…                   …    30
                                       ...
epts
  event processing technical society




Entity Name System (ENS)
                                              Event Processing
                                                  System




                                              entity     e1 id
                                              repository


                    event                               event
                 source A
                                       …     • e1                  …    event
                                                        source B   e1   source C
                                       e1    •…
                                             •…                    …    31
                                       ...
epts
  event processing technical society



Entity Name System (ENS):
Reference Architecture
                                                          Event Processing
                                                              System


                                       ENS node
                                                                Access Services

                                                 Matching Component

                                                                                    Lifecycle
                                               Storage                             Management
                                                          entity     e1 id
                                             Component
                                                          repository


                    event                                               event
                 source A
                                        …                • e1                       …      event
                                                                        source B    e1     source C
                                       e1                •…
                                                         •…                         …       32
                                       ...
epts
     event processing technical society




Entity Name System Service
                                              event consumer



                                               event modeler




                                          event processing medium



                                          event-entity identification   ENS node



                                               event originator
33
epts
     event processing technical society




Entity Name System Service: Use Case
                                                 …
                                                                            correct entity identification:
                                                                            single driver dispatched for
                                                                                    both events
                                 event processing medium                    customers X and Y
                                                                            locations A and B
                                                                          are two different persons
                                                                           are the same address!

                                                                             delivery
                                  event-entity identification                      ENS node
                                                                          location A(=B)


                                          event originator

            customer X                                  customer Y
                                                                             event entities from
              delivery                                        delivery
     event 1 location A                              event 2 location B    different data sources
34
epts
  event processing technical society




Overview – Sample Member Architectures

     • University of Trento - Themis Palpanas
     • TIBCO – Paul Vincent
     • Oracle - Alex Alves
     • IBM - Catherine Moxey
epts
     event processing technical society


TIBCO Reference Architecture(s)




36
epts
       event processing technical society


TIBCO Reference Architecture(s)



                                            Architect: Functional             End User: Functional
                                            (abstract derivative)                  (abstract)




     Architect: Component                                           Architect: Component
           (abstract)                                               (detailed agent-level)

37
epts
      event processing technical society


     TIBCO Example Architecture
      Input
     Events                                         Complex / Event Processor:
        +
     Content                                          -Event relationships
                                           - Complex Event patterns and sequences
                                                  - (Optional) Inference Rules
                                               - (Optional) Business Decisions        Output
                                                                                      Events
                                                                                    (including
                                                                                        UI)


                                           Services         Input Data


                                              Svc




38
epts
      event processing technical society


 TIBCO Example Architecture for FFD
      Input
     Events                                         Complex / Event Processor:
        +
     Content                                           -Event relationships
                                           - Complex Event patterns and sequences
                                               Operations:
                                                  - (Optional) Inference Rules
 Channels:
 TBA
                                               Enrich Delivery Request
                                               - (Optional) Business Decisions          Output
                                               Create + filter Bid List                 Events
 Events:                                       Create Bid Request
 Delivery Request                                                                     (including
 GPS Location                                                                             UI)

                                                                                       Channels:
                                           Services         Input Data                 TBA
                                                                                       Events:
                                                                                       Bid Request
                                              Svc           Data:                      Rejection
                                                            Florist Minimum Ranking
                                                            Driver Ranking
                                                            Van Region Location
                                                            Van-Driver association
                                                            Bid List, Driver List
39
epts
     event processing technical society


 TIBCO Example Architecture detail
                                                                                                              Agent
                                                                                                        Arrangement
                                           Events and                                                   + Choreography
                                           Data                     Event Operations
                                                             components for detecting patterns
 Input                                     includes
                                                              via queries, rules (ECA, inference,
                                           static
 Channel                                   relationships
                                                                        constraint etc),
 +Destination                              / hierarchies
                                                                orchestrations (process, state),            Display
                                                            with optional reactions and decisions
 +Payload                                  / ontology
 +Metadata
 +Timestamp(s)                                         Complex / Event Processor

                                                                                                           Outbound
                                                                                                           Channel
                                                                                                           + Destination
                                                                                                           + Payload
                                                 Svc
                                                                                                           + Metadata
                                                              Data
                                          Services            (Services)             Event and
                                          + Interfaces        + Interfaces           Data State
                                          +                   +                      Local Store
                                          Choreographies      Choreographies         + Choreographies
                                          required            required               / locks required
40
epts
       event processing technical society


TIBCO Architecture – FFD detail
                                                                                                                   Agent
                                                                                                             Arrangement
                                             Events and                                                        + Choreography
                                                                       Event Operations               Agent structure:
                                             Data
                                                               components for detecting patterns      based on process load /
 Input                                       includes
                                                                via queries, rules (ECA, inference,   freq requirements
                                             static
 Channel                                     relationships
                                                                          constraint etc),
 +Destination
                                                                  orchestrations (process, state),
                                            Data / Event structures:
                                             / hierarchies
                                                                                                                   Display
                                                              with optional reactions and decisions
                                            depends on operations
 +Payload                                    / ontology
 +Metadata                                  needed
 +Timestamp(s)                                           Complex / Event Processor
                                                                         Processing operations:
     Event format:                                                       rules, states etc based on
     depends on channel and                                              operations needed
                                                                                                                 Outbound
     source requirements etc
                                                                                                                 Channel
     TimeToLive :                                                                                                + Destination
     depends on processing                                                                                       + Payload
     approach used                                 Svc
                                                                                                                 + Metadata
                                                                 Data
                                            Services             (Services)
                                                                      Data location:
                                                                                     Event and
                                            + Interfaces         + Interfaces latency, State
                                                                      depends on Data
                                            +                    +                   Local Store
                                                                      + external sharing
                                            Choreographies             requirements + Choreographies
                                                                 Choreographies
                                            required             required           / locks required
41
epts
  event processing technical society




Example Deployment Architecture




                                       Distributed Event Processing Agents – see ref.
epts
  event processing technical society




Overview – Sample Member Architectures

     • University of Trento - Themis Palpanas
     • TIBCO – Paul Vincent
     • Oracle - Alex Alves
     • IBM - Catherine Moxey
epts
      event processing technical society



     Conceptual View

                                                                   publishes                               sends through
                                                                                   Event Producer
                    Event                  catalogued in
                   Repository



                                                           Event
                                                                       processes      Event         uses
                             stored in                                                                         Event Channel
                                                                                     Processor


                   Event
                 Specification             described By

                                                                   receives                           receives through
                                                                                   Event Consumer




                                                                                                                 © 2010 Oracle Corporation
44
epts
        event processing technical society



       Logical View
                                                                              Processor
  EDA is composed of several processing steps intermingled                     Rule
                                                                                 Query
  with user logic.

                                                                  Processor                               Listener
                                             STREAM                Rule                                    (Dash-
                                                                     Query
                                                                                                           board)
                                                                              Processor
       Event Source                      Adapter                               Rule
                                                      Processor                  Query

                                                        Query

                                                        Query

       Data Source                       Adapter



                                                                              Processor                 Listener
                                                                  Cache        Rule                      (Event
                                                                  Rule           Query
                                                                                                           Bus)
This arrangement or network of event processing components is
called an event processing network (EPN).


                                                                                          © 2010 Oracle Corporation
  45
epts
 event processing technical society



Functional View




                                      © 2010 Oracle Corporation
epts
event processing technical society



Deployment View




                                     © 2010 Oracle Corporation
epts
      event processing technical society




     Fast-Flower-Delivery Use-case




                                           © 2010 Oracle Corporation


48
epts
  event processing technical society




Overview – Sample Member Architectures

     • University of Trento - Themis Palpanas
     • TIBCO – Paul Vincent
     • Oracle - Alex Alves
     • IBM - Catherine Moxey
epts
  event processing technical society




IBM Conceptual Architecture for EP
epts
   event processing technical society



  IBM EP Architecture – mapping to methodology
System:                                 event processing system – event emitter, event bus, event handler
Environment:                            initial producers of events and their ultimate consumers
Stakeholders                            Concerns                    Architectural View
Event consumers – use                   Receive correct events in   Event Handler (adapters, consumer-side EP
events to get information,              consumable format, with     services, orchestration), Event Processing
take action, detect problems,           relevant data               Services, Repositories, Event Emitters.
etc.                                                                Publication, Subscription, Notification services to
                                                                    receive events
Event “Analysts” – create               Means to specify the        Event Instantiator, Event Processing Services,
and understand event                    events to be produced,      Event Information Management and Query
specifications and                      the patterns to be          Services, Event Security Services, Event
definitions, how events                 identified, event           Registries. Event Repositories to do pattern-
should be processed, etc.               enrichment etc.             matching over time
Event System managers –                 Security, Performance,      Event Security Services, Event Governance and
manage and control the EP               Configurability             related Security Services
system
Event System operators –                Abilities to monitor and    Event Monitoring and Analytic Infrastructure,
operate the EP system                   configure system,           Event Channels, Publication, Subscription,
                                        diagnose and solve          Notification Services
                                        problems
Event Producers                         Generated events are        Event Instantiator, producer-side EP services,
                                        made use of in some way     Event Adapters, Event Bus…
epts
  event processing technical society




IBM Conceptual Architecture for Decision Maker stakeholder
epts
  event processing technical society



IBM Conceptual Architecture for Architect stakeholder
epts
  event processing technical society

FFD Use Case mapped to IBM RA
Showing some of the Event Flows and Event Processing Agents
epts
     event processing technical society




Agenda

        • Introduction to architectures, architecture
             methodologies, and event processing
        • Member architectures and salient features
        • Skeleton reference architecture from EPTS
             Reference Architecture Working Group
        • Summary and future work of the EPTS Reference
             Architecture Working Group




55
epts
          event processing technical society




Step 1: collect Event Processing Reference Architectures…
                                                                                          EP Network
Events:                                        Downstre                                   (parallel & pipelined)
Tuples,                                           am
Packets                                        Respons
 , Data            Applicati                   e/Applicat
Stream                                            ion
    s
                   on Layer
                      API’s,
                     Plug-ins
                                                                                                                                                                                     Query
                                                                                                                                                                                     Clients
             Data Management                                                                                                           Q                                             Subscription
                   Layer:                                                                                                              Q                                             Clients
                                                                                                                                       Q




                                                                                                                                                                         Networked
                                                                                                              Networked
             StreamSystem
                    Processing




                                                                                                                                                                          Adapter
                                                                                                               Adapter




                                                                                                                                                                                       Output Event
                                                                                                                                                                                       Destinations
                                                                            Input Event
                                                                              Sources
                       Calls
                                                                                                                                              Analysis
                                                                                                                                              Modules




                                                                                                                          In-Process
                    System




                                                                                                                                                            In-Process
                                                                                                                            Adapter




                                                                                                                                                              Adapter
                                                                                                                                             Event
                   Resource                                                                                                                Processin
                                                                                                                                               g
                     Layer                                                                                                                  Engine


                                                                                             Historical & Reference Data                                 Event Storage

                                                                           56                                      Aleri RA
          Streambase RA                                     Microsoft RA



                                                                   …



                          TIBCO RA
                                                                                                                      IBM RA
epts
  event processing technical society




Step 2: identify methodology to abstract an EPTS version…


 Recipe
                                        eventZero RA
                                                              Aleri RA
                                          Streambase RA

                                                       Microsoft RA

                                            TIBCO RA

                                       IBM RA                  Progress RA

                                                                      Oracle RA
epts
  event processing technical society




Step 3: distil the common EPTS Reference Architecture(s)
epts
      event processing technical society




Reference Architecture Viewpoints recap
     Viewpoint                                                   Viewpoint
     Element                               Engineering EP   Managing EP         Business with EP
                                           Architecture     Architecture        Architecture
     Concepts                              How to           How to apply?       How to utilize / sell /
                                           implement?                           own?
     Stakeholders Architects /                              Project Manager Decision Maker,
                  Engineers                                                 Customer, Provider
     Concerns                              Effective        Operational         Strategic and
                                           construction and Management          tactical
                                           deployment                           management
     Techniques /                          Modeling,        IT (service/appl)   Monitoring,
     Languages                             Engineering      management,         Enterprise Decision
                                                            project             Management,
                                                            management          Governance
59
epts
      event processing technical society




Stakeholders and Viewpoints covered

                                                    Viewpoints (for CEP / stateful EP)
     Stakeholder                           Component                  Functional
     Decision Maker                        [Not applicable]           Inputs, Outputs and
     / End User                                                       Processing Requirements
                                                                      View
     Architect /                           Solution Components        Functions carried out in
     Designer                              View                       CEP View




60
epts
      event processing technical society




Stakeholders and Viewpoints covered

                                                    Viewpoints (for CEP / stateful EP)
     Stakeholder                           Component                  Functional
     Decision Maker                        [Not applicable]           Inputs, Outputs and
     / End User                                                       Processing Requirements
                                                                      View
     Architect /                           Solution Components        Functions carried out in
     Designer                              View                       CEP View




61
epts
  event processing technical society




Logical View

     • View of the Event Processing Network?
     • A Network Diagram?
     • Currently under consideration!
epts
     event processing technical society




 Example Logical View




63
     Source: WestGlobal
epts
      event processing technical society




Stakeholders and Viewpoints covered

                                                    Viewpoints (for CEP / stateful EP)
     Stakeholder                           Component                  Functional
     Decision Maker                        [Not applicable]           Inputs, Outputs and
     / End User                                                       Processing Requirements
                                                                      View
     Architect /                           Solution Components        Functions carried out in
     Designer                              View                       CEP View




64
epts
  event processing technical society




Functional View

     • Architect and Developer perspective
          – includes the 3 main functions
            (development, run-time and administration),
          – targets primarily the automated event processing
            operations
     • Run-time functions in 2 main groups:
          – the event infrastructure (sources and consumers)
            external to the event processor under consideration,
          – the event processor.
epts
       event processing technical society




Functional View source: definitions of EP
                                                              Event Processing, Complex
                                                               Event Processing, Event
                                                                 Stream Processing
                           Event Processing Rules, etc.
 Event Type Definitions,




                                                             Event Abstraction, Event Pattern
                                                              Detection, Event Composition




                                                                                                             Event Management
                                                                           etc. etc.



                                                                Events                 Derived
                                                                                       Events



                                                          Event Producer           Event Consumer
                                                          (Event Source,           (Event sink, event
                                                          Event Emitter)             handler, event
                                                                                       listener)

Design time                                                                Run time                     Administration
epts
            event processing technical society




Reference Architecture: Functional View
 Definition, Modeling, (continuous) Improvement
                                                             Event Reaction
                                                      Assessment, Routing, Prediction,
       (Pattern, Control, Rule, Query, RegEx.etc)



                                                            Discovery, Learning




                                                                                                                       Event Process Monitoring, Control
                                                                                         0..*




                                                                                                State Management
                                                                                                State Management
                                                        Complex Event Detection
              Event and Complex Event




                                                                                                                             Process Monitoring, Control
                                                        Consolidation, Composition,
                                                                Aggregation
                                                                                         0..*
                                                              Event Analysis
                                                      Analytics, Transforms, Tracking,
                                                       Scoring, Rating, Classification
                                                                                         0..*
                                                             Event Preparation
                                                     Identification, Selection, Filtering,
                                                          Monitoring, Enrichment
                                                                                         0..*


                                                    Event Production       Event Consumption
                                                       Publication,         Dashboard, Apps,
                                                        Retrieval            External Reaction


Design time                                                         Run time                                       Administration
epts
  event processing technical society




Reference Architecture: Functional View / Runtime

                                              Event Reaction
                                       Assessment, Routing, Prediction,
                                             Discovery, Learning
                                                                           0..*




                                                                                  State Management
                                                                                  State Management
                                          Complex Event Detection
                                          Consolidation, Composition,
                                                  Aggregation
                                                                           0..*
                                                Event Analysis
                                        Analytics, Transforms, Tracking,
                                         Scoring, Rating, Classification
                                                                           0..*
                                               Event Preparation
                                       Identification, Selection, Filtering,
                                            Monitoring, Enrichment
                                                                           0..*


                                   Event Production          Event Consumption
                                      Publication,            Dashboard, Apps,
                                       Retrieval               External Reaction


                                                      Run time
epts
  event processing technical society




Reference Architecture: Functional View / Runtime


Event Production: the source of events for event processing.
• Event Publication: As a part of event production, events may be published
  onto a communication mechanism (eg event bus) for use by event
  consumers (including participants in event processing). This is analagous to
  a "push" system for obtaining events.
• Event Retrieval: As a part of event production, events may be explicitly
  retrieved from some detection system. This is analagous to a "pull" system
  for obtaining events.



                                   Event Production   Event Consumption
                                      Publication,     Dashboard, Apps,
                                       Retrieval        External Reaction
epts
  event processing technical society




Reference Architecture: Functional View / Runtime

Event Consumption: the process of using events from event publication and
processing. Event processing itself can be an event consumer, although for
the purposes of the reference architecture, event consumers are meant to
indicate downstream consumers of events generated in event processing.
• Dashboard: a type of event consumer that displays events as they occur to
  some user community.
• Applications: a type of event consumer if it consumes events for its own
  processes.
• External Reaction: caused through some event consumption, as the result
  of some hardware or software process.

                                   Event Production   Event Consumption
                                      Publication,     Dashboard, Apps,
                                       Retrieval        External Reaction
epts
    event processing technical society




Reference Architecture: Functional View / Runtime
Event Preparation: the process of preparing the event and associated payload and
metadata for further stages of event processing.
• Entity Identification: incoming events will need to be identified relative to prior events, such
  as associating events with particular sources or sensors.
• Event Selection: particular events may be selected for further analysis. Different parts of
  event processing may require different selections of events. See also event filtering.
• Event Filtering: a stream or list of events may be filtered on some payload or metadata
  information such that some subset is selected for further processing.
• Event Monitoring: particular types of events may be monitored for selection for further
  processing. This may utilise specific mechanisms external to the event processing such as
  exploiting event production features.
• Event Enrichment: events may be "enriched" through knowledge gained through previous
  events or data.



          Event Preparation
  Identification, Selection, Filtering,
       Monitoring, Enrichment
epts
   event processing technical society




Reference Architecture: Functional View / Runtime
 Event Analysis: the process of analysing suitably prepared events and their
 payloads and metadata for useful information.
 • Event Analytics: the use of statistical methods to derive additional information about an
   event or set of events.
 • Event Transforms: processes carried out on event payloads or data, either related to
   event preparation, analysis or processing.
 • Event Tracking: where events related to some entity are used to identify state changes in
   that entity.
 • Event Scoring: the process by which events are ranked using a score, usually as a part of
   a statistical analysis of a set of events. See also Event Analytics
 • Event Rating: where events are compared to others to associate some importance or
   other, possibly relative, measurement to the event.
 • Event Classification: where events are associated with some classification scheme for
   use in downstream processing.

          Event Analysis
  Analytics, Transforms, Tracking,
   Scoring, Rating, Classification
epts
   event processing technical society




Reference Architecture: Functional View / Runtime

Complex Event Detection: the process by which event analysis results in the
creation of new event information, or the update of existing complex events.
• Event Consolidation: combining disparate events together into a "main" or "primary"
  event. See also event aggregation.
• Event Composition: composing new, complex events from existing, possibly source,
  events.
• Event Aggregation: combining events to provide new or useful information, such as trend
  information and event statistics. Similar to event consolidation.




    Complex Event Detection
    Consolidation, Composition,
            Aggregation
epts
   event processing technical society




Reference Architecture: Functional View / Runtime
Event Reaction: the process subsequent to event analysis and complex event
detection to handle the results of analysis and detection.
• Event Assessment: the process by which an event is assessed for inclusion in some
  process, incorporation in some other event, etc.
• Event Routing: the process by which an event is redirected to some process, computation
  element, or other event sink.
• Event Prediction: where the reaction to some event processing is that some new event is
  predicted to occur.
• Event Discovery: where the reaction to some event processing is the disclosure of a new,
  typically complex, event type.
      • Note that event prediction is predicting some future event, usually of a known type, whereas event
        discovery is the uncovering of a new event type. See also event-based learning.

• Event-based Learning: the reaction to some event processing that uses new event
  information to add to some, typically statistical-based, understanding of events.
                                                • Note that event-based learning is a specialisation of
                                                  general machine learning and predictive analytics.
        Event Reaction
 Assessment, Routing, Prediction,
       Discovery, Learning
epts
            event processing technical society




Reference Architecture: Functional View / Design time
 Definition, Modeling, (continuous) Improvement

                                                    Covers the definition, modeling, improvement /
       (Pattern, Control, Rule, Query, RegEx.etc)




                                                    maintenance of the artifacts
                                                    used in event processing:
              Event and Complex Event




                                                    • event definitions, including event metadata and payloads,
                                                    • event and event object organisations and structures,
                                                    • event processing transformations / queries / rules /
                                                      procedures / flows / states / decisions /
                                                      expressions (although these can sometimes be
                                                      considered as administrative updates in some situations)




Design time
epts
  event processing technical society




Reference Architecture: Functional View / Administration

   Administrative concepts of




                                                                      Event Process Monitoring, Control
   monitoring and control. This may involve
   • starting and stopping the application and event




                                                                            Process Monitoring, Control
     processing elements, including application monitors
   • providing and updating security levels to event inputs and
     outputs (also can design-time)
   • management of high availability and reliability resources,
     such as hot standby processes
   • resource utilisation monitoring of the event processing
     components
   • process updates, such as how-swapping of event
     processing definitions to newer versions.

                                                                  Administration
epts
     event processing technical society




Agenda

        • Introduction to architectures, architecture
             methodologies, and event processing
        • Member architectures and salient features
        • Skeleton reference architecture from EPTS
             Reference Architecture Working Group
        • Summary and future work of the EPTS Reference
             Architecture Working Group




77
epts
     event processing technical society




Current Works

 •    Terminology and Methodology
      – ANSI/IEEE Std 1471 :: ISO/IEC 42010 Terminology
      – EPTS-RA Terminology
      – Concepts from the EPTS Glossary
 •    Reference Architecture Discovery
 •    Reference Architecture Comparison
      – Identify commonalities in proposed architectures
      – Collect core and additional CEP system functions and components
 •    Member Reference Architecture Descriptions
      – Using the EPTS-RA Methodology and Terminology
 •    Application of RA Descriptions to Use Cases
epts
    event processing technical society




Next Steps
•   Generalize the EPTS Reference Architecture descriptions
    – addressing important stakeholders and their views
    – using rigorous RA methodology and terminology (+ glossary)
•   Evaluate the EPTS-RA descriptions
    – apply on selected EPTS use cases and compare to member experience
    – compare with Best Practice Guidelines
•   Review Logical Architecture requirements for Reference purposes
•   Possible Outputs:
    – EPTS-RA Description document
    – EPTS-RA Design Patterns and Best Practice Guidelines document
    – Wiley Book "Pattern Oriented Software Architecture: Architectures,
      Models and Patterns for Event Processing" (already in preparation)
    – Input for Event Processing Metamodels and associated standards
epts
     event processing technical society




Summary

        • Reference Architecture provides a common set of
             EP Functions that may be included in
             – EP systems
             – EP-related tools
        • Describes the mapping from Glossary of Terms to
             Implementation
        • Provides the basis for EP agents, operators and
             languages



80
epts
     event processing technical society




           Thank you !

                                          Acknowledgment to the Event Processing Technical Society
                                          Reference Architecture working group: David Tucker (Event
                                          Zero), John Morrell (Coral8, Inc.), Baden Hughes (Event Zero),
                                          Dieter Gawlick (Oracle), Alex Alves (Oracle), Anand Srinivasan
                                          (Oracle), Shailendra Mishra (Oracle), Zbigniew Jerzak (TU
                                          Dresden), Hans-Arno Jacobsen (University Toronto), Albert
                                          Mavashev (Nastel Technologies Inc.), Simon Courtenage
                                          (University of Westminster), Catherine Moxey (IBM United
                                          Kingdom Limited), Richard Tibbetts (Streambase), Brian Connell
                                          (WestGlobal), Guy Sharon (IBM), Themis Palpanas (University
                                          of Trento)
81
epts
    event processing technical society




References
•   EPTS Glossary
     http://www.ep-ts.com/component/option,com_docman/task,doc_download/gid,66/Itemid,84/

•   University of Trento Architecture section:
    1. Heiko Stoermer, Themis Palpanas, George Giannakopoulos. The Entity Name System:
       Enabling the Web of Entities. International Workshop on Data Engineering meets the
       Semantic Web (DESWeb), in conjunction with the IEEE International Conference on Data
       Engineering (ICDE), Long Beach, CA, USA, March 2010
    2. Zoltan Miklos, Nicolas Bonvin, Paolo Bouquet, Michele Catasta, Daniele Cordioli, Peter
       Fankhauser, Julien Gaugaz, Ekaterini Ioannou, Hristo Koshutanski, Antonio Mana,
       Claudia Niederee, Themis Palpanas, Heiko Stoermer. From Web Data to Entities and
       Back. International Conference on Advanced Information Systems Engineering (CAiSE),
       Hammamet, Tunisia, June 2010
•   IBM Architecture section:
    Conceptual Architecture paper: http://www.redbooks.ibm.com/redpapers/pdfs/redp4642.pdf
•   TIBCO Architecture section:
    Sergio Gonik & Dan Buchko, BusinessEvents Distributed Processing
    Implementation Patterns, Tips, and Techniques, TUCON TIBCO User Conference, Las
    Vegas, May 2010

Contenu connexe

En vedette

Hive integration: HBase and Rcfile__HadoopSummit2010
Hive integration: HBase and Rcfile__HadoopSummit2010Hive integration: HBase and Rcfile__HadoopSummit2010
Hive integration: HBase and Rcfile__HadoopSummit2010Yahoo Developer Network
 
UW Presentation - Architecture Trade-off Analysis Method
UW Presentation - Architecture Trade-off Analysis MethodUW Presentation - Architecture Trade-off Analysis Method
UW Presentation - Architecture Trade-off Analysis MethodShrikant Palkar
 
A Reference Architecture for IoT: How to create a resilient, secure IoT cloud
A Reference Architecture for IoT: How to create a resilient, secure IoT cloudA Reference Architecture for IoT: How to create a resilient, secure IoT cloud
A Reference Architecture for IoT: How to create a resilient, secure IoT cloudWSO2
 
SCM Patterns for Agile Architectures
SCM Patterns for Agile ArchitecturesSCM Patterns for Agile Architectures
SCM Patterns for Agile ArchitecturesBrad Appleton
 
Introduction to ARCHITECTURAL LANGUAGES
Introduction to ARCHITECTURAL LANGUAGESIntroduction to ARCHITECTURAL LANGUAGES
Introduction to ARCHITECTURAL LANGUAGESIvano Malavolta
 
Software Architecture Views and Viewpoints
Software Architecture Views and ViewpointsSoftware Architecture Views and Viewpoints
Software Architecture Views and ViewpointsHenry Muccini
 
Fundamentos de la arquitectura de software
Fundamentos de la arquitectura de softwareFundamentos de la arquitectura de software
Fundamentos de la arquitectura de softwareRoger Villegas
 
Arquitecturas de software - Parte 1
Arquitecturas de software - Parte 1Arquitecturas de software - Parte 1
Arquitecturas de software - Parte 1Marta Silvia Tabares
 
Arquitectura de software
Arquitectura de softwareArquitectura de software
Arquitectura de softwareLiliana Pacheco
 
A Reference Architecture for ETL 2.0
A Reference Architecture for ETL 2.0 A Reference Architecture for ETL 2.0
A Reference Architecture for ETL 2.0 DataWorks Summit
 
Reference architecture for Internet of Things
Reference architecture for Internet of ThingsReference architecture for Internet of Things
Reference architecture for Internet of ThingsSujee Maniyam
 
Open API and API Management - Introduction and Comparison of Products: TIBCO ...
Open API and API Management - Introduction and Comparison of Products: TIBCO ...Open API and API Management - Introduction and Comparison of Products: TIBCO ...
Open API and API Management - Introduction and Comparison of Products: TIBCO ...Kai Wähner
 
A reference architecture for the internet of things
A reference architecture for the internet of thingsA reference architecture for the internet of things
A reference architecture for the internet of thingsCharles Gibbons
 
A Reference Architecture for IoT
A Reference Architecture for IoT A Reference Architecture for IoT
A Reference Architecture for IoT WSO2
 
Structured Vs, Object Oriented Analysis and Design
Structured Vs, Object Oriented Analysis and DesignStructured Vs, Object Oriented Analysis and Design
Structured Vs, Object Oriented Analysis and DesignMotaz Saad
 
Semantic Complex Event Processing
Semantic Complex Event ProcessingSemantic Complex Event Processing
Semantic Complex Event ProcessingAdrian Paschke
 

En vedette (19)

Hive integration: HBase and Rcfile__HadoopSummit2010
Hive integration: HBase and Rcfile__HadoopSummit2010Hive integration: HBase and Rcfile__HadoopSummit2010
Hive integration: HBase and Rcfile__HadoopSummit2010
 
Sdd template
Sdd templateSdd template
Sdd template
 
factsheet cha chà_September 2009.doc.pdf
factsheet cha chà_September 2009.doc.pdffactsheet cha chà_September 2009.doc.pdf
factsheet cha chà_September 2009.doc.pdf
 
UW Presentation - Architecture Trade-off Analysis Method
UW Presentation - Architecture Trade-off Analysis MethodUW Presentation - Architecture Trade-off Analysis Method
UW Presentation - Architecture Trade-off Analysis Method
 
A Reference Architecture for IoT: How to create a resilient, secure IoT cloud
A Reference Architecture for IoT: How to create a resilient, secure IoT cloudA Reference Architecture for IoT: How to create a resilient, secure IoT cloud
A Reference Architecture for IoT: How to create a resilient, secure IoT cloud
 
SCM Patterns for Agile Architectures
SCM Patterns for Agile ArchitecturesSCM Patterns for Agile Architectures
SCM Patterns for Agile Architectures
 
Introduction to ARCHITECTURAL LANGUAGES
Introduction to ARCHITECTURAL LANGUAGESIntroduction to ARCHITECTURAL LANGUAGES
Introduction to ARCHITECTURAL LANGUAGES
 
Software Architecture Views and Viewpoints
Software Architecture Views and ViewpointsSoftware Architecture Views and Viewpoints
Software Architecture Views and Viewpoints
 
Fundamentos de la arquitectura de software
Fundamentos de la arquitectura de softwareFundamentos de la arquitectura de software
Fundamentos de la arquitectura de software
 
Arquitecturas de software - Parte 1
Arquitecturas de software - Parte 1Arquitecturas de software - Parte 1
Arquitecturas de software - Parte 1
 
Arquitectura de software
Arquitectura de softwareArquitectura de software
Arquitectura de software
 
A Reference Architecture for ETL 2.0
A Reference Architecture for ETL 2.0 A Reference Architecture for ETL 2.0
A Reference Architecture for ETL 2.0
 
Reference architecture for Internet of Things
Reference architecture for Internet of ThingsReference architecture for Internet of Things
Reference architecture for Internet of Things
 
Open API and API Management - Introduction and Comparison of Products: TIBCO ...
Open API and API Management - Introduction and Comparison of Products: TIBCO ...Open API and API Management - Introduction and Comparison of Products: TIBCO ...
Open API and API Management - Introduction and Comparison of Products: TIBCO ...
 
A reference architecture for the internet of things
A reference architecture for the internet of thingsA reference architecture for the internet of things
A reference architecture for the internet of things
 
A Reference Architecture for IoT
A Reference Architecture for IoT A Reference Architecture for IoT
A Reference Architecture for IoT
 
Structured Vs, Object Oriented Analysis and Design
Structured Vs, Object Oriented Analysis and DesignStructured Vs, Object Oriented Analysis and Design
Structured Vs, Object Oriented Analysis and Design
 
4+1 view model
4+1 view model4+1 view model
4+1 view model
 
Semantic Complex Event Processing
Semantic Complex Event ProcessingSemantic Complex Event Processing
Semantic Complex Event Processing
 

Similaire à Debs2010 tutorial on epts reference architecture v1.1c

Software architecture simplified
Software architecture simplifiedSoftware architecture simplified
Software architecture simplifiedPrasad Chitta
 
Event Driven Architecture : Managing Business Dynamics for Adaptive Enterprise
Event Driven Architecture : Managing Business Dynamics for Adaptive EnterpriseEvent Driven Architecture : Managing Business Dynamics for Adaptive Enterprise
Event Driven Architecture : Managing Business Dynamics for Adaptive EnterpriseDassana Wijesekara
 
Event Driven Architecture : Managing Business Dynamics for an Adaptive Enterp...
Event Driven Architecture : Managing Business Dynamics for an Adaptive Enterp...Event Driven Architecture : Managing Business Dynamics for an Adaptive Enterp...
Event Driven Architecture : Managing Business Dynamics for an Adaptive Enterp...Dassana Wijesekara
 
Open Data Center Alliance Workgroups, Usage Models and Roadmap Structure
Open Data Center Alliance Workgroups, Usage Models and Roadmap StructureOpen Data Center Alliance Workgroups, Usage Models and Roadmap Structure
Open Data Center Alliance Workgroups, Usage Models and Roadmap StructureOpen Data Center Alliance
 
Arch06 1
Arch06 1Arch06 1
Arch06 1nazn
 
NISI Agile Software Architecture Slide Deck
NISI Agile Software Architecture Slide DeckNISI Agile Software Architecture Slide Deck
NISI Agile Software Architecture Slide DeckUtrecht University
 
Icalt2010 hoel hollins
Icalt2010 hoel hollinsIcalt2010 hoel hollins
Icalt2010 hoel hollinsTore Hoel
 
ESSENSE – A Kernel of Essentials for Software Engineering
ESSENSE – A Kernel of Essentials for Software EngineeringESSENSE – A Kernel of Essentials for Software Engineering
ESSENSE – A Kernel of Essentials for Software EngineeringBrian Elvesæter
 
Fostering MBSE in Engineering Culture
Fostering MBSE in Engineering CultureFostering MBSE in Engineering Culture
Fostering MBSE in Engineering CultureObeo
 
Modular Documentation Joe Gelb Techshoret 2009
Modular Documentation Joe Gelb Techshoret 2009Modular Documentation Joe Gelb Techshoret 2009
Modular Documentation Joe Gelb Techshoret 2009Suite Solutions
 
EDA With Glassfish ESB Jfall IEP Intelligent Event Processing
EDA With Glassfish ESB Jfall IEP Intelligent Event ProcessingEDA With Glassfish ESB Jfall IEP Intelligent Event Processing
EDA With Glassfish ESB Jfall IEP Intelligent Event ProcessingEugene Bogaart
 
Towards tool support for situational engineering of agile methodology
Towards tool support for situational engineering of agile methodologyTowards tool support for situational engineering of agile methodology
Towards tool support for situational engineering of agile methodologySandhiya Rajagopal
 
Knowledge-Based Analysis and Design (KBAD): An Approach to Rapid Systems Engi...
Knowledge-Based Analysis and Design (KBAD): An Approach to Rapid Systems Engi...Knowledge-Based Analysis and Design (KBAD): An Approach to Rapid Systems Engi...
Knowledge-Based Analysis and Design (KBAD): An Approach to Rapid Systems Engi...Elizabeth Steiner
 
An Introductory Session on Enterprise Architecture
An Introductory Session on Enterprise ArchitectureAn Introductory Session on Enterprise Architecture
An Introductory Session on Enterprise Architecturekeshav2011
 
CASE tools_Se lect15 btech
CASE tools_Se lect15 btechCASE tools_Se lect15 btech
CASE tools_Se lect15 btechIIITA
 
Thoughts On Architecting V4 2
Thoughts On Architecting V4 2Thoughts On Architecting V4 2
Thoughts On Architecting V4 2bmercer
 
Technical Architecture
Technical ArchitectureTechnical Architecture
Technical Architecturescmiyer
 
Software Architecture
Software ArchitectureSoftware Architecture
Software ArchitectureVikas Dhyani
 

Similaire à Debs2010 tutorial on epts reference architecture v1.1c (20)

Software architecture simplified
Software architecture simplifiedSoftware architecture simplified
Software architecture simplified
 
Event Driven Architecture : Managing Business Dynamics for Adaptive Enterprise
Event Driven Architecture : Managing Business Dynamics for Adaptive EnterpriseEvent Driven Architecture : Managing Business Dynamics for Adaptive Enterprise
Event Driven Architecture : Managing Business Dynamics for Adaptive Enterprise
 
Event Driven Architecture : Managing Business Dynamics for an Adaptive Enterp...
Event Driven Architecture : Managing Business Dynamics for an Adaptive Enterp...Event Driven Architecture : Managing Business Dynamics for an Adaptive Enterp...
Event Driven Architecture : Managing Business Dynamics for an Adaptive Enterp...
 
Open Data Center Alliance Workgroups, Usage Models and Roadmap Structure
Open Data Center Alliance Workgroups, Usage Models and Roadmap StructureOpen Data Center Alliance Workgroups, Usage Models and Roadmap Structure
Open Data Center Alliance Workgroups, Usage Models and Roadmap Structure
 
Arch06 1
Arch06 1Arch06 1
Arch06 1
 
NISI Agile Software Architecture Slide Deck
NISI Agile Software Architecture Slide DeckNISI Agile Software Architecture Slide Deck
NISI Agile Software Architecture Slide Deck
 
Icalt2010 hoel hollins
Icalt2010 hoel hollinsIcalt2010 hoel hollins
Icalt2010 hoel hollins
 
ESSENSE – A Kernel of Essentials for Software Engineering
ESSENSE – A Kernel of Essentials for Software EngineeringESSENSE – A Kernel of Essentials for Software Engineering
ESSENSE – A Kernel of Essentials for Software Engineering
 
Fostering MBSE in Engineering Culture
Fostering MBSE in Engineering CultureFostering MBSE in Engineering Culture
Fostering MBSE in Engineering Culture
 
Modular Documentation Joe Gelb Techshoret 2009
Modular Documentation Joe Gelb Techshoret 2009Modular Documentation Joe Gelb Techshoret 2009
Modular Documentation Joe Gelb Techshoret 2009
 
EDA With Glassfish ESB Jfall IEP Intelligent Event Processing
EDA With Glassfish ESB Jfall IEP Intelligent Event ProcessingEDA With Glassfish ESB Jfall IEP Intelligent Event Processing
EDA With Glassfish ESB Jfall IEP Intelligent Event Processing
 
Towards tool support for situational engineering of agile methodology
Towards tool support for situational engineering of agile methodologyTowards tool support for situational engineering of agile methodology
Towards tool support for situational engineering of agile methodology
 
Knowledge-Based Analysis and Design (KBAD): An Approach to Rapid Systems Engi...
Knowledge-Based Analysis and Design (KBAD): An Approach to Rapid Systems Engi...Knowledge-Based Analysis and Design (KBAD): An Approach to Rapid Systems Engi...
Knowledge-Based Analysis and Design (KBAD): An Approach to Rapid Systems Engi...
 
An Introductory Session on Enterprise Architecture
An Introductory Session on Enterprise ArchitectureAn Introductory Session on Enterprise Architecture
An Introductory Session on Enterprise Architecture
 
11.ppt
11.ppt11.ppt
11.ppt
 
CASE tools_Se lect15 btech
CASE tools_Se lect15 btechCASE tools_Se lect15 btech
CASE tools_Se lect15 btech
 
Thoughts On Architecting V4 2
Thoughts On Architecting V4 2Thoughts On Architecting V4 2
Thoughts On Architecting V4 2
 
Technical Architecture
Technical ArchitectureTechnical Architecture
Technical Architecture
 
Brochure for pmvt
Brochure for pmvtBrochure for pmvt
Brochure for pmvt
 
Software Architecture
Software ArchitectureSoftware Architecture
Software Architecture
 

Dernier

Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 

Dernier (20)

Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 

Debs2010 tutorial on epts reference architecture v1.1c

  • 1. epts event processing technical society Tutorial v1.1 epts event processing technical society Content by: members of the EPTS Reference Architecture Working Group Event Processing Architectures leading to an EPTS Reference Architecture Courtesy of: Adrian Paschke (Freie Universitaet Berlin) Paul Vincent (TIBCO Software) Catherine Moxey (IBM) Alex Alves (Oracle) Themis Palpanas (University of Trento)
  • 2. epts event processing technical society Tutorial • Event Processing is an increasingly important area in the field of IT • Event Processing Architectures have evolved to handle the needs of low-latency / high-throughput event processing • Event Processing Architecture diagrams are used to describe the functions and component layouts of event processing systems • Various providers and suppliers use their own architectural descriptions, and EPTS has collated and refined these into a candidate “reference” architecture 2
  • 3. epts event processing technical society Agenda • Introduction to architectures, architecture methodologies, and event processing • Member architectures and salient features • Skeleton reference architecture from EPTS Reference Architecture Working Group • Summary and future work of the EPTS Reference Architecture Working Group 3
  • 4. epts event processing technical society Agenda • Introduction to architectures, architecture methodologies, and event processing • Member architectures and salient features • Skeleton reference architecture from EPTS Reference Architecture Working Group • Summary and future work of the EPTS Reference Architecture Working Group 4
  • 5. epts event processing technical society Introduction to Event Processing • Event-centric view of IT • Events are – Sent and Received – Aggregated, Transformed into Data, Deleted – Processed in queries, rules etc – Cause actions like processes, service invocations, etc 5
  • 6. epts event processing technical society Introduction to Event Processing Event Processing, Complex Event Processing, Event Stream Processing Event Processing Rules, etc. Event Type Definitions, Event Abstraction, Event Pattern Detection, Event Composition Event Management etc. etc. Events Derived Events Event Producer Event Consumer (Event Source, (Event sink, event Event Emitter) handler, event listener) Design time Run time Administration
  • 7. epts event processing technical society About the EPTS Reference Architecture Working Group • Started March, 2009 – 18 members, co-chairs are Adrian Paschke (RuleML) and Paul Vincent (TIBCO) – July 09 added responsibilities from Metamodel Working Group • Scope – Define architecture patterns that are compatible with EPTS members’ Event Processing solutions and products. – Define terminology and components regarding Event Processing in accordance with EPTS – Identify and utilize best practices and methods for Technical Architecture descriptions and interchange – Liaise with relevant standards bodies for EP metamodels and reference architectures • Current work is focused on – Discovery of existing Event Processing Architectures • collected RAs from e.g. IBM, Oracle, Tibco, Streambase, Aleri, Microsoft… – Definition of Terminology and Methodology for comparing and describing Event Processing Reference Architectures 7
  • 8. epts event processing technical society Relationship with the other EPTS Groups / Areas • Application and Analysis – EPTS Use Case WG • Value and ROI – EPTS Business Value WG • Terminology – EPTS Glossary WG • Architecture & Metamodels – EPTS Reference Architecture WG • Features and Components – EPTS Language WG
  • 9. epts event processing technical society Reference Architecture and Reference Model • Reference Architecture A reference architecture models the abstract architectural elements in the domain independent of the technologies, protocols, and products that are used to implement the domain. • Reference Model A reference model describes the important concepts and relationships in the domain focusing on what distinguishes the elements of the domain. 9
  • 10. epts event processing technical society Motivation and Benefits • Motivation – Event Processing is evolving from many existing technologies and creating or adapting different tools – Potential adopters (stakeholders) may have problems understanding and adequately defining EP-based architectures and solutions. • Benefits – a Reference Architecture aids efficient Event Processing solution development, by predefining customizable abstract frames of reference for specific stakeholder concerns and application domains. • aids in reusability of successful EP architectures for frequently occurring EP design problems • enables easier comparison of proposed EP solutions – Underlying Reference Model defines / explains the terminology and components in Event Processing architectures
  • 11. epts event processing technical society ANSI/IEEE Std 1471 :: ISO/IEC 42010 Methodology • Recommended Practice for Architectural Description of Software-intensive Systems – Now an ISO/IEC 42010:2007 standard – Includes 6 elements 1. Architectural description 2. System stakeholders and their concerns 3. One or more architectural views 4. Viewpoints 5. A record of all known inconsistencies among the architectural description’s required constituents 6. A rationale for selection of the architecture 11
  • 12. epts event processing technical society ISO/IEC 42010:2007 Terminology (1) • Architecture – The fundamental organization of a system embodied in its components, their relationships to each other, and to the environment, and the principles guiding ist design and evolution. • Architectural Description – A collection of products that document the architecture. • System – A collection of components organized to accomplish a specific function or set of functions. • System Stakeholder – A system stakeholder is an individual, team, or organization (or classes thereof) with interests in, or concerns relative to, a system 12
  • 13. epts event processing technical society ISO/IEC 42010:2007 Terminology (2) • View – A representation of the whole system from the perspective of a related set of concerns. • Viewpoint – A specification of the conventions for constructing and using a view - a pattern or template which to develop individual views by establishing the purposes and audience for a view and the techniques for its creation and analysis. • Model – A view may consist of one or more models and a model may participate in one or more views. – Each model is defined according to the methods established in the corresponding viewpoint definition. 13
  • 14. epts event processing technical society Conceptual model of architectural description from IEEE Std 1471- 2000 (ISO/IEC 42010:2007 ) 14
  • 15. epts event processing technical society Declaring a Viewpoint • Each viewpoint is specified by: – Viewpoint name – The stakeholders addressed by the viewpoint – The stakeholder concerns to be addressed by the viewpoint – The viewpoint language, modeling techniques, or analytical methods used – The source, if any, of the viewpoint (e.g., author, literature citation) • A viewpoint may also include: – Any consistency or completeness checks associated with the underlying method to be applied to models within the view – Any evaluation or analysis techniques to be applied to models within the view – Any heuristics, patterns, or other guidelines which aid in the synthesis of an associated view or its models 15
  • 16. epts event processing technical society Architecture Views - Examples • Domain Architecture domain models represent domain requirement and logic • e.g. process representation, use case analysis, free text • Application Architecture structural, logic architecture – Functional Layering, e.g. functional multi-tier structure – Functional Decomposition, e.g. Component Architecture, UML (class/activity/sequence) • System Architecture – Concrete implementation of an architecture in a target platform 16
  • 17. epts event processing technical society Example UML Architectural Views and Diagrams UML defines 13 diagram models that describe 4+1 architectural views 4+1 architectural views model by Philippe Kruchten, IBM
  • 18. epts event processing technical society Event Processing Reference Architecture Viewpoints Viewpoint Viewpoint Engineering EP Managing EP Business with EP Element Architecture Architecture Architecture Concepts How to How to apply? How to utilize / sell / implement? own? Stakeholders Architects / Project Manager Decision Maker, Engineers Customer, Provider Concerns Effective Operational Strategic and construction and Management tactical deployment management Techniques / Modeling, IT (service/appl) Monitoring, Languages Engineering management, Enterprise Decision project Management, management Governance 18
  • 19. epts event processing technical society Agenda • Introduction to architectures, architecture methodologies, and event processing • Member architectures and salient features • Skeleton reference architecture from EPTS Reference Architecture Working Group • Summary and future work of the EPTS Reference Architecture Working Group 19
  • 20. epts event processing technical society Overview – Sample Member Architectures • University of Trento - Themis Palpanas • TIBCO – Paul Vincent • Oracle - Alex Alves • IBM - Catherine Moxey
  • 21. epts event processing technical society Example Use Case Courtesy of Opher Etzion & Peter Niblett, IBM 21
  • 22. epts event processing technical society Use Case Detail: FFD delivery and bid requests GPS Delivery Location Request Van Florist System Bid Driver Request 22
  • 23. epts event processing technical society FDD Use Case: Stakeholders, Events, and Details 23
  • 24. epts event processing technical society Overview – Sample Member Architectures • University of Trento - Themis Palpanas • TIBCO – Paul Vincent • Oracle - Alex Alves • IBM - Catherine Moxey
  • 25. epts event processing technical society Entity Name System Service for CEP Entity Identification • what is the problem? – different pieces of data may refer to the same real world entity • entity: person, organization, location, conference, piece of hardware, etc – references to entities are often in an event’s payload 25
  • 26. epts event processing technical society Entity Identification • what is the problem? – different pieces of data may refer to the same real world entity • entity: person, organization, location, conference, piece of hardware, etc – references to entities are often in an event’s payload • why is this problem important? – incorrect identification of entities may lead to • erroneous decisions in subsequent processing steps • reduced overall performance quality 26
  • 27. epts event processing technical society Entity Identification • what is the problem? – different pieces of data may refer to the same real world entity • entity: person, organization, location, conference, piece of hardware, etc – references to entities are often in an event’s payload • why is this problem important? – incorrect identification of entities may lead to • erroneous decisions in subsequent processing steps • reduced overall performance quality • why is this problem relevant? – events originate from several different sources – sources may refer to same real-world entity in different ways – if entity identification problem not tackled, event filtering/correlation/etc. will produce wrong results 27
  • 28. epts event processing technical society Our solution • the Entity Name System (see References [1][2]) – scalable infrastructure for assigning and managing unique global identifiers for named entities • basic ideas – any description of an entity is “resolved” into its global ID – not a universal knowledge base about entities • stores minimal amount of information about an entity for – distinguishing entities one from another – finding entities and their identifiers as a result of a query – similar to lightweight master data management system 28
  • 29. epts event processing technical society Entity Name System (ENS) entity repository 29
  • 30. epts event processing technical society Entity Name System (ENS) entity e1 id repository event event source A … • e1 … event source B e1 source C e1 •… •… … 30 ...
  • 31. epts event processing technical society Entity Name System (ENS) Event Processing System entity e1 id repository event event source A … • e1 … event source B e1 source C e1 •… •… … 31 ...
  • 32. epts event processing technical society Entity Name System (ENS): Reference Architecture Event Processing System ENS node Access Services Matching Component Lifecycle Storage Management entity e1 id Component repository event event source A … • e1 … event source B e1 source C e1 •… •… … 32 ...
  • 33. epts event processing technical society Entity Name System Service event consumer event modeler event processing medium event-entity identification ENS node event originator 33
  • 34. epts event processing technical society Entity Name System Service: Use Case … correct entity identification: single driver dispatched for both events event processing medium customers X and Y locations A and B are two different persons are the same address! delivery event-entity identification ENS node location A(=B) event originator customer X customer Y event entities from delivery delivery event 1 location A event 2 location B different data sources 34
  • 35. epts event processing technical society Overview – Sample Member Architectures • University of Trento - Themis Palpanas • TIBCO – Paul Vincent • Oracle - Alex Alves • IBM - Catherine Moxey
  • 36. epts event processing technical society TIBCO Reference Architecture(s) 36
  • 37. epts event processing technical society TIBCO Reference Architecture(s) Architect: Functional End User: Functional (abstract derivative) (abstract) Architect: Component Architect: Component (abstract) (detailed agent-level) 37
  • 38. epts event processing technical society TIBCO Example Architecture Input Events Complex / Event Processor: + Content -Event relationships - Complex Event patterns and sequences - (Optional) Inference Rules - (Optional) Business Decisions Output Events (including UI) Services Input Data Svc 38
  • 39. epts event processing technical society TIBCO Example Architecture for FFD Input Events Complex / Event Processor: + Content -Event relationships - Complex Event patterns and sequences Operations: - (Optional) Inference Rules Channels: TBA Enrich Delivery Request - (Optional) Business Decisions Output Create + filter Bid List Events Events: Create Bid Request Delivery Request (including GPS Location UI) Channels: Services Input Data TBA Events: Bid Request Svc Data: Rejection Florist Minimum Ranking Driver Ranking Van Region Location Van-Driver association Bid List, Driver List 39
  • 40. epts event processing technical society TIBCO Example Architecture detail Agent Arrangement Events and + Choreography Data Event Operations components for detecting patterns Input includes via queries, rules (ECA, inference, static Channel relationships constraint etc), +Destination / hierarchies orchestrations (process, state), Display with optional reactions and decisions +Payload / ontology +Metadata +Timestamp(s) Complex / Event Processor Outbound Channel + Destination + Payload Svc + Metadata Data Services (Services) Event and + Interfaces + Interfaces Data State + + Local Store Choreographies Choreographies + Choreographies required required / locks required 40
  • 41. epts event processing technical society TIBCO Architecture – FFD detail Agent Arrangement Events and + Choreography Event Operations Agent structure: Data components for detecting patterns based on process load / Input includes via queries, rules (ECA, inference, freq requirements static Channel relationships constraint etc), +Destination orchestrations (process, state), Data / Event structures: / hierarchies Display with optional reactions and decisions depends on operations +Payload / ontology +Metadata needed +Timestamp(s) Complex / Event Processor Processing operations: Event format: rules, states etc based on depends on channel and operations needed Outbound source requirements etc Channel TimeToLive : + Destination depends on processing + Payload approach used Svc + Metadata Data Services (Services) Data location: Event and + Interfaces + Interfaces latency, State depends on Data + + Local Store + external sharing Choreographies requirements + Choreographies Choreographies required required / locks required 41
  • 42. epts event processing technical society Example Deployment Architecture Distributed Event Processing Agents – see ref.
  • 43. epts event processing technical society Overview – Sample Member Architectures • University of Trento - Themis Palpanas • TIBCO – Paul Vincent • Oracle - Alex Alves • IBM - Catherine Moxey
  • 44. epts event processing technical society Conceptual View publishes sends through Event Producer Event catalogued in Repository Event processes Event uses stored in Event Channel Processor Event Specification described By receives receives through Event Consumer © 2010 Oracle Corporation 44
  • 45. epts event processing technical society Logical View Processor EDA is composed of several processing steps intermingled Rule Query with user logic. Processor Listener STREAM Rule (Dash- Query board) Processor Event Source Adapter Rule Processor Query Query Query Data Source Adapter Processor Listener Cache Rule (Event Rule Query Bus) This arrangement or network of event processing components is called an event processing network (EPN). © 2010 Oracle Corporation 45
  • 46. epts event processing technical society Functional View © 2010 Oracle Corporation
  • 47. epts event processing technical society Deployment View © 2010 Oracle Corporation
  • 48. epts event processing technical society Fast-Flower-Delivery Use-case © 2010 Oracle Corporation 48
  • 49. epts event processing technical society Overview – Sample Member Architectures • University of Trento - Themis Palpanas • TIBCO – Paul Vincent • Oracle - Alex Alves • IBM - Catherine Moxey
  • 50. epts event processing technical society IBM Conceptual Architecture for EP
  • 51. epts event processing technical society IBM EP Architecture – mapping to methodology System: event processing system – event emitter, event bus, event handler Environment: initial producers of events and their ultimate consumers Stakeholders Concerns Architectural View Event consumers – use Receive correct events in Event Handler (adapters, consumer-side EP events to get information, consumable format, with services, orchestration), Event Processing take action, detect problems, relevant data Services, Repositories, Event Emitters. etc. Publication, Subscription, Notification services to receive events Event “Analysts” – create Means to specify the Event Instantiator, Event Processing Services, and understand event events to be produced, Event Information Management and Query specifications and the patterns to be Services, Event Security Services, Event definitions, how events identified, event Registries. Event Repositories to do pattern- should be processed, etc. enrichment etc. matching over time Event System managers – Security, Performance, Event Security Services, Event Governance and manage and control the EP Configurability related Security Services system Event System operators – Abilities to monitor and Event Monitoring and Analytic Infrastructure, operate the EP system configure system, Event Channels, Publication, Subscription, diagnose and solve Notification Services problems Event Producers Generated events are Event Instantiator, producer-side EP services, made use of in some way Event Adapters, Event Bus…
  • 52. epts event processing technical society IBM Conceptual Architecture for Decision Maker stakeholder
  • 53. epts event processing technical society IBM Conceptual Architecture for Architect stakeholder
  • 54. epts event processing technical society FFD Use Case mapped to IBM RA Showing some of the Event Flows and Event Processing Agents
  • 55. epts event processing technical society Agenda • Introduction to architectures, architecture methodologies, and event processing • Member architectures and salient features • Skeleton reference architecture from EPTS Reference Architecture Working Group • Summary and future work of the EPTS Reference Architecture Working Group 55
  • 56. epts event processing technical society Step 1: collect Event Processing Reference Architectures… EP Network Events: Downstre (parallel & pipelined) Tuples, am Packets Respons , Data Applicati e/Applicat Stream ion s on Layer API’s, Plug-ins Query Clients Data Management Q Subscription Layer: Q Clients Q Networked Networked StreamSystem Processing Adapter Adapter Output Event Destinations Input Event Sources Calls Analysis Modules In-Process System In-Process Adapter Adapter Event Resource Processin g Layer Engine Historical & Reference Data Event Storage 56 Aleri RA Streambase RA Microsoft RA … TIBCO RA IBM RA
  • 57. epts event processing technical society Step 2: identify methodology to abstract an EPTS version… Recipe eventZero RA Aleri RA Streambase RA Microsoft RA TIBCO RA IBM RA Progress RA Oracle RA
  • 58. epts event processing technical society Step 3: distil the common EPTS Reference Architecture(s)
  • 59. epts event processing technical society Reference Architecture Viewpoints recap Viewpoint Viewpoint Element Engineering EP Managing EP Business with EP Architecture Architecture Architecture Concepts How to How to apply? How to utilize / sell / implement? own? Stakeholders Architects / Project Manager Decision Maker, Engineers Customer, Provider Concerns Effective Operational Strategic and construction and Management tactical deployment management Techniques / Modeling, IT (service/appl) Monitoring, Languages Engineering management, Enterprise Decision project Management, management Governance 59
  • 60. epts event processing technical society Stakeholders and Viewpoints covered Viewpoints (for CEP / stateful EP) Stakeholder Component Functional Decision Maker [Not applicable] Inputs, Outputs and / End User Processing Requirements View Architect / Solution Components Functions carried out in Designer View CEP View 60
  • 61. epts event processing technical society Stakeholders and Viewpoints covered Viewpoints (for CEP / stateful EP) Stakeholder Component Functional Decision Maker [Not applicable] Inputs, Outputs and / End User Processing Requirements View Architect / Solution Components Functions carried out in Designer View CEP View 61
  • 62. epts event processing technical society Logical View • View of the Event Processing Network? • A Network Diagram? • Currently under consideration!
  • 63. epts event processing technical society Example Logical View 63 Source: WestGlobal
  • 64. epts event processing technical society Stakeholders and Viewpoints covered Viewpoints (for CEP / stateful EP) Stakeholder Component Functional Decision Maker [Not applicable] Inputs, Outputs and / End User Processing Requirements View Architect / Solution Components Functions carried out in Designer View CEP View 64
  • 65. epts event processing technical society Functional View • Architect and Developer perspective – includes the 3 main functions (development, run-time and administration), – targets primarily the automated event processing operations • Run-time functions in 2 main groups: – the event infrastructure (sources and consumers) external to the event processor under consideration, – the event processor.
  • 66. epts event processing technical society Functional View source: definitions of EP Event Processing, Complex Event Processing, Event Stream Processing Event Processing Rules, etc. Event Type Definitions, Event Abstraction, Event Pattern Detection, Event Composition Event Management etc. etc. Events Derived Events Event Producer Event Consumer (Event Source, (Event sink, event Event Emitter) handler, event listener) Design time Run time Administration
  • 67. epts event processing technical society Reference Architecture: Functional View Definition, Modeling, (continuous) Improvement Event Reaction Assessment, Routing, Prediction, (Pattern, Control, Rule, Query, RegEx.etc) Discovery, Learning Event Process Monitoring, Control 0..* State Management State Management Complex Event Detection Event and Complex Event Process Monitoring, Control Consolidation, Composition, Aggregation 0..* Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification 0..* Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment 0..* Event Production Event Consumption Publication, Dashboard, Apps, Retrieval External Reaction Design time Run time Administration
  • 68. epts event processing technical society Reference Architecture: Functional View / Runtime Event Reaction Assessment, Routing, Prediction, Discovery, Learning 0..* State Management State Management Complex Event Detection Consolidation, Composition, Aggregation 0..* Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification 0..* Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment 0..* Event Production Event Consumption Publication, Dashboard, Apps, Retrieval External Reaction Run time
  • 69. epts event processing technical society Reference Architecture: Functional View / Runtime Event Production: the source of events for event processing. • Event Publication: As a part of event production, events may be published onto a communication mechanism (eg event bus) for use by event consumers (including participants in event processing). This is analagous to a "push" system for obtaining events. • Event Retrieval: As a part of event production, events may be explicitly retrieved from some detection system. This is analagous to a "pull" system for obtaining events. Event Production Event Consumption Publication, Dashboard, Apps, Retrieval External Reaction
  • 70. epts event processing technical society Reference Architecture: Functional View / Runtime Event Consumption: the process of using events from event publication and processing. Event processing itself can be an event consumer, although for the purposes of the reference architecture, event consumers are meant to indicate downstream consumers of events generated in event processing. • Dashboard: a type of event consumer that displays events as they occur to some user community. • Applications: a type of event consumer if it consumes events for its own processes. • External Reaction: caused through some event consumption, as the result of some hardware or software process. Event Production Event Consumption Publication, Dashboard, Apps, Retrieval External Reaction
  • 71. epts event processing technical society Reference Architecture: Functional View / Runtime Event Preparation: the process of preparing the event and associated payload and metadata for further stages of event processing. • Entity Identification: incoming events will need to be identified relative to prior events, such as associating events with particular sources or sensors. • Event Selection: particular events may be selected for further analysis. Different parts of event processing may require different selections of events. See also event filtering. • Event Filtering: a stream or list of events may be filtered on some payload or metadata information such that some subset is selected for further processing. • Event Monitoring: particular types of events may be monitored for selection for further processing. This may utilise specific mechanisms external to the event processing such as exploiting event production features. • Event Enrichment: events may be "enriched" through knowledge gained through previous events or data. Event Preparation Identification, Selection, Filtering, Monitoring, Enrichment
  • 72. epts event processing technical society Reference Architecture: Functional View / Runtime Event Analysis: the process of analysing suitably prepared events and their payloads and metadata for useful information. • Event Analytics: the use of statistical methods to derive additional information about an event or set of events. • Event Transforms: processes carried out on event payloads or data, either related to event preparation, analysis or processing. • Event Tracking: where events related to some entity are used to identify state changes in that entity. • Event Scoring: the process by which events are ranked using a score, usually as a part of a statistical analysis of a set of events. See also Event Analytics • Event Rating: where events are compared to others to associate some importance or other, possibly relative, measurement to the event. • Event Classification: where events are associated with some classification scheme for use in downstream processing. Event Analysis Analytics, Transforms, Tracking, Scoring, Rating, Classification
  • 73. epts event processing technical society Reference Architecture: Functional View / Runtime Complex Event Detection: the process by which event analysis results in the creation of new event information, or the update of existing complex events. • Event Consolidation: combining disparate events together into a "main" or "primary" event. See also event aggregation. • Event Composition: composing new, complex events from existing, possibly source, events. • Event Aggregation: combining events to provide new or useful information, such as trend information and event statistics. Similar to event consolidation. Complex Event Detection Consolidation, Composition, Aggregation
  • 74. epts event processing technical society Reference Architecture: Functional View / Runtime Event Reaction: the process subsequent to event analysis and complex event detection to handle the results of analysis and detection. • Event Assessment: the process by which an event is assessed for inclusion in some process, incorporation in some other event, etc. • Event Routing: the process by which an event is redirected to some process, computation element, or other event sink. • Event Prediction: where the reaction to some event processing is that some new event is predicted to occur. • Event Discovery: where the reaction to some event processing is the disclosure of a new, typically complex, event type. • Note that event prediction is predicting some future event, usually of a known type, whereas event discovery is the uncovering of a new event type. See also event-based learning. • Event-based Learning: the reaction to some event processing that uses new event information to add to some, typically statistical-based, understanding of events. • Note that event-based learning is a specialisation of general machine learning and predictive analytics. Event Reaction Assessment, Routing, Prediction, Discovery, Learning
  • 75. epts event processing technical society Reference Architecture: Functional View / Design time Definition, Modeling, (continuous) Improvement Covers the definition, modeling, improvement / (Pattern, Control, Rule, Query, RegEx.etc) maintenance of the artifacts used in event processing: Event and Complex Event • event definitions, including event metadata and payloads, • event and event object organisations and structures, • event processing transformations / queries / rules / procedures / flows / states / decisions / expressions (although these can sometimes be considered as administrative updates in some situations) Design time
  • 76. epts event processing technical society Reference Architecture: Functional View / Administration Administrative concepts of Event Process Monitoring, Control monitoring and control. This may involve • starting and stopping the application and event Process Monitoring, Control processing elements, including application monitors • providing and updating security levels to event inputs and outputs (also can design-time) • management of high availability and reliability resources, such as hot standby processes • resource utilisation monitoring of the event processing components • process updates, such as how-swapping of event processing definitions to newer versions. Administration
  • 77. epts event processing technical society Agenda • Introduction to architectures, architecture methodologies, and event processing • Member architectures and salient features • Skeleton reference architecture from EPTS Reference Architecture Working Group • Summary and future work of the EPTS Reference Architecture Working Group 77
  • 78. epts event processing technical society Current Works • Terminology and Methodology – ANSI/IEEE Std 1471 :: ISO/IEC 42010 Terminology – EPTS-RA Terminology – Concepts from the EPTS Glossary • Reference Architecture Discovery • Reference Architecture Comparison – Identify commonalities in proposed architectures – Collect core and additional CEP system functions and components • Member Reference Architecture Descriptions – Using the EPTS-RA Methodology and Terminology • Application of RA Descriptions to Use Cases
  • 79. epts event processing technical society Next Steps • Generalize the EPTS Reference Architecture descriptions – addressing important stakeholders and their views – using rigorous RA methodology and terminology (+ glossary) • Evaluate the EPTS-RA descriptions – apply on selected EPTS use cases and compare to member experience – compare with Best Practice Guidelines • Review Logical Architecture requirements for Reference purposes • Possible Outputs: – EPTS-RA Description document – EPTS-RA Design Patterns and Best Practice Guidelines document – Wiley Book "Pattern Oriented Software Architecture: Architectures, Models and Patterns for Event Processing" (already in preparation) – Input for Event Processing Metamodels and associated standards
  • 80. epts event processing technical society Summary • Reference Architecture provides a common set of EP Functions that may be included in – EP systems – EP-related tools • Describes the mapping from Glossary of Terms to Implementation • Provides the basis for EP agents, operators and languages 80
  • 81. epts event processing technical society Thank you ! Acknowledgment to the Event Processing Technical Society Reference Architecture working group: David Tucker (Event Zero), John Morrell (Coral8, Inc.), Baden Hughes (Event Zero), Dieter Gawlick (Oracle), Alex Alves (Oracle), Anand Srinivasan (Oracle), Shailendra Mishra (Oracle), Zbigniew Jerzak (TU Dresden), Hans-Arno Jacobsen (University Toronto), Albert Mavashev (Nastel Technologies Inc.), Simon Courtenage (University of Westminster), Catherine Moxey (IBM United Kingdom Limited), Richard Tibbetts (Streambase), Brian Connell (WestGlobal), Guy Sharon (IBM), Themis Palpanas (University of Trento) 81
  • 82. epts event processing technical society References • EPTS Glossary http://www.ep-ts.com/component/option,com_docman/task,doc_download/gid,66/Itemid,84/ • University of Trento Architecture section: 1. Heiko Stoermer, Themis Palpanas, George Giannakopoulos. The Entity Name System: Enabling the Web of Entities. International Workshop on Data Engineering meets the Semantic Web (DESWeb), in conjunction with the IEEE International Conference on Data Engineering (ICDE), Long Beach, CA, USA, March 2010 2. Zoltan Miklos, Nicolas Bonvin, Paolo Bouquet, Michele Catasta, Daniele Cordioli, Peter Fankhauser, Julien Gaugaz, Ekaterini Ioannou, Hristo Koshutanski, Antonio Mana, Claudia Niederee, Themis Palpanas, Heiko Stoermer. From Web Data to Entities and Back. International Conference on Advanced Information Systems Engineering (CAiSE), Hammamet, Tunisia, June 2010 • IBM Architecture section: Conceptual Architecture paper: http://www.redbooks.ibm.com/redpapers/pdfs/redp4642.pdf • TIBCO Architecture section: Sergio Gonik & Dan Buchko, BusinessEvents Distributed Processing Implementation Patterns, Tips, and Techniques, TUCON TIBCO User Conference, Las Vegas, May 2010