Swan(sea) Song – personal research during my six years at Swansea ... and bey...
RuleML2011 CEP Standards Reference Model
1. Standards for Complex Event Processing and
Reaction Rules
Adrian Paschke1, Paul Vincent2, Florian Springer3
1Freie Universitaet Berlin 2Tibco Software 3Senacor AG
RuleML 2011 America, Ft Lauderdale, Florida, USA
2. AGENDA
Introduction
Overall Picture (CEP Model) and it„s Standards
Use Case (Examples)
Next Steps
Seite 2
3. DEFINITION OF A STANDARD
WHY ARE STANDARDS NECESSARY OR USEFUL?
Definition taken from ISO/IEC Guide 2:2004:
A standard is a
“document, established by consensus and approved by a recognized body, that provides, for
common and repeated use, rules, by
established guidelines or characteristics for activities or their results,
ratified by a
aimed at the achievement of the optimum degree of order in a given context.”
consensus recognized body
Standard
provide rules, guidelines, etc. aimed at the achievement
for common and repeated use of some optimum
NOTE:
Standards should be based on the consolidated results of science, technology and experience,
and aimed at the promotion of optimum community benefits.
Source: ISO/IEC INFORMATION CENTRE (http://www.standardsinfo.net)
Seite 3
4. AGENDA
Introduction
Overall Picture (CEP Model) and it‘s Standards
Use Case (Examples)
Next Steps
Seite 5
5. EPTS CEP (/Reaction Rule) Standards Reference Model
Business & Domain Standards
Domain Reference Model
Domain Reference Model
Domain
Domain Use Case
Domain Use Case Technology
Standards
Common Standards, Laws
Common Standards, Laws
Computer Independent Model (CIM)
Computer Independent Model (CIM) Platform Independent Model (PIM)
Platform Independent Model (PIM) Platform Specific
Platform Specific
Model (PSM)
Model (PSM)
Behavior Process
EPL
Standard Process Decisions
-CQL
Strategy
Vocabulary, Description
Strategy
Reactions
Ontology - ECA-Rules,
Reaction Data Events - PR
- State, Process
Complex Events
Decision
Design Agent (Process) Components
Complex Events - EPA
Architecture (EPN)
Simple Events Transport Channel Messaging and
Persistence
General
Subscription
Functional Model, Design Guidelines and Patterns
Functional Model, Design Guidelines and Patterns
(Domain Independent)
(Domain Independent)
Business Perspective Technical Perspective
(Focus: Design, Modeling) (Infrastructure, Execution)
Seite 6
6. EPTS CEP Standards Classification
Common Standards, Laws
Common standards and laws provide conditions and regulations which have to
be considered in the business, e.g. laws for stock trading. Development of a
software application has to follow these laws.
Domain Reference Model
Domain Reference Models provide a subset of best practices, reference business
processes, etc. for a specific business domain. These models help adaptors in
defining a proper application based on well proven best practices, e.g. a well
proven reference business process for the manufacturing domain.
Domain Use Case
The use case describes how a problem can be solved, e.g. how to detect fraud
in stock trading. It is often derived out of the business strategy and the
reference model.
7. EPTS CEP Standards Classification
Strategy
The strategy defines the individual business strategy of a company which has to
be considered in developing new applications. The strategy consists of business
(business motivations, goals, policies, rules) as well of technical (applications
infrastructure, allowed frameworks) conditions.
Functional Model
The functional model provides domain independent best practices and guidelines
helping to design and develop a proper application, e.g. "Design Patterns -
Elements of Reusable Object-Oriented Software"
8. EPTS CEP Standards Classification - CIM
Computer Independent Model
The Computer Independent Model (CIM) describes the business functionality or
system without reference to IT specifics. CIM should provide a common
understanding between business users and software developers. In the CEP
model it consists of following components:
• Standard Vocabulary, Ontology The ontology provides a set of definitions
and terms enabling a common understanding for all stakeholders.
• Process description Description of activities and its flow to produce a specific
product or service enhanced with events occurring in it or influencing it.
• Reaction Definition of the activities which have to be initiated based on the
previous taken decision.
• Decision Definition of rules, what has to be done if a relevant situation was
detected by a pattern matching
• Complex Events Definition of event correlations for detection of relevant
situations defined by patterns representing knowledge from source events, e.g.
for detection of fraud.
• Simple Events Definition of attributes and types consisting of a simple event,
e.g. the event "Stock Price" consists of the fields WKN, amount, etc.
9. EPTS CEP Standards Classification - PIM
Platform Independent Model Standards (PIM)
The Platform Independent Model layer represents behavior, data,
design, and messaging, independent from a particular EP platform.
• Event-Driven Behavior Effects of events lead to some (state)
changes in the properties of the world which can be abstracted into
situations.
• Event Data Platform-independent representation of events and their
data is crucial for the interoperation between EP tools and between
domain boundaries.
• Event Processing Design Platform-independent (reference)
architecture and design models addressing different views for different
stakeholders.
• Messaging PIM Messaging is addressing transport protocols and
routing, coordination / negotiation mechanisms.
10. COMMON FRAMEWORK AND GUIDELINES
FOR DISCUSSION
General Frameworks and Guidelines for
Developing Systems
Existing Benefits of
Description Example Standards (+Gaps) Standardization
Common Industry standards, laws, Several laws (often country A lot of existing Regulation of Use
Standards, etc. specific), e.g. BörsG for stock standards are Cases
trading in Germany available No Benefit for CEP
Laws
CEP Specific
standards not
necessary
Strategy Overlying business Reduction of fraud by 10 OMG BMM Clarify the Business
UseCase motivations, goals, percent, … May need more strategy
strategies, policies and emphasis on
business rules events and
decision rules
Reference Collection of best practices Reference End-to-End Various per Helps in developing
Model and reference business process for logistics, e.g. domain for data an efficient
processes for a specific Pickup to Delivery Rarely handles application
domain events and rules
Use Case Description of how to solve a Fraud detection for stock UML (not CEP or Create a common
problem trading rules specific) understanding
EPTS UseCase between business
Template user and software
developer
Functional Domain independent Design Patterns for software Lots of different Helps in
Model, Design description of best practices, engineering (Gamma) ones! implementing a
guidelines, etc. EPTS Functional proper application
Patterns ..
Ref Architecture
Seite 11
11. COMPUTER INDEPENDENT MODEL 1/2
FOR DISCUSSION
Computer Independent Model / Business Model
for designing CEP Systems and Processes
Existing Benefits of
Description Example Standards (+Gaps) Standardization
Standard Common definitions of words Stock Order: Transaction for Glossary, Common
Vocabulary, enabling a common buying a specific amount of ontologies for understanding for
Ontology understanding for all stocks for price x events, time … al stakeholders
stakeholders OMG SBVR, W3C involved
OWL
Process Description of activities and Buy Stocks requires Text descriptions Create a common
Description its flow to produce a specific “estimate risk”, “check of “events” understanding
product or service enhanced price”, “check deposit”, BPMN, EPC between business
with events occurring in it “check availability”, etc. user and software
Insufficient
Events: StockPrice, … developer on a “big
details on events
picture”
and rules
Decision Definition of rules, what has If a highly risk for the order Decision Tree Common
to be done if a relevant was detected by a pattern Decision Table understanding for
situation was detected by a the reaction (~DMN) all stakeholders
pattern matching “dropStockOrder” has to be involved
Decision Rules
done
standard missing
Reaction Definition of the activities The stock order has to be None (Text Common
which have to be done, dropped and the trader has based understanding of
based on the previous taken to be informed description) possible outcomes
decision CIM Standard for all stakeholders
Reaction Rules is involved
missing
Seite 12
12. COMPUTER INDEPENDENT MODEL 2/2
FOR DISCUSSION
Computer Independent Model / Business Model
for designing CEP Systems and Processes
Existing Benefits of
Description Example Standards (+Gaps) Standardization
Complex Events Definition of event Stocks of companies, where the None but Better
correlations for detection of stock price increase 3% in - OMG EMP understanding of
relevant situations defined by the last three days and proposed relation between
patterns representing which have production New standard involved events
knowledge from source facilities in Europe and required and roles
events, e.g. for detection of produce products out of
fraud. metal and have more than
10,000 employees
Simple Events Definition of attributes and StockPrice: UML (Gaps in Create a common
types consisting an simple - WKN (String 10) specifics needed understanding
Event - ISIN (String 15) for EP, ~EMP) across business
- Name (String 50) Improve users / event
- Date/Time: (Time) modelling sources, and
- Price: (Decimal) languages eg developers
- Currency: (String 3) NEAR
…
Seite 13
13. PLATFORM INDEPENDENT MODEL 1/4
FOR DISCUSSION
Platform Independent Model / IT Model
for design of CEP Systems
Existing Benefits of
Description Example Standards (+Gaps) Standardization
Behavior Event-Driven Behavior and Rules: W3C RIF + Declarative, explicit
Decisions: Reaction RuleML; representation
– focused on rule-
Event occurences lead based behaviour Publication and
Event E is used
to state changes in OMG PRR interchange of
in Pattern P = “E then F“
patterns, processes decisions and
Event E causes process R to – lacks explicit events
(= situations) reactive behavior
start – specific to production
rules
Decisions represent the ...
choices a system can After Pattern P is detected OMG Decision Modeling
after patterns detected we must decide how to Notation
or process states respond to the situation S – no explicit event
change OMG UML2 Behavioral
View Diagrams
Actions are triggered / – Limited
Process R must be invoked
performed as event expressivness of
reactions as the output events and events =
of decisions and actions/activities
state/situation changes
OMG BPEL
– specific to business
process execution
with WS
W3C WS Choreoagraphy
– Specific to WS
and further EDA
standards
Seite 14
14. PLATFORM INDEPENDENT MODEL 2/4
FOR DISCUSSION
Platform Independent Model / IT Model
for design of CEP Systems
Existing Benefits of
Description Example Standards (+Gaps) Standardization
Data Information carried Event Pattern P uses Software Engineering: Declarative
by events and used properties of event E1, E2 UML Structural View diagrams representation,
in event patterns Processing of event E1 – not specific to events translation and
and processes requires operations on its Knowledge Representation: interchange of events
May be processed properties Interchange and
W3C RDFS/OWL
as a part of event Reaction to event E1 interoperation between
processing in the ISO CL, ISO Topic Maps, different EP tools
requires sending data D to
context of CEP/EP system S OMG ODM UML/OWL Profiles preserving the semantic
Archived +analysed + many existing event interpretation of the
for machine ontologies event data
learning, BI, etc Rules: W3C RIF + Reaction Interchange events
Some of ontologies which include events RuleML over domain boundaries
– specific to rule-based EP which have different
CIDOC CRM: museums and libraries
domain semantics /
ABC Ontology: digital libraries OASIS WS Topics
models
Event Ontology: digital music – Topic-based pubsub only
Prerequisits Semantics:
DOLCE+DnS Ultralite: event aspects in social reality OMG Event Meta Model
Event-Model-F: event-based systems Explicit domain
– not yet an RFP in OMG semantics
VUevent Model: An extension of DOLCE and other event
conceptualizations OASIS Common Base Event Expressiveness
IPTC. EventML: structured event information – For business enterprise apps Extensibility
GEM: geospatial events OASIS WS Notification and W3C Reusability,
Event MultiMedia: multimedia WS Eventing specialisation and
LODE: events as Linked Data – specific to WS mapping
CultureSampo: Publication System of Cultural Heritage Rules: further standardization in
OpenCyC Ontology: human consensus reality, upper ontology with lots of W3CRIF/RuleML
terms and assertions
Standards for other domains
Semantic Complex Event Processing: top level meta model / ontology
for semantic CEP needed, e.g. stream processing
Seite 15
15. PLATFORM INDEPENDENT MODEL 3/4
FOR DISCUSSION
Platform Independent Model / IT Model
for design of CEP Systems
Existing Benefits of
Description Example Standards (+Gaps) Standardization
Design Platfrom-Independent Reference Architecture: ISO/IEC 42010:2007 Abstraction from
(Reference) Architecture and models the abstract Recommended Practice for the PSM design
Model Descriptions addressing architectural design Architectural Description of Increases
different views for different elements Software-intensive Systems understandability
stakeholders Reference Model: – Needs to be tailored to and reusability
describes the important EP architecture Agent model is an
concepts and relationships descriptions abstraction from
in the design space UML 2 Implementation View technical IT
EPN: Distributed Network Diagrams components to
Architecture – limited expressivness role-based agents
Agents: abstract on a more abstract
– Not specialised for EP
components given a level
design; only general
specific abstract role and component and
behavior/responsibilities composite structure
design
Agents: OMG Agent
Metamodel; FIPA Agent
Model; …
– agent specific
Workflow Management
Coalition Reference Model
– workflow model
Seite 16
16. PLATFORM INDEPENDENT MODEL 4/4
FOR DISCUSSION
Platform Independent Model / IT Model
for design of CEP Systems
Existing Benefits of
Description Example Standards (+Gaps) Standardization
Messaging Transport protocols and synchronous vs. Many transport protocols: well understood
routing, coordination / asynchronous transport JMS, JDBC, TCP, UDP, and standardized
negotiation mechanisms Push vs. pul multicast, http, servlet, transport protocols
SMTP, POP3, file, XMPP
Coordination: Publish
Subscribe, ContractNet, – just needed for
… transport
… Message Routing Patterns
– message routing
Coordination and
Negotiation
Mechanisms/Patterns
Seite 17
17. PLATFORM SPECIFIC MODEL 1/3
FOR DISCUSSION
Platform Specific Model (PSM)
describes the concrete Implementation of a CEP Application
Existing Benefits of
Description Example Standards (+Gaps) Standardisation
EPL Continuous query language SELECT x, y from REUTERS ANSI SQL Interchange
CQL running queries against WHERE x.id == y.id extended for queries between
policies such as time AND x.price > y.price * 1.1 continuous vendor platforms
windows, numbers of events, IN WINDOW(600,“secs“) policy (design,
etc Vendor specific versions ODBMS OQL deployment)
include Oracle CQL, extended for [IT department]
Streambase CQL, Sybase continuous Encourage training
CQL, TIBCO BQL policy / education on CQL
(cf SQL takeup)
[university,
student]
Dependancies:
common data
descriptors
EPL Event-Condition-Action rules IF event. SigMvDet W3C RIF PRD Interchange rules
ECA Rules triggered by creation of AND SigMvDet.stock = IBM extended for between vendor
complex events representing AND SigMvDet.change = up ECA (RIF ECA) platforms (design,
event patterns (eg stream THEN Reaction RuleML deployment)
outputs), determing reqBuy(SigMvDet.stock) [IT department]
JSR-94 rule
reactions based on certain Vendor examples include Encourage training
execution API
conditions TIBCO BE, IBM WSBE, / education on ECA
Progress Apama rules (cf Java
Open Source, e.g. Prova, takeup)
Drools [university,
student]
Dependancies:
common data
descriptors
Seite 18
18. PLATFORM SPECIFIC MODEL 2/3
FOR DISCUSSION
Platform Specific Model (PSM)
describes the concrete Implementation of a CEP Application
Existing Benefits of
Description Example Standards (+Gaps) Standardisation
EPL Temporal Regular WHEN event. A, event.B Perl Regex with Interchange rules
TRE Expressions defining complex WHERE event.A (*2) temporal between vendor
events in terms of event / FOLLOWED BY no event.B extensions platforms (design,
non-event sequences over IN time(200,ms) deployment)
time Vendor examples include [IT department]
Oracle CQL (TRE embedded Encourage training
in SQL), Progress Apama / education on TRE
(TRE), TIBCO PML patterns (cf Java
takeup)
[university,
student]
Dependancies:
common data
descriptors
Application Application component Agent 1 type inference OMG AMP Agent Compare
Architecture layouts / configurations, contains rulesets R1, R2 Metamodel and application designs
specifying various processes with destinations D1, D2 Profile (draft) across vendors
/ engines / agents and how Deployment C1 is [IT operations,
they are connected. of 4 x Agent 1 Enterprise
Architecture may also be role active-active Architects]
considered platform Vendor examples include Dependancies:
independent in terms of TIBCO CDD Cluster common agent and
general requirements, but Deployment Descriptor interface
platform specific in terms of descriptors
associating specific logical
layouts, interfaces, and
agent / engine numbers to
achieve specified SLAs
Seite 19
19. PLATFORM SPECIFIC MODEL 3/3
FOR DISCUSSION
Platform Specific Model (PSM)
describes the concrete Implementation of a CEP Application
Existing Benefits of
Description Example Standards (+Gaps) Standardisation
Messaging Middleware type and JMS includes header JMS (Java spec) Late selection of
System configuration information and payload OMG DDS messaging type
information, the latter which [IT, IT operations]
AMQP
includes XML document per Dependancies:
some XSD. JMS extensions common payload
could be for guaranteed Gap: some
descriptors
delivery, timeout etc. abstract
definition of a
messaging
software per the
above
Seite 20
20. AGENDA
Introduction
Overall Picture (CEP Model) and it„s Standards
Use Case (Examples)
Next Steps
Seite 21
21. USE CASE EXAMPLE – PART 1
Application: Logistics and Supply Chain Track & Trace
Standards: Business
2. Specification 3. Adopt/adapt
of use case using appropriate
EP principles industry ref model
Reference Model
Reference Model
Common Standards „
Use Case
and Laws
Use Case
Common Standards, Laws
Common Standards, Laws
Computer Independent Model (CIM)
Computer Independent Model (CIM) Platform Independent Model (PIM)
Platform Independent Model (PIM) Platform Specific
Platform Specific
Model (PSM)
Model (PSM)
Strategy UseCase
Strategy UseCase
Behavior Process
EPL
Standard Process Decisions
Vocabulary, Description (CQL, ECA-Rules,
Reactions
Ontology etc.)
Reaction Data Events
1. Business Strategy Decision
Complex Events
goals / rules etc
Design Agent (Process) Application
Complex Events Architecture
Architecture (EPN)
eg in OMG BMM Simple Events Messaging Event Transport Messaging System
Event Subscription
4. Ref Model + Functional Model, Design Guidelines and Patterns
Functional Model, Design Guidelines and Patterns
Glossary (Domain Independent)
(Domain Independent)
eg in OMG SBVR
eg in W3C OWL
5. Value chain / 6. Business Event 7. Decision 8. Reaction
process / org models / pattern definitions definitions definitions
eg in BPMN eg in BPMN, NEAx eg in OMG DMN eg in BPMN
Seite 22
22. USE CASE EXAMPLE – PART 2
Application: Logistics and Supply Chain Track & Trace
Standards: IT
2. Event patterns
1. Event+
Specification
Data Specification
eg in UML State,
eg in UML Class,
OMG EMP,
OMG EMP, ER
Reaction RuleML
Reference Model
Reference Model
Use Case
Use Case
3. Event-based
Common Standards, Laws
Common Standards, Laws
Computer Independent Model (CIM)
Computer Independent Model (CIM) Platform Independent Model (PIM)
Platform Independent Model (PIM) Platform Specific
Platform Specific
behaviours Model (PSM)
Model (PSM)
Strategy UseCase
Strategy UseCase
Behavior Process
eg in BPMN/UML AD
Standard Process EPL
Decisions
Vocabulary, Description (CQL, ECA-Rules,
Reactions
etc.)
eg in UML PRR
Ontology
Reaction Data Events
Complex Events
Decision
4. Design aspects
Design Agent (Process) Application
Complex Events Architecture
Architecture (EPN)
eg in UML Simple Events Messaging Event Transport Messaging System
Interaction, AMP Event Subscription
Functional Model, Design Guidelines and Patterns
Functional Model, Design Guidelines and Patterns
(Domain Independent)
(Domain Independent)
5. Communication
aspects
eg in OMG EMP
6. Platform-specific
implementation aspects
eg in ANSI CQL or
Reaction RuleML
Seite 23
23. USE CASE EXAMPLE – PART 3
Application: Logistics and Supply Chain Track & Trace
Standards: Community
1. Create appropriate
industry ref model
eg some DSL / in XML...
Reference Model
Reference Model
Use Case
Use Case
Common Standards, Laws
Common Standards, Laws
Computer Independent Model (CIM)
Computer Independent Model (CIM) Platform Independent Model (PIM)
Platform Independent Model (PIM) Platform Specific
Platform Specific
Model (PSM)
Model (PSM)
Strategy UseCase
Strategy UseCase
Behavior Process
EPL
Standard Process Decisions
Vocabulary, Description (CQL, ECA-Rules,
Reactions
Ontology etc.)
Reaction Data Events
Complex Events
Decision
Design Agent (Process) Application
Complex Events Architecture
Architecture (EPN)
2. Extract methodologies Simple Events Messaging Event Transport Messaging System
/ mappings Event Subscription
Functional Model, Design Guidelines and Patterns
Functional Model, Design Guidelines and Patterns
(Domain Independent)
(Domain Independent)
3. Refine reference materials
for teaching
Seite 24
24. USE CASE EXAMPLE – For Systems and Tools
Use Case: Event / Rule Interchange
Event / Rule
Events / serialize Interchange de-serial. Events /
Rules Format Rules
Data model,
EP Ontology EP system
System 1 (XSD,ODM/OW 2
(Tibco) L, XMI, …) (Prova)
Data serialize <XML doc> de-serial. Data
Application A data Application B
Seite 25
25. USE CASE EXAMPLE – For Systems and Tools
Use Case: Tool Interoperation
react
Rule Prova Event Processing Rule PSM
Processing Engine
translate
Event Interchange Format PIM
translate
Complex Event Complex Event Complex Event
Stream PSM
Processing Esper Stream Processing Engine
detect
Raw Event Stream
Seite 26
26. USE CASE EXAMPLE – For Systems and Tools
Heterogeneous Distributed Event Processing Network
Tibco Padres
EIF via JMS
Prova
EIF via
Oracle
HTTP
EIF via EIF via
HTTP HTTP
Esper
Apama
Amit
Seite 27
27. AGENDA
Introduction
Overall Picture (CEP Model) and it„s Standards
Use Case (Examples)
Next Steps
Seite 28
28. Next Steps
Areas for Standards (Activities) Responsible
Functional Model, Design, Guidelines EPTS Reference Architecture
Event Processing Language (PSM) EPTS Language Group, (?)
Data/Behavior RuleML/ W3C Rif
Functional Model.. (Terminology) EPTS
Modeling Events (PIM) OMG EMP, NEAR extension
Several PhD‟s
Domain Specific Reference Models
…
)
Reference Model
Reference Model
Use Case
Use Case
Common Standards, Laws
Common Standards, Laws
Computer Independent Model (CIM)
Computer Independent Model (CIM) Platform Independent Model (PIM)
Platform Independent Model (PIM) Platform Specific
Platform Specific
Model (PSM)
Strategy UseCase
Model (PSM)
Strategy UseCase
Behavior Process
EPL
Standard Process Decisions
Vocabulary, Description Reactions
(CQL, ECA -Rules,
Ontology etc.)
Reaction Data Events
Complex Events
Decision
Design Agent (Process) Application
Complex Events Architecture
Architecture (EPN)
Simple Events Messaging Event Transport Messaging System
Event Subscription
Functional Model, Design Guidelines and Patterns
Functional Model, Design Guidelines and Patterns
(Domain Independent)
(Domain Independent)
Seite 29