1
Drive Smarter Decisions with Big Data
Using Complex Event Processing
Eric Roch ▪ July 16, 2013

Our Speaker
Eric Roch
• Principal for Perficient’s Connected Solutions Practice –
SOA – Cloud - Mobile
• 25+ years of experience in Information Technology
• Previous roles include: executive level management,
technical architect, and software development in top tier
technology organizations including TIBCO Software and
Deloitte Consulting
• Strategic planning and commercialization of
methodologies and software
• Technical architecture for multi-platform application and
systems integration at organizations
• Guest speaker and author
2

3
Perficient is a leading information technology consulting firm serving clients
throughout North America.
We help clients implement business-driven technology solutions that integrate
business processes, improve worker productivity, increase customer loyalty and
create a more agile enterprise to better respond to new business opportunities.
About Perficient

4
• Founded in 1997
• Public, NASDAQ: PRFT
• 2012 revenue $327 million
• Major market locations throughout North America
• Atlanta, Boston, Charlotte, Chicago, Cincinnati, Cleveland, Columbus, Dallas, Denver,
Detroit, Fairfax, Houston, Indianapolis, Los Angeles, Minneapolis, New Orleans, New York
City, Northern California, Philadelphia, Southern California, St. Louis, Toronto and
Washington, D.C.
• Global delivery centers in China, Europe and India
• ~2,000 colleagues
• Dedicated solution practices
• ~85% repeat business rate
• Alliance partnerships with major technology vendors
• Multiple vendor/industry technology and growth awards
Perficient Profile

5
Business Solutions
• Business Intelligence
• Business Process Management
• Customer Experience and CRM
• Enterprise Performance Management
• Enterprise Resource Planning
• Experience Design (XD)
• Management Consulting
Technology Solutions
• Business Integration/SOA
• Cloud Services
• Commerce
• Content Management
• Custom Application Development
• Education
• Information Management
• Mobile Platforms
• Platform Integration
• Portal & Social
Our Solutions Expertise

Agenda
• Big Data Trends and Categories
• Analysis of large amounts of complex, unstructured and semi-
structured data
• Harnessing the power big data, social/mobile data stores and BI
projects for real-time decision-making
• Predictive Analytics and Event Processing for decision management
6

Evolution of Big-Data
• Mainframe
• Client-Server
• Web
• Mobile
• Cloud
• Social
• Internet of Things
7
Source: Go-Globe.com

State of Technology Adoption
8

Categories of Big-Data
9
Source: splunk

10
Characteristics of Big Data
• Data in motion
analyzes data
before storage
• Data at rest
analytics are
based on a
historic snapshot
Source: IBM

Big Data Technologies
• MapReduce frameworks implements
pattern recognition though
classification algorithms – what
happened?
• Data Visualization presents
information views graphically and/or
statistically – what happened and what
might happen?
• Predictive Analytics uses
mathematical pattern recognition in
historical data – what’s going to
happen?
• Complex Event Processing uses
pattern recognition on event streams
and can apply rules to predict logical
events – what is going to happen and
what do we do about it?
11
Source: TIBCO Spotfire

Log Analysis vs. Business Analytics
• Ingest – Versus ETL
• Big Data – Bidirectional
integration with Hadoop
• Query language – MapReduce
function on unstructured data
• Drill anywhere – Investigate on
all the data versus a predefined
schema or cube
• Information discovery –
Discover relationships based on
patterns in the data
• Ad-hoc versus dimensional –
Log analysis is not based a
predefined structure based a
point-in-time set of requirements
12
Source: splunk Implementation

Predictive Analysis
• Predict the future state of
variables associated with
business goals
• Describe human
detectable patterns
• Data mining techniques
• Rule Discovery – describe
• Pattern Discovery – describe
• Clustering – describe
• Classification – predict
• Regression – predict
• Deviation – predict
13
Source: InformationBuilders

Event-driven Architecture
• Event-driven architecture
(EDA) is a software
architecture pattern promoting
the production, detection,
consumption of, and reaction
to events
• Complex event processing
(CEP) consists in processing
many events happening
across all the layers of an
organization, identifying the
most meaningful events within
the event cloud, analyzing their
impact, and taking subsequent
action in real time.
14

A Holist View of Decision Optimization
15
Source: James Taylor
http://www.decisionmanagementsolutions.com

Barriers to Big Data Analytics
• Information throughout the
enterprise
• Silos of data
• Decentralized control
• No one single solution
• No cohesive strategy
• Legacy systems difficult to
make part of the strategy
16

SOA and Integration
17
HTTP
HTTP/S
SOAP/HTTP
SOAP/JMS
FTP
SMTP
EMS/JMS
EDI
Enterprise Service Bus (ESB)
Credit
Check
Place
Order
Check
Quantity
Issue
Invoice
Alert
Large
Order
Notify
Customer
Process
Order
Check
Customer
Account
• Connect
• Transport
• Route
Services Backbone
Enterprise Service Bus
(ESB)
• Mediate
• Event notification
• Exception Handling
Abstract the data format and the behavior of legacy systems to publish events

The SOA Information Gap
″SOA by itself does nothing to address the question
of how data should be managed within this
architecture. ... data remains fragmented despite
the best efforts to rationalize it. This issue is
motivating the creation of a new class of
middleware that Forrester calls the information
fabric.”
The Forrester Report Information Fabric:
Enterprise Data Virtualization
18
″ You will waste your investment in SOA unless you have enterprise
information that SOA can exploit.”
Gartner

Data Virtualization Layer
19
Data
Warehouse
Packaged
Application
Legacy
Application
• Master Data Management and Data Virtualization
• Data federation for consistent packaging of data
• Leverages understanding of metadata relationships
• Applies consistent rules to data
• Centralized control and maintenance
• Flexibility to change information sources and formatsar
Create Quote
Process Flow
Trigger
Create Estimate
Process Flow
Trigger
Information as a Service
(Shared Metadata)

Business Process Management and
Workflow
• The term Business Process Management refers to activities
performed by businesses to optimize and adapt their processes.
• Although it can be said that organizations have always been using
BPM, a new impetus based on the advent of software tools which
allow for
• Direct execution of the business processes without a costly and time
intensive development of the required software.
• In addition, these tools can also monitor the execution of the business
processes, providing managers of an organization with the means to
analyze their performance and make changes to the original processes in
real-time
• BPM has a tight link to componentized and service oriented IT
architecture
20

BPM and Services
21
Service
X
Service
U
Service
Y
Service
Z
Human Task
A
Human Task
D
Human Task
F
Human Task
B
Human Task
C
Workflow
Invoke
Invoke Invoke Invoke
• Workflows implement business processes
• Workflow engine navigates the network of activities
• Typically invoking automatic (service choreography) or manual activities
• Mostly visual programming/modeling

Process Orchestration Layer - BPMS
• Designer and repository
• Execution engine
• Database – case state
• Database – case history
• Case history reporting – KPIs,
task timings, timings by role
• Starting a new case is
resource intensive
22
State
Management
Design
Repository
Process
History
Execution
Engine

BRMS Architecture
• Manages the lifecycle of the
rules
• Author rules
• Execute stateless rules
• Statistical reports about rule
execution
• Rule execution is embedded in
business applications – e.g. a
decision service
23
Source: IBM

Using BRMS in BPMS
• Lifecycle of rules are
external to the BPMS
• Business processes “call”
rules e.g. via services
• Rules make a stateless
decision
• Rules have to have a
driving workflow or
application
24
Rule
Repository
Rule
Engine
Rule
Authoring
BPMS

CEP Architecture
25
• Consistent operational
rules applied to business
events
• Declarative rules and
implicit state management
• Event driven, non-linear,
closed-loop, agile business
processes
• Component failure (fine
grain) – outage (logical
/predictive)
Concept
State
Rule
Bases
BPMS
CEP
Engine
Logical Events – Notifications, Consequences Actions
SOA
Business Applications
Fine-grain
Business Events
System(s) of Record
Integration and
Business Components
Flexible Workflows
ESB
Event
Channel(s)

CEP High-level Architecture
Patterns
• Situation awareness is
about "knowing" the state of
the product, person,
document, or entity of interest
at any point in time.
• Sense and respond is about
detecting some significant
fact about the product,
person, document or entity of
interest, and responding
accordingly
• Track and trace is about
tracking the product, person,
document or entity of interest
over time and tracing
pertinent facts
Source: TIBCO Software
26

CEP Benefits
• Manage events, state transitions, and event correlation reducing
code in the application layer
• Control logic
• Persistence logic
• Business Rules
• Drive business processes with correlated events
• Create operational efficiencies with the same events and drive
longer-term strategic decision support
• Less complex rules with the event driven concepts
• Persistent business objects
• Known context of the event
27

CEP Roadmap and Methodology
• Target critical business events for process
automation and decision optimization
• Inventory relevant events, rules and concepts
• Identify candidate business (sub)process to
automate
• Project LoE(s) and Roadmap
• Integrate systems used in key business
processes
• Instrument applications to emit events
• Define process activities
• Mine candidate rules – code and predictive
analytics
• Model events, rules and concepts
• Iterate through business processes
28

Telco CEP Case Study
• Provisioning
• Track missing provisioning notifications and sends complex events to Billing Ops on
missing notifications
• Open Orders
• Used to track Orders that have not been closed due to a missing event. CEP detects
the missing event and auto closes the Order in the Payment Processing system.
• Pending Payments
• Used to process payments that are pended by the payment processing system. CEP
stores the payment data within the cache and closes the payment at a later via
SOA.
• Customer Coupon Offers
• CEP is used to monitor, alert and prevent Stores from going over a threshold of the
discount funds that they are allocated.
• Logistics Alerts
• CEP is used to track location and Product updates from logistics and to invoke GEH
to republish failed messages
• CEP Framework
• Created CEP developer guide and logging framework to log and search events in
Splunk
29

TIBCO BusinessEvents is a CEP Platform
• Platform Features
• Stateful Rule Engine
• State for Temporal Rules
• Persistence Object Manager
• High Performance Rules Engine
• TIBCO integration platform
• 150+ Adapters
• Channels approach
• Continuous queries
and Event Stream Patterns
• Decision Manager for Business
User Rules Authoring (can write
can upload rules from Excel!)
• Distributed Agents Architecture
for dynamic scalability
• Data Grid
• BE Views (Dashboard)
30
Source: TIBCO Software

Event Enabled Enterprise
31
Transformation
Projects
2009-2011
Business
Solutions
2011-2012
• Last minute addition
• Concept to launch in 6 weeks
• Decoupled architecture – no risk
Implementation
• Customers: Ensure timely
activations
• Operations: Immediate visibility
into order provisioning times
Customer Service
• Stores: Reduce inventory issues
• Operations: Automate fall out of
shipping notices
Supply Chain
• Customers: Added security to account
access
• Operations: Report/alert on
suspicious access attempts
Security/CPNI
• Customers: More access to discounts
• Revenue: Manage discount limits by
individual location
Retail Sales
• Customers: Use IVR to set up
payment agreements
• Customer Service: Reduced call
center volumes
Self-Service
Event
Enabled
2013
• Proven success in real-time, value-
based activities – ready for prime-
time!
• Sense. Model. Respond.
The Tipping Point
• Adapt and respond to real-time
customer behaviors/experiences
• Example: Proactive retention offers
Fast Response
• Abandon one-size-fits-all customer
limitations
• Enable event-driven decisions
for best customer experience
Customer Flexibility

CEP Solution Architecture
32

CEP References
• http://it.toolbox.com/blogs/the-soa-blog/complex-
event-processing-reference-materials-48348
• http://it.toolbox.com/blogs/the-soa-blog/complex-
event-processing-patterns-message-routing-
48987
• http://complexevents.com/wp-
content/uploads/2008/02/1-a-short-history-of-cep-
part-1.pdf
• http://complexevents.com/wp-
content/uploads/2008/07/2-final-a-short-history-of-
cep-part-2.pdf
33

Questions
34

Daily unique content
about content
management, user
experience, portals
and other enterprise
information technology
solutions across a
variety of industries.
Perficient.com/SocialMedia
Facebook.com/Perficient
Twitter.com/Perficient
35

Thank you for your time
and attention today.
Please visit us at Perficient.com
36