1. High Performance CEP:
The SAP Sybase Event Stream Processor
Technical Product Overview
Speaker’s Name/Department (delete if not needed)
Month 00, 2011

2. Insight from fast-moving data: Continuous Intelligence
Analyze events as they occur
• Continuous insight
• Immediate response
Rapid application development
• Reduce/eliminate dependence on specialist
programming skills
• Cut implementation/deployment time
• Improve business agility
Non-intrusive deployment
• Event-driven architecture
• Adapt to existing data models
© 2012 SAP AG. All rights reserved. 2

3. The SAP Sybase Event Stream Processor
INPUT Studio
STREAMS (Authoring)
Reporting
Sensor Data
Tools
SAP® Dashboard
?
Transactions
Sybase® Applications
Message
Events
ESP Bus
SAP HANA, Sybase IQ
Reference
Data
• Unlimited number of input • Send output to apps, dashboards
streams
• Capture data in SAP HANA, Sybase IQ,
• Incoming data is processed as it Sybase RAP for historical analysis,
arrives, according to the compliance
business logic defined using high
level authoring tools
© 2012 SAP AG. All rights reserved. 3

4. Sample Architecture of a CEP-based Application:
Real-time P&L
Market Prices
 Reuters Live Dashboard
Market Data
Middleware
 Bloomberg
 Nasdaq
 …
Order Management Systems
Alerts
Trading Systems
Sybase Historical Analytics,
Ad-hoc Analysis,
IQ
Compliance
© 2012 SAP AG. All rights reserved. 4

5. Two approaches to CEP
Continuous
Rules
Queries
© 2012 SAP AG. All rights reserved. 5

6. Key Concepts
Input Streams Derived Streams,
Events arrive on input streams Windows
Apply continuous query
operators to one or more
input streams to produce a
new stream
Database Analogy Windows can Have State
• Streams are like tables • Retention rules define how many or how
• Events are rows in the table long events are kept
• Fields in an event message are • Opcodes in events can indicate
columns in the table insert/update/delete and can be
automatically applied to the window
© 2012 SAP AG. All rights reserved. 6

7. ESP Studio: Both visual and textual authoring
Visual Dataflow Authoring Language-based Authoring
• ANALYST-LEVEL SKILLS (EXCEL, VBA) • RAPID PROGRAMMING
• EASY TO UNDERSTAND COMPLEX MODELS • EASY TO USE LANGUAGE (CCL)
• NO NEED TO LEARN LANGUAGE SYNTAX • MODULAR, PROJECT BASED APPROACH
ESP Studio supports both Visual and Textual authoring
• Visual diagram produces CCL (code-behind approach)
• User can switch back and forth between diagram and language
• Eclipse-based
© 2012 SAP AG. All rights reserved. 7

8. CCL: Continuous Computation Language
SQL-based Event Processing Language
Leverage familiarity and simplicity of SQL
Instead of “snapshot” queries, continuous queries
Filter (WHERE), Aggregate (GROUP BY), Join, Compute
Extensions for event streams
Windows, Patterns
Example CCL Query
Calculation of 10 CREATE LOCAL WINDOW VWAP10
minute VWAP PRIMARY KEY DEDUCED
AS
SELECT
TICKER.Symbol AS Symbol ,
Sum ( TICKER.Price * TICKER.Size) / Sum ( TICKER.Size)
AS VWAP
FROM TICKER KEEP 10 MINUTES
GROUP BY TICKER.Symbol ;
© 2012 SAP AG. All rights reserved. 8

9. Key CCL Language Features
• Stateless and stateful elements. • Full support for JOINS
– STREAMS (stateless without keys) – ANSI JOIN Syntax
– WINDOWS (stateful with keys) – Stream-Window Joins and Window-Window joins
– Windows push updates and automatically process
opcodes. • Filter
• Retention policies on Windows • Aggregate
– Named windows (windows created with “CREATE – Group By, Group Filter, Group Order
WINDOW..”)
– Unnamed windows (FROM A KEEP 5 ROWS) • Pattern Matching
– Time and Count based retention policies – Watch for specific sequence of events within time
interval
– Custom windows with EventCache
– Within or across streams
• Parameters, Variables – Pattern can include missing events
– Variables can maintain state
• Transform/Project
– Change event schema, compute new fields
© 2012 SAP AG. All rights reserved. 9

10. SPLASH scripting adds extensibility to CCL
SPLASH (Stream Processing LAnguage SHell) was introduced in Aleri to provide
extensibility and overcome the limitations of SQL
It complements CCL, providing the ability to write custom functions and stream/window
operators using a simple, procedural language that is optimized for stream processing
Syntax is similar to C and Java, it’s spirit is closer to little languages like AWK or Perl
ESP 5.0 has full support for SPLASH
Control Structures: Data Structures: Flex Streams:
if, while, switch
if (temperature < 0) { - Variables - Custom event “handlers”
display := 'below zero'; - Vectors written in SPLASH
} else if (temperature = 0) { - Dictionaries
display := 'zero';
} else { - Event Cache
display := 'above zero';
}
For more info see the SPLASH Tutorial on sybase.com (in the Aleri Product Documentation/Manuals)
© 2012 SAP AG. All rights reserved. 10

11. Modularity
Benefits:
CREATE MODULE
Facilitates re-use • Defines a module consisting of
windows and streams
Simplifies team development • Define inputs, outputs,
parameters
Improves manageability of
large projects
IMPORT
• Equiv. to #include – include
external .ccl file(s)
• Beyond modules, re-use
declarations, etc
LOAD MODULE
• At load point, define bindings
and parameters
• Same module can be loaded
multiple times
© 2012 SAP AG. All rights reserved. 11

12. Support for External function libraries (UDF)
• Call out to external function libraries
• Support for C and Java
• Can be used to supplement built-in functions
• Can also be used to “trigger” external actions
© 2012 SAP AG. All rights reserved. 12

13. ESP Studio: RUN/TEST Tools
Single click to Run a project
Rich Suite of Testing, Tuning and Debugging tools
Stream viewer
Record/playback
Manual event input
Performance Monitor
Event Tracer
Debugger with Breakpoints
Multi-server control
View/monitor projects running on multiple servers and clusters
© 2012 SAP AG. All rights reserved. 13

14. ESP Advanced Cluster Architecture
Manager Private
Cloud
Manager Manager
Manager
Controller Controller Controller
Project Project Project
Project Project Project
Project Project
Project Project
Project Project Project
Project Project
© 2012 SAP AG. All rights reserved. 14

15. ESP Cluster Architecture
Key Features
Manager
• Equi-peer management of cluster.
• Program deployment and lifecycle control.
• Program load balancing at deployment.
• Launcher failure detection.
• Container failure detection.
• Program failover.
• Publishes lifecycle updates for Managers, Controllers and Programs.
• Name resolution.
Controller
• Acts as a daemon for Managers to launch Container processes.
• Allows for launching of any configured Program type.
Container
• Process that hosts a Program (i.e. – esp_server).
© 2012 SAP AG. All rights reserved. 15

16. Choice of high availability configurations
FT Cluster
Manager
Configurable to your level of
availability and flexibility needs
Instant failover to "hot" spare
Synchronized
• highest degree of availability Hot Spare
• recovery in seconds; no data loss
Shared resources "warm" spares
• flexible configurations
• recovery in minutes
• parallel & pipeline configurations
Parallel & Pipelined Shared
State
Shared Warm Spares
All managed by fault tolerant
cluster manager
© 2012 SAP AG. All rights reserved. 16

17. ADAPTERS
Standard Optional Custom
(Built-in) (Add-on) (SDK)
• JMS, MQ, • Market Data • C/C++
• Reuters, NYSE Tech
TIBCO • FIX • Java
• Sockets • Rep Server • .NET
• Databases
• Files
• MS Excel
• Email (out)
© 2012 SAP AG. All rights reserved. 17

18. Integration/Connectivity
Application with
In-Process
External Adapters Native ESP
Adapters
Support
• Entirely under the • Independent of • Embed ESP
control of the ESP ESP Server pub/sub API
server. • Uses ESP directly in
• Performance pub/sub API producer or
advantages • TCP socket consumer
• No hot-standby at connectivity to applications
the adapter level ESP Server
• C/C++ only • Auto-failover in
hot-standby
configurations
• C/C++, Java,
.NET
© 2012 SAP AG. All rights reserved. 18

19. USING Sybase Replication Server for Change Capture
Real-time change capture: turn database transactions into event streams
Transactions Database
(insert, update, Replication Replication
Agent Server Real-time Event
delete)
Stream
SAP Sybase
ESP
© 2012 SAP AG. All rights reserved. 19

20. High Level ESP Architecture
Cluster Manager
Security Load Balancing Failover Lifecycle/Status
Distributed Cache SDK Interface URI Resolution
Client
Launcher
SDK - .NET JAVAC++
ESP Project
Security (Kerberos, PAM, SSL)
Adapter Query Engine
Command & Control
SQL Query
Gateway SDK
Studio Window Adapter
External
Stores Framework Adapter
Tools s
Internal
Adapter
Compiler s
Log
File Data
Source/Destination
© 2012 SAP AG. All rights reserved. 20

21. What sets SAP Sybase ESP apart
• Performance • State Management
− Scalable for extreme throughput − Unique ability to automatically apply
− Consistent low-latency incoming events as
− Only CEP vendor to submit to STAC inserts/updates/deletes to a table
benchmarking − Simplifies modeling
• SPLASH scripting − Extremely efficient
− Overcome limitations of SQL • Dynamic
− Flexibility and productivity − Add continuous queries to live system
• Authoring flexibility • High Availability
− Language or UI/IDE − Hot-Hot with Auto-Failover
• “Private Cloud” Architecture • Advanced Subscriptions
− Scalable, Dynamic − Initial state followed by updates
− Subscriptions with predicates
© 2012 SAP AG. All rights reserved. 21

22. High Performance
STAC Certified measurement of throughput and latency
The only CEP vendor to publish STAC benchmarks
• Linear scaling with addition of more
cores:
180,000 messages/sec on 16 cores
300,000 messages/sec on 24 cores
• Consistent/predicable low latency.
Only minor increase in latency from
20,000 to 300,000 messages/sec:
Mean: 1.296 - 1.568 ms
Medium: 1.177 - 1.502 ms
Std Dev: 0.567 - 1.104 ms
Min: 0.091 – 0.115 ms
99th percentile: 2.479 – 3.165 ms
Source: STAC Research Sep 08 – conducted against Aleri 3.1; 5.0 performance is
similar
© 2012 SAP AG. All rights reserved. 22

23. PANOPTICON EX
REAL-TIME DATA VISUALIZATION

24. Visually Rich Real-time Dashboards with
Historical Drill-down and Analysis
© 2012 SAP AG. All rights reserved. 24

25. Panopticon EX Features
Visual Data Analysis
Real-time streaming from Sybase ESP
Historic Time Series data from Sybase RAP or IQ
Drag-and-Drop Designer
Rich palette of charting objects
Simple configuration
No coding required
Multi-dimensional Analysis (in-memory OLAP)
Delivered via Desktop (thick client) or Web (browser)
SDK provides APIs for ability to customize, embed
© 2012 SAP AG. All rights reserved. 25

26. Panopticon EX for ESP
Native integration with Sybase
ESP, RAP, and IQ Panopticon EX
Reduces Insight
True real-time dashboards, Latency
subscribing to ESP streaming
output  It’s Not Just the Time it
Takes to Get the Data
Seamless drill-down into  It’s also the time it
historical time series data in takes to understand the
Sybase RAP data
Rich Palette of
Visualization Tools
Designed to Make it
Easier to Understand the
Data
© 2012 SAP AG. All rights reserved. 26

27. Thank you
Contact information:
F name MI. L name
Title
Address
Phone number