Presentation given at Microsoft's Drive Customer Connections events - May 26th 2010, New York

1. StreamInsight
A Capital Markets Tale

2. Agenda
2 StreamInsight: A Capital Markets Tale
About Lab49
(Very Brief) Overview Of Microsoft StreamInsight Complex Event Processing (CEP)
Capital Markets StreamInsight Solutions
• Market Strategy Engine
• Market Real-Time Risk (RTR) - SQL Server 2008 R2
• Market Real-Time Risk - Windows AppFabric Caching (Velocity)
Lessons Learnt

3. About Lab49

4. Lab49
4 About Us
We create advanced solutions for the We combine the skills of our practices to
financial services industry through strategic deliver innovation:
and technical consulting: • Strategy group
• Working with the world’s largest investment • User Experience practice
banks, hedge funds and exchanges.
• Development practices
• Designing and delivering some of the most
alongside the adoption of some key tenets:
sophisticated and forward thinking financial
applications. • Domain Driven Development
• Applying user-centred principles to the world of • Lean Agile Process
bespoke software. Our History
• From concept to complete, ui to server. Lab49 was founded in 2002 and now comprises
approximately 200 people, with offices in
New York and London.

5. 5 About Us
Matt Davey
Part of the TAP program since
July 2009
Requester for various
StreamInsight V1 features:
• Count Window
• Dynamic Query
Composition at Runtime

6. Overview Of Microsoft StreamInsight
Complex Event Processing (CEP)

7. CEP in Finance
7 CEP in Finance
• Banks are using CEP for feeding data into algorithmic trading systems as well as for finding dealing
opportunities and trading pairs across asset class.
• Risk monitoring, fraud detection and surveillance are also growth areas for CEP

8. StreamInsight Features
8 StreamInsight Features
• Process large volumes of events across multiple data streams in < 1 second
• Provides built-in support for different event types and rich query semantics
• Reduces development cost by utilising existing skill sets and investment in Microsoft development
platform
• Flexible deployment options

9. StreamInsight (CEP) Application at Runtime
CEP Engine
Event Standing Queries
Event Event
Event
Input Output
Event
Adapters Adapters
9 StreamInsight
Event Event
Event
Event

10. 10 Market Strategy Engine
StreamInsight Deployment

11. Capital Markets StreamInsight Solutions

12. Market Strategy Engine

13. Markets are Changing
13 Market Strategy Engine
• Markets are becoming more complex
• StreamInsight offers a cleaner way to process events in this complex world
• Algorithmic trading has become mainstream
• StreamInsight enhances the cleansing/validation of market data, order execution/management,
pricing engines and more
• Speed and latency are critical to maximising profitability
• StreamInsight’s leverages the Microsoft development platform to improve speed of development
coupled with Microsoft’s commitment to “Process large volumes of events across multiple data
streams” in “low latency”
• Back testing in the search for new strategies
• StreamInsight’s adapter architecture allows easy access to historical market data repositories
e.g. kdb+

14. The Problem - Butterfly Strategy
14 Market Strategy Engine
Leg Buy/Sell Working Leg Price Weight Size Weighting
Bobl Buy Yes 1 1
Bund Sell No -0.28 0.28
Schatz Sell No -1.19 1.19
Buy 10 Bobl
10 Fly @ -49
Sell 3 Bund
Sell 12 Schatz

15. StreamInsight Engine Flow
15 Market Strategy Engine
(1)
(3)
(2)

17. Take Aways
17 Take Aways
Benefits over existing non-CEP based strategy engines
• StreamInsight offers the advantage and benefits of CEP principles
• Working and reasoning with Time - dealing with out of order arrivals
Benefits over existing competitor CEP based strategy engines
• LINQ/.NET implementation
• Easy integration to other Microsoft products
• Lower total cost of ownership

18. Market Real-Time Risk: SQL Server 2008

19. Market Risk
19 Market Risk
Market risk is the risk that the value of a portfolio, either an investment portfolio or a trading portfolio, will
decrease due to the change in value of the market risk factors

20. Why The Need For Real-Time Risk (RTR)?
20 Why The Need For Real-Time Risk (RTR)?
Investment banks need to hedge their positions
Latency
T-1 and warehousing - PnL Explain
Servers are cheaper and faster than workstations
Improved Aggregation and drill-down

21. High Level Architecture
21 High Level Architecture
Streaminsight
F# Agent
Market Depth Strategy Strategy Definition
LINQ
Orders
Agent
Agent (F#)
Market Agent (F#)
Agent (F#)
LINQ Leg Orders
Trade
LINQ
Cancel
Aggregation Shocking Curve
Engine
Market Risk HPC
(GPU?)

22. Aggregation Engine Architecture
22 Aggregation Engine Architecture
Static
Data
Microsoft StreamInsight
Position Database
Microsoft Analysis
Input Output SQL
Adapter Adapter Server
2008 R2
Services
LINQ
Risk Real-Time OLAP
Input Push Output Cube
Adapter Adapter
XMLA Server
Client
Excel

23. RTR Highlights
23 RTR Highlights
• StreamInsight process Risk data, feeding SQL Server 2008 R2
• StreamInsight provides the XMLA service with real-time push notifications - e.g. portfolio ID’s
• “XMLA” Server sits between the client and SQL Server intercepting client requests, and providing “push”
services (smart polling, subscription manager)

24. Live Demo

25. Take Aways
25 Take Aways
Compared To Other Solutions
• The Microsoft solution offers a modularised real-time risk solution that is customisable
• It’s a complete Microsoft (.NET) stack
• Leveraging SQL Server 2008 R2 Analysis Services - (ROLAP for T, and MOLAP T-x)
Performance
• We can improve further by adding another Microsoft product......

26. Market Real-Time Risk: Windows
AppFabric Caching (Velocity)

27. Velocity RTR Architecture
27 Velocity RTR Architecture
Static Risk Warehousing
Data
Microsoft StreamInsight
Position Database
SQL
Microsoft Analysis
Input Output
Adapter Adapter Server
2008 R2
Services
LINQ
Risk Real-Time
Input Push Output OLAP
Adapter Adapter Cube
Windows
Server
XMLA Server AppFabric
XMLA Server
Caching
(Velocity)
Client
Excel

28. Database vs Cache
28 Database vs Cache
Database Cache
Notification Notification Service deprecated in SQL Server 2008 Windows AppFabric Cache Notification
OLAP Cube Analysis Service Custom build of an in-memory cube
Scalability Out the box partitioning, hot-hot support Distributed Cache

30. Take Aways
30 Take Aways
Compared To Other Solutions
• The Microsoft solution offers a modularised real-time risk solution that is customisable
• It’s a complete Microsoft (.NET) stack
• Best of both worlds
• SQL Server - backup, warehousing (ROLAP for T, and MOLAP T-x)
• Distributed Cache - performance

31. Closing

32. Lessons Learnt
32 Lessons Learnt
Business
• StreamInsight improves the delivery timeline of business solutions
Technology
• CTI and AdvancedTimeImportSetting
• V1 Ugly Code Issues
• Nested classes
• Static UDF
• UDF’s offer lots of extensibility
• Reactive Framework (Rx) usage
• .NET 4 Not currently supported

33. thank you for your time…
any questions?
matt davey (matt.davey@lab49.com)
mdavey.wordpress.com | Tales from a trading desk

34. Principles of CEP
34 Principles of CEP
• Events are technology-neutral occurrences of interest, such as a new purchase, a change of address, or an attempt to break into a network. Events
can come from people, devices, applications, networks, or databases.
• Events have context, that is, an instance of an event implies timing (when it happened, both in absolute terms and relative to other events),
sequence (again relative to other events), and linking relationships to other events (patterns of events, either expected or implied).
• Events can also carry information about themselves. For example, a "purchase" event might contain the product purchased and the purchaser.
• Events can be evaluated, either based on their context, their data, or additional data that may not accompany the event but is relevant (for example,
is the purchaser a gold customer?).
• Events form patterns in time and/or sequence that may be interesting. For example, a change of address followed by the reporting of a lost ATM
card may indicate an attempt to profit from identity theft.
• Events comprise ad hoc processes, a sequence of activities that results in the execution of a process. For example, a revenue recognition process
may be composed of the events "contract signing," "purchase order received," and "product shipped." An explicitly defined process automated using
events is called an Event Flow.
• Events may be abstracted into other events. For example, a change of address event followed by the reporting of a lost ATM card event may be
represented by the compound event "Attempted Fraud."
• Events can optionally generate responses, or actions. For example, an "Attempted Fraud" event may trigger, in some cases, a "Put Account on
Referral" action to make sure downstream account activity is legitimate.