SlideShare une entreprise Scribd logo
1  sur  37
Télécharger pour lire hors ligne
Stream Processing in Action
Webinar
Nirmal Fernando
Senior Lead Solutions Engineer, WSO2
Nov, 2018
Contents
• Introduction
• WSO2 Stream Processor Overview
• Industry Use Cases
• Demo
Diverse Industries -> Unique Challenges
http://www.solgenie.com/industries/
“The price of light is less than the cost of
darkness.”
- Arthur C. Nielsen,
Market Researcher & Founder of ACNielsen
Value of Insights Degrade Fast
http://www.history.com/news/ask-history/who-determined-the-speed-of-light
http://www.bluntmoms.com/mom-guilt-theres-one-solution/
Stream Processor Core
WSO2 ANALYTICS OFFERING
7
▪ Consumes events, and publish alerts and
summarizations to and from various
enterprise systems.
▪ Event Processor Core with Streaming
Complex Event Processing, Incremental
Time Series Aggregations, and Streaming
Machine Learning.
▪ Stream Processing Functionalities via
Extension Store
▪ High Available and Scalable Analytics
Fabric
▪ Prebuilt and custom analytics solutions
Events
JMS, Thrift, SMTP, HTTP, MQTT, Kafka
Analytics Fabric
Complex Event
Processing
Incremental Time
Series Aggregation
Machine
Learning
Extension Store
FinancialandBanking
Analytics
RetailAnalytics
LocationAnalytics
OperationalAnalytics
SmartEnergyAnalytics
Custom
Analytics
Solutions
...
Solutions
WSO2 STREAM PROCESSOR
WSO2 Stream Processor
An open source, cloud-native
analytics product optimized to create
real-time, actionable insights for agile
digital businesses.
9
WSO2 Stream Processor (WSO2 SP)
1. Data collection
2. Data cleansing
3. Data transformation
4. Data enrichment
5. Data summarization
6. Rule processing
7. Machine Learning & Artificial Intelligence
8. Data pipelining
9. Data Publishing
10. On demand processing
11. Data Presentation
Stream Processing Patterns
● Lightweight, lean, and high performance
● Best suited for
○ Streaming Data Integration,
○ Streaming Analytics
● Streaming SQL & graphical drag-and-drop editor
● Multiple deployment options
○ Process data at the edge (java, python)
○ Micro Stream Processing
○ High availability with 2 nodes
○ Highly scalable distributed deployments
● Support for streaming ML & Long running aggregations
● Monitoring tools and citizen integration options
WSO2 Stream Processor
• Source and Sinks
– HTTP, Kafka, TCP, Email, JMS, File, Rabbitmq,
MQTT, Web-Socket, Twitter, Amazon SQS
• Message Formats
– JSON, XML, Text, Binary, Key-value, CSV
• Data Stores
– RDBMS, Solr, MongoDB, HBase, Cassandra,
Elasticsearch, Hazelcast, Redis
Supported connectors
(Streaming) Machine Learning
▪ Running PMML Models for predictions
- Built via Apache Spark MLlib, Python, H2O.ai (for deep learning algos) or R
- Export as PMML
- Load precreated PMML Model into Siddhi to predict in Realtime
▪ Supporting native models for predictions
- Spark MLlib Models, Java based Tensorflow Models
▪ Online Learning and predictions
- Regression Analytics - Data Classification
- K-Means Clustering - Markov Models
- Anomaly Detections - … more on the way
Supported Extensions
• Geo graphical processing
• NLP
• Graph
• Reordering
• Timeseries
• Import Machine Learning models
– PMML, TensorFlow, etc
• Streaming Machine Learning
– Clustering, Classification, Regression
– Makove Models, Anomaly detection, etc.
• .. more
Image : http://www.weewatch.com/wp-content/uploads/2016/01/28fc45d.jpg
60+
https://store.wso2.com/store/assets/analyticsextension/list
High Availability with 2 Nodes
• 2 node minimum HA
– Process upto 100k
events/sec
– While most other stream processing
systems need around 5+ nodes
• Does not require Kafka
• Incremental state persistence
and recovery
• Multi data center support
Stream Processor
Stream Processor
Event Sources
Dashboard
Notification
Invocation
Data Source
Siddhi App
Siddhi App
Siddhi App
Siddhi App
Siddhi App
Siddhi App
Event
Store
• Exactly-once
processing
• Fault tolerance
• Highly scalable
• No back pressure
• Distributed via
annotations
• Native support for
Kubernetes
Scaling with Distributed Deployment
INDUSTRY USE CASES
Finance and Banking
Use Case 1 - Fraud Detection
• Detecting fraud via known patterns using generic
rules
• Detecting unknown types of fraud via machine
learning
• Detecting rare activity sequences using Markov
Modeling
• Reduce false alarms using fraud scoring
• Caught them in the act - what next?
Demo: https://goo.gl/xo6Wf5
Use Case 2 - Risk Management
• Finding real-time Value at Risk (VaR)
– Historical simulation
– Variance-covariance
– Monte Carlo simulation
• Identifying Front Running with Patterns
Use Case 3 - Stock Market Surveillance
Hey Jude, Mike is
going to buy large qty
of ABC at $21. You
better buy now!
Great! I
bought. ABC
is just $18.9
right now!
Trade 1
Followed by
Trade 2
Jude sells to Mike at
$21.
Broker: Bob
Client Client
MikeJude
Use Case 3 - Stock Market Surveillance
• Identifying Pump with Regression
• Identifying signs of Insider Dealing
• Model “Perfect Trader” in order to detect
fraudsters
Retail
Use Case 1 - Recommendations
• Recommendations based on the buying
products
• Recommendations based on the buying history
of the customer
• Seasonal recommendations
• Contextual, intelligent recommendations
Use Case 2 - Ad Optimization
• Display personalized advertisements on online
shopping stores
– by identifying person’s living location
– by identifying person’s buying history
– by identifying person’s interest
• Optimize displaying advertisements on shopping
stores
– what impact the Ad made? Is it worth the cost?
• Send personalized information when a
customer enters the store
• If a customer is spending a considerable
amount of time near a shop, suggest offers,
send an agent to him etc.
Use Case 3 - Proximity Marketing
• Detect that the customer leaves the store and send
him a gift coupon
• Number of customers & employees in each floor
(heat maps)
• Number of customers & average time spent per
product
• Shopping path, purchased items, average time spent
by each customer
• Overall statistics about given and used offers
Use Case 3 - Proximity Marketing
Location Analytics
Use Case - Fleet Management
• Real-time monitoring - where is your fleet now?,
Visualize your fleet
• Geo-fencing based alerts - get alerted if a driver
exceeds a defined speed limit within an interactively
given geo area
• Predicting travel times - historical data can be used
to build a machine learning model in order to
predict travel times in advance and alert the
subscribers
Operations Analytics
Use Cases
• Collect data from different stages of business
operations
• Tracking the progress of your business operations
• Detect Service Level Agreement violations at each
stage and generate alerts
• Visualize your business operations real-time
• Generate alerts and notify the subscribers
DEMO
Setup
Download and Tryout
WSO2 Stream Processor
https://wso2.com/analytics-and-stream-processing/
Documentation
Further Reading
● WSO2 Stream Processor: Making Real-time Stream Processing Available to the Masses
● Making Real-Time Applications Simpler with WSO2 Stream Processor
● How to use a Stream Processor as a Notification Manager?
● Synchronous Request-Response based Real-time Processing with WSO2 SP
● Distributed Stream Processing with WSO2 SP
THANK YOU
wso2.com

Contenu connexe

Tendances

apidays LIVE New York 2021 - Managing the usage of Asynchronous APIs: What do...
apidays LIVE New York 2021 - Managing the usage of Asynchronous APIs: What do...apidays LIVE New York 2021 - Managing the usage of Asynchronous APIs: What do...
apidays LIVE New York 2021 - Managing the usage of Asynchronous APIs: What do...apidays
 
Hybrid integration platform reference architecture
Hybrid integration platform reference architectureHybrid integration platform reference architecture
Hybrid integration platform reference architectureChanaka Fernando
 
Role of API Management in an API led Digital Economy
Role of API Management in an API led Digital EconomyRole of API Management in an API led Digital Economy
Role of API Management in an API led Digital EconomyWSO2
 
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...WSO2
 
Api centric enterprises
Api centric enterprisesApi centric enterprises
Api centric enterprisesWSO2
 
Best Practices for Productizing APIs with API Management and Automated Testing
Best Practices for Productizing APIs with API Management and Automated TestingBest Practices for Productizing APIs with API Management and Automated Testing
Best Practices for Productizing APIs with API Management and Automated TestingWSO2
 
Achieving Microservices Maturity
Achieving Microservices MaturityAchieving Microservices Maturity
Achieving Microservices MaturityNordic APIs
 
How to Enable Monetization of Your API Ecosystem
How to Enable Monetization of Your API EcosystemHow to Enable Monetization of Your API Ecosystem
How to Enable Monetization of Your API EcosystemWSO2
 
Leveraging Async APIs to deliver cross domain agile collaboration
Leveraging Async APIs to deliver cross domain agile collaboration Leveraging Async APIs to deliver cross domain agile collaboration
Leveraging Async APIs to deliver cross domain agile collaboration Nuwan Dias
 
API Services: Harness the Power of Enterprise Infrastructure
API Services: Harness the Power of Enterprise InfrastructureAPI Services: Harness the Power of Enterprise Infrastructure
API Services: Harness the Power of Enterprise InfrastructureApigee | Google Cloud
 
[WSO2 API Manager Community Call: Streaming API Support in WSO2 API Manager 4.0
[WSO2 API Manager Community Call: Streaming API Support in WSO2 API Manager 4.0[WSO2 API Manager Community Call: Streaming API Support in WSO2 API Manager 4.0
[WSO2 API Manager Community Call: Streaming API Support in WSO2 API Manager 4.0WSO2
 
AWS Api Gateway by Łukasz Marchewka Scalacc
AWS Api Gateway by Łukasz Marchewka ScalaccAWS Api Gateway by Łukasz Marchewka Scalacc
AWS Api Gateway by Łukasz Marchewka ScalaccScalac
 
Apigee Insights: Data & Context-Driven Actions
Apigee Insights: Data & Context-Driven ActionsApigee Insights: Data & Context-Driven Actions
Apigee Insights: Data & Context-Driven ActionsApigee | Google Cloud
 
5 Pillars of Building Enterprise0grade APIs
5 Pillars of Building Enterprise0grade APIs5 Pillars of Building Enterprise0grade APIs
5 Pillars of Building Enterprise0grade APIsWSO2
 
Lessons Learned from Building Enterprise APIs (Gustaf Nyman)
Lessons Learned from Building Enterprise APIs (Gustaf Nyman)Lessons Learned from Building Enterprise APIs (Gustaf Nyman)
Lessons Learned from Building Enterprise APIs (Gustaf Nyman)Nordic APIs
 
Which APIs? which business models - A real-world guide for African banks.
Which APIs? which business models - A real-world guide for African banks.Which APIs? which business models - A real-world guide for African banks.
Which APIs? which business models - A real-world guide for African banks.WSO2
 
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
[WSO2 API Day Toronto 2019] Cloud-native Integration for the EnterpriseWSO2
 
API workshop by AWS and 3scale
API workshop by AWS and 3scaleAPI workshop by AWS and 3scale
API workshop by AWS and 3scale3scale
 

Tendances (20)

apidays LIVE New York 2021 - Managing the usage of Asynchronous APIs: What do...
apidays LIVE New York 2021 - Managing the usage of Asynchronous APIs: What do...apidays LIVE New York 2021 - Managing the usage of Asynchronous APIs: What do...
apidays LIVE New York 2021 - Managing the usage of Asynchronous APIs: What do...
 
Hybrid integration platform reference architecture
Hybrid integration platform reference architectureHybrid integration platform reference architecture
Hybrid integration platform reference architecture
 
Role of API Management in an API led Digital Economy
Role of API Management in an API led Digital EconomyRole of API Management in an API led Digital Economy
Role of API Management in an API led Digital Economy
 
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...
 
Architecting SaaS
Architecting SaaSArchitecting SaaS
Architecting SaaS
 
Api centric enterprises
Api centric enterprisesApi centric enterprises
Api centric enterprises
 
Best Practices for Productizing APIs with API Management and Automated Testing
Best Practices for Productizing APIs with API Management and Automated TestingBest Practices for Productizing APIs with API Management and Automated Testing
Best Practices for Productizing APIs with API Management and Automated Testing
 
Achieving Microservices Maturity
Achieving Microservices MaturityAchieving Microservices Maturity
Achieving Microservices Maturity
 
Flavours of APIs
Flavours of APIs Flavours of APIs
Flavours of APIs
 
How to Enable Monetization of Your API Ecosystem
How to Enable Monetization of Your API EcosystemHow to Enable Monetization of Your API Ecosystem
How to Enable Monetization of Your API Ecosystem
 
Leveraging Async APIs to deliver cross domain agile collaboration
Leveraging Async APIs to deliver cross domain agile collaboration Leveraging Async APIs to deliver cross domain agile collaboration
Leveraging Async APIs to deliver cross domain agile collaboration
 
API Services: Harness the Power of Enterprise Infrastructure
API Services: Harness the Power of Enterprise InfrastructureAPI Services: Harness the Power of Enterprise Infrastructure
API Services: Harness the Power of Enterprise Infrastructure
 
[WSO2 API Manager Community Call: Streaming API Support in WSO2 API Manager 4.0
[WSO2 API Manager Community Call: Streaming API Support in WSO2 API Manager 4.0[WSO2 API Manager Community Call: Streaming API Support in WSO2 API Manager 4.0
[WSO2 API Manager Community Call: Streaming API Support in WSO2 API Manager 4.0
 
AWS Api Gateway by Łukasz Marchewka Scalacc
AWS Api Gateway by Łukasz Marchewka ScalaccAWS Api Gateway by Łukasz Marchewka Scalacc
AWS Api Gateway by Łukasz Marchewka Scalacc
 
Apigee Insights: Data & Context-Driven Actions
Apigee Insights: Data & Context-Driven ActionsApigee Insights: Data & Context-Driven Actions
Apigee Insights: Data & Context-Driven Actions
 
5 Pillars of Building Enterprise0grade APIs
5 Pillars of Building Enterprise0grade APIs5 Pillars of Building Enterprise0grade APIs
5 Pillars of Building Enterprise0grade APIs
 
Lessons Learned from Building Enterprise APIs (Gustaf Nyman)
Lessons Learned from Building Enterprise APIs (Gustaf Nyman)Lessons Learned from Building Enterprise APIs (Gustaf Nyman)
Lessons Learned from Building Enterprise APIs (Gustaf Nyman)
 
Which APIs? which business models - A real-world guide for African banks.
Which APIs? which business models - A real-world guide for African banks.Which APIs? which business models - A real-world guide for African banks.
Which APIs? which business models - A real-world guide for African banks.
 
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
[WSO2 API Day Toronto 2019] Cloud-native Integration for the Enterprise
 
API workshop by AWS and 3scale
API workshop by AWS and 3scaleAPI workshop by AWS and 3scale
API workshop by AWS and 3scale
 

Similaire à Stream Processing in Action

Analytics Patterns for Your Digital Enterprise
Analytics Patterns for Your Digital EnterpriseAnalytics Patterns for Your Digital Enterprise
Analytics Patterns for Your Digital EnterpriseSriskandarajah Suhothayan
 
WSO2Con USA 2017: Analytics Patterns for Your Digital Enterprise
WSO2Con USA 2017: Analytics Patterns for Your Digital EnterpriseWSO2Con USA 2017: Analytics Patterns for Your Digital Enterprise
WSO2Con USA 2017: Analytics Patterns for Your Digital EnterpriseWSO2
 
Solutions Using WSO2 Analytics
Solutions Using WSO2 AnalyticsSolutions Using WSO2 Analytics
Solutions Using WSO2 AnalyticsWSO2
 
Advanced Topics - Session 3 - Optimizing AWS Applications
Advanced Topics - Session 3 - Optimizing AWS ApplicationsAdvanced Topics - Session 3 - Optimizing AWS Applications
Advanced Topics - Session 3 - Optimizing AWS ApplicationsAmazon Web Services
 
WSO2Con USA 2017: Scalable Real-time Complex Event Processing at Uber
WSO2Con USA 2017: Scalable Real-time Complex Event Processing at UberWSO2Con USA 2017: Scalable Real-time Complex Event Processing at Uber
WSO2Con USA 2017: Scalable Real-time Complex Event Processing at UberWSO2
 
Webinar: Analytics with NoSQL: Why, for What, and When?
Webinar: Analytics with NoSQL: Why, for What, and When?Webinar: Analytics with NoSQL: Why, for What, and When?
Webinar: Analytics with NoSQL: Why, for What, and When?MongoDB
 
Clickstream analytics with Markov Chains
Clickstream analytics with Markov ChainsClickstream analytics with Markov Chains
Clickstream analytics with Markov ChainsAlex Papageorgiou
 
Data Science Out of The Box : Case Studies in the Telecommunication by Anand ...
Data Science Out of The Box : Case Studies in the Telecommunication by Anand ...Data Science Out of The Box : Case Studies in the Telecommunication by Anand ...
Data Science Out of The Box : Case Studies in the Telecommunication by Anand ...Data Con LA
 
Large scale Click-streaming and tranaction log mining
Large scale Click-streaming and tranaction log miningLarge scale Click-streaming and tranaction log mining
Large scale Click-streaming and tranaction log miningitstuff
 
IEEE.BigData.Tutorial.2.slides
IEEE.BigData.Tutorial.2.slidesIEEE.BigData.Tutorial.2.slides
IEEE.BigData.Tutorial.2.slidesNish Parikh
 
Fraud prevention is better with TigerGraph inside
Fraud prevention is better with  TigerGraph insideFraud prevention is better with  TigerGraph inside
Fraud prevention is better with TigerGraph insideTigerGraph
 
Optimizing Your AWS Apps & Usage to Reduce Costs - IP Expo
Optimizing Your AWS Apps & Usage to Reduce Costs - IP ExpoOptimizing Your AWS Apps & Usage to Reduce Costs - IP Expo
Optimizing Your AWS Apps & Usage to Reduce Costs - IP ExpoAmazon Web Services
 
Events and microservices
Events and microservicesEvents and microservices
Events and microservicesSaul Caganoff
 
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon Kinesis
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon KinesisDay 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon Kinesis
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon KinesisAmazon Web Services
 
Bigdata based fraud detection
Bigdata based fraud detectionBigdata based fraud detection
Bigdata based fraud detectionMk Kim
 
Hadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorialHadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorialhadooparchbook
 
SnappyData @ Seattle Spark Meetup
SnappyData @ Seattle Spark MeetupSnappyData @ Seattle Spark Meetup
SnappyData @ Seattle Spark MeetupSnappyData
 
AWS re:Invent 2016: Beeswax: Building a Real-Time Streaming Data Platform on ...
AWS re:Invent 2016: Beeswax: Building a Real-Time Streaming Data Platform on ...AWS re:Invent 2016: Beeswax: Building a Real-Time Streaming Data Platform on ...
AWS re:Invent 2016: Beeswax: Building a Real-Time Streaming Data Platform on ...Amazon Web Services
 

Similaire à Stream Processing in Action (20)

Analytics Patterns for Your Digital Enterprise
Analytics Patterns for Your Digital EnterpriseAnalytics Patterns for Your Digital Enterprise
Analytics Patterns for Your Digital Enterprise
 
WSO2Con USA 2017: Analytics Patterns for Your Digital Enterprise
WSO2Con USA 2017: Analytics Patterns for Your Digital EnterpriseWSO2Con USA 2017: Analytics Patterns for Your Digital Enterprise
WSO2Con USA 2017: Analytics Patterns for Your Digital Enterprise
 
Solutions Using WSO2 Analytics
Solutions Using WSO2 AnalyticsSolutions Using WSO2 Analytics
Solutions Using WSO2 Analytics
 
Advanced Topics - Session 3 - Optimizing AWS Applications
Advanced Topics - Session 3 - Optimizing AWS ApplicationsAdvanced Topics - Session 3 - Optimizing AWS Applications
Advanced Topics - Session 3 - Optimizing AWS Applications
 
WSO2Con USA 2017: Scalable Real-time Complex Event Processing at Uber
WSO2Con USA 2017: Scalable Real-time Complex Event Processing at UberWSO2Con USA 2017: Scalable Real-time Complex Event Processing at Uber
WSO2Con USA 2017: Scalable Real-time Complex Event Processing at Uber
 
Webinar: Analytics with NoSQL: Why, for What, and When?
Webinar: Analytics with NoSQL: Why, for What, and When?Webinar: Analytics with NoSQL: Why, for What, and When?
Webinar: Analytics with NoSQL: Why, for What, and When?
 
Clickstream analytics with Markov Chains
Clickstream analytics with Markov ChainsClickstream analytics with Markov Chains
Clickstream analytics with Markov Chains
 
Algorithmic Trading
Algorithmic TradingAlgorithmic Trading
Algorithmic Trading
 
Data Science Out of The Box : Case Studies in the Telecommunication by Anand ...
Data Science Out of The Box : Case Studies in the Telecommunication by Anand ...Data Science Out of The Box : Case Studies in the Telecommunication by Anand ...
Data Science Out of The Box : Case Studies in the Telecommunication by Anand ...
 
Large scale Click-streaming and tranaction log mining
Large scale Click-streaming and tranaction log miningLarge scale Click-streaming and tranaction log mining
Large scale Click-streaming and tranaction log mining
 
IEEE.BigData.Tutorial.2.slides
IEEE.BigData.Tutorial.2.slidesIEEE.BigData.Tutorial.2.slides
IEEE.BigData.Tutorial.2.slides
 
Fraud prevention is better with TigerGraph inside
Fraud prevention is better with  TigerGraph insideFraud prevention is better with  TigerGraph inside
Fraud prevention is better with TigerGraph inside
 
Operational-Analytics
Operational-AnalyticsOperational-Analytics
Operational-Analytics
 
Optimizing Your AWS Apps & Usage to Reduce Costs - IP Expo
Optimizing Your AWS Apps & Usage to Reduce Costs - IP ExpoOptimizing Your AWS Apps & Usage to Reduce Costs - IP Expo
Optimizing Your AWS Apps & Usage to Reduce Costs - IP Expo
 
Events and microservices
Events and microservicesEvents and microservices
Events and microservices
 
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon Kinesis
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon KinesisDay 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon Kinesis
Day 5 - Real-time Data Processing/Internet of Things (IoT) with Amazon Kinesis
 
Bigdata based fraud detection
Bigdata based fraud detectionBigdata based fraud detection
Bigdata based fraud detection
 
Hadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorialHadoop application architectures - Fraud detection tutorial
Hadoop application architectures - Fraud detection tutorial
 
SnappyData @ Seattle Spark Meetup
SnappyData @ Seattle Spark MeetupSnappyData @ Seattle Spark Meetup
SnappyData @ Seattle Spark Meetup
 
AWS re:Invent 2016: Beeswax: Building a Real-Time Streaming Data Platform on ...
AWS re:Invent 2016: Beeswax: Building a Real-Time Streaming Data Platform on ...AWS re:Invent 2016: Beeswax: Building a Real-Time Streaming Data Platform on ...
AWS re:Invent 2016: Beeswax: Building a Real-Time Streaming Data Platform on ...
 

Plus de WSO2

Ballerina Tech Talk - May 2023
Ballerina Tech Talk - May 2023Ballerina Tech Talk - May 2023
Ballerina Tech Talk - May 2023WSO2
 
Platform Strategy to Deliver Digital Experiences on Azure
Platform Strategy to Deliver Digital Experiences on AzurePlatform Strategy to Deliver Digital Experiences on Azure
Platform Strategy to Deliver Digital Experiences on AzureWSO2
 
Modernizing the Student Journey with Ethos Identity
Modernizing the Student Journey with Ethos IdentityModernizing the Student Journey with Ethos Identity
Modernizing the Student Journey with Ethos IdentityWSO2
 
Choreo - Build unique digital experiences on WSO2's platform, secured by Etho...
Choreo - Build unique digital experiences on WSO2's platform, secured by Etho...Choreo - Build unique digital experiences on WSO2's platform, secured by Etho...
Choreo - Build unique digital experiences on WSO2's platform, secured by Etho...WSO2
 
CIO Summit Berlin 2022.pptx.pdf
CIO Summit Berlin 2022.pptx.pdfCIO Summit Berlin 2022.pptx.pdf
CIO Summit Berlin 2022.pptx.pdfWSO2
 
Fueling the Digital Experience Economy with Connected Products
Fueling the Digital Experience Economy with Connected ProductsFueling the Digital Experience Economy with Connected Products
Fueling the Digital Experience Economy with Connected ProductsWSO2
 
A Reference Methodology for Agile Digital Businesses
 A Reference Methodology for Agile Digital Businesses A Reference Methodology for Agile Digital Businesses
A Reference Methodology for Agile Digital BusinessesWSO2
 
Workflows in WSO2 API Manager - WSO2 API Manager Community Call (12/15/2021)
Workflows in WSO2 API Manager - WSO2 API Manager Community Call (12/15/2021)Workflows in WSO2 API Manager - WSO2 API Manager Community Call (12/15/2021)
Workflows in WSO2 API Manager - WSO2 API Manager Community Call (12/15/2021)WSO2
 
Lessons from the pandemic - From a single use case to true transformation
 Lessons from the pandemic - From a single use case to true transformation Lessons from the pandemic - From a single use case to true transformation
Lessons from the pandemic - From a single use case to true transformationWSO2
 
Adding Liveliness to Banking Experiences
Adding Liveliness to Banking ExperiencesAdding Liveliness to Banking Experiences
Adding Liveliness to Banking ExperiencesWSO2
 
Building a Future-ready Bank
Building a Future-ready BankBuilding a Future-ready Bank
Building a Future-ready BankWSO2
 
WSO2 API Manager Community Call - November 2021
WSO2 API Manager Community Call - November 2021WSO2 API Manager Community Call - November 2021
WSO2 API Manager Community Call - November 2021WSO2
 
[API World ] - Managing Asynchronous APIs
[API World ] - Managing Asynchronous APIs[API World ] - Managing Asynchronous APIs
[API World ] - Managing Asynchronous APIsWSO2
 
[API World 2021 ] - Understanding Cloud Native Deployment
[API World 2021 ] - Understanding Cloud Native Deployment[API World 2021 ] - Understanding Cloud Native Deployment
[API World 2021 ] - Understanding Cloud Native DeploymentWSO2
 
[API Word 2021] - Quantum Duality of “API as a Business and a Technology”
[API Word 2021] - Quantum Duality of “API as a Business and a Technology”[API Word 2021] - Quantum Duality of “API as a Business and a Technology”
[API Word 2021] - Quantum Duality of “API as a Business and a Technology”WSO2
 
API Revisions - WSO2 API Manager Community Call (10/27/2021)
API Revisions - WSO2 API Manager Community Call (10/27/2021)API Revisions - WSO2 API Manager Community Call (10/27/2021)
API Revisions - WSO2 API Manager Community Call (10/27/2021)WSO2
 
[2021 Somos Summit] - Rethinking Identity Access Management and The Rise of t...
[2021 Somos Summit] - Rethinking Identity Access Management and The Rise of t...[2021 Somos Summit] - Rethinking Identity Access Management and The Rise of t...
[2021 Somos Summit] - Rethinking Identity Access Management and The Rise of t...WSO2
 
[ICT Spring 2021] - Managed Crowd: The Future of Business as We Know It!
[ICT Spring 2021] - Managed Crowd: The Future of Business as We Know It![ICT Spring 2021] - Managed Crowd: The Future of Business as We Know It!
[ICT Spring 2021] - Managed Crowd: The Future of Business as We Know It!WSO2
 
[EIC 2021] Securing the Digital Double - The Path to a Trusted Digital Ecosystem
[EIC 2021] Securing the Digital Double - The Path to a Trusted Digital Ecosystem[EIC 2021] Securing the Digital Double - The Path to a Trusted Digital Ecosystem
[EIC 2021] Securing the Digital Double - The Path to a Trusted Digital EcosystemWSO2
 
[EIC 2021] The Rise of the Developer in IAM
[EIC 2021] The Rise of the Developer in IAM[EIC 2021] The Rise of the Developer in IAM
[EIC 2021] The Rise of the Developer in IAMWSO2
 

Plus de WSO2 (20)

Ballerina Tech Talk - May 2023
Ballerina Tech Talk - May 2023Ballerina Tech Talk - May 2023
Ballerina Tech Talk - May 2023
 
Platform Strategy to Deliver Digital Experiences on Azure
Platform Strategy to Deliver Digital Experiences on AzurePlatform Strategy to Deliver Digital Experiences on Azure
Platform Strategy to Deliver Digital Experiences on Azure
 
Modernizing the Student Journey with Ethos Identity
Modernizing the Student Journey with Ethos IdentityModernizing the Student Journey with Ethos Identity
Modernizing the Student Journey with Ethos Identity
 
Choreo - Build unique digital experiences on WSO2's platform, secured by Etho...
Choreo - Build unique digital experiences on WSO2's platform, secured by Etho...Choreo - Build unique digital experiences on WSO2's platform, secured by Etho...
Choreo - Build unique digital experiences on WSO2's platform, secured by Etho...
 
CIO Summit Berlin 2022.pptx.pdf
CIO Summit Berlin 2022.pptx.pdfCIO Summit Berlin 2022.pptx.pdf
CIO Summit Berlin 2022.pptx.pdf
 
Fueling the Digital Experience Economy with Connected Products
Fueling the Digital Experience Economy with Connected ProductsFueling the Digital Experience Economy with Connected Products
Fueling the Digital Experience Economy with Connected Products
 
A Reference Methodology for Agile Digital Businesses
 A Reference Methodology for Agile Digital Businesses A Reference Methodology for Agile Digital Businesses
A Reference Methodology for Agile Digital Businesses
 
Workflows in WSO2 API Manager - WSO2 API Manager Community Call (12/15/2021)
Workflows in WSO2 API Manager - WSO2 API Manager Community Call (12/15/2021)Workflows in WSO2 API Manager - WSO2 API Manager Community Call (12/15/2021)
Workflows in WSO2 API Manager - WSO2 API Manager Community Call (12/15/2021)
 
Lessons from the pandemic - From a single use case to true transformation
 Lessons from the pandemic - From a single use case to true transformation Lessons from the pandemic - From a single use case to true transformation
Lessons from the pandemic - From a single use case to true transformation
 
Adding Liveliness to Banking Experiences
Adding Liveliness to Banking ExperiencesAdding Liveliness to Banking Experiences
Adding Liveliness to Banking Experiences
 
Building a Future-ready Bank
Building a Future-ready BankBuilding a Future-ready Bank
Building a Future-ready Bank
 
WSO2 API Manager Community Call - November 2021
WSO2 API Manager Community Call - November 2021WSO2 API Manager Community Call - November 2021
WSO2 API Manager Community Call - November 2021
 
[API World ] - Managing Asynchronous APIs
[API World ] - Managing Asynchronous APIs[API World ] - Managing Asynchronous APIs
[API World ] - Managing Asynchronous APIs
 
[API World 2021 ] - Understanding Cloud Native Deployment
[API World 2021 ] - Understanding Cloud Native Deployment[API World 2021 ] - Understanding Cloud Native Deployment
[API World 2021 ] - Understanding Cloud Native Deployment
 
[API Word 2021] - Quantum Duality of “API as a Business and a Technology”
[API Word 2021] - Quantum Duality of “API as a Business and a Technology”[API Word 2021] - Quantum Duality of “API as a Business and a Technology”
[API Word 2021] - Quantum Duality of “API as a Business and a Technology”
 
API Revisions - WSO2 API Manager Community Call (10/27/2021)
API Revisions - WSO2 API Manager Community Call (10/27/2021)API Revisions - WSO2 API Manager Community Call (10/27/2021)
API Revisions - WSO2 API Manager Community Call (10/27/2021)
 
[2021 Somos Summit] - Rethinking Identity Access Management and The Rise of t...
[2021 Somos Summit] - Rethinking Identity Access Management and The Rise of t...[2021 Somos Summit] - Rethinking Identity Access Management and The Rise of t...
[2021 Somos Summit] - Rethinking Identity Access Management and The Rise of t...
 
[ICT Spring 2021] - Managed Crowd: The Future of Business as We Know It!
[ICT Spring 2021] - Managed Crowd: The Future of Business as We Know It![ICT Spring 2021] - Managed Crowd: The Future of Business as We Know It!
[ICT Spring 2021] - Managed Crowd: The Future of Business as We Know It!
 
[EIC 2021] Securing the Digital Double - The Path to a Trusted Digital Ecosystem
[EIC 2021] Securing the Digital Double - The Path to a Trusted Digital Ecosystem[EIC 2021] Securing the Digital Double - The Path to a Trusted Digital Ecosystem
[EIC 2021] Securing the Digital Double - The Path to a Trusted Digital Ecosystem
 
[EIC 2021] The Rise of the Developer in IAM
[EIC 2021] The Rise of the Developer in IAM[EIC 2021] The Rise of the Developer in IAM
[EIC 2021] The Rise of the Developer in IAM
 

Dernier

Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimizationarrow10202532yuvraj
 
Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Juan Carlos Gonzalez
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5DianaGray10
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
The Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementThe Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementNuwan Dias
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideHironori Washizaki
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 

Dernier (20)

Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization
 
Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
The Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementThe Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API Management
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 

Stream Processing in Action

  • 1. Stream Processing in Action Webinar Nirmal Fernando Senior Lead Solutions Engineer, WSO2 Nov, 2018
  • 2. Contents • Introduction • WSO2 Stream Processor Overview • Industry Use Cases • Demo
  • 3. Diverse Industries -> Unique Challenges http://www.solgenie.com/industries/
  • 4. “The price of light is less than the cost of darkness.” - Arthur C. Nielsen, Market Researcher & Founder of ACNielsen
  • 5. Value of Insights Degrade Fast http://www.history.com/news/ask-history/who-determined-the-speed-of-light
  • 7. Stream Processor Core WSO2 ANALYTICS OFFERING 7 ▪ Consumes events, and publish alerts and summarizations to and from various enterprise systems. ▪ Event Processor Core with Streaming Complex Event Processing, Incremental Time Series Aggregations, and Streaming Machine Learning. ▪ Stream Processing Functionalities via Extension Store ▪ High Available and Scalable Analytics Fabric ▪ Prebuilt and custom analytics solutions Events JMS, Thrift, SMTP, HTTP, MQTT, Kafka Analytics Fabric Complex Event Processing Incremental Time Series Aggregation Machine Learning Extension Store FinancialandBanking Analytics RetailAnalytics LocationAnalytics OperationalAnalytics SmartEnergyAnalytics Custom Analytics Solutions ... Solutions
  • 9. WSO2 Stream Processor An open source, cloud-native analytics product optimized to create real-time, actionable insights for agile digital businesses. 9
  • 11. 1. Data collection 2. Data cleansing 3. Data transformation 4. Data enrichment 5. Data summarization 6. Rule processing 7. Machine Learning & Artificial Intelligence 8. Data pipelining 9. Data Publishing 10. On demand processing 11. Data Presentation Stream Processing Patterns
  • 12. ● Lightweight, lean, and high performance ● Best suited for ○ Streaming Data Integration, ○ Streaming Analytics ● Streaming SQL & graphical drag-and-drop editor ● Multiple deployment options ○ Process data at the edge (java, python) ○ Micro Stream Processing ○ High availability with 2 nodes ○ Highly scalable distributed deployments ● Support for streaming ML & Long running aggregations ● Monitoring tools and citizen integration options WSO2 Stream Processor
  • 13. • Source and Sinks – HTTP, Kafka, TCP, Email, JMS, File, Rabbitmq, MQTT, Web-Socket, Twitter, Amazon SQS • Message Formats – JSON, XML, Text, Binary, Key-value, CSV • Data Stores – RDBMS, Solr, MongoDB, HBase, Cassandra, Elasticsearch, Hazelcast, Redis Supported connectors
  • 14. (Streaming) Machine Learning ▪ Running PMML Models for predictions - Built via Apache Spark MLlib, Python, H2O.ai (for deep learning algos) or R - Export as PMML - Load precreated PMML Model into Siddhi to predict in Realtime ▪ Supporting native models for predictions - Spark MLlib Models, Java based Tensorflow Models ▪ Online Learning and predictions - Regression Analytics - Data Classification - K-Means Clustering - Markov Models - Anomaly Detections - … more on the way
  • 15. Supported Extensions • Geo graphical processing • NLP • Graph • Reordering • Timeseries • Import Machine Learning models – PMML, TensorFlow, etc • Streaming Machine Learning – Clustering, Classification, Regression – Makove Models, Anomaly detection, etc. • .. more Image : http://www.weewatch.com/wp-content/uploads/2016/01/28fc45d.jpg 60+ https://store.wso2.com/store/assets/analyticsextension/list
  • 16. High Availability with 2 Nodes • 2 node minimum HA – Process upto 100k events/sec – While most other stream processing systems need around 5+ nodes • Does not require Kafka • Incremental state persistence and recovery • Multi data center support Stream Processor Stream Processor Event Sources Dashboard Notification Invocation Data Source Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Event Store
  • 17. • Exactly-once processing • Fault tolerance • Highly scalable • No back pressure • Distributed via annotations • Native support for Kubernetes Scaling with Distributed Deployment
  • 20. Use Case 1 - Fraud Detection • Detecting fraud via known patterns using generic rules • Detecting unknown types of fraud via machine learning • Detecting rare activity sequences using Markov Modeling • Reduce false alarms using fraud scoring • Caught them in the act - what next? Demo: https://goo.gl/xo6Wf5
  • 21. Use Case 2 - Risk Management • Finding real-time Value at Risk (VaR) – Historical simulation – Variance-covariance – Monte Carlo simulation
  • 22. • Identifying Front Running with Patterns Use Case 3 - Stock Market Surveillance Hey Jude, Mike is going to buy large qty of ABC at $21. You better buy now! Great! I bought. ABC is just $18.9 right now! Trade 1 Followed by Trade 2 Jude sells to Mike at $21. Broker: Bob Client Client MikeJude
  • 23. Use Case 3 - Stock Market Surveillance • Identifying Pump with Regression • Identifying signs of Insider Dealing • Model “Perfect Trader” in order to detect fraudsters
  • 25. Use Case 1 - Recommendations • Recommendations based on the buying products • Recommendations based on the buying history of the customer • Seasonal recommendations • Contextual, intelligent recommendations
  • 26. Use Case 2 - Ad Optimization • Display personalized advertisements on online shopping stores – by identifying person’s living location – by identifying person’s buying history – by identifying person’s interest • Optimize displaying advertisements on shopping stores – what impact the Ad made? Is it worth the cost?
  • 27. • Send personalized information when a customer enters the store • If a customer is spending a considerable amount of time near a shop, suggest offers, send an agent to him etc. Use Case 3 - Proximity Marketing
  • 28. • Detect that the customer leaves the store and send him a gift coupon • Number of customers & employees in each floor (heat maps) • Number of customers & average time spent per product • Shopping path, purchased items, average time spent by each customer • Overall statistics about given and used offers Use Case 3 - Proximity Marketing
  • 30. Use Case - Fleet Management • Real-time monitoring - where is your fleet now?, Visualize your fleet • Geo-fencing based alerts - get alerted if a driver exceeds a defined speed limit within an interactively given geo area • Predicting travel times - historical data can be used to build a machine learning model in order to predict travel times in advance and alert the subscribers
  • 32. Use Cases • Collect data from different stages of business operations • Tracking the progress of your business operations • Detect Service Level Agreement violations at each stage and generate alerts • Visualize your business operations real-time • Generate alerts and notify the subscribers
  • 33. DEMO
  • 34. Setup
  • 35. Download and Tryout WSO2 Stream Processor https://wso2.com/analytics-and-stream-processing/ Documentation
  • 36. Further Reading ● WSO2 Stream Processor: Making Real-time Stream Processing Available to the Masses ● Making Real-Time Applications Simpler with WSO2 Stream Processor ● How to use a Stream Processor as a Notification Manager? ● Synchronous Request-Response based Real-time Processing with WSO2 SP ● Distributed Stream Processing with WSO2 SP