SlideShare une entreprise Scribd logo
1  sur  14
Télécharger pour lire hors ligne
Photo by Leo Rivas on Unsplash
OfirManor
Whatis(data)streaming
Dataintegrationof backendsystems
“realtimeETL”
Oppositeofbatchprocessing
Continuous,incrementalcomputation
Low-latency
“humanrealtime”
Improveoperationaldecisions
by combining historical and real-time data
Retail
real-timepersonalization
real-timesupplychainandlogisticsoptimizations
IOT/ OnlineGaming/Cyber
Healthcare
collect,analyzeandrespondtoreadingsfrom
multiplepatientdevices(biometrics)or
multiplehospitaldevices(Smarthospitals)
FinancialServices
real-timefrauddetection
real-timemobilenotifications
Ad Tech
real-timeusertargeting
IT
EnterpriseServiceBus
STREAMINGPIPELINESOURCES TARGETS
Oracle
DB/2
SQLServer
SAP
Files
Salesforce
EDW
DataLake
(Hadoop)
DataMarts
DataLake(S3)
FileServer
PERSIST
TheStreamingPipelineBuildingBlocks
enterpriseoperationalsourcesanalyticsenvironments
PROCESS
Stateless/Stateful
Code / SQL / UI
CAPTURE
Events
WRITE
Bulk Write
STREAMINGPIPELINESOURCES
Oracle
DB/2
SQLServer
SAP
Files
Salesforce
TARGETS
EDW
DataLake
(Hadoop)
DataMarts
DataLake(S3)
FileServer
PERSIST
PROCESS
CAPTURE
WRITE
Database
MigrationService
Others
Kinesis
DataAnalytics
Kinesis
DataFirehose
Kinesis
DataStreams
EMR/Glue
Example: AWS-centricarchitecture
STREAMINGPIPELINESOURCES
Oracle
DB/2
SQLServer
SAP
Files
Salesforce
TARGETS
EDW
DataLake
(Hadoop)
DataMarts
DataLake(S3)
FileServer
PERSIST
PROCESS
CAPTURE
WRITE
Example: Kafka-centricarchitecture
KafkaStreams
KSQL(Confluent)
KafkaConnect
KafkaConnect
Debezium
KafkaCore
STREAMINGPIPELINESOURCES
Oracle
DB/2
SQLServer
SAP
Files
Salesforce
TARGETS
EDW
DataLake
(Hadoop)
DataMarts
DataLake(S3)
FileServer
PERSIST
PROCESS
CAPTURE
Example: Kafka-centricvariant
KafkaConnect
Debezium
KafkaCore
ApacheSpark
ApacheFlink
WRITE
KafkaConnect
Justpick yourtools
GoogleCloudDataflow
Cloud
services
Open
source
technologies
Commercial
products
Summary
What?
Continuouslymovingdatabetweensystemswhileapplyingcomputations
Why?
Improveoperationaldecisionsbycombininghistoricalandreal-timedata
How?
CAPTURE  PERSIST  PROCESS  WRITE
Developer/Analyst/ ETLuser ↔ code/SQL/ drag-and-dropUI
Cloudservices↔Products Assemble↔Distribution Specialsources/targets
Photo by Leo Rivas on Unsplash
OfirManor
@ofirm
slideshare.net/ofirm

Contenu connexe

Tendances

using big-data methods analyse the Cross platform aviation
 using big-data methods analyse the Cross platform aviation using big-data methods analyse the Cross platform aviation
using big-data methods analyse the Cross platform aviation
ranjit banshpal
 

Tendances (19)

Clustrix Infographic
Clustrix InfographicClustrix Infographic
Clustrix Infographic
 
Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...
Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...
Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...
 
Data Science: Philosopher's Stone
Data Science: Philosopher's StoneData Science: Philosopher's Stone
Data Science: Philosopher's Stone
 
using big-data methods analyse the Cross platform aviation
 using big-data methods analyse the Cross platform aviation using big-data methods analyse the Cross platform aviation
using big-data methods analyse the Cross platform aviation
 
Spark nas-nuvens
Spark nas-nuvensSpark nas-nuvens
Spark nas-nuvens
 
ITCamp 2018 - Magnus Mårtensson - Azure Global Application Perspectives
ITCamp 2018 - Magnus Mårtensson - Azure Global Application PerspectivesITCamp 2018 - Magnus Mårtensson - Azure Global Application Perspectives
ITCamp 2018 - Magnus Mårtensson - Azure Global Application Perspectives
 
Cloudant
CloudantCloudant
Cloudant
 
"The Future of Manufacturing" by Sujeet Chand, SVP&CTO, Rockwell Automation
"The Future of Manufacturing" by Sujeet Chand, SVP&CTO, Rockwell Automation"The Future of Manufacturing" by Sujeet Chand, SVP&CTO, Rockwell Automation
"The Future of Manufacturing" by Sujeet Chand, SVP&CTO, Rockwell Automation
 
Who is spending on cloud report
Who is spending on cloud reportWho is spending on cloud report
Who is spending on cloud report
 
Apache Kafka in the Healthcare Industry
Apache Kafka in the Healthcare IndustryApache Kafka in the Healthcare Industry
Apache Kafka in the Healthcare Industry
 
Data management trends to watch out for in 2019
Data management trends to watch out for in 2019Data management trends to watch out for in 2019
Data management trends to watch out for in 2019
 
Clustrix Presents at Under the Radar 2013
Clustrix Presents at Under the Radar 2013Clustrix Presents at Under the Radar 2013
Clustrix Presents at Under the Radar 2013
 
International Journal on Cloud Computing: Services and Architecture (IJCCSA)
International Journal on Cloud Computing: Services and Architecture (IJCCSA)International Journal on Cloud Computing: Services and Architecture (IJCCSA)
International Journal on Cloud Computing: Services and Architecture (IJCCSA)
 
BDW Chicago 2016 - John K. Thompson, GM for Advanced Analytics Dell Statisti...
BDW Chicago 2016 - John K. Thompson, GM for Advanced Analytics  Dell Statisti...BDW Chicago 2016 - John K. Thompson, GM for Advanced Analytics  Dell Statisti...
BDW Chicago 2016 - John K. Thompson, GM for Advanced Analytics Dell Statisti...
 
Apache Kafka in the Healthcare Industry
Apache Kafka in the Healthcare IndustryApache Kafka in the Healthcare Industry
Apache Kafka in the Healthcare Industry
 
What is Big Data?
What is Big Data?What is Big Data?
What is Big Data?
 
Databricks delta
Databricks deltaDatabricks delta
Databricks delta
 
Edmc use cases 2018 v2
Edmc use cases 2018   v2Edmc use cases 2018   v2
Edmc use cases 2018 v2
 
Big Data analytics
Big Data analyticsBig Data analytics
Big Data analytics
 

Similaire à Data Streaming: What? Why? How?

Unifying Analytics
Unifying AnalyticsUnifying Analytics
Unifying Analytics
Data Con LA
 
Connected IoT and Intelligent Solutions
Connected IoT and Intelligent SolutionsConnected IoT and Intelligent Solutions
Connected IoT and Intelligent Solutions
Amazon Web Services
 
Introduction to Big Data An analogy between Sugar Cane & Big Data
Introduction to Big Data An analogy  between Sugar Cane & Big DataIntroduction to Big Data An analogy  between Sugar Cane & Big Data
Introduction to Big Data An analogy between Sugar Cane & Big Data
Jean-Marc Desvaux
 
BSides Lisbon - Data science, machine learning and cybersecurity
BSides Lisbon - Data science, machine learning and cybersecurity BSides Lisbon - Data science, machine learning and cybersecurity
BSides Lisbon - Data science, machine learning and cybersecurity
Tiago Henriques
 

Similaire à Data Streaming: What? Why? How? (20)

Picking the right streaming tools for the job
Picking the right streaming tools for the jobPicking the right streaming tools for the job
Picking the right streaming tools for the job
 
Unifying Analytics
Unifying AnalyticsUnifying Analytics
Unifying Analytics
 
Connected IoT and Intelligent Solutions
Connected IoT and Intelligent SolutionsConnected IoT and Intelligent Solutions
Connected IoT and Intelligent Solutions
 
Unify Analytics: Combine Strengths of Data Lake and Data Warehouse
Unify Analytics: Combine Strengths of Data Lake and Data WarehouseUnify Analytics: Combine Strengths of Data Lake and Data Warehouse
Unify Analytics: Combine Strengths of Data Lake and Data Warehouse
 
Introduction to Big Data An analogy between Sugar Cane & Big Data
Introduction to Big Data An analogy  between Sugar Cane & Big DataIntroduction to Big Data An analogy  between Sugar Cane & Big Data
Introduction to Big Data An analogy between Sugar Cane & Big Data
 
Architecting for Real-Time Big Data Analytics
Architecting for Real-Time Big Data AnalyticsArchitecting for Real-Time Big Data Analytics
Architecting for Real-Time Big Data Analytics
 
Hitachi Data Systems Big Data Roadmap
Hitachi Data Systems Big Data RoadmapHitachi Data Systems Big Data Roadmap
Hitachi Data Systems Big Data Roadmap
 
Denodo DataFest 2017: Lowering IT Costs with Big Data and Cloud Modernization
Denodo DataFest 2017: Lowering IT Costs with Big Data and Cloud ModernizationDenodo DataFest 2017: Lowering IT Costs with Big Data and Cloud Modernization
Denodo DataFest 2017: Lowering IT Costs with Big Data and Cloud Modernization
 
Mai-Lan Tomsen Bukovec- Keynote-AWS Summit Manila
Mai-Lan Tomsen Bukovec- Keynote-AWS Summit ManilaMai-Lan Tomsen Bukovec- Keynote-AWS Summit Manila
Mai-Lan Tomsen Bukovec- Keynote-AWS Summit Manila
 
BSides Lisbon - Data science, machine learning and cybersecurity
BSides Lisbon - Data science, machine learning and cybersecurity BSides Lisbon - Data science, machine learning and cybersecurity
BSides Lisbon - Data science, machine learning and cybersecurity
 
Using Elastic to Monitor Everything - Christoph Wurm, Elastic - DevOpsDays Te...
Using Elastic to Monitor Everything - Christoph Wurm, Elastic - DevOpsDays Te...Using Elastic to Monitor Everything - Christoph Wurm, Elastic - DevOpsDays Te...
Using Elastic to Monitor Everything - Christoph Wurm, Elastic - DevOpsDays Te...
 
Salesforce iot
Salesforce iotSalesforce iot
Salesforce iot
 
Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3
 
Big Data LDN 2018: THE THIRD REVOLUTION IN ANALYTICS
Big Data LDN 2018: THE THIRD REVOLUTION IN ANALYTICSBig Data LDN 2018: THE THIRD REVOLUTION IN ANALYTICS
Big Data LDN 2018: THE THIRD REVOLUTION IN ANALYTICS
 
Are you ready for IoT disruption? by Ana Seliškar
Are you ready for IoT disruption? by Ana SeliškarAre you ready for IoT disruption? by Ana Seliškar
Are you ready for IoT disruption? by Ana Seliškar
 
“A Distributed Operational and Informational Technological Stack”
“A Distributed Operational and Informational Technological Stack” “A Distributed Operational and Informational Technological Stack”
“A Distributed Operational and Informational Technological Stack”
 
AWS Loft Dublin Welcome
AWS Loft Dublin WelcomeAWS Loft Dublin Welcome
AWS Loft Dublin Welcome
 
10-Hot-Data-Analytics-Tre-8904178.ppsx
10-Hot-Data-Analytics-Tre-8904178.ppsx10-Hot-Data-Analytics-Tre-8904178.ppsx
10-Hot-Data-Analytics-Tre-8904178.ppsx
 
Big Data Companies and Apache Software
Big Data Companies and Apache SoftwareBig Data Companies and Apache Software
Big Data Companies and Apache Software
 
Opening Keynote - AWS Summit SG 2017
Opening Keynote - AWS Summit SG 2017Opening Keynote - AWS Summit SG 2017
Opening Keynote - AWS Summit SG 2017
 

Dernier

Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
amitlee9823
 
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
amitlee9823
 
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
AroojKhan71
 
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
amitlee9823
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
amitlee9823
 
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
amitlee9823
 
Probability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter LessonsProbability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter Lessons
JoseMangaJr1
 
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
amitlee9823
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
amitlee9823
 

Dernier (20)

ELKO dropshipping via API with DroFx.pptx
ELKO dropshipping via API with DroFx.pptxELKO dropshipping via API with DroFx.pptx
ELKO dropshipping via API with DroFx.pptx
 
Capstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics ProgramCapstone Project on IBM Data Analytics Program
Capstone Project on IBM Data Analytics Program
 
Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% Secure
Call me @ 9892124323  Cheap Rate Call Girls in Vashi with Real Photo 100% SecureCall me @ 9892124323  Cheap Rate Call Girls in Vashi with Real Photo 100% Secure
Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% Secure
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
 
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
 
Invezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signalsInvezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signals
 
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
 
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...
 
Midocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFxMidocean dropshipping via API with DroFx
Midocean dropshipping via API with DroFx
 
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...
 
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...
 
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
 
Accredited-Transport-Cooperatives-Jan-2021-Web.pdf
Accredited-Transport-Cooperatives-Jan-2021-Web.pdfAccredited-Transport-Cooperatives-Jan-2021-Web.pdf
Accredited-Transport-Cooperatives-Jan-2021-Web.pdf
 
Probability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter LessonsProbability Grade 10 Third Quarter Lessons
Probability Grade 10 Third Quarter Lessons
 
CebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxCebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptx
 
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
 
Edukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxEdukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFx
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
Week-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionWeek-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interaction
 

Data Streaming: What? Why? How?