Soumettre la recherche
Mettre en ligne
Data Microservices with Spring Cloud Stream, Task, and Data Flow #jsug #springday
•
20 j'aime
•
22,428 vues
Toshiaki Maki
Suivre
Spring Cloud Data Flow #jsug #springday
Lire moins
Lire la suite
Technologie
Signaler
Partager
Signaler
Partager
1 sur 117
Télécharger maintenant
Télécharger pour lire hors ligne
Recommandé
Yahoo! JAPANのコンテンツプラットフォームを支えるSpring Cloud Streamによるマイクロサービスアーキテクチャ #jsug #sf_52
Yahoo! JAPANのコンテンツプラットフォームを支えるSpring Cloud Streamによるマイクロサービスアーキテクチャ #jsug #sf_52
Yahoo!デベロッパーネットワーク
マイクロサービスに必要な技術要素はすべてSpring Cloudにある #DO07
マイクロサービスに必要な技術要素はすべてSpring Cloudにある #DO07
Toshiaki Maki
大規模・長期保守を見据えたエンタープライズシステム開発へのSpring Frameworkの適用
大規模・長期保守を見据えたエンタープライズシステム開発へのSpring Frameworkの適用
apkiban
AWS で Presto を徹底的に使いこなすワザ
AWS で Presto を徹底的に使いこなすワザ
Noritaka Sekiyama
今こそ知りたいSpring Web(Spring Fest 2020講演資料)
今こそ知りたいSpring Web(Spring Fest 2020講演資料)
NTT DATA Technology & Innovation
Apache Kafkaって本当に大丈夫?~故障検証のオーバービューと興味深い挙動の紹介~
Apache Kafkaって本当に大丈夫?~故障検証のオーバービューと興味深い挙動の紹介~
NTT DATA OSS Professional Services
AWS Transfer Family SFTP and FTPS
AWS Transfer Family SFTP and FTPS
Kameda Harunobu
今だから!Amazon CloudFront 徹底活用
今だから!Amazon CloudFront 徹底活用
Yasuhiro Araki, Ph.D
Recommandé
Yahoo! JAPANのコンテンツプラットフォームを支えるSpring Cloud Streamによるマイクロサービスアーキテクチャ #jsug #sf_52
Yahoo! JAPANのコンテンツプラットフォームを支えるSpring Cloud Streamによるマイクロサービスアーキテクチャ #jsug #sf_52
Yahoo!デベロッパーネットワーク
マイクロサービスに必要な技術要素はすべてSpring Cloudにある #DO07
マイクロサービスに必要な技術要素はすべてSpring Cloudにある #DO07
Toshiaki Maki
大規模・長期保守を見据えたエンタープライズシステム開発へのSpring Frameworkの適用
大規模・長期保守を見据えたエンタープライズシステム開発へのSpring Frameworkの適用
apkiban
AWS で Presto を徹底的に使いこなすワザ
AWS で Presto を徹底的に使いこなすワザ
Noritaka Sekiyama
今こそ知りたいSpring Web(Spring Fest 2020講演資料)
今こそ知りたいSpring Web(Spring Fest 2020講演資料)
NTT DATA Technology & Innovation
Apache Kafkaって本当に大丈夫?~故障検証のオーバービューと興味深い挙動の紹介~
Apache Kafkaって本当に大丈夫?~故障検証のオーバービューと興味深い挙動の紹介~
NTT DATA OSS Professional Services
AWS Transfer Family SFTP and FTPS
AWS Transfer Family SFTP and FTPS
Kameda Harunobu
今だから!Amazon CloudFront 徹底活用
今だから!Amazon CloudFront 徹底活用
Yasuhiro Araki, Ph.D
[CTO Night & Day 2019] AWS で構築するデータレイク基盤と amazon.com での導入事例 #ctonight
[CTO Night & Day 2019] AWS で構築するデータレイク基盤と amazon.com での導入事例 #ctonight
Amazon Web Services Japan
Azure API Management 俺的マニュアル
Azure API Management 俺的マニュアル
貴志 上坂
Garbage First Garbage Collection (G1 GC) #jjug_ccc #ccc_cd6
Garbage First Garbage Collection (G1 GC) #jjug_ccc #ccc_cd6
Yuji Kubota
At least onceってぶっちゃけ問題の先送りだったよね #kafkajp
At least onceってぶっちゃけ問題の先送りだったよね #kafkajp
Yahoo!デベロッパーネットワーク
20200303 AWS Black Belt Online Seminar AWS Cloud Development Kit (CDK)
20200303 AWS Black Belt Online Seminar AWS Cloud Development Kit (CDK)
Amazon Web Services Japan
[Aurora事例祭り]Amazon Aurora を使いこなすためのベストプラクティス
[Aurora事例祭り]Amazon Aurora を使いこなすためのベストプラクティス
Amazon Web Services Japan
Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push
Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push
Lucas Jellema
NGINXをBFF (Backend for Frontend)として利用した話
NGINXをBFF (Backend for Frontend)として利用した話
Hitachi, Ltd. OSS Solution Center.
単なるキャッシュじゃないよ!?infinispanの紹介
単なるキャッシュじゃないよ!?infinispanの紹介
AdvancedTechNight
AWSのログ管理ベストプラクティス
AWSのログ管理ベストプラクティス
Akihiro Kuwano
20201028 AWS Black Belt Online Seminar Amazon CloudFront deep dive
20201028 AWS Black Belt Online Seminar Amazon CloudFront deep dive
Amazon Web Services Japan
はじめてのDynamoDBスキーマ設計
はじめてのDynamoDBスキーマ設計
Yoichi Toyota
Getting Started GraalVM / GraalVM超入門 #jjug_ccc #ccc_c2
Getting Started GraalVM / GraalVM超入門 #jjug_ccc #ccc_c2
tamtam180
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
Amazon Web Services Japan
わたくし、やっぱりCDKを使いたいですわ〜CDK import編〜.pdf
わたくし、やっぱりCDKを使いたいですわ〜CDK import編〜.pdf
ssuser868e2d
AWS Black Belt Online Seminar 2018 Amazon DynamoDB Advanced Design Pattern
AWS Black Belt Online Seminar 2018 Amazon DynamoDB Advanced Design Pattern
Amazon Web Services Japan
20200930 AWS Black Belt Online Seminar Amazon Kinesis Video Streams
20200930 AWS Black Belt Online Seminar Amazon Kinesis Video Streams
Amazon Web Services Japan
Spring CloudとZipkinを利用した分散トレーシング
Spring CloudとZipkinを利用した分散トレーシング
Rakuten Group, Inc.
20191001 AWS Black Belt Online Seminar AWS Lake Formation
20191001 AWS Black Belt Online Seminar AWS Lake Formation
Amazon Web Services Japan
モノリスからマイクロサービスへの移行 ~ストラングラーパターンの検証~(Spring Fest 2020講演資料)
モノリスからマイクロサービスへの移行 ~ストラングラーパターンの検証~(Spring Fest 2020講演資料)
NTT DATA Technology & Innovation
SpringOne 2GX 2014 参加報告 & Spring 4.1について #jsug
SpringOne 2GX 2014 参加報告 & Spring 4.1について #jsug
Toshiaki Maki
Event Driven Microservices with Spring Cloud Stream #jjug_ccc #ccc_ab3
Event Driven Microservices with Spring Cloud Stream #jjug_ccc #ccc_ab3
Toshiaki Maki
Contenu connexe
Tendances
[CTO Night & Day 2019] AWS で構築するデータレイク基盤と amazon.com での導入事例 #ctonight
[CTO Night & Day 2019] AWS で構築するデータレイク基盤と amazon.com での導入事例 #ctonight
Amazon Web Services Japan
Azure API Management 俺的マニュアル
Azure API Management 俺的マニュアル
貴志 上坂
Garbage First Garbage Collection (G1 GC) #jjug_ccc #ccc_cd6
Garbage First Garbage Collection (G1 GC) #jjug_ccc #ccc_cd6
Yuji Kubota
At least onceってぶっちゃけ問題の先送りだったよね #kafkajp
At least onceってぶっちゃけ問題の先送りだったよね #kafkajp
Yahoo!デベロッパーネットワーク
20200303 AWS Black Belt Online Seminar AWS Cloud Development Kit (CDK)
20200303 AWS Black Belt Online Seminar AWS Cloud Development Kit (CDK)
Amazon Web Services Japan
[Aurora事例祭り]Amazon Aurora を使いこなすためのベストプラクティス
[Aurora事例祭り]Amazon Aurora を使いこなすためのベストプラクティス
Amazon Web Services Japan
Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push
Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push
Lucas Jellema
NGINXをBFF (Backend for Frontend)として利用した話
NGINXをBFF (Backend for Frontend)として利用した話
Hitachi, Ltd. OSS Solution Center.
単なるキャッシュじゃないよ!?infinispanの紹介
単なるキャッシュじゃないよ!?infinispanの紹介
AdvancedTechNight
AWSのログ管理ベストプラクティス
AWSのログ管理ベストプラクティス
Akihiro Kuwano
20201028 AWS Black Belt Online Seminar Amazon CloudFront deep dive
20201028 AWS Black Belt Online Seminar Amazon CloudFront deep dive
Amazon Web Services Japan
はじめてのDynamoDBスキーマ設計
はじめてのDynamoDBスキーマ設計
Yoichi Toyota
Getting Started GraalVM / GraalVM超入門 #jjug_ccc #ccc_c2
Getting Started GraalVM / GraalVM超入門 #jjug_ccc #ccc_c2
tamtam180
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
Amazon Web Services Japan
わたくし、やっぱりCDKを使いたいですわ〜CDK import編〜.pdf
わたくし、やっぱりCDKを使いたいですわ〜CDK import編〜.pdf
ssuser868e2d
AWS Black Belt Online Seminar 2018 Amazon DynamoDB Advanced Design Pattern
AWS Black Belt Online Seminar 2018 Amazon DynamoDB Advanced Design Pattern
Amazon Web Services Japan
20200930 AWS Black Belt Online Seminar Amazon Kinesis Video Streams
20200930 AWS Black Belt Online Seminar Amazon Kinesis Video Streams
Amazon Web Services Japan
Spring CloudとZipkinを利用した分散トレーシング
Spring CloudとZipkinを利用した分散トレーシング
Rakuten Group, Inc.
20191001 AWS Black Belt Online Seminar AWS Lake Formation
20191001 AWS Black Belt Online Seminar AWS Lake Formation
Amazon Web Services Japan
モノリスからマイクロサービスへの移行 ~ストラングラーパターンの検証~(Spring Fest 2020講演資料)
モノリスからマイクロサービスへの移行 ~ストラングラーパターンの検証~(Spring Fest 2020講演資料)
NTT DATA Technology & Innovation
Tendances
(20)
[CTO Night & Day 2019] AWS で構築するデータレイク基盤と amazon.com での導入事例 #ctonight
[CTO Night & Day 2019] AWS で構築するデータレイク基盤と amazon.com での導入事例 #ctonight
Azure API Management 俺的マニュアル
Azure API Management 俺的マニュアル
Garbage First Garbage Collection (G1 GC) #jjug_ccc #ccc_cd6
Garbage First Garbage Collection (G1 GC) #jjug_ccc #ccc_cd6
At least onceってぶっちゃけ問題の先送りだったよね #kafkajp
At least onceってぶっちゃけ問題の先送りだったよね #kafkajp
20200303 AWS Black Belt Online Seminar AWS Cloud Development Kit (CDK)
20200303 AWS Black Belt Online Seminar AWS Cloud Development Kit (CDK)
[Aurora事例祭り]Amazon Aurora を使いこなすためのベストプラクティス
[Aurora事例祭り]Amazon Aurora を使いこなすためのベストプラクティス
Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push
Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push
NGINXをBFF (Backend for Frontend)として利用した話
NGINXをBFF (Backend for Frontend)として利用した話
単なるキャッシュじゃないよ!?infinispanの紹介
単なるキャッシュじゃないよ!?infinispanの紹介
AWSのログ管理ベストプラクティス
AWSのログ管理ベストプラクティス
20201028 AWS Black Belt Online Seminar Amazon CloudFront deep dive
20201028 AWS Black Belt Online Seminar Amazon CloudFront deep dive
はじめてのDynamoDBスキーマ設計
はじめてのDynamoDBスキーマ設計
Getting Started GraalVM / GraalVM超入門 #jjug_ccc #ccc_c2
Getting Started GraalVM / GraalVM超入門 #jjug_ccc #ccc_c2
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
20200331 AWS Black Belt Online Seminar AWS Elemental MediaConvert
わたくし、やっぱりCDKを使いたいですわ〜CDK import編〜.pdf
わたくし、やっぱりCDKを使いたいですわ〜CDK import編〜.pdf
AWS Black Belt Online Seminar 2018 Amazon DynamoDB Advanced Design Pattern
AWS Black Belt Online Seminar 2018 Amazon DynamoDB Advanced Design Pattern
20200930 AWS Black Belt Online Seminar Amazon Kinesis Video Streams
20200930 AWS Black Belt Online Seminar Amazon Kinesis Video Streams
Spring CloudとZipkinを利用した分散トレーシング
Spring CloudとZipkinを利用した分散トレーシング
20191001 AWS Black Belt Online Seminar AWS Lake Formation
20191001 AWS Black Belt Online Seminar AWS Lake Formation
モノリスからマイクロサービスへの移行 ~ストラングラーパターンの検証~(Spring Fest 2020講演資料)
モノリスからマイクロサービスへの移行 ~ストラングラーパターンの検証~(Spring Fest 2020講演資料)
En vedette
SpringOne 2GX 2014 参加報告 & Spring 4.1について #jsug
SpringOne 2GX 2014 参加報告 & Spring 4.1について #jsug
Toshiaki Maki
Event Driven Microservices with Spring Cloud Stream #jjug_ccc #ccc_ab3
Event Driven Microservices with Spring Cloud Stream #jjug_ccc #ccc_ab3
Toshiaki Maki
Spring Day 2016 - Web API アクセス制御の最適解
Spring Day 2016 - Web API アクセス制御の最適解
都元ダイスケ Miyamoto
Lineにおけるspring frameworkの活用
Lineにおけるspring frameworkの活用
Tokuhiro Matsuno
Spring Day 2016 springの現在過去未来
Spring Day 2016 springの現在過去未来
Yuichi Hasegawa
Spring bootで学ぶ初めてのwebアプリ開発
Spring bootで学ぶ初めてのwebアプリ開発
terahide
アメブロの大規模システム刷新と それを支えるSpring
アメブロの大規模システム刷新と それを支えるSpring
Takuya Hattori
Spring 5に備えるリアクティブプログラミング入門
Spring 5に備えるリアクティブプログラミング入門
Takuya Iwatsuka
Springを使ったwebアプリにリファクタリングしよう
Springを使ったwebアプリにリファクタリングしよう
土岐 孝平
Application Re-Architecture Technology ~ StrutsからSpring MVCへ ~
Application Re-Architecture Technology ~ StrutsからSpring MVCへ ~
Yuichi Hasegawa
Grailsでドメイン駆動設計を実践する時の勘所
Grailsでドメイン駆動設計を実践する時の勘所
Takuma Watabiki
Cloud Foundry x Wagby
Cloud Foundry x Wagby
Yoshinori Nie
楽天トラベルとSpring(Spring Day 2016)
楽天トラベルとSpring(Spring Day 2016)
Rakuten Group, Inc.
ビッグじゃなくても使えるSpark Streaming
ビッグじゃなくても使えるSpark Streaming
chibochibo
Distributed Tracing Velocity2016
Distributed Tracing Velocity2016
Reshmi Krishna
Distributed tracing - get a grasp on your production
Distributed tracing - get a grasp on your production
nklmish
形態素解析
形態素解析
Works Applications
WalB: Real-time and Incremental Backup System for Block Devices
WalB: Real-time and Incremental Backup System for Block Devices
uchan_nos
3000社の業務データ絞り込みを支える技術
3000社の業務データ絞り込みを支える技術
Ryo Mitoma
離れた場所でも最高のチームワークを実現する方法 ーサイボウズ開発チームのリモートワーク事例ー
離れた場所でも最高のチームワークを実現する方法 ーサイボウズ開発チームのリモートワーク事例ー
Teppei Sato
En vedette
(20)
SpringOne 2GX 2014 参加報告 & Spring 4.1について #jsug
SpringOne 2GX 2014 参加報告 & Spring 4.1について #jsug
Event Driven Microservices with Spring Cloud Stream #jjug_ccc #ccc_ab3
Event Driven Microservices with Spring Cloud Stream #jjug_ccc #ccc_ab3
Spring Day 2016 - Web API アクセス制御の最適解
Spring Day 2016 - Web API アクセス制御の最適解
Lineにおけるspring frameworkの活用
Lineにおけるspring frameworkの活用
Spring Day 2016 springの現在過去未来
Spring Day 2016 springの現在過去未来
Spring bootで学ぶ初めてのwebアプリ開発
Spring bootで学ぶ初めてのwebアプリ開発
アメブロの大規模システム刷新と それを支えるSpring
アメブロの大規模システム刷新と それを支えるSpring
Spring 5に備えるリアクティブプログラミング入門
Spring 5に備えるリアクティブプログラミング入門
Springを使ったwebアプリにリファクタリングしよう
Springを使ったwebアプリにリファクタリングしよう
Application Re-Architecture Technology ~ StrutsからSpring MVCへ ~
Application Re-Architecture Technology ~ StrutsからSpring MVCへ ~
Grailsでドメイン駆動設計を実践する時の勘所
Grailsでドメイン駆動設計を実践する時の勘所
Cloud Foundry x Wagby
Cloud Foundry x Wagby
楽天トラベルとSpring(Spring Day 2016)
楽天トラベルとSpring(Spring Day 2016)
ビッグじゃなくても使えるSpark Streaming
ビッグじゃなくても使えるSpark Streaming
Distributed Tracing Velocity2016
Distributed Tracing Velocity2016
Distributed tracing - get a grasp on your production
Distributed tracing - get a grasp on your production
形態素解析
形態素解析
WalB: Real-time and Incremental Backup System for Block Devices
WalB: Real-time and Incremental Backup System for Block Devices
3000社の業務データ絞り込みを支える技術
3000社の業務データ絞り込みを支える技術
離れた場所でも最高のチームワークを実現する方法 ーサイボウズ開発チームのリモートワーク事例ー
離れた場所でも最高のチームワークを実現する方法 ーサイボウズ開発チームのリモートワーク事例ー
Similaire à Data Microservices with Spring Cloud Stream, Task, and Data Flow #jsug #springday
Spring Cloud Servicesの紹介 #pcf_tokyo
Spring Cloud Servicesの紹介 #pcf_tokyo
Toshiaki Maki
Meetup: Streaming Data Pipeline Development
Meetup: Streaming Data Pipeline Development
Timothy Spann
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
Timothy Spann
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
Timothy Spann
Spring Framework 5.0による Reactive Web Application #JavaDayTokyo
Spring Framework 5.0による Reactive Web Application #JavaDayTokyo
Toshiaki Maki
Delivering the power of data using Spring Cloud DataFlow and DataStax Enterpr...
Delivering the power of data using Spring Cloud DataFlow and DataStax Enterpr...
VMware Tanzu
Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...
Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...
Docker, Inc.
Hadoop application architectures - using Customer 360 as an example
Hadoop application architectures - using Customer 360 as an example
hadooparchbook
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
VMware Tanzu
Episode 3: Kubernetes and Big Data Services
Episode 3: Kubernetes and Big Data Services
Mesosphere Inc.
Azure HDlnsight에서 R 및 Spark를 이용하여 확장 가능한 머신러닝
Azure HDlnsight에서 R 및 Spark를 이용하여 확장 가능한 머신러닝
OSS On Azure
Big Data Analytics with Spark
Big Data Analytics with Spark
DataStax Academy
Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...
Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...
Amazon Web Services
The Heart of the Data Mesh Beats in Real-Time with Apache Kafka
The Heart of the Data Mesh Beats in Real-Time with Apache Kafka
Kai Wähner
Analyzing and processing streaming data with Amazon EMR - ADB204 - New York A...
Analyzing and processing streaming data with Amazon EMR - ADB204 - New York A...
Amazon Web Services
Data Science at Scale on MPP databases - Use Cases & Open Source Tools
Data Science at Scale on MPP databases - Use Cases & Open Source Tools
Esther Vasiete
Fraud Detection for Israel BigThings Meetup
Fraud Detection for Israel BigThings Meetup
Gwen (Chen) Shapira
DevOps Days Tel Aviv - Serverless Architecture
DevOps Days Tel Aviv - Serverless Architecture
Antons Kranga
Spark Streaming with Azure Databricks
Spark Streaming with Azure Databricks
Dustin Vannoy
Spark Summit EU talk by Christos Erotocritou
Spark Summit EU talk by Christos Erotocritou
Spark Summit
Similaire à Data Microservices with Spring Cloud Stream, Task, and Data Flow #jsug #springday
(20)
Spring Cloud Servicesの紹介 #pcf_tokyo
Spring Cloud Servicesの紹介 #pcf_tokyo
Meetup: Streaming Data Pipeline Development
Meetup: Streaming Data Pipeline Development
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
Spring Framework 5.0による Reactive Web Application #JavaDayTokyo
Spring Framework 5.0による Reactive Web Application #JavaDayTokyo
Delivering the power of data using Spring Cloud DataFlow and DataStax Enterpr...
Delivering the power of data using Spring Cloud DataFlow and DataStax Enterpr...
Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...
Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...
Hadoop application architectures - using Customer 360 as an example
Hadoop application architectures - using Customer 360 as an example
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
Real-time Analysis of Data Processing Pipelines with Spring Cloud Data Flow a...
Episode 3: Kubernetes and Big Data Services
Episode 3: Kubernetes and Big Data Services
Azure HDlnsight에서 R 및 Spark를 이용하여 확장 가능한 머신러닝
Azure HDlnsight에서 R 및 Spark를 이용하여 확장 가능한 머신러닝
Big Data Analytics with Spark
Big Data Analytics with Spark
Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...
Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...
The Heart of the Data Mesh Beats in Real-Time with Apache Kafka
The Heart of the Data Mesh Beats in Real-Time with Apache Kafka
Analyzing and processing streaming data with Amazon EMR - ADB204 - New York A...
Analyzing and processing streaming data with Amazon EMR - ADB204 - New York A...
Data Science at Scale on MPP databases - Use Cases & Open Source Tools
Data Science at Scale on MPP databases - Use Cases & Open Source Tools
Fraud Detection for Israel BigThings Meetup
Fraud Detection for Israel BigThings Meetup
DevOps Days Tel Aviv - Serverless Architecture
DevOps Days Tel Aviv - Serverless Architecture
Spark Streaming with Azure Databricks
Spark Streaming with Azure Databricks
Spark Summit EU talk by Christos Erotocritou
Spark Summit EU talk by Christos Erotocritou
Plus de Toshiaki Maki
From Spring Boot 2.2 to Spring Boot 2.3 #jsug
From Spring Boot 2.2 to Spring Boot 2.3 #jsug
Toshiaki Maki
Concourse x Spinnaker #concourse_tokyo
Concourse x Spinnaker #concourse_tokyo
Toshiaki Maki
Serverless with Spring Cloud Function, Knative and riff #SpringOneTour #s1t
Serverless with Spring Cloud Function, Knative and riff #SpringOneTour #s1t
Toshiaki Maki
決済システムの内製化への旅 - SpringとPCFで作るクラウドネイティブなシステム開発 #jsug #sf_h1
決済システムの内製化への旅 - SpringとPCFで作るクラウドネイティブなシステム開発 #jsug #sf_h1
Toshiaki Maki
Spring Boot Actuator 2.0 & Micrometer #jjug_ccc #ccc_a1
Spring Boot Actuator 2.0 & Micrometer #jjug_ccc #ccc_a1
Toshiaki Maki
Spring Boot Actuator 2.0 & Micrometer
Spring Boot Actuator 2.0 & Micrometer
Toshiaki Maki
Open Service Broker APIとKubernetes Service Catalog #k8sjp
Open Service Broker APIとKubernetes Service Catalog #k8sjp
Toshiaki Maki
Spring Cloud Function & Project riff #jsug
Spring Cloud Function & Project riff #jsug
Toshiaki Maki
Introduction to Spring WebFlux #jsug #sf_a1
Introduction to Spring WebFlux #jsug #sf_a1
Toshiaki Maki
BOSH / CF Deployment in modern ways #cf_tokyo
BOSH / CF Deployment in modern ways #cf_tokyo
Toshiaki Maki
Why PCF is the best platform for Spring Boot
Why PCF is the best platform for Spring Boot
Toshiaki Maki
Zipkin Components #zipkin_jp
Zipkin Components #zipkin_jp
Toshiaki Maki
実例で学ぶ、明日から使えるSpring Boot Tips #jsug
実例で学ぶ、明日から使えるSpring Boot Tips #jsug
Toshiaki Maki
Spring ❤️ Kotlin #jjug
Spring ❤️ Kotlin #jjug
Toshiaki Maki
Managing your Docker image continuously with Concourse CI
Managing your Docker image continuously with Concourse CI
Toshiaki Maki
Short Lived Tasks in Cloud Foundry #cfdtokyo
Short Lived Tasks in Cloud Foundry #cfdtokyo
Toshiaki Maki
今すぐ始めるCloud Foundry #hackt #hackt_k
今すぐ始めるCloud Foundry #hackt #hackt_k
Toshiaki Maki
Team Support in Concourse CI 2.0 #concourse_tokyo
Team Support in Concourse CI 2.0 #concourse_tokyo
Toshiaki Maki
From Zero to Hero with REST and OAuth2 #jjug
From Zero to Hero with REST and OAuth2 #jjug
Toshiaki Maki
Consumer Driven Contractsで REST API/マイクロサービスをテスト #m3tech
Consumer Driven Contractsで REST API/マイクロサービスをテスト #m3tech
Toshiaki Maki
Plus de Toshiaki Maki
(20)
From Spring Boot 2.2 to Spring Boot 2.3 #jsug
From Spring Boot 2.2 to Spring Boot 2.3 #jsug
Concourse x Spinnaker #concourse_tokyo
Concourse x Spinnaker #concourse_tokyo
Serverless with Spring Cloud Function, Knative and riff #SpringOneTour #s1t
Serverless with Spring Cloud Function, Knative and riff #SpringOneTour #s1t
決済システムの内製化への旅 - SpringとPCFで作るクラウドネイティブなシステム開発 #jsug #sf_h1
決済システムの内製化への旅 - SpringとPCFで作るクラウドネイティブなシステム開発 #jsug #sf_h1
Spring Boot Actuator 2.0 & Micrometer #jjug_ccc #ccc_a1
Spring Boot Actuator 2.0 & Micrometer #jjug_ccc #ccc_a1
Spring Boot Actuator 2.0 & Micrometer
Spring Boot Actuator 2.0 & Micrometer
Open Service Broker APIとKubernetes Service Catalog #k8sjp
Open Service Broker APIとKubernetes Service Catalog #k8sjp
Spring Cloud Function & Project riff #jsug
Spring Cloud Function & Project riff #jsug
Introduction to Spring WebFlux #jsug #sf_a1
Introduction to Spring WebFlux #jsug #sf_a1
BOSH / CF Deployment in modern ways #cf_tokyo
BOSH / CF Deployment in modern ways #cf_tokyo
Why PCF is the best platform for Spring Boot
Why PCF is the best platform for Spring Boot
Zipkin Components #zipkin_jp
Zipkin Components #zipkin_jp
実例で学ぶ、明日から使えるSpring Boot Tips #jsug
実例で学ぶ、明日から使えるSpring Boot Tips #jsug
Spring ❤️ Kotlin #jjug
Spring ❤️ Kotlin #jjug
Managing your Docker image continuously with Concourse CI
Managing your Docker image continuously with Concourse CI
Short Lived Tasks in Cloud Foundry #cfdtokyo
Short Lived Tasks in Cloud Foundry #cfdtokyo
今すぐ始めるCloud Foundry #hackt #hackt_k
今すぐ始めるCloud Foundry #hackt #hackt_k
Team Support in Concourse CI 2.0 #concourse_tokyo
Team Support in Concourse CI 2.0 #concourse_tokyo
From Zero to Hero with REST and OAuth2 #jjug
From Zero to Hero with REST and OAuth2 #jjug
Consumer Driven Contractsで REST API/マイクロサービスをテスト #m3tech
Consumer Driven Contractsで REST API/マイクロサービスをテスト #m3tech
Dernier
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
presentation ICT roal in 21st century education
presentation ICT roal in 21st century education
jfdjdjcjdnsjd
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
WSO2
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
Dropbox
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
Zilliz
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Orbitshub
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Juan lago vázquez
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Edi Saputra
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
AnitaRaj43
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
apidays
Architecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
Christopher Logan Kennedy
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Orbitshub
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
Samir Dash
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
Khushali Kathiriya
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
Zilliz
Dernier
(20)
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
presentation ICT roal in 21st century education
presentation ICT roal in 21st century education
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Architecting Cloud Native Applications
Architecting Cloud Native Applications
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
Data Microservices with Spring Cloud Stream, Task, and Data Flow #jsug #springday
1.
‹#›© 2016 Pivotal
Software, Inc. All rights reserved. ‹#›© 2016 Pivotal Software, Inc. All rights reserved. Data Microservices with Spring Cloud Stream, Task, and Data Flow Toshiaki Maki (@making) 2016-11-18 Spring Day 2016 #springday #jsug
2.
© 2016 Pivotal
Software, Inc. All rights reserved. Who am I ? • Toshiaki Maki (@making) http://blog.ik.am • Sr. Solutions Architect @Pivotal • Spring Framework enthusiast bit.ly/hajiboot2
3.
© 2016 Pivotal
Software, Inc. All rights reserved. Input Motivation HTTP TCP S3 JDBC Rabbit JMS Twitter Syslog Output HDFS Cassandra HAWQ Greenplum JDBC S3 TCP Gemfire ❓❔❓ • Real Time Analysis • Data Ingestion • ETL Process
4.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring XD (eXtreme Data)
5.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring XD (eXtreme Data) 🤔
6.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring XD Architecture Container Container gpfdist Cassandra jms http ZooKeeper Message Broker XD Admin stream1 = http | cassandra stream2 = jms | gpfdist On Metal/VMs
7.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring XD Architecture Container Container gpfdist Cassandra jms http ZooKeeper Message Broker XD Admin stream1 = http | cassandra stream2 = jms | gpfdist On Metal/VMs 😇
8.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring XD Architecture Container Container gpfdist Cassandra jms http ZooKeeper Message Broker XD Admin stream1 = http | cassandra stream2 = jms | gpfdist On Metal/VMs
9.
© 2016 Pivotal
Software, Inc. All rights reserved. Modern Platforms Cloud Foundry YARN Kubernetes Mesos Docker Swarm Nomad OpenShift
10.
© 2016 Pivotal
Software, Inc. All rights reserved. Cloud Native Redesign •From multiple modules embedded in a container to standalone executable applications •From our own runtime to delegating to existing modern platforms
11.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head
12.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head
13.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing
14.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing
15.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers
16.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers
17.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers on the modern platform
18.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers on the modern platform
19.
© 2016 Pivotal
Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers on the modern platform
20.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Data Flow • Microservices-based Distributed Data Pipelines •Long Lived Stream Applications •Short Lived Task Applications
21.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Data Flow • Microservices-based Distributed Data Pipelines •Long Lived Stream Applications •Short Lived Task Applications Spring Cloud Stream
22.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Data Flow • Microservices-based Distributed Data Pipelines •Long Lived Stream Applications •Short Lived Task Applications Spring Cloud Stream Spring Cloud Task
23.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Data Flow • Microservices-based Distributed Data Pipelines •Long Lived Stream Applications •Short Lived Task Applications Spring Cloud Stream Spring Cloud Task Orchestration Layer
24.
© 2016 Pivotal
Software, Inc. All rights reserved. Structure Spring Cloud Data Flow Spring Cloud Deployer (SPI) Spring Cloud Stream Spring Cloud Task Spring Integration Spring Boot Spring Batch
25.
© 2016 Pivotal
Software, Inc. All rights reserved. Structure Spring Cloud Data Flow Spring Cloud Deployer (SPI) Spring Cloud Stream Spring Cloud Task Spring Integration Spring Boot Spring Batch • Spring Cloud Deployer Local • Spring Cloud Deployer Cloud Foundry • Spring Cloud Deployer Yarn • Spring Cloud Deployer Kubernetes • Spring Cloud Deployer Mesos
26.
© 2016 Pivotal
Software, Inc. All rights reserved. SCDF Deployment Platform Spring Cloud Data Flow Server Deployer SPI REST API SCDFShell SCDF Flo
27.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Data Flow Architecture gpfdist cassandra jms http stream1 = http | cassandra stream2 = jms | gpfdist Message Broker Data Flow Server DB Platform Runtime
28.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Data Flow Architecture gpfdist cassandra jms http stream1 = http | cassandra stream2 = jms | gpfdist Message Broker Data Flow Server DB Platform Runtime
29.
‹#›© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Stream
30.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Stream • Event-driven microservice framework • Built on battle-tested components (Spring Boot / Spring Integration) • Opinionated primitives for streaming applications • Persistent Pub/Sub • Consumer Groups • Partitioning Support • Pluggable messaging middleware bindings source | processor | sink
31.
© 2016 Pivotal
Software, Inc. All rights reserved. JJUG CCC in 2 weeks
32.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Stream Applications Twitter Stream Cassandra java -jar twittersource.jar --server.port=8080 --consumerKey=XYZ --consumerSecret=ABC --spring.cloud.stream.bindings. output.destination=ingest Source Sink java -jar cassandrasink.jar --server.port=8081 --spring.cassandra.keyspace=tweet --spring.cloud.stream.bindings. input.destination=ingest
33.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Stream Applications Twitter Stream Cassandra java -jar twittersource.jar --server.port=8080 --consumerKey=XYZ --consumerSecret=ABC --spring.cloud.stream.bindings. output.destination=ingest Source Sink java -jar cassandrasink.jar --server.port=8081 --spring.cassandra.keyspace=tweet --spring.cloud.stream.bindings. input.destination=ingest Twitter Stream Cassandraingest
34.
© 2016 Pivotal
Software, Inc. All rights reserved. Message Binders • @EnableBinding • Binder Implementations • Production-Ready • Rabbit MQ • Apache Kafka • Experimental • JMS • Google PubSub
35.
© 2016 Pivotal
Software, Inc. All rights reserved. Programming Model (Sink) @SpringBootApplication @EnableBinding(Sink.class) public class DemoSinkApp { @StreamListener(Sink.INPUT) void receive(Message<String> message) { System.out.println("Received " + message); } public static void main(String[] args) { SpringApplication.run(DemoSinkApp.class, args); } }
36.
© 2016 Pivotal
Software, Inc. All rights reserved. Sink Properties (Sink) spring.cloud.stream.bindings.input.destination=demo-strm demo- strm input
37.
© 2016 Pivotal
Software, Inc. All rights reserved. Programming Model (Source) @SpringBootApplication @RestController @EnableBinding(Source.class) public class DemoSourceApp { @Autowired Source source; @GetMapping void send(@RequestParam String text) { source.output() .send(MessageBuilder.withPayload(text).build()); } public static void main(String[] args) { SpringApplication.run(DemoSourceApp.class, args); } }
38.
© 2016 Pivotal
Software, Inc. All rights reserved. Programming Model (Source) @SpringBootApplication @RestController @EnableBinding(Source.class) public class DemoSourceApp { @Bean @InboundChannelAdapter(channel = Source.OUTPUT, poller = @Poller(fixedDelay = "1000", maxMessagesPerPoll = "1")) MessageSource<String> source() { return () -> MessageBuilder.withPayload("Hi").build()); } public static void main(String[] args) { SpringApplication.run(DemoSourceApp.class, args); }}
39.
© 2016 Pivotal
Software, Inc. All rights reserved. Properties (Source) spring.cloud.stream.bindings.output.destination=demo-strm demo- strm Source output
40.
© 2016 Pivotal
Software, Inc. All rights reserved. Binder (RabbitMQ) <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-stream-rabbit</artifactId> </dependency> demo- strm
41.
© 2016 Pivotal
Software, Inc. All rights reserved. Binder (Apache Kafka) <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-stream-kafka</artifactId> </dependency> demo- strm
42.
© 2016 Pivotal
Software, Inc. All rights reserved. Sink Pipeline demo- strm input Source output source | sink
43.
© 2016 Pivotal
Software, Inc. All rights reserved. Programming Model (Processor) @SpringBootApplication @EnableBinding(Processor.class) public class DemoProcessorApp { @StreamListener(Processor.INPUT) @SendTo(Processor.OUTPUT) void receive(String text) { return "[[" + text + "]]"; } public static void main(String[] args) { SpringApplication.run(DemoProcessorApp.class, args); } }
44.
© 2016 Pivotal
Software, Inc. All rights reserved. Properties (Processor) spring.cloud.stream.bindings.output.destination=my-source spring.cloud.stream.bindings.input.destination=my-source spring.cloud.stream.bindings.output.destination=my-proc spring.cloud.stream.bindings.input.destination=my-proc Source Processor Sink
45.
© 2016 Pivotal
Software, Inc. All rights reserved. Pipeline my- source Source output source | processor | sink Processor output input my-proc Sink input
46.
© 2016 Pivotal
Software, Inc. All rights reserved. Reactive API Support @SpringBootApplication @EnableBinding(Processor.class) public class DemoProcessorRxApp { @StreamListener @Output(Processor.OUTPUT) public Flux<String> receive(@Input(Processor.INPUT) Flux<String> stream) { return stream.map(text -> "[[" + text + "]]"); } public static void main(String[] args) { SpringApplication.run(DemoProcessorRxApp.class, args); } }
47.
© 2016 Pivotal
Software, Inc. All rights reserved. Reactive API Support @StreamListener @Output(Processor.OUTPUT) public Flux<AverageData> receive(@Input(Processor.INPUT) Flux<SensorData> stream) { return stream.window(Duration.ofSecond(20), Duration.ofSecond(10)) .flatMap(win -> win.groupBy(sensor -> sensor.id)) .flatMap(group -> calcAverage(group)); }
48.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Stream App Starters Source Processor Sink file ftp gemfire gemfire-cq http jdbc jms load-generator loggregator mail mongodb rabbit s3 sftp syslog tcp tcp-client time trigger triggertask twitterstream bridge filter groovy-filter groovy- transform httpclient pommel scriptable- transform splitter tcp-client transform aggregate- counter cassandra counter field-value- counter file ftp gemfire gpfdist hfs hdfs-dataset jdbc log rabbit redis-pubsub router s3 sftp task-launcher- local task-lancher- yarn tcp throughput web socket http://cloud.spring.io/spring-cloud-stream-app-starters/
49.
© 2016 Pivotal
Software, Inc. All rights reserved. Stream Orchestration in SCDF ingest = twitterstream | cassandra dataflow:> stream create --name=ingest --definition="twitterstream | cassandra" dataflow:> stream deploy --name=ingest |
50.
© 2016 Pivotal
Software, Inc. All rights reserved. Import apps dataflow:> app import --uri http://bit.ly/1-0-4- GA-stream-applications-rabbit-maven rabbit maven kafka docker stream-applications- http://cloud.spring.io/spring-cloud-stream-app-starters/
51.
© 2016 Pivotal
Software, Inc. All rights reserved. Register your own apps dataflow:> app register --name foo --type source --uri http://example.com/foo-source-1.0.jar dataflow:> app register --name foo --type source --uri maven://com.example:foo-source:1.0 dataflow:> app register --name foo --type source --uri docker:myapps/foo-source:1.0 dataflow:> app register --name foo --type source --uri file:///tmp/foo-source:1.0
52.
© 2016 Pivotal
Software, Inc. All rights reserved. Instance Count dataflow:> stream create --name=s1 --definition="http | work | hdfs" dataflow:> stream deploy --name=s1 -- properties="app.http.count=2,app.work.count=3,app .hdfs.count=4"
53.
© 2016 Pivotal
Software, Inc. All rights reserved. Instance Count http work hdfs http work work hdfs hdfs hdfs LoadBalancer
54.
© 2016 Pivotal
Software, Inc. All rights reserved. Resource Management dataflow:> stream deploy --name=s1 -- properties="app.work.spring.cloud.deployer.cloudf oundry.memory=2048"
55.
© 2016 Pivotal
Software, Inc. All rights reserved. Resource Management http http work work work hdfs hdfs hdfs hdfs
56.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Flo http://projects.spring.io/spring-flo/
57.
© 2016 Pivotal
Software, Inc. All rights reserved. Stream Core Features •Persistent Pub-Sub •Consumer Group •Partitioning Support
58.
© 2016 Pivotal
Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average Top Ns1.http s1.ave Message Broker
59.
© 2016 Pivotal
Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top Ns1.http s1.ave Message Broker
60.
© 2016 Pivotal
Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker
61.
© 2016 Pivotal
Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38}
62.
© 2016 Pivotal
Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38}{"id":1, "temperature":38}
63.
© 2016 Pivotal
Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38}
64.
© 2016 Pivotal
Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature": {"id":1, "temperature":38} {"id":1, "temperature":38}
65.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS
66.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} Average Average HDFS HDFS
67.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38}{"id":1, "temperature":38} Average Average HDFS HDFS
68.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} {"id":1, "temperature":38} Average Average HDFS HDFS {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38}
69.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} {"id":1, "temperature":38} Average Average HDFS HDFS {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} 😩
70.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} {"id":1, "temperature":38} Average Average HDFS HDFS {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} 😩 Consumer Group
71.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} {"id":1, "temperature":38} Average Average HDFS HDFS {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} 😩 spring.cloud.stream.bindings.<channelName>.group=ave Spring Cloud Stream group is configured by default 😁 Spring Cloud Data Flow Consumer Group
72.
© 2016 Pivotal
Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs
73.
© 2016 Pivotal
Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38}
74.
© 2016 Pivotal
Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38}{"id":1, "temperature":38}
75.
© 2016 Pivotal
Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38} {"id":1, "temperature":38}
76.
© 2016 Pivotal
Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38} {"id":1, "temperature":38} 🤓
77.
© 2016 Pivotal
Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38} {"id":1, "temperature":38} 🤓consumer group subscriptions are durable 😁
78.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average
79.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
80.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
81.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
82.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
83.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
84.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37} 😩
85.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37} 😩 Partitioning Support(Stateful Stream)
86.
© 2016 Pivotal
Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37} 😩stream deploy --name=xxxx --properties= "app.http.producer.partitionKeyExpression=payload.id" spring.cloud.stream.bindings.<channelName>.producer.par titionKeyExpression=payload.id Spring Cloud Stream Spring Cloud Data Flow Partitioning Support(Stateful Stream)
87.
© 2016 Pivotal
Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average
88.
© 2016 Pivotal
Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
89.
© 2016 Pivotal
Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
90.
© 2016 Pivotal
Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
91.
© 2016 Pivotal
Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
92.
© 2016 Pivotal
Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
93.
© 2016 Pivotal
Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37} 🤓
94.
© 2016 Pivotal
Software, Inc. All rights reserved. Simple Real Time Analytics tweets = twitterstream | hdfs analytics = :ingest.twitterstream > field-value-counter --fieldName=lang HTTP s1.http HDFS COUNTER Data Flow Server REST API
95.
‹#›© 2016 Pivotal
Software, Inc. All rights reserved. Demo (Twitter Stream!)
96.
© 2016 Pivotal
Software, Inc. All rights reserved.
97.
‹#›© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Task
98.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Task • Spring Boot based framework for short lived task • Spring Boot's CommandLineRunner or Spring Batch • Result of each process persists in Task Repository • Well Integrated with Spring Batch (Job Repository)
99.
© 2016 Pivotal
Software, Inc. All rights reserved. Programming Model (CLR) @SpringBootApplication @EnableTask public class DemoTaskApp { @Bean CommandLineRunner clr() { return args -> System.out.println("Task!"); } public static void main(String[] args) { SpringApplication.run(DemoTaskApp.class, args); } } spring.application.name=hello
100.
© 2016 Pivotal
Software, Inc. All rights reserved. Task Execution List Task Name ID Start Time End Time Exit Code hello 3 Tue Nov 15 16:49:08 JST 2016 Tue Nov 15 16:49:08 JST 2016 0 hello 2 Tue Nov 15 16:48:54 JST 2016 Tue Nov 15 16:48:55 JST 2016 0 hello 1 Tue Nov 15 16:48:23 JST 2016 Tue Nov 15 16:48:23 JST 2016 0
101.
© 2016 Pivotal
Software, Inc. All rights reserved. Programming Model (Spring Batch) @SpringBootApplication @EnableBatchProcessing @EnableTask public class DemoBatchApp { @Autowired JobBuilderFactory jobBuilderFactory; @Autowired StepBuilderFactory stepBuilderFactory; @Bean Step step1() { /* ... */ } @Bean Step step2() { /* ... */ } @Bean Job job() { return jobBuilderFactory.get("job") .start(step1()).next(step2()).build()} public static void main(String[] args) { SpringApplication.run(DemoTaskApp.class, args); }} https://github.com/making/cf-spring-batch-demo spring.application.name=hello-batch
102.
© 2016 Pivotal
Software, Inc. All rights reserved. Task Execution List Task Name ID Start Time End Time Exit Code hello- batch 5 Tue Nov 15 17:28:49 JST 2016 Tue Nov 15 17:28:49 JST 2016 0 hello- batch 4 Tue Nov 15 17:27:56 JST 2016 Tue Nov 15 17:27:57 JST 2016 0
103.
© 2016 Pivotal
Software, Inc. All rights reserved. Job Execution List ID Task ID Start Time Step Execution Count Definition Status 2 5 Tue Nov 15 17:28:49 JST 2016 2 Created 1 4 Tue Nov 15 17:27:56 JST 2016 2 Created
104.
© 2016 Pivotal
Software, Inc. All rights reserved. Step Execution List ID Step Name Job Exec Id Start Time End Time Status 1 step1 1 Tue Nov 15 17:28:49 JST 2016 Tue Nov 15 17:28:49 JST 2016 COMPLETED 2 step2 1 Tue Nov 15 17:27:56 JST 2016 Tue Nov 15 17:27:57 JST 2016 COMPLETED
105.
© 2016 Pivotal
Software, Inc. All rights reserved. Task Orchestration in SCDF >task create hello --definition="timestamp --format=¥"yyyy¥"" >task launch hello timestamp Data Flow Server DB Task Name Start Time End Time Exit Code Exit Message Last Updated Time Parameters Message Broker
106.
© 2016 Pivotal
Software, Inc. All rights reserved. Task Execution List
107.
© 2016 Pivotal
Software, Inc. All rights reserved. Job Execution List
108.
© 2016 Pivotal
Software, Inc. All rights reserved. Job Execution Details
109.
© 2016 Pivotal
Software, Inc. All rights reserved. Spring Cloud Data Flow -> Task http | tasklaunchrequest --uri=... | task-lancher timestamp Data Flow Server DB Message Broker
110.
‹#›© 2016 Pivotal
Software, Inc. All rights reserved. Demo (Scalable Tasks)
111.
© 2016 Pivotal
Software, Inc. All rights reserved. Pivotal Cloud Foundry PDF PDF PDF PDF S3 Spring Cloud Data Flow Task Task Task Task PDF PDF Spring Cloud Task Spring Batch index index source sink
112.
‹#›© 2016 Pivotal
Software, Inc. All rights reserved. Getting Started with SCDF on Cloud Foundry
113.
© 2016 Pivotal
Software, Inc. All rights reserved. Install PCF Dev • https://docs.pivotal.io/pcf-dev • Cloud Foundry on your laptop • Included • Redis / RabbitMQ / MySQL • Spring Cloud Services • Install with cf dev start
114.
© 2016 Pivotal
Software, Inc. All rights reserved. cf create-service p-mysql 512mb df-mysql cf create-service p-rabbitmq standard df-rebbitmq cf create-service p-redis shared-vm df-redis cf push dataflow-server -p spring-cloud-dataflow- server-cloudfoundry-1.0.1.RELEASE.jar Deploy SCDF Server on PCF Dev
115.
© 2016 Pivotal
Software, Inc. All rights reserved. Tutorial • 📗 https://blog.ik.am/entries/396 • 💎 https://github.com/making-demo-scdf/spring-cloud- dataflow-cookbook
116.
© 2016 Pivotal
Software, Inc. All rights reserved. Upcoming Features • Some ‘porting’ from XD • Batch Job DSL + Designer • Role based access • Looking forward • Spring Cloud Sleuth • JavaDSL • In-place application version upgrades with Spinnaker • Application Groups • Polyglot • Expanded analytics with Redis and Python/R ecosystem • More provided apps/tasks •SCDF Tile for Pivotal Cloud Foundry
117.
© 2016 Pivotal
Software, Inc. All rights reserved. Thanks! • References • http://www.slideshare.net/SpringCentral/data-microservices-in-t cloud ✨ • https://cloud.spring.io/spring-cloud-dataflow/ • http://cloud.spring.io/spring-cloud-stream/ • http://cloud.spring.io/spring-cloud-stream-app-starters/ • https://github.com/spring-cloud/spring-cloud-task/ • http://cloud.spring.io/spring-cloud-dataflow-server-cloudfoundry
Télécharger maintenant