"Data Mesh in Kubernetes", Andrii Syniuk

Fwdays
FwdaysFwdays
"Data Mesh in Kubernetes", Andrii Syniuk
1. Introduction
2. Data storage and analytics evolution
3 What is Data Mesh?
4. Motivation
5. Components, Implementation, Evolution
6. Challenges
7. Q&A
Agenda
Introduction
1
whoami
https://www.linkedin.com/in/andrii-syniuk/
Domain Driven Design enthusiast
Software Engineer
Site Reliability Engineer
Data Ecosystem Tech Lead
Data curious gold digger
Data Storage and Analytic
evolution
2
Modern Software architectures and data in it
Monolithic Architecture
microservices.io
Data Warehouse
Modern Software architectures and data in it
What is Data Mesh?
3
Main principles
• Data as a Product
• Federated Governance
• Domain Ownership
• Self serve
Data Mesh
9
Motivation
4
Why bother? It was working just fine.
- Reusing data products
- Faster time to market
- Decentralized data teams
Components,
Implementation,
Evolution
5
Components
Landscape
DB
Data products
Pipelines
Governance Self Service
Challenges
6
Challenges
- Policy As A Code
- Data quality
- A lot of small databases
- Avoiding vendor lock-in
- Platformization
- Get it work together
Q&A
7
1 sur 17

Recommandé

Enterprise Data Architecture Deliverables par
Enterprise Data Architecture DeliverablesEnterprise Data Architecture Deliverables
Enterprise Data Architecture DeliverablesLars E Martinsson
28.5K vues16 diapositives
Hybrid SharePoint Strategy with Office 365 par
Hybrid SharePoint Strategy with Office 365Hybrid SharePoint Strategy with Office 365
Hybrid SharePoint Strategy with Office 365Netwoven Inc.
3K vues36 diapositives
A Key to Real-time Insights in a Post-COVID World (ASEAN) par
A Key to Real-time Insights in a Post-COVID World (ASEAN)A Key to Real-time Insights in a Post-COVID World (ASEAN)
A Key to Real-time Insights in a Post-COVID World (ASEAN)Denodo
113 vues39 diapositives
Introduction To SQL Server 2014 par
Introduction To SQL Server 2014Introduction To SQL Server 2014
Introduction To SQL Server 2014Vishal Pawar
2K vues34 diapositives
Feature Store as a Data Foundation for Machine Learning par
Feature Store as a Data Foundation for Machine LearningFeature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine LearningProvectus
349 vues43 diapositives
Building the Artificially Intelligent Enterprise par
Building the Artificially Intelligent EnterpriseBuilding the Artificially Intelligent Enterprise
Building the Artificially Intelligent EnterpriseDatabricks
253 vues59 diapositives

Contenu connexe

Similaire à "Data Mesh in Kubernetes", Andrii Syniuk

Curiosity Software and RCG Global Services Present - Solving Test Data: the g... par
Curiosity Software and RCG Global Services Present - Solving Test Data: the g...Curiosity Software and RCG Global Services Present - Solving Test Data: the g...
Curiosity Software and RCG Global Services Present - Solving Test Data: the g...Curiosity Software Ireland
44 vues21 diapositives
When and How Data Lakes Fit into a Modern Data Architecture par
When and How Data Lakes Fit into a Modern Data ArchitectureWhen and How Data Lakes Fit into a Modern Data Architecture
When and How Data Lakes Fit into a Modern Data ArchitectureDATAVERSITY
686 vues33 diapositives
Data Systems Integration & Business Value Pt. 2: Cloud par
Data Systems Integration & Business Value Pt. 2: CloudData Systems Integration & Business Value Pt. 2: Cloud
Data Systems Integration & Business Value Pt. 2: CloudDATAVERSITY
1.5K vues78 diapositives
Data Systems Integration & Business Value Pt. 2: Cloud par
Data Systems Integration & Business Value Pt. 2: CloudData Systems Integration & Business Value Pt. 2: Cloud
Data Systems Integration & Business Value Pt. 2: CloudData Blueprint
593 vues78 diapositives
All Together Now: A Recipe for Successful Data Governance par
All Together Now: A Recipe for Successful Data GovernanceAll Together Now: A Recipe for Successful Data Governance
All Together Now: A Recipe for Successful Data GovernanceInside Analysis
593 vues66 diapositives
awari-ds-aula1.pdf par
awari-ds-aula1.pdfawari-ds-aula1.pdf
awari-ds-aula1.pdfMarcos993896
5 vues58 diapositives

Similaire à "Data Mesh in Kubernetes", Andrii Syniuk(20)

When and How Data Lakes Fit into a Modern Data Architecture par DATAVERSITY
When and How Data Lakes Fit into a Modern Data ArchitectureWhen and How Data Lakes Fit into a Modern Data Architecture
When and How Data Lakes Fit into a Modern Data Architecture
DATAVERSITY686 vues
Data Systems Integration & Business Value Pt. 2: Cloud par DATAVERSITY
Data Systems Integration & Business Value Pt. 2: CloudData Systems Integration & Business Value Pt. 2: Cloud
Data Systems Integration & Business Value Pt. 2: Cloud
DATAVERSITY1.5K vues
Data Systems Integration & Business Value Pt. 2: Cloud par Data Blueprint
Data Systems Integration & Business Value Pt. 2: CloudData Systems Integration & Business Value Pt. 2: Cloud
Data Systems Integration & Business Value Pt. 2: Cloud
Data Blueprint593 vues
All Together Now: A Recipe for Successful Data Governance par Inside Analysis
All Together Now: A Recipe for Successful Data GovernanceAll Together Now: A Recipe for Successful Data Governance
All Together Now: A Recipe for Successful Data Governance
Inside Analysis593 vues
Data Virtualization enabled Data Fabric: Operationalize the Data Lake (APAC) par Denodo
Data Virtualization enabled Data Fabric: Operationalize the Data Lake (APAC)Data Virtualization enabled Data Fabric: Operationalize the Data Lake (APAC)
Data Virtualization enabled Data Fabric: Operationalize the Data Lake (APAC)
Denodo 106 vues
Strata San Jose 2017 - Ben Sharma Presentation par Zaloni
Strata San Jose 2017 - Ben Sharma PresentationStrata San Jose 2017 - Ben Sharma Presentation
Strata San Jose 2017 - Ben Sharma Presentation
Zaloni 2.6K vues
Building a Data Driven Culture and AI Revolution With Gregory Little | Curren... par HostedbyConfluent
Building a Data Driven Culture and AI Revolution With Gregory Little | Curren...Building a Data Driven Culture and AI Revolution With Gregory Little | Curren...
Building a Data Driven Culture and AI Revolution With Gregory Little | Curren...
Cloud Computing - Demystified par LadderMinds
Cloud Computing - DemystifiedCloud Computing - Demystified
Cloud Computing - Demystified
LadderMinds274 vues
How Data Drives Business at Choice Hotels par Cloudera, Inc.
How Data Drives Business at Choice HotelsHow Data Drives Business at Choice Hotels
How Data Drives Business at Choice Hotels
Cloudera, Inc.1.4K vues
Hadoop Migration to databricks cloud project plan.pptx par yashodhannn
Hadoop Migration to databricks cloud project plan.pptxHadoop Migration to databricks cloud project plan.pptx
Hadoop Migration to databricks cloud project plan.pptx
yashodhannn81 vues
Intersection of Business Intelligence and CRM vsr12 par David J Rosenthal
Intersection of Business Intelligence and CRM vsr12Intersection of Business Intelligence and CRM vsr12
Intersection of Business Intelligence and CRM vsr12
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha... par DATAVERSITY
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...
DATAVERSITY485 vues
Architect’s Open-Source Guide for a Data Mesh Architecture par Databricks
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
Databricks3.1K vues
Data Lakehouse, Data Mesh, and Data Fabric (r1) par James Serra
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
James Serra5.4K vues
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris... par DATAVERSITY
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
DATAVERSITY829 vues
Data-Ed: Data Architecture Requirements par Data Blueprint
Data-Ed: Data Architecture Requirements Data-Ed: Data Architecture Requirements
Data-Ed: Data Architecture Requirements
Data Blueprint1.9K vues

Plus de Fwdays

"The role of CTO in a classical early-stage startup", Eugene Gusarov par
"The role of CTO in a classical early-stage startup", Eugene Gusarov"The role of CTO in a classical early-stage startup", Eugene Gusarov
"The role of CTO in a classical early-stage startup", Eugene GusarovFwdays
31 vues43 diapositives
"Cross-functional teams: what to do when a new hire doesn’t solve the busines... par
"Cross-functional teams: what to do when a new hire doesn’t solve the busines..."Cross-functional teams: what to do when a new hire doesn’t solve the busines...
"Cross-functional teams: what to do when a new hire doesn’t solve the busines...Fwdays
30 vues29 diapositives
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad... par
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad..."Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad...
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad...Fwdays
40 vues30 diapositives
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur par
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur
"Thriving Culture in a Product Company — Practical Story", Volodymyr TsukurFwdays
40 vues31 diapositives
"Fast Start to Building on AWS", Igor Ivaniuk par
"Fast Start to Building on AWS", Igor Ivaniuk"Fast Start to Building on AWS", Igor Ivaniuk
"Fast Start to Building on AWS", Igor IvaniukFwdays
36 vues76 diapositives
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ... par
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ..."Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ...
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ...Fwdays
33 vues39 diapositives

Plus de Fwdays(20)

"The role of CTO in a classical early-stage startup", Eugene Gusarov par Fwdays
"The role of CTO in a classical early-stage startup", Eugene Gusarov"The role of CTO in a classical early-stage startup", Eugene Gusarov
"The role of CTO in a classical early-stage startup", Eugene Gusarov
Fwdays31 vues
"Cross-functional teams: what to do when a new hire doesn’t solve the busines... par Fwdays
"Cross-functional teams: what to do when a new hire doesn’t solve the busines..."Cross-functional teams: what to do when a new hire doesn’t solve the busines...
"Cross-functional teams: what to do when a new hire doesn’t solve the busines...
Fwdays30 vues
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad... par Fwdays
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad..."Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad...
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad...
Fwdays40 vues
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur par Fwdays
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur
Fwdays40 vues
"Fast Start to Building on AWS", Igor Ivaniuk par Fwdays
"Fast Start to Building on AWS", Igor Ivaniuk"Fast Start to Building on AWS", Igor Ivaniuk
"Fast Start to Building on AWS", Igor Ivaniuk
Fwdays36 vues
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ... par Fwdays
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ..."Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ...
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ...
Fwdays33 vues
"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi par Fwdays
"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi
"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi
Fwdays26 vues
"How we switched to Kanban and how it integrates with product planning", Vady... par Fwdays
"How we switched to Kanban and how it integrates with product planning", Vady..."How we switched to Kanban and how it integrates with product planning", Vady...
"How we switched to Kanban and how it integrates with product planning", Vady...
Fwdays61 vues
"Bringing Flutter to Tide: a case study of a leading fintech platform in the ... par Fwdays
"Bringing Flutter to Tide: a case study of a leading fintech platform in the ..."Bringing Flutter to Tide: a case study of a leading fintech platform in the ...
"Bringing Flutter to Tide: a case study of a leading fintech platform in the ...
Fwdays23 vues
"Shape Up: How to Develop Quickly and Avoid Burnout", Dmytro Popov par Fwdays
"Shape Up: How to Develop Quickly and Avoid Burnout", Dmytro Popov"Shape Up: How to Develop Quickly and Avoid Burnout", Dmytro Popov
"Shape Up: How to Develop Quickly and Avoid Burnout", Dmytro Popov
Fwdays60 vues
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy par Fwdays
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy
Fwdays40 vues
From “T” to “E”, Dmytro Gryn par Fwdays
From “T” to “E”, Dmytro GrynFrom “T” to “E”, Dmytro Gryn
From “T” to “E”, Dmytro Gryn
Fwdays34 vues
"Why I left React in my TypeScript projects and where ", Illya Klymov par Fwdays
"Why I left React in my TypeScript projects and where ",  Illya Klymov"Why I left React in my TypeScript projects and where ",  Illya Klymov
"Why I left React in my TypeScript projects and where ", Illya Klymov
Fwdays247 vues
"KillTech project: through innovation to a winning capability", Yelyzaveta B... par Fwdays
"KillTech project: through innovation to a winning capability",  Yelyzaveta B..."KillTech project: through innovation to a winning capability",  Yelyzaveta B...
"KillTech project: through innovation to a winning capability", Yelyzaveta B...
Fwdays230 vues
"Dude, where’s my boilerplate? ", Oleksii Makodzeba par Fwdays
"Dude, where’s my boilerplate? ", Oleksii Makodzeba"Dude, where’s my boilerplate? ", Oleksii Makodzeba
"Dude, where’s my boilerplate? ", Oleksii Makodzeba
Fwdays116 vues
"Pixel-Pushing Pundit Challenges in 2023, or Non-functional Requirements for ... par Fwdays
"Pixel-Pushing Pundit Challenges in 2023, or Non-functional Requirements for ..."Pixel-Pushing Pundit Challenges in 2023, or Non-functional Requirements for ...
"Pixel-Pushing Pundit Challenges in 2023, or Non-functional Requirements for ...
Fwdays89 vues
"Do you really need your test environment?", Vlad Kampov par Fwdays
"Do you really need your test environment?", Vlad Kampov "Do you really need your test environment?", Vlad Kampov
"Do you really need your test environment?", Vlad Kampov
Fwdays213 vues
"Crafting a Third-Party Banking Library with Web Components and React", Germa... par Fwdays
"Crafting a Third-Party Banking Library with Web Components and React", Germa..."Crafting a Third-Party Banking Library with Web Components and React", Germa...
"Crafting a Third-Party Banking Library with Web Components and React", Germa...
Fwdays180 vues
"Generating Types without climbing a tree", Matteo Collina par Fwdays
"Generating Types without climbing a tree", Matteo Collina "Generating Types without climbing a tree", Matteo Collina
"Generating Types without climbing a tree", Matteo Collina
Fwdays89 vues
"You Keep Using That Word", Sam Newman par Fwdays
"You Keep Using That Word", Sam Newman"You Keep Using That Word", Sam Newman
"You Keep Using That Word", Sam Newman
Fwdays37 vues

Dernier

Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa... par
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...The Digital Insurer
28 vues18 diapositives
GigaIO: The March of Composability Onward to Memory with CXL par
GigaIO: The March of Composability Onward to Memory with CXLGigaIO: The March of Composability Onward to Memory with CXL
GigaIO: The March of Composability Onward to Memory with CXLCXL Forum
126 vues12 diapositives
Future of Learning - Yap Aye Wee.pdf par
Future of Learning - Yap Aye Wee.pdfFuture of Learning - Yap Aye Wee.pdf
Future of Learning - Yap Aye Wee.pdfNUS-ISS
38 vues11 diapositives
Java Platform Approach 1.0 - Picnic Meetup par
Java Platform Approach 1.0 - Picnic MeetupJava Platform Approach 1.0 - Picnic Meetup
Java Platform Approach 1.0 - Picnic MeetupRick Ossendrijver
25 vues39 diapositives
Combining Orchestration and Choreography for a Clean Architecture par
Combining Orchestration and Choreography for a Clean ArchitectureCombining Orchestration and Choreography for a Clean Architecture
Combining Orchestration and Choreography for a Clean ArchitectureThomasHeinrichs1
68 vues24 diapositives
TE Connectivity: Card Edge Interconnects par
TE Connectivity: Card Edge InterconnectsTE Connectivity: Card Edge Interconnects
TE Connectivity: Card Edge InterconnectsCXL Forum
96 vues12 diapositives

Dernier(20)

Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa... par The Digital Insurer
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...
GigaIO: The March of Composability Onward to Memory with CXL par CXL Forum
GigaIO: The March of Composability Onward to Memory with CXLGigaIO: The March of Composability Onward to Memory with CXL
GigaIO: The March of Composability Onward to Memory with CXL
CXL Forum126 vues
Future of Learning - Yap Aye Wee.pdf par NUS-ISS
Future of Learning - Yap Aye Wee.pdfFuture of Learning - Yap Aye Wee.pdf
Future of Learning - Yap Aye Wee.pdf
NUS-ISS38 vues
Combining Orchestration and Choreography for a Clean Architecture par ThomasHeinrichs1
Combining Orchestration and Choreography for a Clean ArchitectureCombining Orchestration and Choreography for a Clean Architecture
Combining Orchestration and Choreography for a Clean Architecture
TE Connectivity: Card Edge Interconnects par CXL Forum
TE Connectivity: Card Edge InterconnectsTE Connectivity: Card Edge Interconnects
TE Connectivity: Card Edge Interconnects
CXL Forum96 vues
Understanding GenAI/LLM and What is Google Offering - Felix Goh par NUS-ISS
Understanding GenAI/LLM and What is Google Offering - Felix GohUnderstanding GenAI/LLM and What is Google Offering - Felix Goh
Understanding GenAI/LLM and What is Google Offering - Felix Goh
NUS-ISS39 vues
Empathic Computing: Delivering the Potential of the Metaverse par Mark Billinghurst
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the Metaverse
Microchip: CXL Use Cases and Enabling Ecosystem par CXL Forum
Microchip: CXL Use Cases and Enabling EcosystemMicrochip: CXL Use Cases and Enabling Ecosystem
Microchip: CXL Use Cases and Enabling Ecosystem
CXL Forum129 vues
MemVerge: Memory Viewer Software par CXL Forum
MemVerge: Memory Viewer SoftwareMemVerge: Memory Viewer Software
MemVerge: Memory Viewer Software
CXL Forum118 vues
The Importance of Cybersecurity for Digital Transformation par NUS-ISS
The Importance of Cybersecurity for Digital TransformationThe Importance of Cybersecurity for Digital Transformation
The Importance of Cybersecurity for Digital Transformation
NUS-ISS25 vues
Liqid: Composable CXL Preview par CXL Forum
Liqid: Composable CXL PreviewLiqid: Composable CXL Preview
Liqid: Composable CXL Preview
CXL Forum121 vues
MemVerge: Past Present and Future of CXL par CXL Forum
MemVerge: Past Present and Future of CXLMemVerge: Past Present and Future of CXL
MemVerge: Past Present and Future of CXL
CXL Forum110 vues
Astera Labs: Intelligent Connectivity for Cloud and AI Infrastructure par CXL Forum
Astera Labs:  Intelligent Connectivity for Cloud and AI InfrastructureAstera Labs:  Intelligent Connectivity for Cloud and AI Infrastructure
Astera Labs: Intelligent Connectivity for Cloud and AI Infrastructure
CXL Forum125 vues
JCon Live 2023 - Lice coding some integration problems par Bernd Ruecker
JCon Live 2023 - Lice coding some integration problemsJCon Live 2023 - Lice coding some integration problems
JCon Live 2023 - Lice coding some integration problems
Bernd Ruecker67 vues
How to reduce cold starts for Java Serverless applications in AWS at JCON Wor... par Vadym Kazulkin
How to reduce cold starts for Java Serverless applications in AWS at JCON Wor...How to reduce cold starts for Java Serverless applications in AWS at JCON Wor...
How to reduce cold starts for Java Serverless applications in AWS at JCON Wor...
Vadym Kazulkin70 vues
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV par Splunk
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
Splunk86 vues

"Data Mesh in Kubernetes", Andrii Syniuk