Soumettre la recherche
Mettre en ligne
Microservices: Data & Design - Miguel Cervantes
•
0 j'aime
•
342 vues
Amazon Web Services
Suivre
Speaker: Miguel Cervantes - Associate Solutions Architect, AWS
Lire moins
Lire la suite
Signaler
Partager
Signaler
Partager
1 sur 36
Recommandé
Microservices & Data Design
Microservices & Data Design
Amazon Web Services
Microservices & Data Design: Database Week San Francisco
Microservices & Data Design: Database Week San Francisco
Amazon Web Services
Microservices & Data Design: Database Week SF
Microservices & Data Design: Database Week SF
Amazon Web Services
Microservices and Data Design
Microservices and Data Design
AWS Germany
Deep Dive into AWS X-Ray: Monitor Modern Applications (DEV324) - AWS re:Inven...
Deep Dive into AWS X-Ray: Monitor Modern Applications (DEV324) - AWS re:Inven...
Amazon Web Services
ENT307 Move your Desktops and Apps to AWS with Amazon WorkSpaces and AppStre...
ENT307 Move your Desktops and Apps to AWS with Amazon WorkSpaces and AppStre...
Amazon Web Services
How to Enable Single Sign On to Multiple AWS Accounts and Business Applicatio...
How to Enable Single Sign On to Multiple AWS Accounts and Business Applicatio...
Amazon Web Services
Building Modern Apps Using Amazon DynamoDB Transactions: re:Invent 2018 Recap...
Building Modern Apps Using Amazon DynamoDB Transactions: re:Invent 2018 Recap...
Amazon Web Services
Recommandé
Microservices & Data Design
Microservices & Data Design
Amazon Web Services
Microservices & Data Design: Database Week San Francisco
Microservices & Data Design: Database Week San Francisco
Amazon Web Services
Microservices & Data Design: Database Week SF
Microservices & Data Design: Database Week SF
Amazon Web Services
Microservices and Data Design
Microservices and Data Design
AWS Germany
Deep Dive into AWS X-Ray: Monitor Modern Applications (DEV324) - AWS re:Inven...
Deep Dive into AWS X-Ray: Monitor Modern Applications (DEV324) - AWS re:Inven...
Amazon Web Services
ENT307 Move your Desktops and Apps to AWS with Amazon WorkSpaces and AppStre...
ENT307 Move your Desktops and Apps to AWS with Amazon WorkSpaces and AppStre...
Amazon Web Services
How to Enable Single Sign On to Multiple AWS Accounts and Business Applicatio...
How to Enable Single Sign On to Multiple AWS Accounts and Business Applicatio...
Amazon Web Services
Building Modern Apps Using Amazon DynamoDB Transactions: re:Invent 2018 Recap...
Building Modern Apps Using Amazon DynamoDB Transactions: re:Invent 2018 Recap...
Amazon Web Services
Streamlining Application Development with AWS Service Catalog (DEV328) - AWS ...
Streamlining Application Development with AWS Service Catalog (DEV328) - AWS ...
Amazon Web Services
AWS Dev Lounge: Applying the Twelve-Factor Application Manifesto to Developin...
AWS Dev Lounge: Applying the Twelve-Factor Application Manifesto to Developin...
Amazon Web Services
Eliminate Migration Confusion: Speed Migration with Automated Tracking (ENT31...
Eliminate Migration Confusion: Speed Migration with Automated Tracking (ENT31...
Amazon Web Services
SID305 AWS Certificate Manager Private CA
SID305 AWS Certificate Manager Private CA
Amazon Web Services
2018 10-19-jc conf-embrace-legacy-java-ee-by-aws-serverless
2018 10-19-jc conf-embrace-legacy-java-ee-by-aws-serverless
Kim Kao
Building Real-time Serverless Backends with GraphQL
Building Real-time Serverless Backends with GraphQL
Amazon Web Services
Control for Your Cloud Environment Using AWS Management Tools (ENT226-R1) - A...
Control for Your Cloud Environment Using AWS Management Tools (ENT226-R1) - A...
Amazon Web Services
SRV207 Orchestrating AWS Lambda with Step Functions
SRV207 Orchestrating AWS Lambda with Step Functions
Amazon Web Services
SID201 Overview of AWS Identity, Directory, and Access Services
SID201 Overview of AWS Identity, Directory, and Access Services
Amazon Web Services
Increase the Value of Video with ML & Media Services - SRV322 - New York AWS ...
Increase the Value of Video with ML & Media Services - SRV322 - New York AWS ...
Amazon Web Services
How AQR Capital Uses AWS to Research New Investment Signals
How AQR Capital Uses AWS to Research New Investment Signals
Amazon Web Services
Microsoft SQL Server - SQL Server Migrations Presentation
Microsoft SQL Server - SQL Server Migrations Presentation
Microsoft Private Cloud
Best Practices for Centrally Monitoring Resource Configuration & Compliance (...
Best Practices for Centrally Monitoring Resource Configuration & Compliance (...
Amazon Web Services
Legacy java ee meet lambda
Legacy java ee meet lambda
Kim Kao
AWS Systems Manage: Bridging Operational Models
AWS Systems Manage: Bridging Operational Models
Amazon Web Services
MongoDB World 2018: Tutorial - How to Build Applications with MongoDB Atlas &...
MongoDB World 2018: Tutorial - How to Build Applications with MongoDB Atlas &...
MongoDB
Gain Insights, Make Decisions, and Take Action Across a Streamlined and Autom...
Gain Insights, Make Decisions, and Take Action Across a Streamlined and Autom...
Arraya Solutions
Architecting ASP.NET Core Microservices Applications on AWS (WIN401) - AWS re...
Architecting ASP.NET Core Microservices Applications on AWS (WIN401) - AWS re...
Amazon Web Services
Driving Innovation with Serverless Applications (GPSBUS212) - AWS re:Invent 2018
Driving Innovation with Serverless Applications (GPSBUS212) - AWS re:Invent 2018
Amazon Web Services
AWS Webcast - Datacenter Migration to AWS
AWS Webcast - Datacenter Migration to AWS
Amazon Web Services
Data Design and Modeling for Microservices I AWS Dev Day 2018
Data Design and Modeling for Microservices I AWS Dev Day 2018
AWS Germany
Data Design for Microservices - DevDay Austin 2017 Day 2
Data Design for Microservices - DevDay Austin 2017 Day 2
Amazon Web Services
Contenu connexe
Tendances
Streamlining Application Development with AWS Service Catalog (DEV328) - AWS ...
Streamlining Application Development with AWS Service Catalog (DEV328) - AWS ...
Amazon Web Services
AWS Dev Lounge: Applying the Twelve-Factor Application Manifesto to Developin...
AWS Dev Lounge: Applying the Twelve-Factor Application Manifesto to Developin...
Amazon Web Services
Eliminate Migration Confusion: Speed Migration with Automated Tracking (ENT31...
Eliminate Migration Confusion: Speed Migration with Automated Tracking (ENT31...
Amazon Web Services
SID305 AWS Certificate Manager Private CA
SID305 AWS Certificate Manager Private CA
Amazon Web Services
2018 10-19-jc conf-embrace-legacy-java-ee-by-aws-serverless
2018 10-19-jc conf-embrace-legacy-java-ee-by-aws-serverless
Kim Kao
Building Real-time Serverless Backends with GraphQL
Building Real-time Serverless Backends with GraphQL
Amazon Web Services
Control for Your Cloud Environment Using AWS Management Tools (ENT226-R1) - A...
Control for Your Cloud Environment Using AWS Management Tools (ENT226-R1) - A...
Amazon Web Services
SRV207 Orchestrating AWS Lambda with Step Functions
SRV207 Orchestrating AWS Lambda with Step Functions
Amazon Web Services
SID201 Overview of AWS Identity, Directory, and Access Services
SID201 Overview of AWS Identity, Directory, and Access Services
Amazon Web Services
Increase the Value of Video with ML & Media Services - SRV322 - New York AWS ...
Increase the Value of Video with ML & Media Services - SRV322 - New York AWS ...
Amazon Web Services
How AQR Capital Uses AWS to Research New Investment Signals
How AQR Capital Uses AWS to Research New Investment Signals
Amazon Web Services
Microsoft SQL Server - SQL Server Migrations Presentation
Microsoft SQL Server - SQL Server Migrations Presentation
Microsoft Private Cloud
Best Practices for Centrally Monitoring Resource Configuration & Compliance (...
Best Practices for Centrally Monitoring Resource Configuration & Compliance (...
Amazon Web Services
Legacy java ee meet lambda
Legacy java ee meet lambda
Kim Kao
AWS Systems Manage: Bridging Operational Models
AWS Systems Manage: Bridging Operational Models
Amazon Web Services
MongoDB World 2018: Tutorial - How to Build Applications with MongoDB Atlas &...
MongoDB World 2018: Tutorial - How to Build Applications with MongoDB Atlas &...
MongoDB
Gain Insights, Make Decisions, and Take Action Across a Streamlined and Autom...
Gain Insights, Make Decisions, and Take Action Across a Streamlined and Autom...
Arraya Solutions
Architecting ASP.NET Core Microservices Applications on AWS (WIN401) - AWS re...
Architecting ASP.NET Core Microservices Applications on AWS (WIN401) - AWS re...
Amazon Web Services
Driving Innovation with Serverless Applications (GPSBUS212) - AWS re:Invent 2018
Driving Innovation with Serverless Applications (GPSBUS212) - AWS re:Invent 2018
Amazon Web Services
AWS Webcast - Datacenter Migration to AWS
AWS Webcast - Datacenter Migration to AWS
Amazon Web Services
Tendances
(20)
Streamlining Application Development with AWS Service Catalog (DEV328) - AWS ...
Streamlining Application Development with AWS Service Catalog (DEV328) - AWS ...
AWS Dev Lounge: Applying the Twelve-Factor Application Manifesto to Developin...
AWS Dev Lounge: Applying the Twelve-Factor Application Manifesto to Developin...
Eliminate Migration Confusion: Speed Migration with Automated Tracking (ENT31...
Eliminate Migration Confusion: Speed Migration with Automated Tracking (ENT31...
SID305 AWS Certificate Manager Private CA
SID305 AWS Certificate Manager Private CA
2018 10-19-jc conf-embrace-legacy-java-ee-by-aws-serverless
2018 10-19-jc conf-embrace-legacy-java-ee-by-aws-serverless
Building Real-time Serverless Backends with GraphQL
Building Real-time Serverless Backends with GraphQL
Control for Your Cloud Environment Using AWS Management Tools (ENT226-R1) - A...
Control for Your Cloud Environment Using AWS Management Tools (ENT226-R1) - A...
SRV207 Orchestrating AWS Lambda with Step Functions
SRV207 Orchestrating AWS Lambda with Step Functions
SID201 Overview of AWS Identity, Directory, and Access Services
SID201 Overview of AWS Identity, Directory, and Access Services
Increase the Value of Video with ML & Media Services - SRV322 - New York AWS ...
Increase the Value of Video with ML & Media Services - SRV322 - New York AWS ...
How AQR Capital Uses AWS to Research New Investment Signals
How AQR Capital Uses AWS to Research New Investment Signals
Microsoft SQL Server - SQL Server Migrations Presentation
Microsoft SQL Server - SQL Server Migrations Presentation
Best Practices for Centrally Monitoring Resource Configuration & Compliance (...
Best Practices for Centrally Monitoring Resource Configuration & Compliance (...
Legacy java ee meet lambda
Legacy java ee meet lambda
AWS Systems Manage: Bridging Operational Models
AWS Systems Manage: Bridging Operational Models
MongoDB World 2018: Tutorial - How to Build Applications with MongoDB Atlas &...
MongoDB World 2018: Tutorial - How to Build Applications with MongoDB Atlas &...
Gain Insights, Make Decisions, and Take Action Across a Streamlined and Autom...
Gain Insights, Make Decisions, and Take Action Across a Streamlined and Autom...
Architecting ASP.NET Core Microservices Applications on AWS (WIN401) - AWS re...
Architecting ASP.NET Core Microservices Applications on AWS (WIN401) - AWS re...
Driving Innovation with Serverless Applications (GPSBUS212) - AWS re:Invent 2018
Driving Innovation with Serverless Applications (GPSBUS212) - AWS re:Invent 2018
AWS Webcast - Datacenter Migration to AWS
AWS Webcast - Datacenter Migration to AWS
Similaire à Microservices: Data & Design - Miguel Cervantes
Data Design and Modeling for Microservices I AWS Dev Day 2018
Data Design and Modeling for Microservices I AWS Dev Day 2018
AWS Germany
Data Design for Microservices - DevDay Austin 2017 Day 2
Data Design for Microservices - DevDay Austin 2017 Day 2
Amazon Web Services
Data Design for Microservices
Data Design for Microservices
Amazon Web Services
A Modern Data Architecture for Microservices
A Modern Data Architecture for Microservices
Amazon Web Services
Introduction to Serverless computing and AWS Lambda - Floor28
Introduction to Serverless computing and AWS Lambda - Floor28
Boaz Ziniman
Introduction to Serverless computing and AWS Lambda | AWS Floor28
Introduction to Serverless computing and AWS Lambda | AWS Floor28
Amazon Web Services
2019 03-13-implementing microservices by ddd
2019 03-13-implementing microservices by ddd
Kim Kao
Implementing Microservices by DDD
Implementing Microservices by DDD
Amazon Web Services
Serverless on AWS: Architectural Patterns and Best Practices
Serverless on AWS: Architectural Patterns and Best Practices
Vladimir Simek
Wellington Management: The Journey to All-In, One Data Center at a Time (FSV2...
Wellington Management: The Journey to All-In, One Data Center at a Time (FSV2...
Amazon Web Services
Introduction to Serverless on AWS - Builders Day Jerusalem
Introduction to Serverless on AWS - Builders Day Jerusalem
Amazon Web Services
SRV315 Building Enterprise-Grade Serverless Apps
SRV315 Building Enterprise-Grade Serverless Apps
Amazon Web Services
Coordinating Microservices with AWS Step Functions.pdf
Coordinating Microservices with AWS Step Functions.pdf
Amazon Web Services
Serverless Architectural Patterns and Best Practices
Serverless Architectural Patterns and Best Practices
Amazon Web Services
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Amazon Web Services
Migrating database to cloud
Migrating database to cloud
Amazon Web Services
How can your business benefit from going serverless?
How can your business benefit from going serverless?
Adrian Hornsby
Getting Started with Serverless Architectures with Microservices_AWSPSSummit_...
Getting Started with Serverless Architectures with Microservices_AWSPSSummit_...
Amazon Web Services
Introduction to Serverless
Introduction to Serverless
Steven Bryen
Breaking Down the 'Monowhat'
Breaking Down the 'Monowhat'
Amazon Web Services
Similaire à Microservices: Data & Design - Miguel Cervantes
(20)
Data Design and Modeling for Microservices I AWS Dev Day 2018
Data Design and Modeling for Microservices I AWS Dev Day 2018
Data Design for Microservices - DevDay Austin 2017 Day 2
Data Design for Microservices - DevDay Austin 2017 Day 2
Data Design for Microservices
Data Design for Microservices
A Modern Data Architecture for Microservices
A Modern Data Architecture for Microservices
Introduction to Serverless computing and AWS Lambda - Floor28
Introduction to Serverless computing and AWS Lambda - Floor28
Introduction to Serverless computing and AWS Lambda | AWS Floor28
Introduction to Serverless computing and AWS Lambda | AWS Floor28
2019 03-13-implementing microservices by ddd
2019 03-13-implementing microservices by ddd
Implementing Microservices by DDD
Implementing Microservices by DDD
Serverless on AWS: Architectural Patterns and Best Practices
Serverless on AWS: Architectural Patterns and Best Practices
Wellington Management: The Journey to All-In, One Data Center at a Time (FSV2...
Wellington Management: The Journey to All-In, One Data Center at a Time (FSV2...
Introduction to Serverless on AWS - Builders Day Jerusalem
Introduction to Serverless on AWS - Builders Day Jerusalem
SRV315 Building Enterprise-Grade Serverless Apps
SRV315 Building Enterprise-Grade Serverless Apps
Coordinating Microservices with AWS Step Functions.pdf
Coordinating Microservices with AWS Step Functions.pdf
Serverless Architectural Patterns and Best Practices
Serverless Architectural Patterns and Best Practices
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Migrating database to cloud
Migrating database to cloud
How can your business benefit from going serverless?
How can your business benefit from going serverless?
Getting Started with Serverless Architectures with Microservices_AWSPSSummit_...
Getting Started with Serverless Architectures with Microservices_AWSPSSummit_...
Introduction to Serverless
Introduction to Serverless
Breaking Down the 'Monowhat'
Breaking Down the 'Monowhat'
Plus de Amazon Web Services
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Amazon Web Services
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Amazon Web Services
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Amazon Web Services
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Amazon Web Services
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Amazon Web Services
Open banking as a service
Open banking as a service
Amazon Web Services
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Amazon Web Services
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Amazon Web Services
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Amazon Web Services
Computer Vision con AWS
Computer Vision con AWS
Amazon Web Services
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
Amazon Web Services
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Amazon Web Services
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Amazon Web Services
Tools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
How to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
Building a web application without servers
Building a web application without servers
Amazon Web Services
Fundraising Essentials
Fundraising Essentials
Amazon Web Services
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Amazon Web Services
Come costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWS
Amazon Web Services
Plus de Amazon Web Services
(20)
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Open banking as a service
Open banking as a service
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Computer Vision con AWS
Computer Vision con AWS
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Tools for building your MVP on AWS
Tools for building your MVP on AWS
How to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Building a web application without servers
Building a web application without servers
Fundraising Essentials
Fundraising Essentials
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Come costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWS
Microservices: Data & Design - Miguel Cervantes
1.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Data Design for Microservices Miguel Cervantes migcerva@amazon.com Solutions Architect
2.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Microservices at Amazon • Service-Oriented Architecture (SOA) • Single-purpose • Connect only through APIs • Connect over HTTPS • “Microservices”
3.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Monolithic vs. SOA vs. Microservices Microservices: • Many very small components • Business logic lives inside of single service domain • Simple wire protocols(HTTP with XML/JSON) • API driven with SDKs/Clients SOA: • Fewer more sophisticated components • Business logic can live across domains • Enterprise Service Bus like layers between services • Middleware
4.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Monolithic vs. SOA vs. Microservices SOA Coarse-grained Microservices Fine-grained Monolithic Single Unit
5.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Microservice Challenges • Distributed computing is hard • Transactions – Multiple Databases across multiple services • Eventual Consistency • Lots of moving parts • Service discovery • Increase coordination • Increase message routing
6.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Key Elements of Microservices… • Some core concepts are common to all services – Service registration, discovery, wiring, administration – State management – Service metadata – Service versioning – Caching • Low Friction Deployment • Automated Management and Monitoring
7.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Key Elements of Microservices… • Eliminates any long-term commitment to a technology stack • Polyglot ecosystem • Polyglot persistence – Decompose Databases – Database per microservice pattern • Allows easy use of Canary and Blue-Green deployments
8.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Key Elements of Microservices… • Each microservice is: – Elastic: scales up or down independently of other services – Resilient: services provide fault isolation boundaries – Composable: uniform APIs for each service – Minimal: highly cohesive set of entities – Complete: loosely coupled with other services Controller A Controller B Controller A Controller B Q Q Tight Coupling Loose Coupling
9.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Microservices Benefits • Fast to develop • Rapid deployment • Parallel development & deployment • Closely integrated with DevOps – Now ”DevSecOps” • Improved scalability, availability & fault tolerance • More closely aligned to business domain
10.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved • Two-pizza teams • Full ownership • Full accountability • Aligned incentives • “DevOps” Principles of the Two Pizza Team
11.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved How do Two Pizza Teams work? We call them “Service teams” • Own the “primitives” they build: – Product planning (roadmap) – Development work – Operational/Client support work • “You build it, you run it” • Part of a larger concentrated org – Amazon.com, AWS, Fresh, …
12.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data Architecture Challenges
13.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved
14.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved
15.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Challenge: Centralized Database user-svc account-svccart-svc DB Applications often have a monolithic data store • Difficult to make schema changes • Technology lock-in • Vertical scaling • Single point of failure
16.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Centralized Database – Anti-pattern user-svc account-svccart-svc DB Applications often have a monolithic data store • Difficult to make schema changes • Technology lock-in • Vertical scaling • Single point of failure
17.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Decentralized Data Stores account-svccart-svc DynamoDB RDS user-svc ElastiCache RDS Polyglot Persistence • Each service chooses its data store technology • Low impact schema changes • Independent scalability • Data is gated through the service API
18.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Challenge: Transactional Integrity • Polyglot persistence generally translates into eventual consistency • Asynchronous calls allow non-blocking, but returns need to be handled properly • How about transactional integrity? • Event-sourcing – Capture changes as sequence of events • Staged commit • Rollback on failure ERROR STATE? ROLLBACK?
19.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Best Practice: Use Correlation IDs • 09-02-2015 15:03:24 ui-svc INFO [uuid-123] …… • 09-02-2015 15:03:25 catalog-svc INFO [uuid-123] …… • 09-02-2015 15:03:26 checkout-svc ERROR [uuid-123] …… • 09-02-2015 15:03:27 payment-svc INFO [uuid-123] …… • 09-02-2015 15:03:27 shipping-svc INFO [uuid-123] …… ui-svc catalog-svc checkout- svc shipping- svc payment- svc request correlation id: “uuid-123” correlation id: “uuid-123”
20.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Best Practice: Microservice owns Rollback • Every microservice should expose its own “rollback” method • This method could just rollback changes, or trigger subsequent actions • Could send a notification • If you implement staged commit, also expose a commit function Microservice Function 1 Rollback Commit (optional)
21.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Event-Driven: DynamoDB Streams • If async, consider event-driven approach with DynamoDB Streams • Don’t need to manage function execution failure, DDB Streams automatically retries until successful • “Attach” yourself to the data of interest Microservice
22.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Challenge: Report Errors / Rollback • What if functions fail? (business logic failure, not code failure) • Create a “Transaction Manager” microservice that notifies all relevant microservices to rollback or take action • DynamoDB is the trigger for the clean-up function (could be SQS, Kinesis etc.) • Use Correlation ID to identify relations mm-svc Transaction Manager Function DDB Streams API Call Error Table
23.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Challenge: Report Errors / Rollback ERROR DynamoDB Error Table Transaction Manager Function Kinesis Error Stream SQS Error Queue Rollback (correlation-id) Rollback (correlation-id) Rollback (correlation-id) Rollback (correlation-id)
24.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Challenge: Code Error Lambda Execution Error because of faulty code Leverage Cloudwatch Logs to process error message and call Transaction Manager Set Cloudwatch Logs Metric Filter to look for Error/Exception and call Lambda Handler upon Alarm state ui-svc Cloudwatch Logs Cloudwatch Alarm Transaction Manager Function
25.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Beware: Stream Model with AWS Lambda • DynamoDB Streams and Kinesis streams directly work with AWS Lambda, however AWS Lambda needs to acknowledge processing the message correctly • If Lambda fails to process the message, the stream horizon will not be moved forward, creating a “jam” • Solution: Monitor AWS Lambda Error CloudWatch Metric and react when error rate of same “Correlation ID” keeps increasing
26.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Keep Data Consistent Databases AWS Lambda “Cleanup” Function Cloudwatch Scheduled Event • Perform Master Data Management (MDM) to keep data consistent • Create AWS Lambda function to check consistencies across microservices and “cleanup” • Create CloudWatch Event to schedule the function (e.g. hourly basis)
27.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Choosing a Datastore
28.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Storage & DB options in AWS Amazon RDS Amazon DynamoDB Amazon Elasticsearch Service Amazon S3 Amazon Kinesis Amazon ElastiCache In-Memory NoSQL SQL SearchObject Streaming Amazon Redshift Amazon Glacier
29.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Challenge: What Service to Use? • Many problems can be solved with NoSQL, RDBMS or even in-memory cache technologies • Non-functional requirements can help identify appropriate services • Solution: Classify your organizations non-functional requirements and map them to service capabilities
30.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Determine Your Non-Functional Requirements Requirement Latency > 1s 200 ms -1s 20 ms – 200 ms < 20 ms Durability 99.99 99.999 99.9999 > 99.9999 Storage Scale < 256 GB 256 GB – 1 TB 1 TB – 16 TB > 16 TB Availability 99 99.9 99.95 > 99.95 Data Class Public Important Secret Top Secret Recoverability 12 – 24 hours 1 – 12 hours 5 mins – 1 hour < 5 mins Skills None Average Good Expert This is only an example. Your classifications will be different There will be other requirements such as regulatory compliance too.
31.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Map Non-Functional Requirements to Services Service Latency Durability Storage Availability Recoverability from AZ Failure (RPO, RTO) RDS < 100 ms > 99.8 (EBS) 16 TB 99.95 0s and 90s (MAZ) Aurora < 100 ms > 99.9 64 TB > 99.95 0s and < 30s (MAZ) Aurora + ElastiCache < 1 ms > 99.9 64 TB > 99.95 0s and < 30s (MAZ) DynamoDB < 10 ms > 99.9 No Limit > 99.99 0s and 0s DynamoDB / DAX < 1 ms > 99.9 No Limit > 99.99 0s and 0s ElastiCache Redis < 1 ms N/A 6.1 TiB 99.95 0s and < 30s (MAZ) Elasticsearch < 200 ms > 99.9 1.5 PB 99.95 0s and < 30s (Zone Aware) S3 < 500 ms 99.999999999 No Limit 99.99 0s and 0s The information below is not exact and does not represent SLAs
32.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Finalizing Your Data Store Choices • After mapping your non-functional requirements to services you should have a short list to choose from • Functional requirements such as geospatial data and query support will refine the list further • You may institute standards to make data store selection simpler and also make it easier for people to move between teams. These can still be overridden, but require justification to senior management
33.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Challenge: Reporting and Analytics • Data is now spread across a number of isolated polyglot data stores • Consolidation and aggregation required • Solution: Pull data from required microservices, push data to data aggregation service, use pub/sub, or use a composite service (anti-pattern).
34.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved Aggregation usr svc Pull model Push model Data Aggregation Application usr svc Data Aggregation Application Pub/Sub Composite Composite Data Service usr account cart account svc cart svc account svc cart svc Pub Sub usr svc account svc cart svc Data Aggregation Application Push Pull
35.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved A Few Thoughts • Use Non-Functional Requirements to help identify the right data store(s) for each microservice • Use polyglot persistence to avoid bottlenecks, schema issues and allow independent scalability (and cache) • Embrace eventual consistency and design fault-tolerant business processes which can recover • Think ahead and plan your analytics requirements as part of the overall architecture
36.
© 2018, Amazon
Web Services, Inc. or its Affiliates. All rights reserved aws.amazon.com/activate Everything and Anything Startups Need to Get Started on AWS