Introduction to Amazon Relational Database Service (Amazon RDS)
•
3 j'aime•990 vues
by Yoav Eilat, Product Marketing Manager, AWS
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Introduction to Amazon Relational Database Service (Amazon RDS)
Similaire à Introduction to Amazon Relational Database Service (Amazon RDS) (20)
Plus de Amazon Web Services
Plus de Amazon Web Services (20)
Dernier
Dernier (20)
Introduction to Amazon Relational Database Service (Amazon RDS)
- 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. June 16, 2017 Introduction to Amazon Relational Database Service (Amazon RDS) Yoav Eilat, AWS Product Marketing
- 2. AWS Data Services to Accelerate Your Move to the Cloud RDS Open Source RDS Commercial Aurora Migration for DB Freedom DynamoDB & DAX ElastiCache EMR Redshift Spectrum AthenaElasticsearch Service QuickSightGlue Lex Polly Rekognition Machine Learning Databases to Elevate your Apps Relational Non-Relational & In-Memory Analytics to Engage your Data Inline Data Warehousing Reporting Data Lake Amazon AI to Drive the Future Deep Learning, MXNet Database Migration Schema Conversion
- 3. • Multi-engine support – Aurora, MySQL, MariaDB, PostgreSQL, Oracle, SQL Server • Automated provisioning, Scaling, Patching, Backup/Restore • High availability with RDS Multi-AZ, Auto-Failover – 99.95% SLA for Multi-AZ deployments • Security • Monitoring Amazon RDS
- 4. Provisioning and Effortless Scaling • Handle higher load or lower usage • Naturally grow over time • Control costs
- 5. Read Replicas Bring data close to your customer’s applications in different regions Relieve pressure on your master node for supporting reads and writes Promote a Read Replica to a master for faster recovery in the event of disaster
- 6. Enterprise-grade fault tolerance solution for production databases Automatic failover Synchronous replication Inexpensive & enabled with one click High Availability Multi-AZ Deployments
- 7. Security and Compliance • Network Isolation • Database instance IP firewall protection • AWS IAM based resource-level permission controls • Encryption at rest using AWS KMS or Oracle/Microsoft TDE • SSL protection for data in transit • Assurance programs for finance, healthcare, government and more
- 8. Amazon Virtual Private Cloud (Amazon VPC) Securely control network configuration Availability Zone AWS Region 10.1.0.0/16 10.1.1.0/24 Manage connectivity VPN connection VPC peering Internet gateway MAWS Direct Connect Routing rules
- 9. Security groups Database IP firewall protection Protocol Port Range Source TCP 3306 172.31.0.0/16 TCP 3306 “Application security group” Corporate address admins Application tier M
- 10. IAM governed access You can use AWS Identity and Access Management (IAM) to control who can perform actions on RDS Users and DBAApplications DBA and Ops Your database RDS Controlled with IAMControlled with database grants M
- 11. At Rest Encryption for all RDS Engines AWS Key Management Service (KMS) Two-tiered key hierarchy using envelope encryption: • Unique data key encrypts customer data • AWS KMS master keys encrypt data keys • Available for ALL RDS engines Benefits: • Limits risk of compromised data key • Better performance for encrypting large data • Easier to manage small number of master keys than millions of data keys • Centralized access and audit of key activity Data key 1 Data key 2 Data key 3 Data key 4 Custom application Customer master key(s) Amazon RDS instance 3 Amazon RDS instance 2 Amazon RDS instance 1
- 13. Compliance Aurora SOC 1, 2, 3 ISO 20001/9001 ISO 27107/27018 PCI HIPAA BAA MySQL SOC 1, 2, 3 ISO 20001/9001 ISO 27107/27018 PCI FedRamp HIPAA BAA UK Gov. Programs Singapore MTCS Oracle SOC 1, 2, 3 ISO 20001/9001 ISO 27107/27018 PCI FedRamp HIPAA BAA UK Gov. Programs Singapore MTCS MariaDB SOC 1, 2, 3 ISO 20001/9001 ISO 27107/27018 PCI PostgreSQL SOC 1, 2, 3 ISO 20001/9001 ISO 27107/27018 PCI FedRamp HIPAA BAA UK Gov. Programs Singapore MTCS SQL Server SOC 1, 2, 3 ISO 20001/9001 ISO 27107/27018 PCI UK Gov. Programs Singapore MTCS
- 14. Standard monitoring Amazon CloudWatch metrics for Amazon RDS CPU utilization Storage Memory Swap usage DB connections I/O (read and write) Latency (read and write) Throughput (read and write) Replica lag Many more Amazon CloudWatch Alarms Similar to on-premises custom monitoring tools
- 15. Enhanced Monitoring Access to over 50 new CPU, memory, file system, and disk I/O metrics as low as 1 second intervals
- 17. • Airbnb moved its main MySQL database to Amazon RDS with only 15 minutes of downtime • RDS simplifies much of the time-consuming administrative tasks associated with databases so engineers can spend more time on features • Uses asynchronous master-slave replication to improve website performance launched via the RDS console or an API call • Leverages multi-Availability Zone (Multi-AZ) for high availability Airbnb – Amazon RDS for MySQL
- 18. The Forrester Wave™ is copyrighted by Forrester Research, Inc. Forrester and Forrester Wave™ are trademarks of Forrester Research, Inc. The Forrester Wave™ is a graphical representation of Forrester's call on a market and is plotted using a detailed spreadsheet with exposed scores, weightings, and comments. Forrester does not endorse any vendor, product, or service depicted in the Forrester Wave. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. The Forrester Wave™: Database-As-A-Service, Q2 2017
- 19. Amazon Aurora: Reinventing the Relational Database
- 20. Key Questions We Asked • What if we started from a clean sheet of paper with only constraint being that the database was a relational database? • Could we offer much better performance by leveraging the massive scale of our cloud? • Could we give you a database with designed durability indistinguishable from 100% and availability of 99.99%? • …And could we be better and cheaper than the 30-year old commercial databases in use today?
- 21. Amazon RDS for Aurora • MySQL compatible with up to 5x better performance on the same hardware: 100,000 writes/sec & 500,000 reads/sec • Scalable with up to 64 TB in single database, up to 15 read replicas • Highly available, durable, and fault-tolerant custom SSD storage layer: 6-way replicated across 3 Availability Zones • Transparent encryption for data at rest using AWS KMS • Stored procedures in Amazon Aurora can invoke AWS Lambda functions Fastest growing service in AWS history A new relational database engine, built from the ground up to leverage AWS
- 23. Use case: Near real-time analytics and reporting Master Read Replica Read Replica Read Replica Shared distributed storage volume Reader end-point A customer in the travel industry migrated to Aurora for their core reporting application accessed by ~1,000 internal users. Replicas can be created, deleted and scaled within minutes based on load. Read-only queries are load balanced across replica fleet through a DNS endpoint – no application configuration needed when replicas are added or removed. Low replication lag allows mining for fresh data with no delays, immediately after the data is loaded. Significant performance gains for core analytics queries - some of the queries executing in 1/100th the original time. ► Up to 15 promotable read replicas ► Low replica lag – typically < 10ms ► Reader end-point with load balancing
- 24. Amazon Aurora is now PostgreSQL-compatible • PostgreSQL 9.6 compatibility with support for PostGIS • All the features you expect from Amazon Aurora including 15 read replicas with <10ms lag, shared storage, failover without data loss, 6-way replication across 3 Availability Zones, encryption with AWS KMS • Available now in preview
- 25. Simplify monitoring from the AWS Management Console Database load: Identifies database bottlenecks Easy Powerful Identifies source of bottlenecks Top SQL Adjustable time frame Hour, day, week, and longer Max CPU Amazon RDS Performance Insights
- 26. AWS Database Migration Service • Fully managed service for migration from on-premises to the AWS Cloud with minimal downtime • Migrates data to and from all widely used commercial and open source DBs • Schema Conversion Tool that converts source DB schemas, stored procedures and application code to a different target format • Supports homogenous and heterogeneous data replication • A terabyte-sized DB can be migrated for as little as $3
- 27. Database Conversion Capabilities in SCT Source Database Target Database Microsoft SQL Server Amazon Aurora, MySQL, PostgreSQL MySQL, MariaDB Amazon Aurora, PostgreSQL Oracle Amazon Aurora, MySQL, PostgreSQL Oracle Data Warehouse Amazon Redshift PostgreSQL Amazon Aurora, MySQL Teradata, Netezza, Greenplum Amazon Redshift HP Vertica, SQL Server DW Amazon Redshift MongoDB Amazon DynamoDB
- 28. AWS Database Migration Service Customers
- 29. Heterogeneous Migration • Oracle private DC to RDS PostgreSQL migration • Used the AWS Schema Conversion Tool to convert their database schema • Used on-going replication (CDC) to keep databases in sync until they reached the cutover window • Benefits: • Improved reliability of the cloud environment • Savings on Oracle licensing costs • SCT Assessment Report let them understand the scope of the migration
Notes de l'éditeur
- Intro myself. What I do. Time with Amazon/AWS We are going to go through RDS as a whole attempting to provide you as much education and guidance as possible while also going deeper into some areas to help you get a better understanding on how RDS works so that you can better understand how it can be a beneficial service for you to run your applications and workloads on. Have a few quick poll questions so I better understand who my audience is today. Who is using RDS today Who is looking to move to RDS Who is running a database but on EC2.
- AWS offers a broad portfolio, across Databases, Analytics, and Artificial Intelligence, driven by customer requests. Over 90% of new features and services are driven by customer feedback. Our customers’ demands are usually driven by a need to make application experiences better. Better means faster to deploy, easier to run, more reliable, and less complex & less expensive to operate. For databases, we have a mix of relational Open Source, Commercial Engines, and Amazon-created Aurora; for the greatest scalability and performance we have non-relational Open Source and Amazon-created DynamoDB. Our customers use a range of analytics, as many are migrating from appliances to on-demand Cloud services with decoupled data, again with a number of Open Source offerings plus Amazon-created Redshift and Redshift Spectrum. We also have the support services needed to enable ETL, serverless query, and visualization. Artificial Intelligence is a growing part of the AWS portfolio, for deep learning with platforms of machine learning and finished services to enable developers to easy add voice and pictorial capabilities to applications. Our customers use migration for all purposes: both modernization of existing applications and moving between database options, as much for analytics as relational and non-relational databases.
- Choice of engines. Same database engine as you use on-premise. Should expect Custom applications to work with minimal adjustments. Speaking of the MANAGED database service, the whole point is to enable you to focus on what differentiates your organization. We handle the provisioning, give you click-to-scale capability, handle patching, backup/restore. We cover its HA – w/ Multi-AZ deployment **&** automated failover Security and Monitoring
- you choose Multi-AZ then the service will launch the RDS instance in one of the AZs available to you in the region you are launching in. If you choose single AZ it will launch in the AZ you choose. Call out that you can modify your environment from single AZ to multi-AZ. when you are running in a Muti AZ configuration this is how things work when there is a switch from the primary to the standby. This is a key feature of this being a managed service. When one of the failure conditions that I talked about in the previous slides is met the RDS service is going to switch the DNS name for your database from the primary to the standby. DNS failover typically takes 30-60 seconds. Once this happens you are now running against your standby database with no action required on your part. As part of the failover the standby is going to go through the standard crash recovery that happens for that particular engine, just as it would do if it were running in your data center or running on EC2.
- Provisioning is a great place to start our conversation on Managed Database service. All it takes on AWS, is a handful of clicks on the console or an API call and you have access to a healthy database in a few minutes. But sizing HW for the database is a challenge. Its like trying to swat a fly in mid-air. You know its right there; but by the time you catch up to it, its on a completely different trajectory. With RDS, you address this with Agility. You can pick from a variety of server sizes and storage types to get started. You have the flexibility to change out infrastructure as your applications gain adoption and complexity. You can scale up in anticipation of peak workloads and then back down! Swapping out database servers can now be done in minutes giving you the real ability to optimize cost/capacity. Impossible to do on-premise. Another way to scale the database is to use …
- Read-replicas! RDS makes it easy to create read replicas of your database and automatically keeps them in sync. You can now offload the master database by pointing read workloads to read replicas. Depending on the database engines, you can even setup these read replicas in other regions from the master. You can leverage this to improve end-user experience by satisfying application reads from the nearest local region. In addition to using Read Replication scalability; RDS also provides a simple 1-click mechanism to promote a Read Replica to “Master”. This allows you to mitigate impact to your end-users from failures or issues with your Master Database.. You can think of this as a limited HA. RDS leverages log replication to keep the Read Replicas in-sync. Sometimes there might be replication lag between the Master and the Read Replica. You could loose transactions if there’s a replication lag when the Master fails.
- RDS provides a true Enterprise grade fault tolerance capability with synchronous replication in multi-AZ deployments. It is enabled with just one-click and can be configured for automatic failover.
- At AWS, Security is a top priority. RDS leverages many features of the AWS platform to ensure security for the databases. So lets dive into these components with .. Networking
- First off, RDS instances are launched inside of a Virtual Private Cloud or VPC. As many of you may already know a VPC gives you the ability to define a private network address space within AWS. You can also carve out different subnets within your VPC address space to further segment and isolate your application components, including your database. So with your database launched inside of a VPC you now get to control which users and applications access your database and how they access it. By “HOW” we’re talking about network connectivity! You can create a VPN connection from your corporate data center into the VPC so that you can access the database in a hybrid fashion. Or you can use Direct Connect to link your data center to an AWS region giving you a connection with consistent performance You can also use “VPC peering” to allow applications running in one VPC to access your database in another VPC. You can grant public access to your database by attaching an internet gateway to your VPC You can control the routing of your VPC using route tables that you attach to each of the subnets in your VPC
- Another way to control access to the database is with Security Groups. If you have used EC2 you are probably already familiar with security groups. Think of Security Groups are a virtual firewall that is placed around your database environment. A SG allows you to specify access rules for your database. Each rule is a combination of protocol, port range, and source of the traffic that you allow into your database. For the Source, you can setup an IP address, a particular CIDR block covering multiple IP addresses, or even another security group. Meaning that in this example, your RDS instance will accept traffic from all instances in the “Application security group”. This gives you the flexibility to setup a multi tier archicture where you only allow connections from the parts of the tier that actually need to access the database.
- IAM is about the WHO gets to access the database. RDS uses IAM capabilities like users, roles and privileges to enforce controls around who gets to launch, terminate or modify the database instance. This is different from database or applications users who are typically authenticated by the database itself.
- KMS is managed service Scaleability, Reliability, Durability, Security, Key deletion, Key creation, Key rotation You get to focus on your application and integrating with the service. For RDS you get an extra layer of protection around access to the underlying storage Encryption uses industry standard AES-256 encryption to protect the data. How you enable encryption for RDS CloudTrail integration Not only encrypting the DB Instance Storage but also… Automated Backups Read Replicas Snapshots Available for all six engines at no additional cost -------------------- So here is how RDS provides you the ability to encrypt your data. It is done using the AWS Key Management Service (KMS) KMS is a managed service that provides you the ability to create and manage encryption keys and then encrypt and decrypt your data with those keys. All of these keys are tied to your own AWS account and are fully managed by you. KMS takes care of all the availability, scaling, security, and durability that you would normally have to deal with when implementing your own key store. With KMS performing these management tasks it allows you the ability to focus on using the keys and building out your application. On top of all this the KMS team has also focused on making all of this very easy for you to use. Specific to RDS KMS gives you an additional layer of protection to guard against un-authorized access to the underlying storage of your RDS instance. For the data encryption KMS uses industry standard AES-256 encryption to protect the data that is stored on the underlying host that your RDS instance is running on. A little bit on how this works. We are using a two key hierarchy for encrypting data on your RDS instance. When you are launching an RDS instance you can choose to make the database encrypted. If you do nothing else besides turning on encryption this will result in an AWS managed key for RDS being created or the existing RDS customer master key will be used. This is good if you just want encryption and don’t want to think about anything else related to the key. Once this key is created it can only be used for RDS encryption and not with any other AWS service so you right away have scoped down the use of the key If you want you can create your own customer master key within KMS and then reference that key when you are creating your RDS instance. When you go the approach of creating your own key you get much more control over the use of that key such as when it is enabled or disabled, when the key is rotated, and what the access policies are for the key. Whatever your approach for the master key these keys cannot leave the service. You cannot export a plaintext version of that key out of the service. All you can do is ask that encrypt and decrypt operations happen against that key. When RDS wants to encrypt data on the instance it will make a call to KMS using the necessary credentials. KMS will then give RDS a data key that is actually used to encrypt the data on that instance. This data key is encrypted using the master key that was created when you launched the instance or that you created and specified for use during the instance creation. This data key is specific to that RDS instance and will not be used with another RDS instance. Without the presence and access to this data key the data within RDS is useless to someone accessing it throughout the normal path. So with this you have an overall master key that is then vending out data keys for each instance that you choose to encrypt. ------- Some of the benefits of using this service. Encryption and decryption are handled transparently so you don’t have to modify your application to access your data. Limited risks of a compromised data key Better performance for encrypting large data sets and there is no performance penalty for using KMS encryption with your RDS instance. You only have to manage a few master keys and not many data keys You get centralized access and audit of key activity via cloudtrail so that you can see every time a key is accessed in used from your KMS configuration. -------- Talk about why you should trust AWS with your keys There are no tools in place to access your physical key material. Your plaintext keys are never stored in nonvolatile memory. You control who has permissions to use your keys. Separation of duties between systems that use master keys and ones that use data keys. Multiparty controls for all maintenance of KMS systems that use your master keys. Third-party evidence of these controls: Service Organization Control (SOC 1) PCI-DSS See AWS Compliance packages for details ---- Enhance this slide to show some different RDS clusters each with a different data key.
- One of the reasons that customers are concerned with security is that they have certain compliance requirements that they have to meet AWS has lots of different customers in the startup, enterprise, and public sector space running a wide range of applications and workloads across many different industries. Some of these applications and workloads require that they meet certain regulatory requirements. To support the audit and compliance requirements of these customers AWS has worked to achieve certification that you can run your workloads on RDS and be able to achieve a full compliant application. So for RDS we currently offer 9 different compliance attestations including Soc1, 2, & 3, FedRamp, PCI/DSS, and HIPAA. With these compliance certifications for RDS it means that you can build and run your applications and workloads, related to these compliance bodies, on RDS. However, it does not relieve you of the responsibility of making sure that your application meets the appropriate compliance requirements. AWS takes responsibility for the compliance of the RDS service and the infrastructure related to it and you, the customer, take care of the application that you have built on top of AWS. With this model you can take the audit findings from your application and combine them with the appropriate attestation from AWS’s third party auditors and have a complete verification of compliance for your application running on AWS. ---------------- Chart of what is certified and what is not is here: https://w.amazon.com/index.php/AWS_Security_Assurance/Compliance/Scope
- Here is a listing of our compliance certifications and which database engines they currently apply to on RDS. You can see here that we have a wide range of certifications covering MySQL, Oracle, SQL Server, and PostgreSQL. As you can see there is a lot of coverage among these four different engines around key areas of compliance.
- This slide needs to flow relatively quick. For everyone using RDS you get a collection of 15-18 metrics that are automatically made available to you, depending on the type of instance that you are using. RDS sends the necessary information for these metrics to Amazon CloudWatch and then you are able to view the metrics in the RDS console, in the CloudWatch console, or via the CloudWatch APIs. These metrics are made available to you at one minute intervals. With this monitoring you can keep an eye on the performance of your database around key items like CPU utilization, memory, storage, latency, and any lag between your master and read replica databases. You can view this information in individual graphs, multiple graphs, or even pull them into your own monitoring tool. Additionally, you can also take these metrics and build alarms based on thresholds that are meaningful to you. Then whenever you go above that threshold you can have a notification or other action take place that helps to respond to that metric being outside of its normal boundaries. We will actually take a look at an example of what you can do there a bit later in this presentation. --------------- Basic cloudwatch metrics - http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/rds-metricscollected.html
- This slide needs to flow relatively quick Turning on “Enhanced Monitoring” gives you access to 37 additional metrics. This also gives you fine grained granularity from 1 min down to 1 sec. A few of the metrics that you get with enhanced monitoring are: Free Memory, Active Memory, Load Average, and how much of the file system you have used. You might be asking the differences between basic monitoring and enhanced monitoring and why they are not all delivered the same. The reason is that standard monitoring is based on what the Hypervisor that is running your database instance can see. There are metrics related to your OS that the hypervisor cannot see and we utilize an agent running on your RDS instance to collect the necessary metrics to give you enhanced monitoring. Enhanced monitoring is available for: all six engines ------- Discussed in a section on this page: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_Monitoring.html Available on all classes except t1.micro and m1.small
- Having leveraged AWS extensively, they also have built some home grown solutions that helped them monitor the security of AWS configurations. Recently they announced the open-sourcing of this solution. Cloud Security Architect Jason Chan provided some perspective on their use of RDS for PostgreSQL: "We recently announced the open source availability of Security Monkey, our solution for monitoring and analyzing the security of our Amazon Web Services configurations. We leveraged RDS PostgreSQL to capture the security data gathered by our solution. Building an application backed by production ready PostgreSQL database with Multi-AZ high availability, automated backups, patching and upgrades handled by RDS helped us focus on our development to bring this powerful open source solution to the community."
- moved main MySQL database to RDS because it automatically takes care of the time consuming administrative tasks. Easy to handle difficult tasks like Replication and Scaling w/ just simple API calls. AirBnB use Multi-AZ deployments to automate database replication and durability.
- What if we started with a clean sheet of paper? could we create a modern database with massive scalability Durability indistinguishable from 100% and 99.99% availability
- FIVE highlights #1 mySQL compatible w/ 5x better performance #2 Scalable to 64TB w/ 15 read replicas #3 HA, Durable and Fault Tolerant SSD storage w/ 6-way repl across 3 AZs!! #4 Transparent Encryption - DATA AT REST w/ KMS #5 Stored procedures that invoke AWS Lambda
- Use Aurora for near real-time Analytics and reporting app with about a 1000 users. They leverage Aurora's read replicas, creating, scaling and deleting replicas based on load. The app connects via a Reader End point and load balances the read workloads across the instances as the cluster grows and shrinks with no change to the app. Low replication lag; < 10ms means they're always working on current data. Significant performance gains with some queries executing in 1/100th of the original time.
- PostgreSQL 9.6 with support for PostGIS All the same features of Aurora w/ 15 read replicas; <10ms lab, 6-way replication across 3-AZs, KMS for data at rest encryption
- Talk Track: With Amazon Aurora Postgres, we are also announcing a new database monitoring feature called Performance Insights. Performance Insights is designed to help customers quickly assess whether there are any performance bottlenecks in their relational database workloads and where to take action. Collects detailed database performance data through light weight mechanisms and uses the data to drive an intuitive graphical interface that provides a simple and complete view of recent database performance. For questions: Will be rolling out incrementally to all RDS engines over 2017 First release will be on Postgres compatible edition of Aurora See the preview in the demo grounds Feature is free. Will be supported on all instances except micros Feature will keep 35 days of historical data
- We're doing a session at 2:30 on DMS A fully managed solution for migrating databases from on-prem to AWS Cloud with minimal downtime. . Works for OLTP and DW databases; Homogeneous and Hetrogeneous database migrations 1 TB database for
- Moved ORacle from on-premise to AWS RDS Postgres Leveraged SCT. SCT Assessment report to help understand the scope of effort and then with the actual schema migration. Used DMS for moving data with minimal downtime. Benefits: reliability of the cloud Savings on Oracle licensing costs.
- And that’s it. I really appreciate your attendance at this session today and I hope you got a lot of useful information out of this and that it helps you move forward in better utilizing the RDS service. Don’t forget to fill out your surveys as it helps us to make sure that we are delivering the best content possible to you.