Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Steve Abraham, Solutions Architect
Shree Kenghe,...
What to Expect from the Session
Agenda
• Housekeeping
• Migrating to AWS
• SCT Overview
• DMS Overview
• Q&A
• Workshop
Housekeeping
Download:
http://chilp.it/203c7aa
Housekeeping
Migrating to AWS
• Quickly provision databases
• Multiple Availability Zones
• Rapid scaling
• Automated patching
• Easy read replica creat...
• How will my on-premises data migrate to the cloud?
• How can I make it transparent to my users?
• How will on-premises a...
Migration Options
• Lift and shift
• Leverage Amazon EC2 and Amazon S3
• Keep existing DB engine but migrate to Amazon RDS...
Database Migration Process
Phase Description Automation Effort (%)
1 Assessment SCT 5
2 Database schema conversion SCT/DMS 10
3 Application conversio...
AWS Schema Conversion Tool
Overview
AWS Schema Conversion Tool
Features
• Converts schema of one database engine to another
• Database Migration Assessment re...
Convert Tables, Views, and Code
• Sequences
• User Defined Types
• Synonyms
• Packages
• Stored Procedures
• Functions
• T...
Components of the Console
1. Source Schema
2. Action Items
3. Target Schema
4. Schema Element Details
5. Edit Window
Database Migration Assessment
1. Connect Schema
Conversion Tool to
source and target
databases.
2. Run Assessment
Report.
...
Supported Conversions
Source Database Target Database
Microsoft SQL Server Amazon Aurora, Microsoft SQL Server, MySQL, Pos...
$0
for software license
Allowed Use
 Use Schema Conversion Tool to migrate database
schemas to Amazon RDS, Amazon Redshif...
Prerequisites
• Create Databases
• Source
• Target
• Download AWS Schema Conversion Tool
• http://amzn.to/2b2YE2a
• Downlo...
Global Settings – Logging
Global Settings – Drivers
Global Settings – Performance and Memory
Global Settings – Assessment Report
Time Considerations
• Any Hard Dates?
• Planning Time?
• Typically 2-3 Weeks
• Several Iterations
Database Considerations
• Number of Schemas?
• Number of Tables?
• Engine Specific Types?
• Users/Roles/Permissions?
Network Considerations
• Access (Firewalls, Tunnels, VPNs)?
• Which VPC?
• Which Security Groups?
Requirements Considerations
• Engine Selection Criteria?
• Which Tables Need to Move?
• Same Target For All Tables?
Temporary Work Schema
• Added to target DB instance
• After migration delete temporary work schema
• Schemas
• Microsoft S...
DMS Overview
• Start your first migration in 10 minutes or less
• Keep your apps running during the migration
• Replicate within, to, o...
Customer
premises
Application users
AWS
Internet
VPN
• Start a replication instance
• Connect to source and target
databas...
Multi-AZ option for high availability
Customer
premises
or AWS
AWS
Internet
VPN
AWS DMS
AWS DMS
AWS Database Migration service pricing
T2 for developing and periodic data migration tasks
C4 for large databases and mini...
Migration Scenarios and Options
On-Premises Migration Scenarios
• An on-premises database to a database on Amazon RDS DB
instance
• An on-premises databas...
RDS Migration Scenarios
• A database on an Amazon RDS DB instance to an on-premises
database
• A database on an Amazon RDS...
EC2 Migration Scenarios
• A database on an Amazon EC2 instance to an on-premises
database
• A database on an Amazon EC2 in...
DMS Components
• Replication Instances
• Endpoints
• Tasks
Replication Instances
• Performs the work of the migration
• Tasks run on instances
• Can support multiple tasks
• AWS DMS...
Public and Private Replication Instances
• A replication instance should have a public IP address if the source
or target ...
Sources for AWS Database Migration Service
Customers use the following databases as a source for data migration using AWS
...
Targets for AWS Database Migration Service
Customers can use the following databases as a target for data replication usin...
Tasks Overview
• Run on a replication instance
• Contain two and only two endpoints (source and target)
• Different migrat...
Migration Methods
• Migrate existing data
• Migrate existing data and replicate ongoing changes
• Replicate data changes o...
DMS – Change Data Capture (CDC)
“No Touch” design
• Reads recovery log of source database
• Using the engine’s native chan...
Replication
Instance
Source Target
Start Full Load
While Loading Data Also Capture Changes
Source TargetReplication Instance
Update
Load Complete - Apply Captured Changes
Source TargetReplication Instance
Update
Changes Reach Steady State
Source TargetReplication Instance
Update
Cutover - Shut Down Apps & Apply Remaining Changes
Source TargetReplication Instance
Update
Flip!
Source TargetReplication Instance
Update
Changes are Transactional and Come From the Logs
Migration Server
Source Target
Update
t
1
t
2
t
1
t
2
Replication
Instance
Source Target
Multiple Targets
Target
Target
Migration
Server
Source Target
Multiple Sources
Source
Source
Migration
Server
Source
Target
Multiple Sources and Targets
Source
Source
Target
You Don’t Have to Take Everything
Source

Target
Replication Instance
Homogenous or Heterogeneous
Replication
Instance
SQL Server MySQL
Replication
Instance
Oracle Oracle
Replication
Instance
...
Lab
Lab Document
Download: http://chilp.it/499290a
Lab to be run in us-west-2 (Oregon) region
Lab
Thank you!
Remember to complete
your evaluations!
Prochain SlideShare
Chargement dans…5
×

AWS re:Invent 2016: Workshop: Using the Database Migration Service (DMS) for Database Consolidation, Data Distribution and Replication (DAT321)

10 339 vues

Publié le

It can help you do much more. You can use DMS to consolidate multiple databases into a single database or split a single database into multiple databases. You can also use DMS for data distribution to multiple systems. For both of these use cases your source database can be outside of AWS (on premises) or in AWS (EC2 or RDS). DMS can also be used for near real-time replication of data. Replication can be done to one or more targets within AWS, in the same region or across regions. You can also replicate data from databases within AWS to databases outside of AWS. In this session we will discuss all these usage patterns and help you try them out yourselves.

Prerequisites:
You should have good database knowledge and at least some experience with Amazon RDS or Amazon Aurora.

Participants should have an AWS account established and available for use during the workshop.

Please bring your own laptop.

Publié dans : Technologie
  • A professional Paper writing services can alleviate your stress in writing a successful paper and take the pressure off you to hand it in on time. Check out, please ⇒ www.HelpWriting.net ⇐
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Writing good research paper is quite easy and very difficult simultaneously. It depends on the individual skill set also. You can get help from research paper writing. Check out, please ⇒ www.HelpWriting.net ⇐
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

AWS re:Invent 2016: Workshop: Using the Database Migration Service (DMS) for Database Consolidation, Data Distribution and Replication (DAT321)

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Steve Abraham, Solutions Architect Shree Kenghe, Solutions Architect November 29, 2016 Workshop: Using the Database Migration Service (DMS) for Database Consolidation, Data Distribution, and Replication DAT321
  2. 2. What to Expect from the Session Agenda • Housekeeping • Migrating to AWS • SCT Overview • DMS Overview • Q&A • Workshop
  3. 3. Housekeeping Download: http://chilp.it/203c7aa
  4. 4. Housekeeping
  5. 5. Migrating to AWS
  6. 6. • Quickly provision databases • Multiple Availability Zones • Rapid scaling • Automated patching • Easy read replica creation • High durability • Point in time recovery • Detailed metrics • Single-click encryption at rest Amazon RDS Why AWS?
  7. 7. • How will my on-premises data migrate to the cloud? • How can I make it transparent to my users? • How will on-premises and cloud data interact? • How can I integrate my data assets within AWS? • How can I move off of commercial databases? How?
  8. 8. Migration Options • Lift and shift • Leverage Amazon EC2 and Amazon S3 • Keep existing DB engine but migrate to Amazon RDS • For example Oracle on-premises to RDS Oracle • Migrate database engine • Commercial engine to open source • Maintenance window • Maintenance window duration vs. CDC with 0 downtime
  9. 9. Database Migration Process
  10. 10. Phase Description Automation Effort (%) 1 Assessment SCT 5 2 Database schema conversion SCT/DMS 10 3 Application conversion/remediation SCT 25 4 Scripts conversion SCT 6 5 Integration with third-party applications 4 6 Data migration DMS 4 7 Functional testing of the entire system 30 8 Performance tuning SCT 2 9 Integration and deployment 7 10 Training and knowledge 2 11 Documentation and version control 2 12 Post production support 3 Database Migration Phases
  11. 11. AWS Schema Conversion Tool Overview
  12. 12. AWS Schema Conversion Tool Features • Converts schema of one database engine to another • Database Migration Assessment report for choosing the best target engine • Code browser that highlights places where manual edits are required The AWS Schema Conversion Tool helps automate many database schema and code conversion tasks when migrating from Oracle and SQL Server to open source database engines.
  13. 13. Convert Tables, Views, and Code • Sequences • User Defined Types • Synonyms • Packages • Stored Procedures • Functions • Triggers • Schemas • Tables • Indexes • Views
  14. 14. Components of the Console 1. Source Schema 2. Action Items 3. Target Schema 4. Schema Element Details 5. Edit Window
  15. 15. Database Migration Assessment 1. Connect Schema Conversion Tool to source and target databases. 2. Run Assessment Report. 3. Read Executive Summary. 4. Follow detailed instructions.
  16. 16. Supported Conversions Source Database Target Database Microsoft SQL Server Amazon Aurora, Microsoft SQL Server, MySQL, PostgreSQL MySQL MySQL, PostgreSQL Oracle Amazon Aurora, MySQL, Oracle, PostgreSQL Oracle Data Warehouse Amazon Redshift PostgreSQL Amazon Aurora, MySQL, PostgreSQL Teradata Amazon Redshift
  17. 17. $0 for software license Allowed Use  Use Schema Conversion Tool to migrate database schemas to Amazon RDS, Amazon Redshift, or Amazon EC2–based databases  To use Schema Conversion Tool to migrate schemas to other destinations, contact for special pricing Pricing  Free software license  For active AWS customers with accounts in good standing Pricing, Terms & Conditions
  18. 18. Prerequisites • Create Databases • Source • Target • Download AWS Schema Conversion Tool • http://amzn.to/2b2YE2a • Download Drivers • http://amzn.to/2axE0Hn • Update Global Settings
  19. 19. Global Settings – Logging
  20. 20. Global Settings – Drivers
  21. 21. Global Settings – Performance and Memory
  22. 22. Global Settings – Assessment Report
  23. 23. Time Considerations • Any Hard Dates? • Planning Time? • Typically 2-3 Weeks • Several Iterations
  24. 24. Database Considerations • Number of Schemas? • Number of Tables? • Engine Specific Types? • Users/Roles/Permissions?
  25. 25. Network Considerations • Access (Firewalls, Tunnels, VPNs)? • Which VPC? • Which Security Groups?
  26. 26. Requirements Considerations • Engine Selection Criteria? • Which Tables Need to Move? • Same Target For All Tables?
  27. 27. Temporary Work Schema • Added to target DB instance • After migration delete temporary work schema • Schemas • Microsoft SQL Server: AWS_SQLSERVER_EXT • MySQL: AWS_MYSQL_EXT • Oracle: AWS_ORACLE_EXT • PostgreSQL: AWS_POSTGRESQL_EXT
  28. 28. DMS Overview
  29. 29. • Start your first migration in 10 minutes or less • Keep your apps running during the migration • Replicate within, to, or from Amazon EC2 or RDS • Move data to the same or a different database engine AWS Database Migration Service (AWS DMS)
  30. 30. Customer premises Application users AWS Internet VPN • Start a replication instance • Connect to source and target databases • Select tables, schemas, or databases  Let AWS DMS create tables, load data, and keep them in sync  Switch applications over to the target at your convenience Keep your apps running during the migration AWS DMS
  31. 31. Multi-AZ option for high availability Customer premises or AWS AWS Internet VPN AWS DMS AWS DMS
  32. 32. AWS Database Migration service pricing T2 for developing and periodic data migration tasks C4 for large databases and minimizing time T2 pricing starts at $0.018 per hour for T2.micro C4 pricing starts at $0.154 per hour for C4.large 50 GB GP2 storage included with T2 instances 100 GB GP2 storage included with C4 instances Data transfer inbound and within AZ is free Data transfer across AZs starts at $0.01 per GB
  33. 33. Migration Scenarios and Options
  34. 34. On-Premises Migration Scenarios • An on-premises database to a database on Amazon RDS DB instance • An on-premises database to a database on an Amazon EC2 instance • Migration from an on-premises database to another on-premises database is not supported.
  35. 35. RDS Migration Scenarios • A database on an Amazon RDS DB instance to an on-premises database • A database on an Amazon RDS DB instance to a database on an Amazon RDS DB instance • A database on an Amazon RDS DB instance to a database on an Amazon EC2 instance
  36. 36. EC2 Migration Scenarios • A database on an Amazon EC2 instance to an on-premises database • A database on an Amazon EC2 instance to a database on an Amazon EC2 instance • A database on an Amazon EC2 instance to a database on an Amazon RDS DB instance
  37. 37. DMS Components • Replication Instances • Endpoints • Tasks
  38. 38. Replication Instances • Performs the work of the migration • Tasks run on instances • Can support multiple tasks • AWS DMS currently supports T2 and C4 instance classes for replication instances
  39. 39. Public and Private Replication Instances • A replication instance should have a public IP address if the source or target database is located in a network that is not connected to the replication instance's VPC by using a virtual private network (VPN), AWS Direct Connect, or VPC peering. • A replication instance should have a private IP address when both the source and target databases are located in the same network that is connected to the replication instance's VPC by using a VPN, AWS Direct Connect, or VPC peer.
  40. 40. Sources for AWS Database Migration Service Customers use the following databases as a source for data migration using AWS DMS: On-premises and Amazon EC2 instance databases: • Oracle Database 10g – 12c • Microsoft SQL Server 2005 – 2014 • MySQL 5.5 – 5.7 • MariaDB (MySQL-compatible data source) • PostgreSQL 9.4 – 9.5 • SAP ASE 15.7+ RDS instance databases: • Oracle Database 11g – 12c • Microsoft SQL Server 2008R2 - 2014. CDC operations are not supported yet. • MySQL versions 5.5 – 5.7 • MariaDB (MySQL-compatible data source) • PostgreSQL 9.4 – 9.5 • Amazon Aurora (MySQL-compatible data source)
  41. 41. Targets for AWS Database Migration Service Customers can use the following databases as a target for data replication using AWS DMS: On-premises and EC2 instance databases: • Oracle Database 10g – 12c • Microsoft SQL Server 2005 – 2014 • MySQL 5.5 – 5.7 • MariaDB (MySQL-compatible data target) • PostgreSQL 9.3 – 9.5 • SAP ASE 15.7+ RDS instance databases: • Oracle Database 11g – 12c • Microsoft SQL Server 2008 R2 - 2014 • MySQL 5.5 – 5.7 • MariaDB (MySQL-compatible data target) • PostgreSQL 9.3 – 9.5 • Amazon Aurora (MySQL-compatible data target) • Amazon Redshift
  42. 42. Tasks Overview • Run on a replication instance • Contain two and only two endpoints (source and target) • Different migration methods available • Specify selection and/or transformation rules • Can run multiple tasks
  43. 43. Migration Methods • Migrate existing data • Migrate existing data and replicate ongoing changes • Replicate data changes only
  44. 44. DMS – Change Data Capture (CDC) “No Touch” design • Reads recovery log of source database • Using the engine’s native change data capture API • No agent required on the source Some requirements • Oracle: Supplemental logging required • MySQL: Full image row level bin logging required • SQL Server: Recovery model bulk logged or full • Postgres: wal_level = logical; max_replication_slots >= 1; max_wal_Senders >=1; wal_sender_timeout = 0 Changes captured and applied as units of single committed transactions Activated when load starts No changes are applied until load completes, then applied as soon as possible in near real-time
  45. 45. Replication Instance Source Target Start Full Load
  46. 46. While Loading Data Also Capture Changes Source TargetReplication Instance Update
  47. 47. Load Complete - Apply Captured Changes Source TargetReplication Instance Update
  48. 48. Changes Reach Steady State Source TargetReplication Instance Update
  49. 49. Cutover - Shut Down Apps & Apply Remaining Changes Source TargetReplication Instance Update
  50. 50. Flip! Source TargetReplication Instance Update
  51. 51. Changes are Transactional and Come From the Logs Migration Server Source Target Update t 1 t 2 t 1 t 2
  52. 52. Replication Instance Source Target Multiple Targets Target Target
  53. 53. Migration Server Source Target Multiple Sources Source Source
  54. 54. Migration Server Source Target Multiple Sources and Targets Source Source Target
  55. 55. You Don’t Have to Take Everything Source  Target Replication Instance
  56. 56. Homogenous or Heterogeneous Replication Instance SQL Server MySQL Replication Instance Oracle Oracle Replication Instance Oracle Aurora
  57. 57. Lab
  58. 58. Lab Document Download: http://chilp.it/499290a Lab to be run in us-west-2 (Oregon) region
  59. 59. Lab
  60. 60. Thank you!
  61. 61. Remember to complete your evaluations!

×