AWS Database Migration Service (AWS DMS) and AWS Schema Conversion Tool (AWS SCT) can help migrate databases from many supported data sources to supported targets. In this session, we review how the combination of AWS DMS and AWS SCT can help migrate your NoSQL databases, such as MongoDB and Cassandra, to Amazon DynamoDB. We provide an overview of AWS DMS and AWS SCT, and we demonstrate migrating a sample Cassandra database into DynamoDB.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migrating Your NoSQL Database to
Amazon DynamoDB
Arun Thiagarajan
Product Manager
AWS Database Migration Service/Schema Conversion Tool
D A T 3 1 4

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
A quick overview of Amazon DynamoDB
Why you should migrate from self-managed NoSQL databases
An overview of AWS Database Migration Tool (AWS DMS) and AWS
Schema Conversion Tool (AWS SCT)
How to migrate from MongoDB and Cassandra to Amazon DynamoDB
Demo
Q&A

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon DynamoDB
Fully managed nonrelational database for any scale
High performance
• Fast, consistent
performance
• Virtually unlimited
throughput
• Virtually unlimited
storage
Enterprise ready
• Encryption at rest and
transit
• Fine-grained access
control
• PCI, HIPAA, FIPS 140-2
eligible
Fully managed
• Maintenance-free
• Serverless
• Auto scaling
• Backup and
restore
• Global tables

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fully managed
Maintenance-free
Security
• Operating-system patching
• Database patching
• Access control
• Audit
• Encryption
• Compliance
Durability
• Sustain server, rack, and data
center outages
• Re-replicate data quickly upon
hardware failure
• Manage backup and restore
Availability
• High-availability configuration
• Monitoring
• Cross-region replication
Performance
• Performance tuning
• Indexing
• In-memory caching
Scalability
• Capacity planning
• Host provisioning
• Host repair and retirement

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why should you migrate from self managed NoSQL
databases?
With MongoDB/Cassandra, you deal with:
• Outages (infrastructure failures, buggy patches) → Lost revenue
• Lesser performance → Unpredictable user experience
• Extra effort to scale, upgrade, and maintain → Lost time and resources
• Required, specialized training → Less time to develop features that matter

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What are AWS DMS and AWS SCT?
AWS Schema Conversion Tool converts your commercial
database and data warehouse schemas to open-source
engines or AWS-native services, such as Amazon Aurora
and Redshift
AWS Database Migration Service easily and securely
migrates and/or replicate your databases and data
warehouses to AWS

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>100,000 databases migrated with DMS

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Schema Conversion Tool
• Migrate your data
Migrate your Cassandra database using SCT
Data Extractors
Modernize
• Modernize your database schema
Convert your Cassandra database to Amazon
DynamoDB
Amazon DynamoDB

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS SCT product highlights
Assess Plan Convert schema
and code
Optimize Migrate

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Database Migration Service
Migrate
• Migrate business-critical applications
• Migrate data warehouse to Redshift
• Upgrade to a minor version
• Consolidate shards into Aurora
• Archive old data
• Migrate from NoSQL to SQL, SQL to
NoSQL, or NoSQL to NoSQL
Amazon RDS
Amazon
Redshift
Amazon
Aurora
Amazon
DynamoDB
Amazon S3

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Database Migration Service
Replicate
• Create cross-region Read Replicas
• Run your analytics in the cloud
• Populate your data lake

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DMS—Endpoint support expansion
Amazon
Redshift
Amazon
DynamoDB
Amazon S3
Amazon S3
Amazon
Aurora
Amazon
Aurora
Oracle SQL Server Netezza
Greenplum Vertica Teradata
AWS Snowball
Edge
MongoDB Cassandra
Amazon
Elasticsearch
Amazon Kinesis

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DMS product highlights
Verify Validate Ship with
Snowball
Secure

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customer premises AWS
Internet
VPN
Migrating DB using AWS Snowball
AWS DMS
Local replication
agent
(through AWS SCT)
AWS Snowball

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Core Components
MongoDB Amazon DynamoDB
Collection Table
Document Item
Field Attribute
Secondary Index Secondary Index

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration Approaches
Document mode: In this mode, AWS DMS migrates all the JSON data into
a single column named “_doc” in the target DynamoDB table.
Table mode: In this mode, AWS DMS scans a specified number of
documents in the MongoDB database and creates a sample schema with
all the keys and their types. During migration, you can use the object
mapping feature in AWS DMS to transform the original data from
MongoDB to the desired structure in DynamoDB.

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Object Mapping Example

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
MongoDB to Amazon DynamoDB Migration
MongoDB
Shard
MongoDB
Shard
MongoDB
Shard
Amazon DynamoDB
Reference: https://aws.amazon.com/blogs/database/performing-a-live-
migration-from-a-mongodb-cluster-to-amazon-dynamodb/

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cassandra to Amazon DynamoDB Migration
Migrate data from Apache Cassandra and migrate to Amazon DynamoDB
• Extracts data through local migration agents
• Data is optimized for DynamoDB and saved in local files
• Files are loaded to an Amazon S3 bucket (through network or Amazon
Snowball) and then to Amazon DynamoDB
AWS SCT S3 Bucket Amazon DynamoDB
AWS DMS

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cassandra to DynamoDB migration—Step 1
Clone your Cassandra data center:

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cassandra to DynamoDB migration—Step 2
AWS SCTApache Cassandra
Convert your schema from Cassandra to DynamoDB

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cassandra to DynamoDB migration—Step 3
Apache Cassandra AWS SCT
Migrate your data from Cassandra to DynamoDB

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cassandra to DynamoDB migration
Step 1
Step 2
Step 3

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Learn more about DynamoDB in the below sessions
DAT201 – A Deep Dive into What’s New for Amazon DynamoDB
DAT303 – Protecting Your Greatest Asset (Your Data): Security Best Practices on DynamoDB
DAT311– Building Serverless Applications with Amazon DynamoDB & AWS Lambda – Workshop
DAT314 – Migrating Your NoSQL Database to Amazon DynamoDB
DAT320 – Becoming a Nimble Giant: How Amazon DynamoDB Serves Nike at Scale
DAT321 – Amazon DynamoDB Under the Hood: How We Built the Hyper-Scale Database of the Future
DAT325 – How Oath (a Verizon Company) Built a Multi-Region GDPR Application with Amazon DynamoDB
DAT332 – Why GE Aviation Migrated from Cassandra to Amazon DynamoDB
DAT333 – Real-World Use Cases for Amazon DynamoDB
DAT341 – Migrating Financial and Accounting Systems from Oracle to Amazon DynamoDB
DAT342 – Implementing Microservice Architectures with Amazon DynamoDB and AWS Lambda
DAT345 – How GumGum Migrated from Cassandra to Amazon DynamoDB
DAT347 – How Amazon Migrated Items and Offers for Retail, Marketplace, and Digital to DynamoDB
DAT349 – Deep Dive on Amazon DynamoDB Global Tables
DAT352 – Migrate Your Nonrelational Database to AWS
DAT357 – Build Internet-Scale Apps with Amazon DynamoDB
DAT365 – Separating Fact from Fiction: Amazon DynamoDB Scaling and Adaptive Capacity
DAT401 – Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB
DAT404 – Advanced Design Patterns for Amazon DynamoDB – Workshop

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
But wait, there’s more about DynamoDB
• A list of all DynamoDB sessions, workshops, and chalk talks
• Migrating Apache Cassandra databases to DynamoDB
• What’s new with DynamoDB
• Purpose-built databases in AWS
• DynamoDB service level agreement
• Adaptive capacity
• Point-in-time recovery (PITR)
• Global tables

34. Follow @dynamodb to get the latest

35. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Arun Thiagarajan

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.