Migrating 600 databases to AWS and making them better, faster, and cheaper COM311 AWS re:Invent 2022 This dev chat covers the entire journey of a real-life large-scale database migration from Oracle on premises to PostgreSQL on AWS. Learn about the process and unique details, issues, and tips learned from zero-downtime database migration, change data capture, and schema conversion. Find out how the migration not only made the databases cheaper by eliminating licensing costs, but also sped up development and helped the application run faster. Finally, find out about the missing features the migration team needed for PostgreSQL and how they open-sourced them. Date Thursday, December 1 11:00 AM - 11:30 AM Session level 300 - Advanced

1. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL

2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migrating 600 databases to AWS
and making them better, faster,
and cheaper
C O M 3 1 1
Matt Houghton (he/him)
Data Architect
CDL

3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL

4. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
High availability

5. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Enterprise-grade database

6. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Hybrid cloud
Primary Secondary
Read-Only

7. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Move up the stack
• Learn by failing
• Undifferentiated heavy lifting
• Focus on the business problem

8. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Migration steps
• AWS Schema Conversion Tool
(AWS SCT)
• Change data capture (CDC)
• Database objects
• Database code
• Engine-specific functions
• Code/rewrite/rearchitect

9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Change data capture

10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Demo

11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Event-driven
-

12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Migration monitoring

13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Automation
• Infrastructure as code
• Parameterize
• Switchover
• Rollback
• Reduce risk
• Consistency
• Speed of deployment

14. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Tuning
• join_collapse_limit
• from_collapse_limit
• work_mem
• pg_stat_statements
• pg_prewarm
• Long-running SQL
• Tools
• The organization

15. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
What do the DBAs do now?
• Infrastructure as code
• More automation
• Config-driven deployments
• Faster upgrades/fixes
• Scale the business

16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Developer experience
• Self-service
• Database clones
• Privacy automation
• Speed
• Preapproved queries
• Health checks
• Database-as-a-service product
• New skills

17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon RDS Ready

18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Connection pooling

19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Application logging

20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Data encryption

21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Database failure and performance

22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Open source

23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Fast refresh materialized views

24. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Fast refresh materialized views – Walkthrough

25. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Fast refresh materialized views – Walkthrough

26. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Fast refresh materialized views – Walkthrough

27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Fast refresh materialized views – Walkthrough

28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Demo

29. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Final tip

30. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CDL - EXTERNAL
Thank you!
Matt Houghton
@mattdevdba
Please complete
the session survey