How can organisations leverage the cloud for running relational database workloads? In this session we cover architectures and best practices that enable high performance, high availability, and graceful migration for relational database workloads while reducing costs. Hear how customers are using Amazon RDS to manage their databases at scale. This session teaches you to take advantage of features unique to AWS and Amazon RDS to free your databases from the confines of the conventional data centre. Speaker: Adrian Hunter, Solutions Architect, Amazon Web Services Featured Customer - Suncorp

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Adrian Hunter, Solution Architect
Amazon Web Services
Sam Johnston, BI Cloud Architect
Suncorp
Relational Databases Utilising
Amazon RDS
Technical 201

2. Overview of Amazon RDS
PerformanceReliabilitySecurity
Cost
Optimisation
Migration
Suncorp’s Story
Live Demo

3. Overview of Amazon RDS

4. Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
Query Construction
Query Optimisation
Schema Design
Why Managed Databases?
Traditional DC

5. Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
Query Construction
Query Optimisation
Schema Design
Why Managed Databases?
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
Query Construction
Query Optimisation
Schema Design
Traditional DC DB on EC2

6. Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
Query Construction
Query Optimisation
Schema Design
Why Managed Databases?
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
Query Construction
Query Optimisation
Schema Design
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
Query Construction
Query Optimisation
Schema Design
Traditional DC DB on EC2 Amazon RDS

7. Relational Databases
Fully managed
Simple and fast to scale
Fast, predictable performance
Low cost, pay for what you use
Amazon
RDS
Amazon
Aurora

8. Security

9. Network Layer
Amazon VPC: Control subnets, AZ
specificity (DB subnet groups), route
tables and NACLs
Security Groups: Restrict instance
traffic
Public Access: Avoid it or limit it
1
2
3

10. Instance Access
Control: Use AWS Identity and Access
Management (IAM) to control instance
lifecycle permissions, grant least
privileges
Audit: Use AWS CloudTrail to log AWS
API invocations
1
2

11. Data Access
Grant least privileges to applications and
end users
Amazon RDS: Database level
authentication
1
2

12. Encryption
Protect data at rest: Client side
application, TDE, Encrypted DB
instances using AWS KMS
Secure data in transit: Encrypted
connections via SSL
1
2

13. Reliability

14. Backup and Recovery
Automated Backups
• Nightly system snapshots + transaction backup
• Enables point-­in-­time restore to any point in
retention period, up to the last 5 minutes
• Max retention period = 35 days
DB Snapshots
• User-­driven snapshots of database
• Kept until explicitly deleted
1
2

15. Admin Windows
Configurable administrative windows of time
Backup window
at least 30 minutes, once a day
Maintenance window
at least 30 minutes, once a week
1
2

16. SecondaryPrimary
AZ1 AZ2
Application
Availability – Multi-­AZ

17. SecondarySecondary
Physical
Synchronous
Replication
Primary
AZ1 AZ2
Application
Availability – Multi-­AZ

18. SecondarySecondary
Physical
Synchronous
Replication
Primary
AZ1 AZ2
Application Application
Availability – Multi-­AZ

19. SecondarySecondaryPrimary
AZ1 AZ2
Application Application
Availability – Multi-­AZ

20. SecondarySecondary PrimaryPrimary
AZ1 AZ2
Application Application
Availability – Multi-­AZ

21. SecondarySecondary PrimaryPrimary
AZ1 AZ2
DNS
Application Application
Primary Update
Availability – Multi-­AZ

22. SecondarySecondary PrimaryPrimary
AZ1 AZ2
DNS
cname update
Application Application
Primary Update
Availability – Multi-­AZ

23. SecondarySecondary PrimaryPrimary
AZ1 AZ2
DNS
cname update
Application Application
Primary Update
Availability – Multi-­AZ

24. SecondarySecondary Primary
Physical
Synchronous
Replication
AZ1 AZ2
DNS
cname update
Application Application
Primary Update
Availability – Multi-­AZ

25. Minimise Unavailability During Scaling
4:30:58 PM … 4:56:17 PM : 25 minutes 4:43:31 PM … 4:45:16 PM : 105 seconds
m3.large (2 vCPU, 7.5GiB RAM) >> m3.xlarge (4 vCPU, 15GiB RAM)

26. Performance

27. Read Replicas for Scale
Secondary
Application
Secondary
Sync
Replication
Multi-­AZ
Primary

28. Read Replicas for Scale
Secondary
Application
Read
Replica
Read
Replica
Secondary
Sync
Replication
Multi-­AZ
Primary
Async Replication

29. Read Replicas for Scale
Secondary
Read
Replica
Read
Replica
Secondary
Sync
Replication
Multi-­AZ
Primary
Eventually Consistent
Reads
Writes &
Consistent
Reads
Async Replication

30. Read Replicas Improve Availability
Secondary
Read
Replica
Read
Replica
Secondary
Sync
Replication
Multi-­AZ
Primary
Eventually Consistent
Reads
Writes &
Consistent
Reads
Async Replication

31. Read Replicas Improve Availability
Secondary
Read
Replica
Read
Replica
Secondary Primary
Sync
Replication
Multi-­AZ
Eventually Consistent
Reads
Writes &
Consistent
Reads
Async Replication

32. SecondarySecondary Primary
Upgrade
Eventually Consistent
Reads
Writes &
Consistent
Reads
Async Replication
Read Replicas Improve Availability
Read
Replica
Read
Replica

33. Read Replicas Improve Availability
Secondary
Read
Replica
Read
Replica
Secondary Primary
Sync
Replication
Multi-­AZ
Eventually Consistent
Reads
Writes &
Consistent
Reads
Async Replication
Modify

34. Sizing to meet your Workload Demands
What is your Workload’s Demand Pattern?
Constant, steady-­state
Predictable fluctuations, steady-­state
Growing, but predictable
Fluctuating, spiky
What is your Workload’s Query Pattern?
For example: simple key lookups
For example: complex analytics

35. Workload Pattern Impact on Performance
Compute
Capabilities
vCPUs
Memory
Capabilities
GB of RAM
Network
Performance
MB/s (Throughput)
Storage
Performance
I/O Throughput
RDS DB Instance Class
RDS Storage Type

36. Compute Resources – Scaling and Performance
Range of DB
instance types
From: 1 vCPU and
1 GB of RAM
To: 40 vCPUs and
244 GB of RAM
Grouped in instance
families:
Standard
Memory Optimized
Burstable Capacity
Scale up or down by
changing the
instance class
R3

37. RDS Storage Types
Magnetic General Purpose
(SSD)
Provisioned IOPS
(SSD)

38. Monitoring and Alerting
Monitor performance using Amazon CloudWatch
Default Metrics and Enhanced Monitoring
Alarms &
notifications
1
2

39. Cost Optimisation

40. Environments
SyM
A A
RRRR
A A
Production

41. Environments
SyM
A A
RRRR
A A
Production
SyM
A A
RRRR
A A
Pre-­Production

42. Environments
SyM
A A
RRRR
A A SyM
A A
RRRR
A A
Production
Pre-­Production

43. Environments
SyM
A A
RRRR
A A SyM
A A
RRRR
A A
Production
Pre-­Production UAT
yM
A A
RR

44. Environments
SyM
A A
RRRR
A A SyM
A A
RRRR
A A
yM
A
yM
A
yM
A
yM
A
yM
A
yM
A
Production
Pre-­Production
Dev / Test
UAT
yM
A A
RR

45. Environments
SyM
A A
RRRR
A A SyM
A A
RRRR
A A
yM
A
yM
A
yM
A
yM
A
yM
A
yM
A
Production
Pre-­Production
Dev / Test
UAT
yM
A A
RR

46. Burst Mode – T2 and GP2
T2 – Instance Family with Burst Capability
• Base performance + burst
• Earn credits per hour when below base performance
• Can store up to 24 hours’ worth of credits
• Amazon CloudWatch metrics to see credits and usage
GP2 – SSD based Amazon EBS Storage
• 3 IOPS per GB base performance
• Earn credits when usage below base
• Burst to 3000+ IOPS

47. Burst Mode – T2 and GP2
6000 tps
3800 tps
2200 tps
GP2 exhaust credits
T2 exhaust CPU credits
CPU
Utilisation
ReadIOPS

48. Reserved Instances
• Significant discount compared to On Demand rates
• 1 Year or 3 Year options
• Payment options
• All Upfront
• Partial Upfront
• No Upfront
• No changes in the way you operate

49. Lean Reporting as a Service
Sam Johnston
BI Cloud Architect

50. Suncorp Group: A top 20 ASX-­listed company, largest insurer (by gross written premium), 5th largest bank,
relationships with 9 million+ customers, $96 billion in Assets under management, 14,500+ employees

51. Requirements
Regulatory & Contractual Compliance
Data Quality
Data-­Use Approval
Data-­Use Auditability
Self Service
Regulatory & Contractual Compliance
Data Quality
Performance Consistency
Cost Approval & Transparency
Self Service
Platform Owner
Data Security
Performance Consistency
Cost Transparency
Lowest Platform Support Overhead
Ease of Maintenance
Report Developer
Fast Dev/Test Environment Delivery
Easier Access To Data Needed
Performance Consistency
Improved Productivity (Performant Env)
Self Service
Data Owner Report Owner

52. Requirements
Regulatory & Contractual Compliance
Data Quality
Data-­Use Approval
Data-­Use Auditability
Self Service
Regulatory & Contractual Compliance
Data Quality
Performance Consistency
Cost Approval & Transparency
Self Service
Platform Owner
Data Security
Performance Consistency
Cost Transparency
Lowest Platform Support Overhead
Ease of Maintenance
Report Developer
Fast Dev/Test Environment Delivery
Easier Access To Data Needed
Performance Consistency
Improved Productivity (Performant Env)
Self Service
Data Owner Report Owner

53. Requirements
Regulatory & Contractual Compliance
Data Quality
Data-­Use Approval
Data-­Use Auditability
Self Service
Regulatory & Contractual Compliance
Data Quality
Performance Consistency
Cost Approval & Transparency
Self Service
Platform Owner
Data Security
Performance Consistency
Cost Transparency
Lowest Platform Support Overhead
Ease of Maintenance
Report Developer
Fast Dev/Test Environment Delivery
Easier Access To Data Needed
Performance Consistency
Improved Productivity (Performant Env)
Self Service
Data Owner Report Owner

54. Requirements
Regulatory & Contractual Compliance
Data Quality
Data-­Use Approval
Data-­Use Auditability
Self Service
Regulatory & Contractual Compliance
Data Quality
Performance Consistency
Cost Approval & Transparency
Self Service
Platform Owner
Data Security
Performance Consistency
Cost Transparency
Lowest Platform Support Overhead
Ease of Maintenance
Report Developer
Fast Dev/Test Environment Delivery
Easier Access To Data Needed
Performance Consistency
Improved Productivity (Performant Env)
Self Service
Data Owner Report Owner

55. Reporting – Views of Data
Old Way
• Inter-­dependent, multi-­stage ETL
• Cascading Failures / Slowness
• Contention
• Complex = inertia
Domino Effect. Photograph by Martin Fisch. Unmodified.
This work is licensed under a Creative Commons Attribution 4.0 International License.

56. Reporting – Views of Data
New Way
• One platform per Data View
• Isolates failure
• Size compute to job
• Consistent performance
• Lean – reduce waste, focus on value
Old Way
• Inter-­dependent, multi-­stage ETL
• Cascading Failures / Slowness
• Contention
• Complex = inertia

57. Lean Reporting as a Service
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Owner(s)
Multiple Source Systems
Centralised Data Store
Data Governance

58. Batch Tier
RDS
SQL LAKEView
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Flow
Data
Owner(s)
Report
Owner(s)
Batch Workload
Subset of Data
SQL Code
Lean Reporting as a Service

59. Batch TierServing Tier
Report
Visualisation
RDS
View
Rep
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Flow
Data
Flow
Data
Flow
Data
Owner(s)
Report
Owner(s)
Report
Users
Data View Persists
Batch Tier Disposed
Approved Access Only
Lean Reporting as a Service

60. Batch Tier
RDS
SQL LAKEView
Serving Tier
Report
Visualisation
RDS
View
Rep
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Flow
Data
Flow
Data
Flow
Time To Live < 24Hrs TTL < 7 Hrs
Data
Owner(s)
Report
Owner(s)
Report
Users
Variable Service Hours
Stateless Serving Tier
Lean Reporting as a Service

61. Batch Tier
RDS
SQL LAKEView
Serving Tier
RDS
View
Rep
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Flow
Data
Flow
Data
Flow
Report
Users
Batch Tier
RDS
SQL LAKEView
Serving Tier
RDS
View
Rep
Data
Flow
Data
Flow
Data
Flow
Report
Users
Batch Tier
RDS
SQL LAKEView
Serving Tier
Report
Visualisation
RDS
View
Rep
Data
Flow
Data
Flow
Data
Flow
Scale
Out
Data
Owner(s)
Report
Owner(s)
Report
Users
Lean Reporting as a Service
Report
Visualisation
Report
Visualisation

62. Suncorp Data Service
Batch Tier
RDS
SQL LAKEView
Serving Tier
Report
Visualisation
RDS
View
Rep
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Flow
Data
Flow
Data
Flow
Data
Owner(s)
Report
Owner(s)
Report
Users
Batch Tier
RDS
SQL LAKEView
Serving Tier
Report
Visualisation
RDS
View
Rep
Data
Flow
Data
Flow
Data
Flow
Report
Users
Batch Tier
RDS
SQL LAKEView
Serving Tier
Report
Visualisation
RDS
View
Rep
Data
Flow
Data
Flow
Data
Flow
Scale
Out
Report
Users
Lean Reporting as a Service

63. Batch Tier
RDS
SQL LAKEView
Serving Tier
Report
Visualisation
RDS
View
Rep
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Flow
Data
Flow
Data
Flow
Self Service Portal Cost Dashboard
Governance & Control – Ownership
Data
Owner(s)
Report
Owner(s)
• Notification and approval
• Visibility
• Reporting – usage, cost
Report
Users
Lean Reporting as a Service

64. MonitorSuncorp Data Service
Batch Tier
RDS
SQL LAKEView
Serving Tier
Report
Visualisation
RDS
View
Rep
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Flow
Data
Flow
Data
Flow
Move DataMeta-­Data Repo Code Repo DestroyBuild
Data
Owner(s)
Report
Owner(s)
Governance & Control – System
Building
Report
Users
Alarm
• Infrastructure + Transform as Code
• Build/Destroy
• Move approved data only
• Monitoring/Enforcement
Lean Reporting as a Service

65. Suncorp Data Service
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Flow
Data
Flow
Development
Report
Visualisation
RDS
LAKESQLViewRep
Data
Owner(s)
Report
Owner(s)
Report
Developer
Self Service
Controlled Development Access
Scalability per Need
Lean Reporting as a Service

66. Report
Tester
Suncorp Data Service
Data
Lake
System of
Record
System of
Record
System of
Record
Data
Flow
Data
Owner(s)
Report
Owner(s)
Test Batch
Tier
RDS
SQL LAKEView
Test Serving Tier
Report
Visualisation
RDS
View
Rep
Data
Flow
Data
Flow
Data
Flow
Self Service
Controlled Test access
Scalability and Operational Readiness Testing
Lean Reporting as a Service

67. Lessons Learned
Leverage Managed Services
Cost Awareness
Automate
Experiment. Learn. Iterate.

68. Outcomes
Visibility
Ownership
Way of Working
Changing Conversations

69. www.suncorpgroupcareers.com.au

70. Thank you
AWS Summit 2016, Sydney

71. Migration

72. Migration Approaches
One time Migration
Periodic Migration
1
2
3 Ongoing Replication

73. Migration Considerations
Data
Size, Types, Character sets, transfer mechanism
Dependencies
Connected Apps, DB Links, Network, Acceptable
downtime
1
2
3 Native tools
Oracle Datapump, pg_dump, mysqldump, SQL
Server Management Studio

74. Start your first migration in 10 minutes or less
Keep your apps running during the migration
Replicate within, to, or from Amazon EC2 or RDS
Move data to the same or different database engine
AWS
Database Migration
Service
Amazon
Aurora

75. Customer
Premises
Application Users
Internet
VPN
Keep your Apps Running During Data Migration

76. Customer
Premises
Application Users
Internet
VPN
AWS Database
Migration Service
Start a Replication Instance
Keep your Apps Running During Data Migration

77. Customer
Premises
Application Users
Internet
VPN
AWS Database
Migration Service
Connect to Source and Target Databases
Keep your Apps Running During Data Migration

78. Customer
Premises
Application Users
Internet
VPN
AWS Database
Migration Service
Select Tables, Schemas or Databases
Keep your Apps Running During Data Migration

79. Customer
Premises
Application Users
Internet
VPN
AWS Database
Migration Service
Let the AWS Database Migration Service create tables, load data
Keep your Apps Running During Data Migration

80. Customer
Premises
Application Users
Internet
VPN
AWS Database
Migration Service
Let the AWS Database Migration Service keep them in sync
Keep your Apps Running During Data Migration

81. Customer
Premises
Application Users
Internet
VPN
Switch applications over to the target at your convenience
AWS Database
Migration Service
Keep your Apps Running During Data Migration

82. AWS Schema Conversion Tool
Features
Oracle, SQL Server, MySQL and PostgreSQL conversion to MySQL/Aurora/MariaDB/PostgreSQL
Database Migration Assessment report for choosing the best target engine
Code browser that highlights places where manual edits are required
The AWS Schema Conversion Tool helps
automate many database schema and code
conversion tasks when migrating between
database engines.

83. SCT helps with Converting tables, views, and code
Sequences
User Defined Types
Synonyms
Packages
Stored Procedures
Functions
Triggers
Schemas
Tables
Indexes
Views
Amazon
Aurora

84. SCT can tell you How Hard the Migration will be
1. Connect SCT to
Source and Target
databases.
2. Run Assessment
Report.
3. Read Executive
Summary.
4. Follow detailed
instructions.

85. Demo
Migrating Data with the
AWS Database Migration Service
Marc Teichtahl – Solution Architect, AWS

86. What You’ve Learnt Today
• How to run fast and scalable relational database workloads
using Amazon RDS
• How to migrate your data to Amazon RDS
• Rethink what you can do with databases

87. AWS Training & Certification
Intro Videos & Labs
Free videos and labs to
help you learn to work
with 30+ AWS services
– in minutes!
Training Classes
In-­person and online
courses to build
technical skills –
taught by accredited
AWS instructors
Online Labs
Practice working with
AWS services in live
environment –
Learn how related
services work
together
AWS Certification
Validate technical
skills and expertise –
identify qualified IT
talent or show you
are AWS cloud ready
Learn more: aws.amazon.com/training

88. Your Training Next Steps:
ü Visit the AWS Training & Certification pod to discuss your
training plan & AWS Summit training offer
ü Register & attend AWS instructor led training
ü Get Certified
AWS Certified? Visit the AWS Summit Certification Lounge to pick up your swag
Learn more: aws.amazon.com/training

89. Thank You