Technical deep dive in to 10 AWS Cloud best practices with in-depth look at the tips and tricks of architecting on the AWS platform.

1. AWS Government, Education, &
Nonprofits Symposium
Canberra, Australia | May 20, 2014
Black Belt Tips on AWS
Dean Samuels
Solution Architect
Amazon Web Services

2. AWS Pace of Innovation

3. Ninja Tips
• Compute and Networking
• Storage & Content Delivery
• Deployment & Management
• Security
• Big Data & App Services……maybe!

4. Meet Simon
• Black Belt Tip
– Route53 & Elastic Load Balancing
• Cross-Zone Load Balancing….finally!
• Application Failover via DNS….really?
Simon is all about Compute &
Networking
• Design for failure is his motto
• Where possible Simon prefers
scale out vs. scale up
approaches
• Compartmentalization -
minimize blast radius
• Integrates with third-party
providers in the cloud too!

5. • Route 53 DNS Failover
ELB & Route53
• Cross-Zone Load Balancing

6. Meet Simon
• Black Belt Tip
– Route53 & Elastic Load Balancing
• Cross-Zone Load Balancing….finally!
• Application Failover via DNS….really?
• Ninja Tip
– VPC Peering
• Trust thy neighbour!
– VPC peering within an account
– VPC peering between accounts
Simon is all about Compute &
Networking
• Design for failure is his motto
• Where possible Simon prefers
scale out vs. scale up
approaches
• Compartmentalization -
minimize blast radius
• Integrates with third-party
providers in the cloud too!

7. VPC Peering
Simon’s Shared Services VPC
10.1.0.0/16
Simon’s Workspaces VPC
192.168.0.0/20
Simon’s Enterprise Apps VPC
172.16.0.0/16
Simon’s Web Apps VPC
10.11.0.0/16
Simon’s Proxy VPC
10.20.10.0/24
Internet
Dean’s WAF VPC
10.100.0.0/16
Simon’s Test/Dev VPC
10.10.0.0/16

8. This is Jeff
• Black Belt Tip
– Storage Gateway File Shares
• S3 Backed NAS
– Large volume file shares, no upfront cost
– On-premise or in the AWS Cloud
Jeff is ‘Mr Storage’…optimising use of AWS
storage tiers is his thing
• Instance storage for temporary data
• EBS storage for persistent storage
• S3 for backups, serving web & media
and even as a BitTorrent seeder
• Glacier for archiving data
• Hates paying for storage he doesn’t use
• But loves the S3 price reductions!

9. Next Generation Storage
File Servers
Corporate Data center
AWS Cloud
Internet
or
WAN
SSL
On-Premise AWS
Storage Gateway
Cache & Upload Buffer Storage
Direct Attached or Storage Area
Network Disks
iSCSI
Cached-Volumes
Multi-Terabyte
AWS Storage
Gateway Service
“Block” Volumes
@ S3 Prices
“Block” Volumes
@ S3 Prices
Encrypted &
Compressed
Volume
Snapshots
EC2
File Servers
iSCSI
Cached-Volumes
Multi-Terabyte
CIFS/
NFS
Clients
CIFS/
NFS
EC2 Clients
Third-Party options too:
• Riverbed Whitewater
• SoftNAS
• Maginatics
EC2 AWS Cached
Storage Gateway
Cache &
Upload Buffer
EBS PIOPS

10. This is Jeff • Black Belt Tip
– Storage Gateway File Shares
• S3 Backed NAS
– Large volume file shares no upfront cost
– On-premise or in the AWS Cloud
• Ninja Tip
– Instance Storage
• Normally ephemeral storage
– Using replication = durable storage
– EBS PIOPs and Enhanced Networking
Jeff is ‘Mr Storage’…optimising use of AWS
storage tiers is his thing
• Instance storage for temporary data
• EBS storage for persistent storage
• S3 for backups, serving web & media
and even as a BitTorrent seeder
• Glacier for archiving data
• Hates paying for storage he doesn’t use
• But loves the S3 price reductions!

11. High Speed* & High Density*
Instance storage for durable data
Instance Storage with sync to EBS Instance Storage to Instance Storage to EBS
*I2 and C3 Instances:
- Multiple 10s & 100’s GB SSD-based instance storage
- Enhanced Networking = Higher PPS and lower jitter & latency
EBS Optimized
MDADM
RAID 0
array
DRBD
protocol A
(asynchronous)
Up to 50,000 IOPs = 800MBs
General Network
Traffic
EBS PIOPS
SSD Backed
Data Store
EC2 Instance
MDADM
RAID 0 or 1+0
array
HDD
or
SSD (100,000s
IOPS) Enhanced
Networking*

12. Say Hi to Rodos
• Black Belt Tip
– Programmable resources
• AWS Support
– It’s an API too!
• Automated/Self Healing infrastructures
– Servers != Our Pets
Rodos doesn’t like to make mistakes…so
he automates everywhere.
• Uses CloudFormation wherever
possible….but not everything is
supported by CloudFormation?
• AutoScaling! AutoScaling! AutoScaling!
• Interacts with AWS Support to have
things optimised and fixed…but Rodos
doesn’t scale
• Happy to write scripts to interact with
AWS API

13. Programmatic Access to Resources
• Monitoring Your Service Limits
– Via Service API
• aws iam get-account-summary
• aws autoscaling describe-account-limits
• aws ec2 describe-account-attributes
• aws ses get-send-quota
– Via Trusted Advisor
• aws support describe-trusted-advisor-check-result --check-id eW7HH0l7J9
--language en
• Accessing Support via API
– Integrate with your own management/monitoring systems
– Automatically log tickets via CloudFormation

14. Resource Management with Tags
#!/usr/bin/ruby
require 'aws-sdk'
AWS.regions.sort_by(&:name).each do |region|
puts region.name
region.ec2.instances.each do |instance|
if instance.status == :stopped and instance.tags.to_h.has_key?('DevProjectA')
instance.start
puts "t#{instance.id} starting"
end
end
end
for region in $(aws ec2 describe-regions --query 'Regions[*].RegionName' --output text)
do
echo ${region}
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --filters
"Name=instance-state-name,Values=running" "Name=tag-key, Values=Uptime, Name=tag-value,
Values=BusinessHoursOnly" --output text --region ${region} | xargs aws ec2 stop-instances --
instance-ids --region ${region} 2> /dev/null
done
Ruby SDK
AWS CLI

15. Say hi to Rodos • Black Belt Tip
– Programmable resources
• AWS Support
– It’s an API too!
• Automated/Self Healing infrastructures
– Servers != Our Pets
• Ninja Tip
– CloudFormation
• Taking it to the next level!
– Custom Resources
Rodos doesn’t like to make mistakes... so
he automates everywhere.
• Uses CloudFormation wherever
possible….but not everything is
supported by CloudFormation?
• AutoScaling! AutoScaling! AutoScaling!
• Interacts with AWS Support to have
things optimised and fixed but Rodos
doesn’t scale
• Happy to write scripts to interact with
AWS API

16. CloudFormation Custom Resources
Region
SQS Queue
AWS
CloudFormation
Custom Resource
Topic
Auto scaling Group
Custom Resource
Implementation
• Add New Resources
– Including AWS resources not currently
supported by CFN
• Interact with the CloudFormation
Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing
resources
• Data management via
CloudFormation
• It’s really simple if you use
aws-cfn-resource-bridge
– Install or fork from
https://github.com/aws/aws-cfn-resource-bridge
Create
Parameter1:Value1
Parameter2:Value2
….
Parametern:Valuen
Data
Export
Data
Import
DynamoDB S3Datapipeline
1
2 3
4
5
6
Output
Parameter1:Value1
Parameter2:Value2
….
Parametern:Valuen

17. CloudFormation Custom Resources
Region
SQS Queue
AWS
CloudFormation
Custom Resource
Topic
Auto scaling Group
Custom Resource
Implementation
• Add New Resources
– Including AWS resources not currently
supported by CFN
• Interact with the CloudFormation
Workflow
• Inject dynamic data into a stack
• Extend the capabilities of existing
resources
• Data management via
CloudFormation
• It’s really simple if you use
aws-cfn-resource-bridge
– Install or fork from
https://github.com/aws/aws-cfn-resource-bridge
Delete
Parameter1:Value1
Parameter2:Value2
….
Parametern:Valuen
Data
Import
Data
Export
DynamoDB S3Datapipeline
1 2
3
4
5
Output
Parameter1:Value1
Parameter2:Value2
….
Parametern:Valuen
6

18. What’s up Squigg?
• Black Belt Tip
– IAM Roles with EC2
• Don’t leave home without it!
Squigg is always concerned about user
password and credential leaks
• Admin users with no MFA
• Users leaving credentials in software
• Users not rotating their credentials
• Users not using strong password
policies
• Finds it hard to keep track of
individual IAM identifies for users

19. IAM Roles for EC2 Instances
AWS Cloud
Amazon
S3
Amazon
DynamoDB
Your
Application
AWS IAM
Your
Application
Your
Application
Your
Application
Auto
Scaling
Your
Application
Auto
Scaling
Role: RW access to
objects, items and
instances
• Eliminates use of long-term credentials
• Automatic credential rotation
• Less coding – AWS SDK does all the work
• Easier and more Secure!
Amazon
EC2

20. What’s up Squigg?
• Black Belt Tip
– IAM Roles with EC2
• Don’t leave home without it!
• Ninja Tip
– Limit number of IAM Users
• Use IAM Roles instead
– Cross-Account IAM Access
– Identity Federation
Squigg is always concerned about
password and user credential leaks
• Admin users with no MFA
• Users leaving credentials in software
• Users not rotating their credentials
• Users not using strong password
policies
• Finds it hard to keep track of
individual IAM identifies for users

21. dsamuel@amazon.com
Acct ID: 111122223333
ec2-role
{
"Statement":
[
{
"Action":
[
"ec2:StartInstances",
"ec2:StopInstances"
],
"Effect":
"Allow",
"Resource":
"*"
}
]
}
squigg@amazon.com
Acct ID: 123456789012
Authenticate with
squigg access keys
Optionally also with MFA
Get temporary
security credentials
for ec2-role
Call AWS APIs
using temporary
security credentials
of ec2-role
{
"Statement":
[
{
"Effect":
"Allow",
"Action":
"sts:AssumeRole",
"Resource":
"arn:aws:iam::111122223333:role/ec2-­‐role"
}
]
}
{
"Statement":
[
{
"Effect":"Allow",
"Principal":{"AWS":"123456789012"},
"Action":"sts:AssumeRole"
}
]
}
Cross-account API access
ec2-role trusts IAM users from the AWS account
squigg@amazon.com (123456789012)
Permissions assigned to squigg granting him permission
to assume ec2-role in dsamuel@amazon.com account
IAM user: squigg
Permissions assigned
to ec2-role
STS
Amazon EC2

22. Hey there Russell
But you can visit Russell and other AWS Solution
Architects at the SA Corner at the AWS Booth
Russell & Big Data are like Peas & Carrots…..
But unfortunately we are out of time!

23. How to Keep Up to Date
• AWS Podcast
– http://aws.amazon.com/podcasts/aws-podcast/
• Amazon Web Services Blog
– http://aws.amazon.com/blogs/aws/
• What’s New?
– http://aws.amazon.com/new/
• Social Media
– @awscloud & /amazonwebservices
• Your Friendly Solution Architect Team
– Speak to the team today at the SA Corner

24. THANK YOU
Please give us your feedback by filling out the Feedback Forms
AWS Government, Education, &
Nonprofits Symposium
Canberra, Australia | May 20, 2014