Journey through the Cloud - Best Practices Getting Started in the AWS Cloud

Best practices for getting
started with AWS
Ryan Shuttleworth – Technical Evangelist
@ryanAWS

Journey through the cloud

Common use cases & stepping stones into the AWS cloud
Learning from customer journeys
Best practices to bootstrap your projects

Best practices

Simple things to plan for when starting with AWS
Some technical and human considerations
Helping you put your best foot forward from the off

Agenda
8 things you should know
Where you should start
Things to do up front

1
Choose your use
case well

Choose use case that suits you

Low hanging fruit can be easiest way to ‘cut teeth’


Dev & Test
Spin environments up and down
on demand

Decouple development and test
environments from operations
constraints

Explore elasticity in a sandboxed
environment



Dev & Test Backup & DR
Spin environments up and down Take part of your data or
on demand business applications step- by-
step into non-production DR use
environments from operations Understand cloud dynamics and
constraints test during controlled failovers

environment



Dev & Test Backup & DR Greenfield Project
Spin environments up and down Take part of your data or Embody best practice of cloud
on demand business applications step- by- computing in unconstrained
step into non-production DR use greenfield projects
environments from operations Understand cloud dynamics and Self contained web projects,
constraints test during controlled failovers document archiving etc

environment



Dev & Test Backup & DR Greenfield Project Pain point
Spin environments up and down Take part of your data or Embody best practice of cloud Move specific service aspects
on demand business applications step- by- computing in unconstrained causing undue cost or
step into non-production DR use greenfield projects management burden
environments from operations Understand cloud dynamics and Self contained web projects, Workflows, search indexing,
constraints test during controlled failovers document archiving etc media streaming, document
archiving, constrained databases
environment


Plan evolution & set goals

PoC Production Automation

Understand services Implement monitoring Automate corrective measures
Examples

Test performance Change control and management Auto-scaling

Architect for scale Security management Zero downtime deployments

Build cross functional team capabilities Scalability System backup and recovery

Plan evolution & set goals

PoC Production Automation

Understand services Implement monitoring Automate corrective measures
Examples

Test performance Change control and management Auto-scaling

Architect for scale Security management Zero downtime deployments

Build cross functional team capabilities Scalability System backup and recovery

Beanstalk
APIs
Beanstalk Cloud Formation
CLI
Cloud Watch
Auto scaling
IAM

Organize your house

Accounts

Create an account structure
that makes sense
Use accounts like environments
where you need separation and
control

e.g
Dev Sandboxes
Test Environments
Business Units
Products & Services

Organize your house

Accounts Billing

Create an account structure Control access to billing
that makes sense information
Use accounts like environments Use IAM users to keep billing
where you need separation and information in the master account
control
Consolidate billing into a
e.g single account
Dev Sandboxes Let one account pick up the bill for
Test Environments multiple ‘sub accounts’
Business Units
Products & Services Setup billing alerts and
automated bill reporting
Get CloudWatch notifications when
billing reaches a point and output
csv reports to S3 for analysis

Enable CSV &
Billing settings Programmatic Access

Billing
Preferences

Master Account
aws.invoices@mycompany.com

Master Account

consolidated billing information

Division B
admin@divisionB.com
IAM User2
Dev2
Admin2

Master Account

consolidated billing information Tags: (key-
value)
Division B e.g Own=Div
Proj=R
admin@divisionB.com
IAM User2
Dev2
Admin2

Tags: Tags: Tags:
Own=Div Own=Div Own=Div
Proj=P Proj=Q Proj=R

Master Account


Operating Co. A Division B Business Unit C
admin@opcoa.com admin@divisionB.com admin@busUnitC.com
User1 User2 User3
IAM

IAM

IAM
Dev1 Dev2 Dev3
Admin1 Admin2 Admin3

Tags: Tags: Tags: Tags: Tags: Tags: Tags: Tags: Tags:
Own=OpCo Own=OpCo Own=OpCo Own=Div Own=Div Own=Div Own=BusC Own=BusC Own=BusC
Proj=A Proj=B Proj=C Proj=P Proj=Q Proj=R Proj=X Proj=Y Proj=Z

Programmatic billing access
Master Account

S3 CSV


Operating Co. A Division B Business Unit C
admin@opcoa.com admin@divisionB.com admin@busUnitC.com
User1 User2 User3
IAM

IAM

IAM
Dev1 Dev2 Dev3
Admin1 Admin2 Admin3

Tags: Tags: Tags: Tags: Tags: Tags: Tags: Tags: Tags:
Own=OpCo Own=OpCo Own=OpCo Own=Div Own=Div Own=Div Own=BusC Own=BusC Own=BusC
Proj=A Proj=B Proj=C Proj=P Proj=Q Proj=R Proj=X Proj=Y Proj=Z

Organize your house

Accounts Billing Access Keys

Create an account structure Control access to billing Decide upon a key
that makes sense information management strategy
Use accounts like environments Use IAM users to keep billing Control access to EC2 instances via
where you need separation and information in the master account SSH and embedded public key:
control e.g. EC2 Key Pair per group of
Consolidate billing into a instances, EC2 Key Pair per account
e.g single account
Dev Sandboxes Consider SSH key rotation &
Let one account pick up the bill for
automation
Business Units Limit exposure to private key
Products & Services Setup billing alerts and compromise by rotating keys and
replacing authorized_keys
automated bill reporting listings on running instances
Get CloudWatch notifications when Consider bootstrap automation to
billing reaches a point and output grant developer access with
csv reports to S3 for analysis developer unique keypairs

Organize your house

Accounts Billing Access Keys Groups & Roles

Create an account structure Control access to billing Decide upon a key Use IAM Groups to manage
that makes sense information management strategy console users and API access
Use accounts like environments Use IAM users to keep billing Control access to EC2 instances via Provide developers with IAM user
where you need separation and information in the master account SSH and embedded public key: login and unique API access
control e.g. EC2 Key Pair per group of credentials
Consolidate billing into a instances, EC2 Key Pair per account Control & restrict what IAM users
e.g single account can do by placing them in groups
Dev Sandboxes Consider SSH key rotation & with policies
Let one account pick up the bill for
automation
Business Units Limit exposure to private key
Assign EC2 Instances IAM
Products & Services compromise by rotating keys and roles
Setup billing alerts and
replacing authorized_keys Let AWS manage API access
automated bill reporting listings on running instances credentials on running instances by
Get CloudWatch notifications when Consider bootstrap automation to assigning a system entitlement to an
billing reaches a point and output grant developer access with instance
csv reports to S3 for analysis developer unique keypairs e.g instance can only read S3 bucket

Identity & access management

Account

Administrators Developers Applications

Jim Brad Reporting

Bob Mark Console

Susan Tomcat

Kevin


Groups Account


Jim Brad Reporting

Bob Mark Console

Susan Tomcat

Kevin
Multi-factor authentication


Groups Account Roles


Jim Brad Reporting

Bob Mark Console

Susan Tomcat

Kevin
Multi-factor authentication AWS system entitlements

IAM policies

{
"Statement": [
{
"Effect": "Allow",
"Action": [
"elasticbeanstalk:*",
Policy driven "ec2:*",
"elasticloadbalancing:*",
Declarative definition of rights for groups "autoscaling:*",
"cloudwatch:*",
Policies control access to AWS APIs "s3:*",
"sns:*"
],
"Resource": "*"
}
]
}

Shared responsibility

Customer Data

Platform, Applications, Identity & Access Management
You

Operating System, Network & Firewall Configuration

Client-side Data Encryption & Data Server-side Encryption Network Traffic Protection
Integrity Authentication (File System and/or Data) (Encryption/Integrity/Identity)

Foundation Services
Amazon

Compute Storage Database Networking

Availability Zones
AWS Global Edge Locations
Infrastructure Regions

Leverage shared security model

Understand your customer & form security stance



Penetration test requests

Your certifications Your processes
External
audience




External
audience
IAM

Internal
Administration
audience

Architecture




External
audience
IAM AWS Certifications

Internal Regulated
Administration AWS White
audience audience Papers

Architecture AWS QSA Process



Engage with security assessors early in adoption cycle

Don’t fear assessment – AWS meets high standards (PCI, ISO27001, SOC2…)
As with any infrastructure provider, security assessments take time
Derive value from architecture reviews early in deployment cycle




Use comprehensive materials and certifications provided by AWS

http://aws.amazon.com/security/

Risk and compliance paper
AWS security processes paper
CSA consensus assessments
initiative questionnaire




Use comprehensive materials and certifications provided by AWS

Build upon features of AWS and implement a ‘security by design’ environment

Build upon AWS features

Tiered Access Security Groups VPC Direct Connect & VPN

IAM Instance firewalls Subnet control Private connections to VPC
Control users and allow AWS to Firewall control on instances via Create low level networking Secured access to resources in AWS
manage credentials in running Security Groups constraints for resource access, such over software or hardware VPN and
instances for service access as public and private subnets, dedicated network links
(allocation, rotation) CLIs and APIs internet gateways and NATs

Instantly audit your entire AWS
APIs vs Instance infrastructure from scriptable APIs –
Bastion hosts
Provide developer API credentials generate an on-demand IT inventory Only allow access for management
and control access to SSH keys enabled by programmatic nature of of production resources from a
AWS bastion host. Turn off when not
Temporary Credentials needed

Provide developer API credentials
and control access to SSH keys

4
Architect to use cloud
strengths

Architect to use cloud strengths

Review application architectures early – assess fit for cloud
? e.g. variable capacity requirements, ‘standard’ technology stacks, reference architectures*

Can cloud benefits be leveraged with minimum effort outlay?
? e.g. Application performance improvement by migration of static content to S3/CloudFront

Will cloud yield cost savings & agility improvements?
? e.g. Faster development cycles for dev/test, reduced cap-ex for application environments

Can automation lead to a more agile & secure service?
? e.g. fully scripted deployments, IAM & EC2 instance roles, rolling deployments

*http://aws.amazon.com/architecture


Disposable compute

✓✓ Design systems that can suffer
instance loss
Dispose of compute when it is not
✓ ✓ required


Disposable compute

Flexible capacity
✓ ✓ ✓ Design for systems that potentially scale
from zero instances to hundreds
Use Auto-scaling (events, schedules etc) to
✓ ✓ ✓ drive capacity availability


Disposable compute

Flexible capacity
✓ ✓ ✓ Utilize 99.999999999% durability of objects
in S3
Scale databases with RDS and use
Cost effective & reliable storage ✓ ✓ ✓ DynamoDB for high throughput NoSQL


Disposable compute

Flexible capacity
✓ ✓ ✓ Automate everything from scaling to
instance recovery from failure

Cost effective storage

Automation and control

Bootstrapping – custom AMIs
Instanc
e
AMI

Custom machine
1 Create instance for your OS choice image

2 Configure environment
Auto-scaling
Manual deployments
3 Install software Programmatic deployments

4 Create AMI from instance

5 Launch fully configured instances from AMI

Bootstrapping – metadata service
Instanc
e
Metadata service contains wealth of information about an instance AMI

http://169.254.169.254/latest/meta-data Custom or standard
machine image

ami-id local-hostname Receive custom
Metadata
data to drive
ami-launch-index local-ipv4 Service
bootstrapping
ami-manifest-path mac
block-device-mapping network
hostname placement
instance-action profile
instance-id public-hostname
Instance-type public-ipv4
kernel-id public-keys
reservation-id

Instanc
e

machine image

+ user data Receive custom
data to drive
Metadata
Service
bootstrapping
Scripts in user-data field of metadata will be executed on launch
e.g.
#!/bin/sh
yum -y install httpd
chkconfig httpd on
/etc/init.d/httpd start

Or:

<powershell>
…
</powershell>

Instanc
e

machine image

+ user data Receive custom
data to drive
Metadata
Service
bootstrapping
Scripts in user-data field of metadata will be executed on launch

Install software e.g. web server, app server, proxy

Pull data and application packages from S3

Publish metadata for instance to other systems e.g. monitoring systems

Setup security profile of instance based upon intended use e.g. pull latest config

1. Use multiple
availability zones

2. Use RDS with replicas
and slaves

4. Use Elastic Load
Balancing

5. Use Route53 to host
DNS zones


Elastic Load Balancing Route 53 RDS Auto-scaling

Use at regional level Leverage SLA Scale databases without Dynamically scale resources &
Combined with autoscaling will Improve application reliability with admin overhead control costs
balance requests and resource Route 53’s SLA on requests served Choose instance size for databases Only provision the resources that
capacity across availability zones and scale up over time are required with scale up and cool
Weighted routing down policies that match demand
Within VPC Perform A/B analysis, and staged Add high availability from
Use to loadbalance between application roll-outs by moving a management console
application tiers within an portion of traffic to new Create master-slave configurations
availability zone infrastructure and read-replicas. AWS takes care of
the failover and recreation of a new
Instance migrations Control TTLs and updates slave in event of master DB loss
Easily move instances from dev Take absolute control of DNS
environments to test environments updates for more decisive system
by moving between ELBs updates

Services not software

30% 70%

Self Managed
Software & Your Managing All of the
Infrastructure Business “Undifferentiated Heavy Lifting”

AWS
More Time to Focus on Configuring Your
Cloud-Based
Your Business Cloud Assets
Infrastructure & Services

70% 30%


Relational Database Service
Use RDS for databases Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations

DynamoDB Use DynamoDB for
Provisioned throughput NoSQL database high performance key-
Fast, predictable performance
value DB
Fully distributed, fault tolerant architecture

Processing results
Amazon SQS Reliable message
Reliable, highly scalable, queue service
Amazon SQS
queuing without
for storing messages as they travel
between instances
additional software

Processing
task/processing
trigger 1

2

Push inter-process Simple Workflow Task A

workflows into the Reliably coordinate processing steps
Task B 3
across applications
cloud with SWF (Auto-scaling)

Integrate AWS and non-AWS resources
Manage distributed state in complex
systems Task C

Document
Cloud Search Server
Don’t install search
Elastic search engine based upon
software, use
Amazon A9 search engine
CloudSearch Fully managed service with sophisticated
feature set
Search
Scales automatically
Server

Results

Elastic MapReduce
Elastic Hadoop cluster
Process large volumes
Integrates with S3 & DynamoDB of data cost effectively
Leverage Hive & Pig analytics scripts with EMR
Integrates with instance types such as
spot

6
Be elastic and cost
optimized

Be elastic and cost optimized

Elastic Load Balancing Auto-scaling policies

Scalability

Cost Optimization

Availability

Instance types and sizes

Auto-scaling policies

Manually By Schedule
Send an API call or use CLI to Scale up/down based on date
launch/terminate instances – and time
Only need to specify capacity
change (+/-)

By Policy Auto-Rebalance
Scale in response to changing Instances are automatically
conditions, based on user launched/terminated to
configured real-time ensure the application is
monitoring and alerts balanced across multiple Azs

Auto-scaling policies

Manually By Schedule
Preemptive manual scaling
Send an API call or use CLI to Regular scaling up and down
Scale up/down based on date
of capacity
launch/terminate instances – ofand time
instances
Only need marketing event add 10
e.g. before a to specify capacity e.g. scale from 0 to 2 to process SQS
more instances messages every night or double
change (+/-) capacity on a Friday night

By Policy Auto-Rebalance
Scale in response to changing Instances are automatically
Dynamic scale based upon
conditions, based on user Maintain capacity across
launched/terminated to
configuredmetrics
custom real-time availability zones
ensure the application is
e.g. SQS queue depth, Average CPU e.g. Instance availability maintained in
monitoring and alerts
load, ELB latency
balanced across multiple Azs
event of AZ becoming unavailable

Instance types

On-demand instances Reserved instances Spot instances

Unix/Linux instances start at 1- or 3-year terms Bid on unused EC2 capacity
$0.02/hour
Pay low up-front fee, receive significant hourly Spot Price based on supply/demand,
Pay as you go for compute power discount determined automatically

Low cost and flexibility Low Cost / Predictability Cost / Large Scale, dynamic workload handling

Pay only for what you use, no up-front Helps ensure compute capacity is available
commitments or long-term contracts when needed
Use Cases:
Use Cases:
Use Cases: Applications with flexible start and end times
Applications with short term, spiky, or
unpredictable workloads; Applications with steady state or predictable Applications only feasible at very low compute
usage prices
Application development or testing
Applications that require reserved capacity,
including disaster recovery

Everything is programmable

Access everything Achieve the highest levels
via CLI, API or Compute of automation
Console Security Scaling sophistication with ease
CDN Backup
DNS Database
Storage Load Balancing
Workflow Monitoring
Networking
Messaging

Elastic Beanstalk OpsWorks CloudFormation
Quickly deploy and manage apps in AWS…

CloudFormation components & terminology

CloudFormation
Template Stack

JSON formatted file Framework Configured AWS services
Parameter definition Stack creation Comprehensive service support
Resource creation Stack updates Service event aware
Configuration actions Error detection and rollback Customisable

Powerful management framework with Chef support

Stack Layers Apps Management

Managed Collection of Your application Management
environment resources assets services
Definition of environment Blueprint for a Resources to deploy Scaling, cloning, user
such as production or test collection of resources and run in layers access, self healing
(instances, EBS, EIPs
etc)

Offering
Basic 24x7x365 ✓
Developer Forum Access ✓
Documentation ✓
Business Access to support Support for
HealthChecks
Enterprise

Basic Offering
24x7x365 ✓
Documentation ✓
Business Access to support Email

Named Contacts 1
Enterprise
Fastest Response Time 12 Hours

Architecture Support Building Blocks

Best Practice ✓
Diagnostics Tools ✓

Basic Offering
24x7x365 ✓
Documentation ✓
Business Access to support Phone, Chat,
Email
Enterprise Named Contacts 5
Fastest Response Time 1 Hour

Architecture Support Use Case
Guidance

Best Practice ✓
Direct Routing ✓
3rd Party Software ✓
Trusted Advisor ✓

Basic Offering
24x7x365 ✓
Business Documentation ✓
Access to support Phone, Chat,
Email
Enterprise Named Contacts Unlimited

Fastest Response Time 15 Minutes

Architecture Support Application
Architecture

Best Practice ✓
Direct Routing ✓
3rd Party Software ✓
Trusted Advisor ✓
Direct TAM Access ✓
White Glove Case Handling ✓
Management Business Review ✓

Business and Enterprise Support has been enhanced to include best
practice audits via AWS Trusted Advisor

Security Fault Tolerance Cost Optimization

Open ports in Security Groups EBS snapshot age Unused Elastic Ips

World access (/0 CIDR) ELB Optimization Underutilized EC2 instances

IAM use Availability Zones

3rd Party Software Support Enhancements

Operating Systems 3rd Party Software

Operating Systems including: Common application stack components
including:
Amazon Linux
Apache and IIS web servers
Ubuntu Amazon SDKs
Red Hat Enterprise Linux Sendmail
SUSE Linux Postfix
FTP
Microsoft Windows 2003 & 2008 R2
Disk Management tools (LVM, RAID) VPN
Solutions (OpenVPN, RRAS)
Databases (MySQL, SQL Server)

Choose your use case well
Organize your environments
Think security
Architect to cloud strengths
Be elastic & cost optimized
Use frameworks where appropriate
Get supported

Journey through the Cloud - Best Practices Getting Started in the AWS Cloud

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (19)

En vedette

En vedette (20)

Similaire à Journey through the Cloud - Best Practices Getting Started in the AWS Cloud

Similaire à Journey through the Cloud - Best Practices Getting Started in the AWS Cloud (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Dernier

Dernier (20)

Journey through the Cloud - Best Practices Getting Started in the AWS Cloud