AWS 101 Lunch & Learn March 2013

AWS 101
Lunch & Learn
Ryan Shuttleworth
Technical Evangelist
@ryanAWS

Agenda

12:00 - 12:45 Presentation – AWS 101
Introducing the concepts behind AWS such as utility
computing and elasticity

12:45 - 13:00 Working lunch

13:00 - 14:00 Presentation and Demonstration
Live demonstration and interactive walkthrough

What we are going to cover

Keypairs Amazon Machine Images
Security groups S3
EC2 instances CloudFront
Metadata service Elastic Load balancer
Autoscaling RDS

Feel free to follow along on your laptops

Consumer Seller IT Infrastructure
Business Business Business

Tens of millions of Sell on Amazon Cloud computing
active customer websites infrastructure for
accounts hosting web-scale
Use Amazon solutions
technology for your
own retail website
Eight countries: Hundreds of
US, UK, Germany, Leverage Amazon’s thousands of
Japan, France, massive fulfillment registered
Canada, China, Italy center network customers in over
190 countries

About Amazon
How did Amazon…

Web Services

Deep experience in
building and
?
operating global web
scale systems

…get into cloud computing?

Over 10 years in the making

Enablement of sellers on Amazon

Internal need for scalable deployment environment

Early forays proved developers were hungry for more

AWS Mission

Enable businesses and
developers to use web
services* to build scalable,
sophisticated applications.

*What people now call “the cloud”

Each day AWS adds the equivalent server
capacity to power Amazon when it was a
global, $5B enterprise

Each day AWS adds the equivalent server
capacity to power Amazon when it was a
global, $5B enterprise

2003

$5.2B retail business
7,800 employees
A whole lot of servers

Objects in S3
1.3 Trillion

835k peak transactions per second

EMR Jobs

3.7 M clusters
launched since May 2010
4,000,000

3,500,000

3,000,000

2,500,000

2,000,000

1,500,000

1,000,000

500,000

0

Utility computing

On demand Pay as you go

Uniform Available

Utility computing

On demand Pay as you go
Compute
Scaling
Security
CDN Backup
DNS Database
Storage Load Balancing
Workflow Monitoring
Networking
Uniform Messaging Available

On a global footprint

Region

US-WEST (N. California) EU-WEST (Ireland)

GOV CLOUD ASIA PAC
(Tokyo)

US-EAST (Virginia)

US-WEST (Oregon)

ASIA PAC
(Singapore)
SOUTH AMERICA (Sao
Paulo)
ASIA PAC
(Sydney)


Availability Zone


Edge Locations

London(2)
Seattle South Bend New York (3) Amsterdam (2)
Newark Stockholm
Dublin
Palo Alto
Tokyo (2)

San Jose
Frankfurt(2)
Paris(2)
Ashburn(2) Milan
Madrid Osaka
Los Angeles (2) Jacksonville

Dallas(2) Hong Kong

St.Louis

Miami Singapore(2)

Sydney
Sao Paulo

At the end of a web service

ec2-run-instances ami-b232d0db ec2-run-instances ami-b232d0db
--instance-count 3 --instance-count 5
--availability-zone eu-west-1a --availability-zone eu-west-1c
--instance-type m1.small --instance-type m1.medium


ec2-run-instances ami-b232d0db
--instance-count 2
--availability-zone us-east-1d
--instance-type m1.xlarge
ec2-run-instances ami-b232d0db
--instance-count 2
--availability-zone us-east-1b
--instance-type m1.xlarge


as-create-auto-scaling-group MyGroup
ec2-authorize default -p 80 --launch-configuration MyConfig
--availability-zones eu-west-1c
--min-size 2
--max-size 200

elb-create-lb
myLoadBalancer

Elastic capacity Traditional IT
capacity
Capacity

Time
Your IT needs

Elastic capacity

On and Off Fast Growth

Variable peaks Predictable peaks

Elastic capacity
WASTE

On and Off Fast Growth

Variable peaks Predictable peaks

CUSTOMER DISSATISFACTION

Elastic capacity

Capacity
Traditional
IT capacity

Elastic cloud capacity
Time

Your IT needs

503
Service Temporarily Unavailable

The server is temporarily unable
to service your request due to
maintenance downtime or capacity
problems. Please try again later.

Typical weekly traffic to
Amazon.com

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

November traffic to Amazon.com

November

Provisioned capacity

November

76%
Provisioned capacity

November
24%

November 10th 2010
Turned off last physical web server of
Amazon.com

November 10th 2010
Turned off last physical web server of
Amazon.com

October 31st 2011
Turned off last web servers supporting
European business

Time: +00h

<10 cores

Elastic Capacity

Time: +24h
>1500
cores

Elastic Capacity

Time: +72h

<10 cores

Elastic Capacity

Time: +120h

>600 cores

Elastic Capacity

40 servers to 5000 in 3 days
EC2 scaled to peak
of 5000 instances
Number of EC2 Instances

“Techcrunched”

Launch of Facebook
modification

Steady state of ~40
instances

4/12/2008 4/13/2008 4/14/2008 4/15/2008 4/16/2008 4/17/2008 4/18/2008 4/19/2008 4/20/2008

Reference Model
security

Deployment & Administration

App Services

Compute Storage Database

Networking

AWS Global Infrastructure

Global infrastructure


App Services

Compute Storage Database Regions
An independent collection of AWS resources in a
Networking defined geography
A solid foundation for meeting location-dependent
AWS Global Infrastructure privacy and compliance requirements



App Services

Compute Storage Database Availability Zones
Designed as independent failure zones
Networking Physically separated within a typical metropolitan
region



App Services

Compute Storage Database Edge Locations
To deliver content to end users with lower latency
Networking A global network of edge locations
Supports global DNS infrastructure (Route53) and
AWS Global Infrastructure Cloud Front CDN

Networking

Direct Connect
Dedicated connection to AWS

VPN Connection
Secure internet connection to AWS

Virtual Private Cloud
App Services Private, isolated section of the AWS Cloud

Route 53
Networking Highly available and scalable Domain Name
System

Compute

Vertical Elastic Compute Cloud (EC2)
Scaling
Basic unit of compute capacity
From $0.02/hr
Range of CPU, memory & local disk options
13 Instance types available, from micro to cluster
compute

Feature Details
Flexible Run windows or linux distributions
Scalable Wide range of instance types from micro to cluster
compute
App Services Machine Images Configurations can be saved as machine images
(AMIs) from which new instances can be created
Full control Full root or administrator rights
Secure Full firewall control via Security Groups

Monitoring Publishes metrics to Cloud Watch
Networking
Inexpensive On-demand, Reserved and Spot instance types

AWS Global Infrastructure VM Import/Export Import and export VM images to transfer
configurations in and out of EC2

Compute
Trigger auto-
scaling policy

as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones eu-west-1a
--min-size 4
--max-size 200

Auto-scaling
Automatic re-sizing of compute clusters based upon
App Services
demand

Compute Storage Database Feature Details
Control Define minimum and maximum instance pool sizes
and when scaling and cool down occurs

Networking Integrated to Use metrics gathered by CloudWatch to drive scaling
CloudWatch
Instance types Run auto scaling for on-demand instances and spot.
Compatible with VPC

Compute

Elastic Load Balancing
Create highly scalable applications
Distribute load across EC2 instances in multiple
availability zones

Feature Details
Auto-scaling Automatically scales to handle request volume
App Services
Available Load balance across instances in multiple availability
zones
Compute Storage Database Health checks Automatically checks health of instances and takes
them in or out of service
Session stickiness Route requests to the same instance
Networking
Secure sockets layer Supports SSL offload from web and application
servers with flexible cipher support
Monitoring Publishes metrics to Cloud Watch

Storage

S3 - Durable storage, any
object
99.999999999% durability of objects
Unlimited storage of objects of any type
Feature Details
Up to 5TB size per object
Flexible object store Buckets act like drives, folder structures within

Access control Granular control over object permissions

Deployment & Administration Server-side encryption 256bit AES encryption of objects

Multi-part uploads Improved throughput & control
App Services
Object versioning Archive old objects and version new ones

Object expiry Automatically remove old objects
Access logging Full audit log of bucket/object actions

Networking Web content hosting Serve content as web site with built in page handling

Notifications Receive notifications on key events

AWS Global Infrastructure Import/Export Physical device import/export service

Storage

Elastic Block Store
High performance block storage device
1GB to 1TB in size
Mount as drives to instances

Feature Details
High performance file Mount EBS as drives and format as required
system
Flexible size Volumes from 1GB to 1TB in size

Secure Private to your instances
App Services
Available Replicated within an Availability Zone

Compute Storage Database Backups Volumes can be snapshotted for point in time restore

Monitoring Detailed metrics captured via Cloud Watch

Networking


Database

Relational Database Service
Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations

Deployment & Administration Feature Details
Platform support Create MySQL, SQL Server and Oracle RDBMS

App Services Preconfigured Get started instantly with sensible default settings

Automated patching Keep your database platform up to date automatically
Backups Automatic backups and point in time recovery and full
DB backups
Backups Volumes can be snapshotted for point in time restore
Networking
Failover Automated failover to slave hosts in event of a failure

AWS Global Infrastructure Replication Easily create read-replicas of your data and
seamlessly replicate data across availability zones

Database

Amazon Relational Database Service
(Amazon RDS) databases stores forum
threads, site content, and project
configuration data.

High availability Multi-AZ database
deployment to handle live game metadata
and user-generated content.
Enterprise-grade fault tolerance for
App Services
protecting customer data.

By managing time-consuming database
administration tasks, Amazon RDS allows
SEGA to focus on business critical
Networking applications.


Database

DynamoDB
Provisioned throughput NoSQL database
Fast, predictable performance
Fully distributed, fault tolerant architecture

Feature Details
Deployment & Administration Provisioned throughput Dial up or down provisioned read/write capacity

Predictable Average single digit millisecond latencies from SSD
performance backed infrastructure
App Services
Strong consistency Be sure you are reading the most up to date values

Fault tolerant Data replicated across availability zones
Monitoring Integrated to Cloud Watch

Networking Secure Integrates with AWS Identity and Access
Management (IAM)
Elastic MapReduce Integrates with Elastic MapReduce for complex
analytics on large datasets

Application Services

Amazon SQS Processing results

Reliable, highly scalable, queue
service for storing messages as they Amazon SQS
travel between instances

Processing
task/processing trig
App Services

Feature Details
Compute Storage Database Reliable Messages stored redundantly across multiple
availability zones
Simple Simple APIs to send and receive messages
Networking
Scalable Unlimited number of messages

Secure Authentication of queues to ensure controlled access


Simple Workflow 1

Reliably coordinate processing steps
across applications Task A
2

Integrate AWS and non-AWS resources
Manage distributed state in complex
systems Task B 3
(Auto-scaling)

Task C

App Services
Feature Details
Process state Maintain application state across complex workflows
in a reliable and available manner
Tracking Tracks executions and log process for audit purposes

Consistency Ensures processing tasks are executed and duplicity of
Networking events does not occur
Simple Simple Decider and Task programming model for
rapid integration

Document
Cloud Search Server
Elastic search engine based upon
Amazon A9 search engine
Fully managed service with
sophisticated feature set
Search
Scales automatically
Server

Deployment & Administration Results

App Services
Feature Details
Auto-scaling Automatically scales based upon request volumes and
data volumes
High performance In memory operation means consistently low latency
for search results

Networking Sophisticated features Support for faceting, stemming, synonyms, stop
words and custom rank expressions
Low cost Elastic service, pay for what you use


CloudFront 3 Served from S3
/images/*
World-wide content distribution
network
Easily distribute content to end users
with low latency, high data transfer
speeds, and no commitments. London 2 Served from EC2
*.php

Paris
1 Single CNAME
NY
www.mysite.com
App Services
Feature Details
Compute Storage Database Fast Multiple world-wide edge locations to serve content
as close to your users as possible
Integrated with other Works seamlessly with S3 and EC2 origin servers
services
Networking
Dynamic content Supports static and dynamic content from origin
servers
Streaming Supports rtmp from S3 and includes support for live
AWS Global Infrastructure streaming from Adobe FMS and Microsoft Media
Server

Deployment & Admin

Elastic Beanstalk
One-click deployment from Eclipse, Visual Studio and
Git
Rapid deployment of applications
All AWS resources automatically created

Feature Details
Platform support Containers for Java, .net and PHP
App Services
Resource creation Creates load balancer, instances, autoscaling and
monitoring automatically
Compute Storage Database Monitoring & Logs Integrated with Cloud Watch and consolidates server
logs
Versioning Manage versions of applications and easily rollback
deployments
Networking
Notifications Receive alerts on key events

Full resource access Access all underlying AWS resources as necessary

Deployment & Admin

OpsWorks
DevOps focused managed application stacks
Underlying Chef recipes allow for complete
customisation

Feature Details
Platform support Chef recipes allows for community expansion for
platform components such as Solr, NgniX etc
App Services Resource creation Customizable deployments, rollback, partial
deployments, patch management, automatic
instance scaling, and auto healing
Compute Storage Database Layered Manage logical application layers and combine
into stacks.

Networking


Deployment & Admin

Cloud Formation
Automate creation of ‘stacks’ in a repeatable way
Scripting framework for AWS resource creation

Feature Details
Platform support Support for AWS resources from EC2 to IAM
App Services
Resource creation Creates AWS resources behind the scenes and reports
on progress
Compute Storage Database Declarative Specify stacks in JSON format and source control your
environments
Customizable Drive stack creation with paramaters
Networking


Deployment & Admin

Identity & Access Management
Granular control of user rights with AWS
Automated granting of EC2 service rights

Software Developer Kits
Comprehensive support of programming models for
Deployment & Administration using AWS services

App Services


Networking


+ others
Simple Email Service
Simple Notification Service
ElastiCache
Elastic MapReduce
CloudWatch
…

Introducing Amazon
Redshift
Data Warehousing the AWS Way

Easily and rapidly analyze Automated deployment &
petabytes of data administration
1/10 the cost of traditional data Compatible with popular BI tools
warehouses

Your Analytics
AWS Data Pipeline

Orchestration service for data-driven workflows
Create automated and scheduled data flows
Pre-integrated with AWS data sources
Easily connect with 3rd party & on-premises sources

Amazon Elastic Transcoder

Transcoding Pipeline 1
Amazon S3
Output Bucket

Amazon S3
Input Bucket Transcoding Pipeline 2

Amazon S3
Output Bucket

Amazon S3
Input Bucket
Transcoding Pipeline 4
Amazon S3
Output Bucket


720p Kindle Fire HD Custom
Transcoding Job n Transcoding Job 2  Transcoding Job 1

Inside a Transcoding Pipeline

Shared responsibility

Foundation Services
Amazon

Compute Storage Database Networking

Availability Zones
AWS Global Edge Locations
Infrastructure Regions

Shared responsibility

Customer Data

Platform, Applications, Identity & Access Management
You

Operating System, Network & Firewall Configuration

Client-side Data Encryption & Data Server-side Encryption Network Traffic Protection
Integrity Authentication (File System and/or Data) (Encryption/Integrity/Identity)

Foundation Services
Amazon

Compute Storage Database Networking

Availability Zones
AWS Global Edge Locations
Infrastructure Regions

Security standards

Certifications Physical Security HW, SW, Network
SOC 1 Type 2 (formerly Datacenters in Systematic change
SAS-70) nondescript facilities management
ISO 27001 Physical access strictly Phased updates
controlled deployment
PCI DSS for EC2, S3,
EBS, VPC, RDS, ELB, Must pass two-factor Safe storage
IAM authentication at least decommission
twice for floor access
FISMA Moderate Automated monitoring
Compliant Controls Physical access logged and self-audit
and audited
HIPAA & ITAR Advanced network
Compliant Architecture protection

http://aws.amazon.com/security

So what are
we going to do
today?

Instance

Availability Zone Region

Instance S3


Cloud
Front

Instance S3


Cloud
Front

Instance S3

RDS


Elastic Load Cloud
Balancer Front

Instance Instance S3

RDS


Elastic Load Cloud
Balancer Front

Instance Instance S3

Auto
scaling
Group

RDS


Go grab some
lunch, come
back and we’ll
crack on

Ask questions
(it will fill time when we wait for
things to launch)

There will be a recording of this
demo on YouTube, so don’t
worry if you miss anything

bootstrapping
passing data to an instance

Instance User
request data

Instance User Meta-data
request data service

Instance User Meta-data
request data service

Instance

Shell script in user-data will be executed on launch:

#!/bin/sh
yum -y install httpd php mysql php-mysql
chkconfig httpd on
/etc/init.d/httpd start

Public Key
Inserted by Amazon into
each EC2 instance that
you launch

EC2
Instance

Comms secured
with private key

Private Key
Downloaded and stored
by you

security groups
instance firewalling

Name
Description
Security Group
Protocol
Port 22
(SSH) Port range
Port 80 IP Address, range,
(HTTP) or another security
group

instance

Elastic Load Balancer

Instance Instance Instance Instance Instance Instance

Availability Zone Availability Zone Availability Zone

Region

auto-scaling
elastic server pool

Launch Configuration Auto-Scaling Group Auto-Scaling Policy

Describes what Auto Auto Scaling managed Parameters for
Scaling will create when grouping of EC2 performing an Auto
adding instances Scaling action
Instances
Automatic health check to Scale Up/Down and by
AMI how much
Instance Type maintain pool size
Security Group ChangeInCapacity (+/- #)
Instance Key Pair Automatically scale the
number of instances by ExactCapacity (#)
policy – Min, Max, Desired ChangeInPercent (+/- %)
Only one active launch
configuration at a time Cool Down (seconds)
Automatic Integration with
Auto Scaling will terminate ELB
Policy can be triggered by
instances with old launch CloudWatch events
configuration first Automatic distribution &
rolling update balancing across AZs

Create a launch configuration:

as-create-launch-config
--image-id ami-50348c51
--instance-type t1.micro
--key apac-tok
--group 'Web Servers’
--launch-config 101-launch-config

Create an auto-scaling group:

as-create-auto-scaling-group 101-as-group
--availability-zones ap-northeast-1a
--launch-configuration 101-launch-config
--load-balancers myELB
--max-size 5
--min-size 1

Create an auto-scaling policy (scale up):

as-put-scaling-policy 101ScaleUpPolicy
--auto-scaling-group 101-as-group
--adjustment=1
--type ChangeInCapacity
--cooldown 300

Create an auto-scaling policy (scale down):

as-put-scaling-policy 101ScaleDownPolicy
--auto-scaling-group 101-as-group
"--adjustment=-1"
--type ChangeInCapacity
--cooldown 300

AWS 101 Lunch & Learn March 2013

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (20)

Similaire à AWS 101 Lunch & Learn March 2013

Similaire à AWS 101 Lunch & Learn March 2013 (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Dernier

Dernier (20)

AWS 101 Lunch & Learn March 2013