2. What you can do after this
Public 2
Relational
Database
Service
AutoScaling CloudWatch
Simple Queue
Service
Load
Balancing
…
Notification
EC2
use
Application
7. Usage of Regions and
Availability Zones
Public 7
closer to specific customers, meet legal etc.
Regions
8. Data Transfer Cost
Public 8
EU (Ireland)
EC2
machine
1
EC2
machine
2
Availability
Zone
US-West (Northern California)
$!
$
$
$!
9. Access Interfaces
Public 9
AWS SDKs
(**)
(*) : not all AWS services
(**) : all AWS services
AWS
Management
Console(*)
Java-based
command
line(*)
AWS SDK(**)
Web Service
(REST or
SOAP API)(**)
12. Amazon EC2 Web Service
Basic Storage
Customized
AMIs
Public 12
13. What is EC2 ?
“Amazon Elastic Compute Cloud (Amazon EC2) is a web service
that provides resizable computing capacity that you use to
build and host your software systems”
Public 13
an EC2 instance
Availability Zone
Availability Zone
14. Amazon Machine Image
& Instances
Public 14
AMI
(Template)
- OS: Ubuntu
- Platform: x86
- Storage devices: null
EC2 Instance
(Large Instance Type)
EC2 Instance
(High-CPU Medium
Instance Type)
- 7.5 GB memory
- 4 EC2 Compute Units (*)
- 850 GB instance storage
- 1.7 GB of memory
- 5 EC2 Compute Units
(*)
- 350 GB of instance storage
A sample AMI
Large Instance Type
High-CPU Medium
Instance Type
(*) 1 EC2 Compute Unit = 1.0 - 1.2 GHz 2007 Opteron or 2007 Xeon processor
launch instance
16. Elastic Block Store
& Instance Store
Public 16
Attach
Attach
instance store
(ephemeral store)
EBS volume
Create snapshot
Instance
Instance A
EBS snapshot in Amazon S3
Create volume
Instance B
New EBS volume
Instance A Instance B
Attach
Detach
Backup
Detaching
Persistence
Instance
EBS volume
17. Root device storage
Public 17
AMI backed by instance store
AMI backed by EBS
Instance A
Instance B
Attach more
Attach more
Root device storage
launch instance
launch instance
18. Elastic IP Addresses
Public 18
10.0.0.170
ec2-122-248-202-170...com
Internet user
10.0.0.190
10.0.0.180
ec2-122-248-202-180...com
1.1.1.1 1.1.1.2
19. Pricing Model
Usage hour per EC2 instance
Data Transfer per EC2 instance (both “in”
and “out”) in different Availability Zone or
Region
Data Transfer per Elastic IP Address (both
“in” and “out”)
Other impact factors: Region, OS, Instance Type, Long-term Contract, Bidding
Public 19
21. Amazon EC2 Web Service
Basic Storage
Customized
AMIs
Public 21
22. Storage Types
Elastic Block Store volume Instance store
Persistent V
Cross-instance
access
V
Back up V
Size limits Up to 1TiB per volume Up to 3.3TiB per
instance
Free V
Public 22
24. Block device mapping
Public 24
OS: Amazon Linux
Kernel: aki-13d5aa41
….
Block device mapping
/dev/sdb=none
AMI template
/dev/sdc=ephemeral0
/dev/sdd=snap-a08912c9:15:true
count up based on the instance type
<device name>=<value>
25. EBS Pricing Model
Storage volume (in GB) per month
I/O request (in million)
Other impact factors: Region
Public 25
26. Amazon EC2 Web Service
Basic Storage
Customized
AMIs
Public 26
27. Needs of customized AMI
To meet the your own needs
To share
To sell
Public 27
28. AMI Creation Process
Windows Linux/UNIX
Root
storage
device
EBS-backed
AMI
Instance store-
backed AMI
From existing AMI
Fresh installation
Public 28
2
1
29. Launch the
instance
Customize
instance
Create image from
customized
instance
Create snapshot of
root device
Register image
from snapshot
(1) Creating EBS-Backed AMIs
Public 29
PROMPT> ec2-create-image instance-id
PROMPT> ec2-register --root-device-name
/dev/sda1 -b /dev/sda1=snap-12345678
OR
30. (2) Creating Windows
instance store-backed AMI
Launch the
instance
Customize
instance
Bundle
customized
instance
to S3
Register
bundled
image
Public 30
PROMPT> ec2-bundle-instance instance-id PROMPT> ec2-register <s3-bucket>/image.manifest.xml
-n image_name
OR OR
32. Amazon CloudWatch Web Service
Public 32
• CPU utilization
• Network traffic
• I/O
• Latency
000
EC2 instances
EBS volumes
CloudWatch service
CloudWatch metrics
RDS instances
Load Balancers
33. CloudWatch modes
Public 33
• At 5-minutes frequency
• Free of charge
Basic
• For EC2 instances
• At 1-minute frequency
• $3.5+ per instance per month
Detailed
35. CloudWatch Alarms
“Watches a single metric over a time period and invokes
actions when the value of the metric exceeds a given
threshold over a number of time periods”
Public 35
Amazon
SNS topic
Auto
Scaling
policy OK state
ALARM state
INSUFFICIENT_DATA state
36. Pricing Model
Per EC2 instance / month
Per custom metric / month
Per Alarm / month
API Requests (per 1,000 Get, List, or Put
requests)
Other impact factors: Region
Public 36
40. Overview for Developer
Sticky Sessions
"X-Forwarded-Port", "X-Forwarded-For"
and "X-Forwarded-Proto" Support
Known issue: HTTP 60 seconds timeout
for request
Public 40
41. Pricing Model
Usage hour per Load Balancer instance
Data processed (in GB) per Load Balancer
instance
Other impact factors: Region
Public 41
50. Amazon Relational Database
Web Service
Resizable capacity
for databases
Amazon firewall
Flexible back up
methods
Replication (only on
MySQL)
Monitoring
Public 50
51. Create new DB Instance
Public 51
Update DB Security Group before
connecting to the instance
53. Pricing Model
Per DB Instance Class / month
Storage (in GB) / month
I/O (in million) / month
Back up storage / month
Bandwidth (in GB both “in” / ”out”) / month
Other impact factors: Region, Multi-AZ Deployment, Reserved Instances
Public 53
57. Pricing Model
Storage (in GB) / month
Requests (in 1000 unit) / month
Bandwidth (in GB both “in” / ”out”) / month
Other impact factors: Region, Reduced Redundancy Storage option
Public 57
60. Your application
Amazon Simple Queue
Web Service
Public 60
Machine A
message (text)
HTTP GET or POST request
Machine B
message (text)
HTTP GET or POST response
Message 1
Message 2
Message 3
Message 4
Amazon Queue
61. Key Features
64 KB of text in a message
Not first in, first out delivery of messages
Locking the message: Visibility Timeout
Control access to a queue
Public 61
62. Pricing Model
$0.01 per 10k requests
"out" bandwidth (in GB)
Other impact factors: Region
Public 62
65. Subscriber
Subscriber
Subscriber
Notification
topic
Amazon Simple Notification
Web Service
Public 65
SNS
message (text)
HTTP GET request
Program A
message (text)
message (text)
message (text)
HTTP POST request
Email
Simple Queue Service
receiver@email.com
Queue
http://receiver.com/message
Message (text)
SMS (in US)
Subscriber
800-201-7575
67. Pricing Model
$0.06 per 100k API Requests (free first
100k requests) / month
Amount of notifications (free first part) /
month
"out" bandwidth (in GB)
Other impact factors: Region
Public 67
Objectives:
Understand how to integrate Amazon services into your application
Deploy /manage your application on EC2
Region
Regions are dispersed (spread in wide area) and located in separate geographic areas (US, EU, etc.). Each EC2 Region is designed to be completely isolated from the other Amazon EC2 Regions.
This achieves the greatest possible failure independence and stability, and it makes the locality of each EC2 resource unambiguous
Region list:
US East (Northern Virginia): us-east-1
US West (Oregon) : us-west-2
US West (Northern California) : us-west-1
EU (Ireland) : eu-west-1
Asia Pacific (Singapore) : ap-southeast-1
Asia Pacific (Tokyo) : ap-northeast-1
South America (Sao Paulo) : sa-east-1
Availability Zone
Availability Zones are distinct locations within a Region that are engineered to be isolated from failures in other Availability Zones and provide inexpensive, low latency network connectivity to other Availability Zones in the same Region
However, failures can occur that affect the availability of instance resources that are in the same location. Although this is rare, if you host all your Amazon EC2 instances in a single location that is affected by such a failure, your instances will be unavailable.
By launching instances in separate Regions, you can design your application to be closer to specific customers or to meet legal or other requirements.
By launching instances in separate Availability Zones, you can protect your applications from the failure of a single location
While standard Amazon EC2 Regional Data Transfer charges of $.01 per GB in/out apply when transferring data between an Amazon EC2 instance and Amazon RDS DB Instance in different Availability Zones of the same Region
There is no additional charge for data transferred between Amazon SimpleDB and other Amazon Web Services within the same Region
The command-line client needs to be installed with some configuration
And each service has its own command-line package, so it needs to download separately
Quote from AWS EC2 Documentation
“resizable”: With EC2, you use and pay for only the capacity that you need. This eliminates the need to make large and expensive hardware purchases, reduces the need to forecast traffic, and enables you to automatically scale your IT resources to deal with changes in requirements or spikes in popularity related to your application or service
Diagram
Simple visualization of EC2 in Amazon cloud platform: “instance” ~ “virtual machine”
An Amazon Machine Image (AMI): is a template that contains a software configuration: operating system, application server, applications. If an instance fails, you can launch a new one from the AMI. Amazon publishes many AMIs that contain common software configurations for public use. In addition, members of the AWS developer community have published their own custom AMIs
Instance Type: a specification that defines the memory, CPU, storage capacity, and hourly cost for an instance. Some instance types are designed for standard applications, whereas others are designed for CPU-intensive applications, or memory-intensive applications, etc.
EC2 instance: an virtual machine
You launch AMIs at your own risk. Amazon cannot vouch for the integrity or security of AMIs shared by other EC2 users. Therefore, you should treat shared
Public AMIs are available from Amazon
Should get a public AMI ID from a trusted source (use at your own risk)
AMIs as you would any foreign code that you might consider deploying in your own data center and perform the appropriate due diligence. Ideally, you should get the AMI ID from a trusted source (a web site, another EC2 user, etc). If you do not know the source of an AMI, we recommend that you search the forums for comments on the AMI before launching it. Conversely, if you have questions or observations about a shared AMI, feel free to use the AWS forums to ask or comment
Amazon Elastic Block Store (Amazon EBS) provides block level storage volumes for use with Amazon EC2 instances. Amazon EBS volumes are highly available and reliable storage volumes that can be attached to any running instance. The attached Amazon EBS volumes are exposed as storage volumes that persist independently from the life of the instance
Persistence : the AMI backed instance store doesn’t support Stopped state. Rebooting doesn’t cause losing data on instance store
The amount of instance store hdd depends on instance type
Private address is reachable from within the Amazon EC2 network
Public address that are directly mapped to each other through Network Address Translation (NAT) and are reachable from the Internet
If you use dynamic DNS to map an existing DNS name to a new instance's public IP address, it might take up to 24 hours for the IP address to propagate through the Internet. As a result, new instances might not receive traffic while terminated instances continue to receive request
You can associate one Elastic IP address with only one instance at a time. When you associate an Elastic IP address with an instance, its current public IP address is released to the Amazon EC2 public IP address pool. If you disassociate an Elastic IP address from the instance, the instance is automatically assigned a new public IP address within a few minutes
All accounts are limited to 5 Elastic IP addresses because public (IPV4) Internet addresses are a scarce public resource
To ensure our customers are efficiently using Elastic IP addresses, we impose a small hourly charge when these IP addresses are not mapped to an instance. When these IP addresses are mapped to an instance, they are free of charge
Limit total storage is applied whichever you reach first
The volume need not be attached to a running instance in order to take a snapshot. The snapshots can also be shared with specific AWS accounts or made public
launch instance from snapshot: demo later
Amazon EBS snapshots are incremental backups, meaning that only the blocks on the device that have changed since your last snapshot will be saved. If you have a device with 100GiB of data, but only 5GiB of data have changed since your last snapshot, only the 5GiB of modified data will be stored back to Amazon S3. Even though snapshots are saved incrementally, the snapshot deletion process is designed so that you need to retain only the most recent snapshot in order to restore the volume
No device: to use this option only when you want to suppress a block device from attaching at launch time
If you customize your instance with ephemeral storage devices or additional EBS volumes besides the root device, the new AMI contains block device mapping information for those storage devices and volumes. When you then launch an instance from your new AMI, the instance automatically launches with the additional devices and volumes
$3.50 per instance per month (the per metric price below x 7 pre-defined metrics per instance). Custom metrics: memory usage, transaction volumes, or error rates …
In the following figure, the alarm threshold is set to 3 and the minimum breach is 3 periods. That is, the alarm invokes its action only when the threshold is breached for 3 consecutive periods. In the figure, this happens with the third through fifth time periods, and the alarm's state is set to ALARM. At period six, the value dips below the threshold, and the state reverts to OK. Later, during the ninth time period, the threshold is breached again, but not for the necessary three consecutive periods. Consequently, the alarm's state remains OK
An alarm has three possible states:
OK—The metric is within the defined threshold
ALARM—The metric is outside of the defined threshold
INSUFFICIENT_DATA—The alarm has just started, the metric is not available, or not enough data is available for the metric to determine the alarm state
Elastic Load Balancing can detect the health of Amazon EC2 instances. When it detects unhealthy load-balanced Amazon EC2 instances, it no longer routes traffic to those Amazon EC2 instances and spreads the load across the remaining healthy Amazon EC2 instances
Elastic Load Balancing supports the ability to stick user sessions to specific EC2 instances
Elastic Load Balancing supports use of both the Internet Protocol version 4 and 6 (IPv4 and IPv6)
Sticky Sessions
Enables the load balancer to bind a user's session to a specific application instance. This ensures that all requests coming from the user during the session will be sent to the same application instance
Load-balancer-generated HTTP cookies, which allow browser-based session lifetimes
Application-generated HTTP cookies, which allow application-specific session lifetimes
"X-Forwarded-Port" , "X-Forwarded-For" and "X-Forwarded-Proto" Support
Because load balancers intercept traffic between clients and servers, your server access logs contain only the IP address of the load balancer. To see the original IP address/port/protocol of the client, use the X-Forwarded-* request header. Elastic Load Balancing stores the IP address of the client in the X-Forwarded-For request header and passes the header along to your server
HTTP 60 seconds timeout for request
When a load balancer forwards a HTTP request to instance, if the instance sends back the response after 60 seconds, the load balancer will automatically kill that HTTP routine and client will receive empty response (no http headers)
$ curl –i http://host/balacing
Auto Scaling allows you to scale your Amazon EC2 capacity up or down automatically according to conditions you define. Auto Scaling monitors the health of each EC2 instance that it launches. If any instance terminates unexpectedly, Auto Scaling detects the termination and launches a replacement instance. This capability helps you maintain a fixed, desired number of EC2 instances automatically.
Auto Scaling is particularly well suited for applications that experience hourly, daily, or weekly variability in usage.
Auto Scaling is enabled by Amazon CloudWatch and available at no additional charge beyond Amazon CloudWatch fees.
Scheduled time in Unix cron syntax format
Resizable capacity for databases : modify cpu, memory and storage at any time, even in running server instance
Amazon firewall : control access to your DB Instances. A DB Security Group acts like a firewall controlling network access to your DB Instance
Flexible back up methods : automated backups and DB Snapshots. Automated backups automatically back up your DB Instance during a specific, user-definable backup window, and keeps the backups for a limited, user-specified period of time (called the backup retention period); you can later recover your database to any point in time during that retention period. DB Snapshots are user-created snapshots that enable you to back up your DB Instance to a known state, and restore to that specific state at any time. Amazon RDS keeps all DB Snapshots until you delete them.
Flexible scaling : Currently, replication is only supported for the MySQL engine. We plan to support replication options for Oracle in the future.
Amazon RDS for MySQL provides two distinct replication options to serve different purposes.
Monitoring: monitor metrics with Amazon CloudWatch
Auto Minor Version Upgrade option enables your DB Instance to receive minor engine version upgrades automatically when they become available
DB Instance Class ~ EC2 Instance Type : indicate the CPU + RAM amount of DB instance
DB Instance Identifier is a customer-supplied identifier for a DB Instance. This identifier specifies a particular DB Instance when interacting with the Amazon RDS API and commands. The DB Instance identifier must be unique for that customer in an AWS region
Database Name depends on the database engine in use:
For the MySQL database engine, the Database Name is the name of a database hosted in your Amazon DB Instance. An Amazon DB Instance can host multiple databases. Databases hosted by the same DB Instance must have a unique name within that instance
For the Oracle database engine, Database Name is used to set the value of ORACLE_SID, which must be supplied when connecting to the Oracle RDS instance.
If you are looking to use replication to increase database availability while protecting your latest database updates against unplanned outages, consider running your DB Instance as a Multi-AZ deployment. When you create or modify your DB Instance to run as a Multi-AZ deployment, Amazon RDS will automatically provision and manage a “standby” replica in a different Availability Zone. In the event of planned database maintenance, DB Instance failure, or an Availability Zone failure, Amazon RDS will automatically failover to the standby so that database operations can resume quickly without administrative intervention. Multi-AZ deployments utilize synchronous replication, making database writes concurrently on both the primary and standby so that the standby will be up-to-date in the event a failover occurs
If you are looking to take advantage of MySQL’s built-in replication to scale beyond the capacity constraints of a single DB Instance for read-heavy database workloads, Amazon RDS makes it easier with Read Replicas. You can create a Read Replica of a given “source” DB Instance using the AWS Management Console or CreateDBInstanceReadReplica API. Once the Read Replica is created, database updates on the source DB Instance will be propagated to the Read Replica. You can create multiple Read Replicas for a given source DB Instance and distribute your application’s read traffic amongst them. In particular, updates are applied to your Read Replica(s) after they occur on the source DB Instance (“asynchronous” replication), and replication lag can vary significantly. This means recent database updates made to a standard (non Multi-AZ) source DB Instance may not be present on associated Read Replicas in the event of an unplanned outage on the source DB Instance. As such, Read Replicas do not offer the same data durability benefits as Multi-AZ deployments. While Read Replicas can provide some read availability benefits, they and are not designed to improve write availability.
Data transferred between Amazon RDS and Amazon EC2 Instances in the same Availability Zone is free.
Data transferred between Availability Zones for replication of Multi-AZ deployments is free.
It allows customer to store and retrieve any amount of data on the web
Control access to buckets and objects
Allow only downloading on anonymous user
Don’t allow specific users to get list of objects in a bucket
Restrict the access to a bucket / object from specific IP address
Versioning objects in a bucket : Versioning is a means of keeping multiple variants of an object in the same bucket. In one bucket, for example, you can have two objects with the same key, but different version IDs, such as photo.gif (version 111111) and photo.gif (version 121212). You might enable versioning to prevent objects from being deleted or overwritten by mistake, or to archive objects so that you can retrieve previous versions of them
Data transferred between Amazon RDS and Amazon EC2 Instances in the same Availability Zone is free.
Data transferred between Availability Zones for replication of Multi-AZ deployments is free.
Run ruby scripts to setup first and remember to clean up
It allows customer to store and retrieve any amount of data on the web
Access control : to grant another AWS account a particular type of access to your queue (e.g., SendMessage) or for a specific period of time
A topic is a communication channel to send messages and subscribe to notifications. It provides an access point for publishers and subscribers to communicate with each other
Currently Amazon SNS will only accept US phone numbers as valid subscription end-points.
Control access: to grant another AWS account a particular type of topic action (e.g., Publish) or to limit subscriptions to your topic to only the HTTPS protocol (avoid spam in email)