Aws introduction tocloudstorage

Amazon Web Services
Introduction to Cloud Storage

Kevin Wittkopf
Enterprise Solutions Architect…

Cloud Storage
AWS offers multiple cloud-based storage options.

• Amazon S3 (Simple Storage Service)

• Amazon EBS (Elastic Block Storage)

• Amazon EC2 Local Instance Store (Ephemeral volumes)

• Amazon Cloudfront

Each has a unique combination of
performance, durability, cost, and interface
Often used to form a Storage Hierarchy

Storage by the Numbers
Cloudfront S3 EBS Local Disk
Scale Unlimited Unlimited Storage 1TB per volume 147 – 1690GB
Temporary 5TB objects ~44TB per 1-5 volumes
Storage 100 name spaces server
(buckets)
Durability N/A 99.999999999% .1-.5% AFR Ephemeral
(standard) design requires
99.99% application-level
(RRS) redundancy
Latency Low High Low Very Low

Management Zero Zero Maintenance Snapshots, Manual
Maintenance RAID, Redundancy
Consistency
Locality Global Region-Centric AZ-Specific In-machine

Interface REST or SOAP REST or SOAP Block Device Block Device
Web Services Web Services

S3
Amazon Simple Storage Service

©2011 Amazon Web Services May not be reused or
redistributed without permission

Amazon

EC2

Amazon Amazon Amazon
Amazon
EBS S3 RDS SimpleDB

©2011 Amazon Web Services May not be reused or redistributed without permission

Amazon Simple Storage Service
Amazon S3: Simple Storage Service
Highly scalable data storage in-the-cloud
Programmatic access via web services API
Is a Web Store, not a file system
 Optimized for WORM
 Eventually consistent
Fast, highly available and durable
Economical

©2011 Amazon Web

Is Your Data Storage This Resilient?
Note: Conceptual drawing only. Actual number of nodes & datacenters may vary

Region

Datacenter Datacenter

Datacenter
Node Node
1 n
...

©2011 Amazon Web

Amazon S3 Adoption Rate: Billions of Objects Stored
Peak Requests:
290,000+
600
per second 6 months!

500 490

400

300
262

200

102
100

40
14
2.9
0
Q4 2006 Q4 2007 Q4 2008 Q4 2009 Q4 2010 Q2 2011

©2011 Amazon Web Services May not be

Amazon S3 Namespace

Amazon S3

bucket bucket

object object object object

bucket

object object
©2011 Amazon Web Services May not be reused

Amazon S3 Namespace

Amazon S3

mculver-images media.mydomain.com

Beach.jpg 2009/maui/sun.jpg img1.jpg img2.jpg

public.blueorigin.com

Installer.msi img/pic1.jpg
©2011 Amazon Web Services May not be

Reduced Redundancy Storage Saves $$

Designed to provide 99.99% durability and 99.99%
availability of objects over a given year
This durability level corresponds to an average annual
expected loss of 0.01% of objects
Designed to sustain the loss of data in a single facility

©2011 Amazon Web

S3 Pricing has Three Dimensions

Storage Data Transfer Requests
Tier Norma RRS Tier Price Type Price
l
Frist 1 $0.14/G $0.093/GB All transfer in Free PUT, $0.01
TB/mo B COPY, per
POST, or 1,000
LIST
Next 49 TB $0.125/G $0.083/GB First 1 GB out Free GET and $0.01
B all other per
requests 10,000
Next 450 $0.11/GB $0.073/GB <= 10 TB out $0.12/G
TB B
Next 500 $0.095/G $0.063/GB Next 40 TB $0.09/G
TB B out B
Next 4000 $0.08/G $0.053/GB Next 100 TB $0.07/G
TB B out
©2011 Amazon Web B

Access Control Lists
• Control access to bucket or to object
• No inheritance from bucket to object
• Set at create, write, or any time
• List of up to 100 grants by:
– Owner
– Email
– Any AWS account
– Anyone
• Permissions:
– READ – Object or bucket
– WRITE – Create, overwrite, delete objects in
bucket
– READ_ACP – Read ACL for bucket or object
– WRITE_ACP – Overwrite ACL for bucket or object
– FULL_CONTROL – All of the above


Bucket Policy
Allow Jason or Mark to do anything in my S3 bucket
when coming from the Office internal network
{"Statement":[{
"Effect":"Allow",
"Principal":{"AWS":["4649-6425", "5243-0045"]},
"Action":"*",
"Resource":"/mybucket/*",
"Condition":{
"IpAddress":{"AWS:SourceIp":"176.13.0.0/12"}
}}]}

©2011 Amazon Web

S3 Multipart Uploads
Improved throughput—upload parts in parallel
Quick recovery from any network issues—Smaller part
size minimizes the impact of restarting a failed upload due to
a network error.
Pause and resume object uploads—
 upload object parts over time
 Once you initiate a multipart upload there is no expiry
 must explicitly complete or abort the multipart upload
Begin an upload before you know the final object size—
can upload as you are create.

©2011 Amazon Web

Even More Amazon S3 (not so simple, is it?)
Website support
Versioning
Browser Upload to S3
Logging
Requestor Pays
Signed URLs
Bittorrent support
IAM
Meta-data

©2011 Amazon Web

AWS Import/Export
Get your data into AWS faster - load it onto a
portable storage device and ship it to an Amazon
data center
Faster than Internet transfer and more cost
effective than upgrading your connectivity
Use cases: data migration, offsite backup, direct
data interchange, disaster recovery

©2011 Amazon Web

AWS Import/Export vs. S3 upload costs

$1,200

$1,000

$800

$600 Import/Export
S3 upload
$400

$200

$0
100Gb 500Gb 1 Tb 2 Tb 3 Tb 5 Tb * 10 Tb **

AWS Import/Export vs. S3 upload time savings

60

50

40
Days

30 Import/Export
20 S3 upload

10

0
100 Gb 500 Gb 1 Tb 2 Tb 3 Tb 5 Tb

EBS Features

Persistent storage
Volume lifetime is independent of any particular EC2 instance.
General purpose
Raw, unformatted, block device. Use from Linux, Solaris or
Windows.
Performance
Equal to or better than local EC2 drive.
High reliability
Built-in redundancy within availability zone.
AFR (Annual Failure Rate) between 0.1% and 1%.
Scalable
Volume sizes ranging from 1 GB to 1 TB.
Easy
Easy to create, attach, back up, restore, and delete volumes.


Amazon Web Services: Durable & Available
Note: Conceptual drawing only. The number of Availability Zones may vary

Japan
US East Region EU West Region

Availabilit Availabilit Availabilit Availabilit
y Zone A y Zone B y Zone A y Zone B Availabilit Availabilit
y Zone A y Zone B

Availabilit US West Region Availabilit Singapore
y Zone C y Zone C

Availabilit Availabilit Availabilit Availabilit
y Zone A y Zone B y Zone A y Zone B

©2011 Amazon Web Services May not be reused or redistributed without written permission

EBS Volumes are in a Single Availability Zone

EBS Volume 1 EBS Volume 2

Availability Zone A Availability Zone B
Note: Conceptual diagram only


How EBS Interacts With EC2 and S3
AWS Cloud

EC2 Availability Zone S3 Storage
Create Snapshot

EBS Snapshot
EBS EBS EBS EBS EBS EBS

EBS Snapshot
Clone From
Snapshot EBS Snapshot

EC2 EC2 EC2
EBS Snapshot

EBS Snapshot

Internet

What’s the Difference Between
Amazon S3 and Amazon EBS?

Amazon S3 EBS
Paradigm Object store File system
Performance Very fast Very very fast
Redundancy Across data centers Within data center
Security Public Key / Private Visible only to your
Key EC2
Pricing $0.14/GB/Mo. stored $0.10/GB/Mo. allocated
Access from the Net? Yes(1) No(2)
Typical use case Write once, read many It’s a disk drive
(1) Only with proper credentials, unless ACLs are world readable
(2) Accessible from Net if mounted to server and set up as FTP, etc.

EBS Pricing
EBS Volumes
 $0.10 per GB-month
of provisioned storage
 $0.10 per 1 million I/O requests
 No charge for mounting/unmounting
volume
EBS Snapshots to Amazon S3
 $0.14 per GB-month of data stored
 $0.01 per 1,000 PUT requests
(when saving a snapshot)
 $0.01 per 10,000 GET requests
(when loading a snapshot)


©2011 Amazon Web Services May not be reused or
redistributed without permission

EBS Volume Lifecycle
Call CreateVolume
1 GB to 1 TB

Create
Call AttachVolume to affiliate
with one EC2 instance

Vast amounts of unused space Attach

• Format from EC2 instance
Attached
OS
&
• Mount formatted drive
Snapshot to Amazon S3 In Use

Deleted
CreateSnapshot

Call DeleteVolume Detach

Call DetachVolume

How Do Snapshots Work?

Time

Snapshot 2 Snapshot 3
Snapshot 1

Chunk 1
Block 1
S3
Chunk 2
Block 2
EBS Volume
Chunk 3
Block 3
Chunk 4
Block 4


Things You Should Know… (Part I)

EBS is a network resource
 Networks have variable performance
Striping will improve performance, although it’s usually fine to
start with
 4-8 drives per logical volume is observed sweet spot
 Great approach for high-performance database workloads
EBS is already redundant behind the scenes. It makes no
sense to implement RAID on top of EBS


Things You Should Know… (Part II)

Lazy loads from snapshots
 When you create a volume, space is allocated immediately
 Data from snapshot written to disk on 1st request for maximum
performance
 For maximum IO performance, run dd to touch all blocks
Like all of AWS EBS is constantly innovating.
These tips may not be appropriate in a few months


AWS Import/Export
Get your data into AWS faster - load it onto a
portable storage device and ship it to an Amazon
data center

Faster than Internet transfer and more cost
effective than upgrading your connectivity

Use cases: data migration, offsite backup, direct
data interchange, disaster recovery

Amazon CloudFront
Easy-to-use content delivery network
Same pay-as-you-go pricing as all of AWS
– No up-front contracts
– No long-term commitments
– Self-service sign-up
Uses Amazon S3 as the origin store
Worldwide network of edge locations
– Seattle, Palo Alto, Los Angeles, St.
Louis, Dallas, Ashburn, Newark, Miami, Amsterdam, Dub
lin, Frankfurt, London, Hong Kong, Tokyo

Aws introduction tocloudstorage

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Aws introduction tocloudstorage

Similaire à Aws introduction tocloudstorage (20)

Dernier

Dernier (20)

Aws introduction tocloudstorage

Notes de l'éditeur