2. Traditional Storage
On-Premise Storage Options
SAN – network-attached block devices: LUNs
DAS – local block devices (disks)
NAS – network attached file systems, filers
Differ in, Performance, Durability, and Cost
Often used to form a Storage Hierarchy
3. Cloud Storage
AWS offers multiple cloud-based storage options.
• Amazon S3 (Simple Storage Service)
• Amazon EBS (Elastic Block Storage)
• Amazon EC2 Local Instance Store (Ephemeral volumes)
Each has a unique combination of performance,
durability, cost, and interface
Often used to form a Storage Hierarchy
4. Other Storage options?
Amazon S3, EBS, and EC2 Instance Store volumes are
the basic storage building blocks, but for more
specialized uses, or managed resources, we have
other options as well:
Cloudfront
Amazon RDS
SimpleDB
SQS
Relational Databases on EC2
5. Storage by the Numbers
S3 EBS Local
Scale Unlimited Storage 1TB per volume 147 – 1690GB
5TB objects ~44TB per server 1-5 volumes
100 name spaces (buckets)
Durability 99.999999999% (standard) .1-.5% AFR Ephemeral design
99.99% requires application-
(RRS) level redundancy
Latency High Low Very Low
Management Zero Maintenance Snapshots, RAID, Manual Redundancy
Consistency
Locality Region-Centric AZ-Specific In-machine
Interface REST or SOAP Block Device Block Device
Web Services
6. S3 (Simple Storage Service)
Pay for exactly what you use
Highly Durable
Limitlessly scalable
Natively On-Line
7. S3 Best Practices
Replicate Metadata
Parallel IO for max speed
Multipart Upload
Sorted Writes
Resource-level IAM permissions
8. S3 Use Cases and Users
Media Serving
Media Sharing
File Storage / Backup
Static Content
Big Data
9. Amazon EBS
Pay for what you provision
POSIX compliant network
attached block device
Native Redundancy and Write
Cache; optimized for random
writes and reads.
10. EBS Best Practices
Pre-warming
Snapshots
Striping (RAID 0)
Block-device-mapping
11. EBS Use Cases and Users
Application hosting
Envoy Media Group Case Stud
Storage for multimedia content delivery on Amazon W
Data Processing
Introd
Establi
that of
Database
media
and po
channe
implem
Large Data
Envoy moved their IT infrastructure from a traditional co-location
advantage of -as-you-
to share, and promptly delivered, Envoy selected Gluster in conce
Challenges
Envoy currently hosts their entire web presence on Amazon Web
which has helped them meet the needs of their growing business
base. One of their biggest challenges was ensuring the IT infrastru
to respond to the highly dynamic nature of demand such as when
launch and then grow exponentially. From a data storage standpo
needed a solution that integrated easily with AWS and provided h
Storing the data locally on EC2 instances is not suffcient since that
persistent and would be lost if a server reboots. Amazon EBS volu
to one instance at a time a problem if your business requires hu
needed to ensure their data was easily shared.
12. EC2 Instance Store or Ephemeral Volumes
Free with your EC2 Instance
Zero Network Overhead; local,
direct attached resource.
High performance for sequential
reads and writes
Temporary; goes away when
your instance does
13. Instance Store Best Practices
App-level replication
Striping (RAID 0, 1)
Block-device-mapping
14. Instance Store Use Cases and Users
Stateless Web Hosts
Transcoding
Caching
HPC
DNFS
15. Better Together: Cross-system patterns
EBS <-> S3: Snapshots
S3 <-> SimpleDB: meta-data store
S3 <-> Local Storage: Caching
16. Short Detour: S3 vs EBS AMIs
Two types of EC2 AMIs and Instances:
Instance store (“S3-backed”)
Original EC2, boot from ephemeral storage
Can Start and Terminate only
All data is ephemeral (unless separate EBS volume attached)
EBS root (“boot from EBS”)
Newer, boot from EBS volume
Can Start, Stop, Create Image, and Terminate
Ephemeral storage still available, but not exposed by default
Use instance Block Device Mapping (command line and API only)
Additional EBS volumes can be attached to
either type
17. So, which should I choose?
Might be all of them!
S3: Static Online Content, Media Serving, Backups,
Big Data, Centralized Resources
EBS: Applications that require POSIX, Databases,
Data Warehouses
Ephemeral: Cache/Swap/Temp, Applications that
require minimum latency
18. Test!
It costs pennies to evaluate these technologies
against your use-case, and making the right choice
could save you millions.
19. How do I get it to the Cloud?
AWS Import/Export
Faster than Internet transfer and more cost effective than
upgrading your connectivity
Use cases: data migration, offsite backup, direct data
interchange, disaster recovery
WAN Optimization
Asperatech, Riverbed, Tsunami-UDP, etc
20. Read the originals!
Storage Options in the AWS Cloud:
https://media.amazonwebservices.com/AWS_Storage_O
ptions.pdf
Storage Use Cases in the AWS Cloud:
https://media.amazonwebservices.com/AWS_Storage_U
se_Cases.pdf
