With AWS you can choose the right storage service for the right use case. Given the myriad of choices, from object storage to block storage, this session will profile details and examples of some of the choices available to you, with details on real world deployments from customers using Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Store (Amazon EBS), Amazon Glacier and AWS Storage Gateway. In addition, this session will also cover all the new AWS storage features introduced in the last 12 months.

1. Choosing the Right Data Storage Solution
Joe Lyons – Sr. Manager Global Storage
November 13, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

2. AWS Storage Options
•
•
•
•
•
Scalable Storage
Inexpensive Archive Storage
Persistent Direct Attached Storage
Turn-key gateway solution
Customer Presentation: iMemories

3. We are constantly producing more data

4. From all types of industries

5. #
1
Object Storage
●○○○○

6. AMAZON S3
SIMPLE STORAGE SERVICE

7. 99.999999999%
Durability

8. Trillions
Of Unique Customer Objects

9. 1.5 Million+
Peak Transactions Per Second

10. Storage Tiers: Buckets + Unlimited Objects

11. Reduced Redundancy Option
99.99% saves ~20%

12. Spotify adds over 20,000 tracks a day - RRS

13. Amazon S3 Website: Static Content

14. Amazon S3 continuous cost reduction
• 16 Price reductions since launch
• TCO: On-premises vs. Amazon S3
– Can be challenging for some customers
– We can help!

15. RAW Storage at On perm vs. Cloud
Storage
1PB Raw Storage
800TB Usable Storage
600TB Allocated Storage
Amazon S3 Only actual
usage is
charged
400TB Written Application Storage

16. Use Amazon S3 When You Need
•
•
•
•
Unlimited storage capacity
High durability
Storage for backups
Single Origin Store with Delivery via Amazon CloudFront

17. AMAZON GLACIER
LOW-COST ARCHIVING SERVICE

18. 1c
Per GB / Month

19. $120
Per TB / Year

20. 99.999999999%
Durability

21. 3-5 Hours
Data Retrieval

22. STORAGE COSTS
VS
RETRIEVAL COSTS

23. Use Amazon Glacier When You Need
•
•
•
•
•
Inexpensive/Long-term archiving
Unlimited storage capacity
Eliminated Tape Museums
Eliminate Tech Refresh
High durability

24. Amazon S3 / Amazon Glacier Integration
POLICY-BASED ARCHIVING SERVICE

25. Lifecycle Rule

26. Archive Recovery Process with Tape
+ Days or Weeks

27. Archive Recovery Process with AWS &
Amazon Glacier
Glacier
S3
EC2
/HPC
CloudFront
Generating
Business
Value
$$
Hours

29. Use Amazon S3 and Amazon Glacier When You Need
•
•
•
HSM in the cloud
Archive data from Amazon S3/RRS to Amazon Glacier by
policy
Delete data from Amazon Glacier by policy

30. #
2
Block Storage
●●○○○

31. #
3
SYNC VOLUMES
●●●○○

32. AMAZON EBS
ELASTIC BLOCK STORAGE

33. Ephemeral Storage

34. 10 GB
1TB


35. Provisioned
IOPS
4000
IOPS

39. Amazon EBS
Snapshots

42. Use Amazon EBS When You Need
•
•
•
•
•
Long-term persistent storage
Data changes frequently
Block storage for your databases – Provisioned IOPS volumes
Filesystem for an instance NTFS, ExtFS, RAID, LVM…
Access to raw, unformatted block-level storage

43. AWS STORAGE GATEWAY

44. AWS Storage Gateway

45. What is AWS Storage Gateway?
Integrates on-prem IT environments with Cloud storage for
departmental and remote office backup and DR
Utilizes a virtual appliance that sits in customer datacenter
Exposes compatible iSCSI interface on front end
Stores primary data on-AWS in Amazon S3 or on-premise
with data backed-up to Amazon S3 as Amazon EBS
snapshots
Amazon Confidential
45

46. Solution Overview
1. Offsite Backup
2. DR
3. Data Mirroring
AWS Storage Gateway works with your existing backup
application and moves your data into Amazon S3 as EBS
snapshots
Run AWS Storage Gateway in Amazon EC2 and access
snapshots up to 32 TB in size
Run apps in the cloud using your uploaded data –
HPC/Hadoop/Analytics
4. Dept. File Share
AWS Storage Gateway – Cached Volumes will help you
create Storage Volumes in cloud and keep most recent
accessed data locally [Reduce SAN footprint for File Shares]
5. Archive/Glacier
VTL version of AWS Storage Gateway would help customer
move data in the form of Virtual Tapes from on premise to
Amazon S3 (VTL – Virtual Tape Library) and then to Amazon
Glacier (VTS – Virtual Tape Shelf)

47. IT’S ALL ABOUT
CHOICE
PERFORMANCE-ORIENTED
COST-ORIENTED

48. Understanding AWS Storage Options
Tamara Monson, iMemories, Inc.
November 13, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

50. Our Story: History
iMemories was founded in 2006
Cloud was in its infancy
–
Handling physical assets:
–
–
–
Using on-premises equipment:
Focus was digitization for web/online
viewing, DVD creation, archiving
2013
•
VCRs, decks
Film projectors
Photo and slide scanners
2012
–
–
–
2010
•
VHS, Betamax, MiniDV tapes
16mm, 8mm film reels
Photos, slides
2008
•
AWS officially launched in 2006: Amazon
S3
2006
•
•
iMemories' Growth

51. Our Story: Success
1.5PB
0.75PB
2013
2012
2010
8TB
2008
•
Growing infrastructure
Reliability not as high as desired
Less-than-ideal customer experience
Tripled full-time resources to manage
Costs for static, infrequently accessed
assets much higher than for morefrequently accessed assets
Rate of growth driving need for
infrastructure elasticity
2006
•
•
•
•
•
iMemories' Growth

52. Our Story: Need
Use Case
• Video and photo
converted and digital
source files
• Short-term local copy
• Access infrequently
– Disc creation
– Additional encodings
Requirements
• Durable, redundant,
secure
• Keep forever
• No or low operational
costs
• No new hardware
• Off-premises

53. Evaluating Options: Calculating TCO
• Operations
– Percentage FTE
– Facilities and power
– Maintenance
• Hardware
– Life span
– Number of copies
– Raw GB vs. usable GB vs.
consumed GB
1.2
1
0.8
0.6
0.4
0.2
0
Time
Raw
Usable
Consumed

54. Evaluating Options: Comparisons
Amazon S3
Amazon Glacier
Cloud Files
Cloud

55. iMemories Evolution
Architecture 1.0
• All storage and processing
on-premises (circa 2006)
• Centered on physical
assets, devices, servers
• Expensive: scale by adding
more
• Non-ideal customer
experience
Architecture 2.x
• Solve scale challenges
• Reduce costs
• Best possible customer
experience

56. Storage 1.0
• On-premises
• ~1.5 PB capacity
• Limitations of scale:
– Add new hardware
– Space and power
Storage Volume
Digitization,
Encoding
Servers
iMemories Scottsdale Data center
File Mgmt

57. Storage 2.0: Amazon Glacier
Amazon SNS
Glacier
Manager
Amazon SQS
Vaults
Regions
AWS Cloud
Storage Volume
Digitization,
Encoding
Servers
File Mgmt
iMemories Scottsdale Data Center

58. Amazon Glacier
Manager
-
Interfaces with Amazon SQS
Tracks restore/submit request
state
Processes Submit requests in
real-time
Processes Restore requests
once/hour
-
-
Uses 10 MB chunks
Performs audits 3x/week
-
-
5% of consumed / 720 hours 
peak hourly rate
Distribute request across 1-hr
windows
One discrepancy
Zero lost files on Glacier (!!!)
Generates internal activity reports
File Management
Glacier Manager
Digitization,
Encoding
Servers

59. 2.x: Move to AWS
v-ec2-east1.imemories.com
v-ec2-west2.imemories.com
S3 Manager
Streaming
Server
Elastic Load
Balancing
Streaming
Servers
Video
Files
Availability Zones
Regions
AWS Cloud
Storage
Volume
Digitization,
Encoding
Servers
File Mgmt
iMemories Scottsdale Data Center

60. 2.x: Move to AWS
Upload
Web App
Video
Files
Video
Processing
Group
u-ec2.imemories.com
API
Elastic Load
Balancing
File Mgmt
Amazon
SWF
Region
AWS Cloud
iMemories Scottsdale Data Center

61. In Closing
•
iMemories Story
–
–
–
•
Evaluating Storage Options
–
–
–
•
History
Success
Need
Define use case and solution requirements
Understand TCO
Design and build for Amazon Glacier’s 4-hr download SLA, 5% restore cap
iMemories Evolution
–
–
–
Amazon Glacier is first full production integration
Migrating more functionality to cloud
Will always have an on-premises component

62. Enabling families to share memories in endless ways.
http://www.imemories.com
tamara.monson@imemories.com

63. Please give us your feedback on this
presentation
STG101
As a thank you, we will select prize
winners daily for completed surveys!