Learn about ways to archive data for compliance or cost savings, and balance retrieval speed and cost to fit your specific use case. We examine concepts such as active archiving (archive storage with fast retrieval times), compliance archiving, and many more.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deep Dive on Data Archiving in
Amazon S3 & Amazon Glacier,
With Special Guest, 20th Century Fox
H e n r y Z h a n g , S e n i o r P r o d u c t M a n a g e r, A m a z o n G l a c i e r
B i l l W a l k e r, V P C l o u d E n g i n e e r i n g , 2 0 t h C e n t u r y F o x F i l m
W h i l R e l i f o r d , V P I n f o r m a t i o n Te c h n o l o g y, 2 0 t h C e n t u r y F o x F i l m
S T G 3 0 4

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data movement
OnlineOfflin
e
Data security
and management
Complete set of data building blocks
Amazon
EFS
Amazon
EBS
AWS Snow family
AWS Storage Gateway
Family
AWS Direct Connect
Amazon EFS File Sync
Amazon S3
Transfer Acceleration
Storage Partners
Amazon Kinesis
Data Streams
Amazon Kinesis
Video Streams
Amazon
S3
Amazon
Glacier
AWS KMS
AWS IAM
AWS CloudWatch
AWS CloudTrail
AWS CloudFormation
AWS Lambda
Amazon Macie
AWS QuickSight

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Benefits of Amazon S3 & Amazon Glacier
Durable, Available, & Scalable
Security & Compliance
Query In Place
Flexible Management Ecosystem
Low-Cost

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Just Say No
X No capital investment
X No commitment
X No capacity planning
X No idle capacity
X No onerous media handling
X No complex technology refreshes
X No undifferentiated heavy lifting

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Global Availability

6. Consideration 1 – Archive with Data Lifecycle Management

7. Archiving in S3 and Glacier w/ Data Lifecycle
S3 Standard
Active data Archive dataInfrequently accessed data
S3 - Infrequent Access Amazon Glacier
Synchronous access Async accessSynchronous access
$0.023/GB/mo. $0.004/GB/mo.$0.0125/GB/mo.

8. - Transition Standard to Standard-IA
- Transition Standard-IA to Amazon Glacier
- Transition based on object tags
- Expiration and versioning
Data lifecycle management
T T+3 days T+5 days T+ 15 days T + 25 days T + 30 days T + 60 days T + 90 days T + 150
days
T + 250
days
T + 365
days
Data access frequency over time
Lifecycle Policy
Storage Class Analysis

9. Save money on storage
• 45% saving over S3 Standard
• 68% saving over S3 Standard-IA
* Assumes the highest public pricing tier

10. Transition older videos to Standard-IA

11. Glacier - Flexible Data Retrieval Options
All of your Glacier data is accessible with any of three retrieval options
Standard Retrieval
• Standard access
• 3-5 hours
• $0.01/GB
Bulk Retrieval
• Batch/Bulk access
• 5-12 hours
• $0.0025/GB
Expedited Retrieval
• Rare urgent access
• 1-5 minutes
• $0.03/GB
On-site tape replacement Off-site tape replacement

12. Consideration 2 – Durability

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
99.999999999%
Durability
Durability for long-term preservation
Built-in Fixity Checking
Automatic recovery

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Traditional Redundancy
DRPrimary

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
S3 and Glacier’s Built-in Durability
AZ1
AZ2
AZ3
 99.999999999% durability
 Objects replicated across at least three
availability zones
 AZ’s separated in power, network, flood domains
 Automated periodic data integrity checks
 Tolerant of concurrent failures
 Storage devices, nodes, racks
 Switch and network
 Datacenters
 Optional cross-region, cross-account replication

16. Cross-Region Replication with Lifecycle Archiving
S3 Bucket A
Amazon Glacier
S3 Bucket B

17. Consideration 3 – Accessibility

18. Multiple Ways to Access S3 and Glacier
1. Use S3 and Glacier via S3 Lifecycle Management
2. Direct Amazon Glacier API/SDK
3. AWS Storage Gateway
4. 3rd party tools and gateways
FastGlacier

19. Amazon Glacier – Direct access/APIs
Create
Vault
Configure
Access
Upload
Archives
Register
Archive ID
Data Upload
Initiate
Retrieval
Async
Retrieval
Completion
Completion
Notification
Download
Data
Data Retrieval

20. Amazon Glacier – Third-Party Tools and Gateways
• Consumer grade: less than $50
• Example: Cloudberry, FastGlacier, Arq (Haystack Software)
• Small / medium business: $500 - $1,000
• Example: Synology, Veeam, QNap
• Enterprise gateway and data management software
• Example: NetApp AltaVault, CommVault, StorNext,
StoreReduce, Vidispine

21. Which Option Should I Choose?
• Use S3 lifecycle managed Amazon Glacier if the S3
object keys are sufficient for index/search capability
• Use Amazon Glacier directly if you already plan to store
more metadata/indices in a database
• Use 3rd party tools to minimize coding

22. Consideration 4 – Compliance

23. Amazon Glacier Vault Lock allows you to easily
set compliance controls on individual vaults and
enforce them via a lockable policy
Time-based retention
MFA authentication
Controls govern all
records in a vault
Immutable policy
Two-step locking
Compliance Storage with Vault Lock

24. Vault Lock for Compliance Storage
• Non-overwrite, non-erasable records
• Time-based retention with “ArchiveAgeInDays” control
• Policy lockdown (strong governance)
• Legal hold with vault-level tags
• Configure optional designated third-party access and grant
temporary access

25. Amazon Glacier received a third-party assessment
from Cohasset Associates on how Amazon Glacier
with Vault Lock can be used to meet the requirements
of SEC Rule 17a-4(f) and CFTC 1.31(b)-(c).

26. Example Control: 1-year Record Retention
• Deny delete archive operation
• From anybody (root, administrators, users, business partners)
• When ArchiveAgeInDays is <= 365 days
Archive age computed from the time an archive lands in a vault

27. Example Control: 1-Year Record Retention

28. Vault Lock in the Amazon Glacier Console

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
20th Century Fox Film
Bill Walker, VP Cloud Engineering
Whil Reliford, VP Information Technology

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
20th Century Fox Film
Bill Walker, VP Cloud Engineering
Whil Reliford, VP Information Technology

32. Who is 20th Century Fox?
32
Theatrical Home Entertainment International Television Distribution
• Highest Grossing Film
• The Most Inspiring Film
• The Most Kick A** Film
• All Major OTT Platforms
• VOD / SVOD / PVOD / EST
• We Still Sell Discs!
• 195 Countries
• 15 Formats / 68 Languages
• 2.5+ Million Files / Year

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Make Early, Make Once, Use Many
Media Types

34. Previous On-Premise Distribution Model
23,000,000
Files
23,000+
B2B
Partners

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FOX Marketing Assets – “Make Once Use Many”

36. Starting Key Art

37. Global 1-Sheet Reuse

38. Theatrical Marketing Reuse to…
Living 1-Sheets
Outdoor Art
Digital Billboard
Marketing
Presentation
Trailer
Countdown Graphics
Online Banner

39. Home Entertainment
DVD / Blue Ray Labels
Publicity
Digital Billboard
Standee
Title treatment
Packaging
Electronic Sell Through

40. TV Distribution
Billboards
Layered Art
Outdoor Art
Key Art
Background

41. Global Trailer Reuse
Video assets such as Trailers and TV Spots can be dubbed and subtitled for localized distribution.

42. 13,188 Assets
510 External Users
Worldwide: $748M
Production: $200M

43. SALES MGMT RIGHTS MGMT BUSINESS MGMT
M E T A D A T A M G M T
MEDIA PROCESSINGDIGITAL ARCHIVE
• Sales pipeline
• Deal Management
• Forecasting
• Windows
• Available
• Rights
• Content catalog
• Global identifier
• Consumer/tech metadata
• Contracts
• Order fulfillment
• Customer Mgmt
• Mastering
• Localization
• Graphics & editorial
SCREENERS
PLATFORM
MARKETING/BQ
PLATFORM
• Sports
• News
• Entertainment
• Customers
• VIP / Press
• Publicity
SALES
PLATFORM
• 23K Customers
• SVOD
• Pay TV
• Basic TV
• Basic TV on
demand
• Free TV
• Free TV on
demand
PI
CONTENT SECURITY
Supported formats
across 68 languages:
• Theatrical
• Non-theatrical
• DVD/BD
• EST
• VOD
• NVOD
• PPV home
• PPV local
API
• Watermarking
• Forensic burn-in
• Piracy search
• 500K Orders • 2.4M Deliveries• Content library
• 80K titles
• Digital masters
20th Century Fox Cloud Transformation

44. Initial Storage Migration – Fox Distribution Cache
23,000
B2B
Partners
23M
Assets
8 PB
Monthly
Deliveries:
200,000+
Territories:
195+

45. Data Center – S3/Glacier Migration
STATS:
8 PB Content
23M Assets
> 60 Days
16 gbps Average
16 gbps / 15 TB
2 x 10 G
DR Asset Repository
8 PB
23M+ ASSETS
Offsite
2300 Tapes
Las
Vegas
Tape Library
Amazon Glacier Amazon S3
Los
Angeles
Production Asset Repository
8 PB
23M+ ASSETS
ON DEMAND

46. End State
REPLICATION
INGEST
100 TB/DAY
FUTURE
DR Asset
Repository
U S E A S T U S W E S T
Transcoding
Amazon Glacier Amazon S3 Amazon Glacier Amazon S3

47. 20th & 21st Century Fox Global Supply Chain
AWS & Storage Architecture
MEDIA PROCESSING MEDIA DISTRIBUTIONCONTENT LIBRARY (ARCHIVE)
Wyoming
DR Digital
Vault
(Tape)
Los Angeles
Prod Digital
Vault & WIPs
(Tape)
S3
GLACI
ER
GLACIER
Fox AWS
Media Operations
Fox AWS
Distribution Cache
S3
GLACIER
OTT Platform
Playout Platforms
FoxFast Platform
Marketing Platforms
News Platforms
B2B Platforms
Sports Platforms
Other 21CF Platforms8 PB (now)
16 PB (6 mos)
23 PB
Finished Assets 
VS Tenant #1
VS Tenant #3
VS Tenant #2
Masters 

48. Benefits of Moving to S3 and Glacier
• Hard cost savings due to move from physical storage and tape to AWS
• Reallocation of highly skilled engineering resources to higher value functions
Cost Savings
• Likelihood of corruption is significantly reduced relative to physical storage
• Diversifying risk by distributing content across multiple data centers
Durability
• In previous scenario, we needed 60 days preparation to provision storage for new use cases
• No need for outages on the environment to add storage to it
Scalability
• Ability to start copying content in days versus months
• Ability to include multiple regions, including international
Speed and Flexibility

49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
STG313 – Big Data Breakthroughs – Wed, 12:15 PM OR 7:00 PM
STG303 – Deep Dive on Amazon Glacier – Thurs, 1:45 PM
STG312 – Best Practices for Building a Data Lake in Amazon S3 & Amazon
Glacier – Thurs, 3:15 PM
Learn more…

50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
For Enterprise Storage Engineers
• Learn how to architect and
manage highly available
solutions on AWS storage
services
• Advance toward AWS
certifications
• Help your organization migrate
to the cloud faster
Online at www.aws.training
• Access 100+ new digital
training courses including
advanced training on storage
• Deep Dives on S3, EFS, and EBS
• Migrating and Tiering Storage
to AWS (Hybrid Solutions)
At re:Invent
• Visit Hands-on Labs at the
Venetian
• Attend a proctored
“Introduction to EFS” Spotlight
Lab on Thursday at 3pm at the
Venetian
• Meet Storage experts at the Ask
the Experts in Hands-on Labs
room at the Venetian
New Storage Training

51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Q&A
Amazon S3 Amazon Glacier

52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!