The document discusses AWS Snowball, Snowball Edge, and Snowmobile - physical data transport solutions for migrating large amounts of data into AWS. Snowball is designed for petabyte-scale data migration, Snowball Edge provides petabyte-scale hybrid storage and compute capabilities, and Snowmobile is for exabyte-scale data migration using a 45-foot shipping container. The document provides details on their capabilities, use cases, security features, and cost. It also includes examples like how Oregon State University uses Snowball to migrate terabytes of oceanic research data and how DigitalGlobe migrated 100 petabytes of satellite imagery to AWS using Snowmobile.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Rashim Gupta, Principal Product Manager, Storage Services
March 2017
Data migration using AWS Snowball, Snowball Edge
& Snowmobile

2. Data Migration, Batch and Stream Processing
Direct
Connect
Snowball,
Snowball Edge,
Snowmobile
3rd Party
Connectors
Transfer
Acceleration
Storage
Gateway
Kinesis Firehose
File
Amazon EFS
Block
Amazon EBS
(persistent)
Object
Amazon GlacierAmazon S3 Amazon EC2
Instance Store
(ephemeral)

3. Why are we here?
Large Customer:
“We want to move 250 PBs into AWS, but it would
take years to do that over the network.
How can we upload our data faster into AWS?”

4. Why are we here?
Data is growing at a tremendous pace!
4k/8k
Genomics
Seismic
Financial
Logs
IoT
100TB over the Internet: 2 years
1PB over the internet: 22 years
100PB over T3: 609 years

5. AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale
data migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale
data migration

6. AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale
data migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale
data migration

7. The AWS data transport solution
AWS Snowball—petabyte scale data transport
• Rugged 8.5G impact case
• Rain and dust resistant
• Data encryption end-to-end
• 50 TB capacity/10G network
• E-ink label
E-ink shipping
label
RE:INVENT 2015 LAUNCH

8. Snowball features released post re:Invent 2015
• Data export
• 80 TB Snowball
• Snowball SDK
• S3 Adaptor
• HDFS import
• HIPAA eligibility

9. How is my data transported securely?
• 256-bit encryption
• Data is encrypted by the
Snowball client before it is
written to the Snowball
appliance
• Keys are managed by AWS and
are never written to the Snowball

10. How is my data transported securely?
• Strong chain of custody
• Tamper-resistant case
• Tamper-resistant electronics
(TPM)
• Each Snowball is erased
according to NIST 800-88 media
sanitization guidelines between
every job

11. How Snowball moves data into and out of AWS
Create
a job
Connect the
Snowball
Copy data to
the Snowball
Your data
moved to
Amazon S3
In transit to you Delivered to you Delivered to AWS At AWS
Job created Job completed

12. Snowball to date
SPEED
COSTLOGISTICS
SECURITY • Circled the earth 100x
• 63 total launches
• 10 AWS Regions
100x

13. When to use AWS Snowball
Cloud
Migration
Disaster
Recovery
Datacenter
Decommission
Content
Distribution

14. Customer use case: S3 as a data lake
Amazon EMR Amazon Kinesis
Amazon
Redshift
Amazon
DynamoDB
Amazon RDS
AWS Data Pipeline
Spark StreamingCassandra Storm
Amazon
S3

15. Case Study: Oregon State University
Use-case:
• Collect and analyze oceanic and coastal images
• 60TB of data per week
• Environmental and ocean ecosystem research
Architecture before Snowball:
• Transferred data with many small hard drives
• Used to take weeks to months to upload data
• $4MM+ in infrastructure investment
• Expensive and inefficient
Snowball lets OSU migrate TBs of data in days at a fraction of the cost

16. What does Snowball cost?
• Per Job shipping
- $250 / Job plus shipping (80TB)
- $200 / Job plus shipping (50TB)
• Includes 10 days to fill the device at your site
• $15/day after the tenth day on site
• Standard S3 charges apply
• $0.00/GB to transfer data in

17. AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale
data migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale
data migration

18. AWS Snowball Edge
Petabyte-scale hybrid device with onboard compute and storage
• 100 TB local storage
• Local compute equivalent to an
Amazon EC2 m4.4xlarge instance
• 10GBase-T, 10/25Gb SFP28, and
40Gb QSFP+ networking
• Ruggedized and rack-mountable
RE:INVENT 2016 LAUNCH

19. Snowball Edge key features
S3-compatible endpoint
File interface (NFS)
Clustering
Run AWS Lambda functions
Faster data transfer
Encryption

20. Hybrid capabilities beyond data migrationMIGRATION
Create job Copy data, Ship to AWS Moved to S3
Collect & Process data
PROCESSING
Create job Ship to AWS Moved to S3

21. Snowball Edge can…
Extend of your
data center
Process data Expedite
move
Encrypted, secure,
and embedded
compute
Write data directly
as data is generated
Offers a fast and
cost effective way to
ensure data can be
quickly transferred to
and from the cloud
Simplify
data transfer
Uses standard
and familiar tools
for the data
transfer process

22. Snowball Edge applicability
• Incorporate Snowball Edge into existing applications, i.e.
• Backup destination
• File/object store
• Data cluster
• …
• Build new, end to end applications with Snowball Edge
• In the manufacturing plant
• In the hospital
• In automobiles
• …

23. Snowball Edge use cases
Offline
Staging
Local Tiering
and Compute
IoT
Local
Transformation

24. Aviation
•Use Cases:
- Planes equipped with cameras
- Fly over regions to build 3D models
- Run algorithms on content for data mining
•Current solution:
- Cameras stream data to massive HDDs which are shipped to DC
•How Snowball helps?
- Migrate directly to AWS
•How Snowball Edge helps?
- Identify operational failures in trips for immediate action

25. Healthcare Industry
 Use Cases:
- Typical hospital's data repository can exceed 100s of TBs
- Expensive (time and $$$) to transfer data to AWS
- How to integrate with solutions like Philips HeathSuite to provide
healthcare professionals with data quickly
 How Snowball helps?
- Can transfer data in days instead of weeks
- Can easily scale to hundreds of hospitals
- Meets HIPAA requirements
 How Snowball Edge helps?
- While data is being stored, can we detect anomalies to instantly
raise alerts?

26. Oil & Gas
Ben Wilson (CTO of GE Oil and Gas):
“PIGs and Snowballs—a match made in heaven! AWS
Snowball 25 TB of pipeline PIG data to be managed at
AWS. That is our GE PIG we pulled some of the data
from. It’s always fun to try new AWS features and try to
break them!”
•Use Cases:
- PIG is a device to inspect cracks in pipelines using sensors
- Device fits inside pipeline and sent through to measure quality
•How Snowball helps?
- Use Snowball to migrate data to AWS

27. Security
• Use Cases:
- Biometric system uses face detection for security systems
- Replacing security guards with automated security systems
- Dozens of video cameras recording 8-10 hours/day
- Tens of thousands of videos a week
• How Snowball helps?
- When video is recorded, goes to local storage and then shipped to AWS
• How Snowball Edge helps?
- Do lot of image processing; Can use Lambda to do on premise
processing

28. Demo

29. Getting started with Snowball Edge
Step 1: Create a Job and wait for Snowball to ship to you

30. Getting started with Snowball Edge
Step 2: Power it on and connect to network
Step 3: Install the Snowball Edge Client Software
Step 4: Note the IP address on the Kindle Fire of device
Step 5: Get the Manifest file and Unlock code from AWS
Console

31. Getting started with Snowball Edge
Step 6: Unlock the device
snowballEdge unlock -i 192.168.1.141 -m C:sfloft.bin -u 906d9-
960bf-3e3f2-99aa2-79bf1
Step 7: Check the status of the Snowball
snowballEdge status -i 192.168.1.141 -m C:sfloft.bin -u 906d9-
960bf-3e3f2-99aa2-79bf1
Step 8: Get the S3 credentials
snowballEdge credentials -i 192.168.1.141 -m C:sfloft.bin -u
906d9-960bf-3e3f2-99aa2-79bf1

32. S3 operations
You can use AWS CLI for S3 commands:
• Store the S3 credentials under .aws/credentials
• aws s3 ls s3://rashimgcmh --endpoint http://192.168.1.141:8080
• aws s3 cp c:bar.txt s3://rashimgcmh --endpoint
http://192.168.1.141:8080

33. Using CloudBerry with Snowball

34. Third-party tools and gateways
•Consumer grade: less than $50
• Example: Cloudberry, Arq (Haystack Software)
•Small / medium business: $500 - $1,000
• Example: Synology, Veeam, QNap
•Enterprise grade gateway (price varies)
• Example: NetApp AltaVault, Commvault, TSM, EMC Networker,
BackupExec

35. Solve Seeding Headaches with
NetApp ® AltaVaultTM
and AWS Snowball
Primary
Storage
Cloud-Integrated
Storage
AltaVault™
3rd Party
Backup and
Recovery
Solution
Encryption
De-duplication
Compression
NetApp® AltaVault
TM
AWS Snowball

36. What does Snowball Edge cost?
• Per Job shipping
- $300 / Job plus shipping
• Includes 10 days to fill the device at your site
• $30/day after the tenth day on site
• Standard S3 charges apply
• $0.00/GB to transfer data in

37. AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale
data migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale
data migration

38. Storage migration for exabytes: AWS Snowmobile

39. What
• 10-100PB in a 45 foot-long ruggedized container truck
Where & When
• Can be made available in all AWS regions
How
• Data transferred via multiple 40Gbps interfaces up to 1Tb/s (100PB in a few weeks)
• Appears as HTTP end point
• End-to-end secure (256-bit encryption)
• Customer orders a Snowmobile, we dispatch it to their site, they hook it up and fill it, it returns
How much does it cost
• $0.005/GB/mo based on provisioned capacity (from site departure to AWS ingestion completion)

40. Snowmobile key features
Massively Scalable
Customized for your needs
Faster data transfer
Strong Encryption
Rugged, Durable, and Secure

41. Snowmobile Case Study: DigitalGlobe
Use-case: Seeing a better world
• DigitalGlobe takes Satellite imagery of the Earth
• 100PB image library = 6 billion square kilometers
• 1PB new image every year
Architecture before Snowmobile:
• Stored data in their own data center
• Needed elastic compute power to retrieve and analyze images
• Wanted to move data to the cloud, but no feasible solution
Snowmobile lets DigitalGlobe migrate 100PB of data to the cloud

42. Summary
Typical scenario Data Migration Data Migration or
Local compute
Data Migration
Storage 50TB, 80TB 100TB 100PB
Compute N/A Comparable to m4.4xlarge N/A
AWS Services available S3 Lambda, File Gateway, S3 N/A
NFS Endpoint N/A Yes Yes
Encryption Yes Yes Yes
Clustering for local use N/A Available N/A
Typical job lifetime Days - weeks Data Migration – Days-Weeks
Local compute – Weeks - Years
Weeks – months
HIPAA Yes Roadmap No
Snowball SnowmobileSnowball Edge

43. Questions?
Thank you