Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Data Transfer Services
Data Ingest Strategie...
Storage is the gravity
for cloud applications
Amazon EFS
File
Amazon EBS
Amazon EC2
Instance Store
Block
Amazon S3 Amazon Glacier
Object
Data Transfer
AWS Direct
Connec...
Internet/VPN ingest
What is Internet/VPN?
Globally available
Default method of ingesting content into Amazon S3
Simple standards-based (HTTP) ...
How does Internet/VPN ingest work?
Accelerate data transfer using
multipart upload
Ingest data directly into S3 buckets
wi...
Amazon S3 transfer acceleration
What is Amazon S3 transfer acceleration?
Network- and protocol-based data transfer service
Acceleration of data ingress/eg...
Ingest & Egress with S3 transfer acceleration
S3 bucket
AWS edge
location
Uploader
Optimized
throughput!
Uses AWS 55 globa...
Amazon
Route 53
Resolve
b1.s3-accelerate.amazonaws.com
HTTPS PUT/POST
upload_files.zip
HTTP/S PUT/POST
“upload_files.zip”
...
Using the service is as easy as 1, 2, 3…
Update application to point to new S3 URL
• Update “bucket.s3.amazonaws.com” to
“...
Rio De
Janeiro
Warsaw New York Atlanta Madrid Virginia Melbourne Paris Los
Angeles
Seattle Tokyo Singapore
Time[hrs]
500 G...
How much will it help me?
Pricing*
Dimension Price/GB
Data transfer in from Internet** $0.04 (Edge location in US, EU, JP)
$0.08 (Edge location in r...
Amazon CloudFront
What is Amazon CloudFront?
Global content delivery network
55 edge locations worldwide
Supports ingest through PUT/POST me...
Using CloudFront to ingest data into AWS
AWS region
Customer client
HTTP/S PUT/POST
“upload_files.zip”
Amazon EC2
S3 bucke...
Amazon CloudFront pricing
Data transfer out of Amazon CloudFront to your origin server billed at the rates
listed in the R...
AWS Direct Connect
What is AWS Direct Connect?
Dedicated, 1 or 10 GE private pipes into AWS
Create private (VPC) or public virtual interfaces...
Physical connection
• Cross-connect at the location
• Single-mode optical fiber
- 1000Base-LX or 10GBASE-LR
• Potential on...
At the Direct Connect location
CORP
AWS Direct
Connect
Routers
Customer
Router
Colocation
DX Location
Customer
network
`
A...
Dedicated port through Direct Connect partner
CORP
AWS Direct
Connect
Routers
Colocation
DX Location
Partner network
AWS b...
Direct Connect Locations
AWS Region AWS Direct Connect Location
Asia Pacific (Singapore) Equinix SG2, GPX, Mumbai
Asia Pac...
Amazon Kinesis Firehose
Amazon Kinesis
Streams
• For technical developers
• Build your own custom
applications that process
or analyze streaming
d...
Amazon Kinesis Firehose
Load massive volumes of streaming data into Amazon S3, Amazon
Redshift and Amazon Elasticsearch Se...
Vertical/use case Accelerated ingest-
load to final destination for analytics
Ad tech/
marketing analytics
Advertising dat...
AWS Storage Gateway
What is AWS Storage Gateway?
Works with your existing applications
Secure and durable storage in AWS
Low latency for frequ...
Common uses for AWS Storage Gateway
Backup and archive
Disaster recovery
Data migration
How does AWS Storage Gateway work?
Amazon EBS
snapshots
Amazon S3
Amazon
Glacier
AWS
Storage Gateway
appliance
Application...
AWS Storage Gateway configurations
iSCSI block storage
Gateway-stored volumes
iSCSI virtual tape storage
Low latency for a...
Gateway-virtual tape library (VTL)
• Replace or augment your aging tape infrastructure with durable object
storage
• Virtu...
AWS Snowball
What is AWS Snowball?
Petabyte-scale data transport
E-ink shipping
label
Ruggedized case
8.5G impact
All data encrypted
en...
How it works
How fast is Snowball?
• Less than 1 day to transfer 200 TB through 3 x 10 GB connections
with 3 Snowballs, less than 1 wee...
Use cases: AWS Snowball
Cloud
migration
Disaster
recovery
Data center
decommissioning
Content
distribution
Pricing
Dimension Price
Usage charge per job $200.00 (50 TB)
$250.00 (80 TB)
Extra day charge (first 10 days* are free) $1...
AWS technology partnerships
Amazon storage partner ecosystem
Gateway/NAS
Data
management
Sync and shareBackup/DR
Content and
acceleration
Archive
File...
Backup to AWS approaches
Amazon S3
Amazon
Glacier
AWS
Direct
Connect
Internet
Amazon S3-IA
Application
servers
Cloud gatew...
CommVault ties together on-premises and cloud-data
strategies
Commvault orchestrates the enterprise
• Back up in the cloud...
Backup to AWS approaches
Amazon S3
Amazon
Glacier
AWS
Direct
Connect
Internet
Amazon S3-IA
Application
servers
Cloud gatew...
NetApp AltaVault backup from on-
premises to S3/Amazon Glacier
Common backup applications integrated with AltaVaultSolve b...
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
unified file services that extend
from endpoints...
Summary: When to use each service
IF YOU NEED: CONSIDER:
An optimized or replacement Internet connection to:
connect direc...
Thank You!
AWS Data Transfer Services: Data Ingest Strategies Into the AWS Cloud
Prochain SlideShare
Chargement dans…5
×

AWS Data Transfer Services: Data Ingest Strategies Into the AWS Cloud

6 042 vues

Publié le

Different types and sizes of data require different strategies. In this session, learn about the various features and services available for migrating data, be it small ongoing transactional data or large multi-petabyte volumes. Come learn how customers are using the latest network, streaming and large scale ingest features for their cloud data migrations to AWS storage services.

Publié dans : Technologie
  • Real Ways To Make Money, Most online opportunities are nothing but total scams! ♣♣♣ http://scamcb.com/ezpayjobs/pdf
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

AWS Data Transfer Services: Data Ingest Strategies Into the AWS Cloud

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Data Transfer Services Data Ingest Strategies into the AWS Cloud Erik Durand Principal, Business Development
  2. 2. Storage is the gravity for cloud applications
  3. 3. Amazon EFS File Amazon EBS Amazon EC2 Instance Store Block Amazon S3 Amazon Glacier Object Data Transfer AWS Direct Connect ISV connectors Amazon Kinesis Firehose Storage Gateway S3 transfer acceleration AWS storage is a platform AWS Snowball Amazon CloudFront Internet/VPN
  4. 4. Internet/VPN ingest
  5. 5. What is Internet/VPN? Globally available Default method of ingesting content into Amazon S3 Simple standards-based (HTTP) connection Use your existing internet connection Available in a VPC for VPN connectivity Acceleration through multipart upload Data transfer into AWS is free VPN connections using VPC virtual private gateway •$0.05 per VPN connection-hour •$0.048 per VPN connection-hour for connections to the Tokyo region
  6. 6. How does Internet/VPN ingest work? Accelerate data transfer using multipart upload Ingest data directly into S3 buckets with existing internet connectivity S3 bucket AWS Region and through console or API customer gateway endpoints VPN connection Internet Internet through VPN + VPC
  7. 7. Amazon S3 transfer acceleration
  8. 8. What is Amazon S3 transfer acceleration? Network- and protocol-based data transfer service Acceleration of data ingress/egress with S3 buckets Typically 50% to 300% faster Feature of S3 enabled at the bucket level Available in all S3 regions worldwide No client/server software required No code changes to your application No firewall exceptions Simple pricing model
  9. 9. Ingest & Egress with S3 transfer acceleration S3 bucket AWS edge location Uploader Optimized throughput! Uses AWS 55 global edge locations AWS determines best edge location Data transfer optimized between edge and customer, and edge and S3 Data is not stored on the edge cache
  10. 10. Amazon Route 53 Resolve b1.s3-accelerate.amazonaws.com HTTPS PUT/POST upload_files.zip HTTP/S PUT/POST “upload_files.zip” Service traffic flow Client to S3 bucket example S3 bucket b1.s3-accelerate.amazonaws.com EC2 proxy AWS region AWS edge location Customer client 1 2 3 4 Data is not cached on the AWS edge location Fully managed file transfer acceleration using all AWS edge locations
  11. 11. Using the service is as easy as 1, 2, 3… Update application to point to new S3 URL • Update “bucket.s3.amazonaws.com” to “<bucket-name>.s3-accelerate.amazonaws.com” • Original bucket location and contents are the same, only namespace changes Or use permissions through API s3:PutAccelerateConfiguration Enable the service in the AWS Management Console Start uploading data to Amazon S3 1 2 3
  12. 12. Rio De Janeiro Warsaw New York Atlanta Madrid Virginia Melbourne Paris Los Angeles Seattle Tokyo Singapore Time[hrs] 500 GB upload from these edge locations to a bucket in Singapore Public internet How fast is S3 transfer acceleration? S3 transfer acceleration
  13. 13. How much will it help me?
  14. 14. Pricing* Dimension Price/GB Data transfer in from Internet** $0.04 (Edge location in US, EU, JP) $0.08 (Edge location in rest of the world) Data transfer out to Internet $0.04 Data transfer out to another AWS region $0.04 Amazon S3 charges Standard data transfer charges apply *Plus standard Amazon S3 data transfer charges apply **Accelerated performance or there is no bandwidth charge
  15. 15. Amazon CloudFront
  16. 16. What is Amazon CloudFront? Global content delivery network 55 edge locations worldwide Supports ingest through PUT/POST methods Works with S3 multipart upload Supports SSL SNI and TLS connections Integrated with ACM and AWS WAF for additional security Proxy ingest to S3, EC2 and even your own origins Tiered and custom pricing models
  17. 17. Using CloudFront to ingest data into AWS AWS region Customer client HTTP/S PUT/POST “upload_files.zip” Amazon EC2 S3 bucket ELB Custom origin CloudFront edge location Ingest content into S3, EC2, ELB, or your own custom origin with Amazon CloudFront Use cache behaviors to direct to the correct origin based on PATH pattern matching Restrict access through geo restriction or AWS WAF Web ACL
  18. 18. Amazon CloudFront pricing Data transfer out of Amazon CloudFront to your origin server billed at the rates listed in the Regional Data Transfer Out to Origin (per GB) table. Data transfer out of Amazon CloudFront to Internet will be charged at rates listed in the Regional Data Transfer Out to Internet (per GB) table. Amazon CloudFront offers additional pricing options through a CloudFront Reserved Capacity (CFRC) contract. Contact sales for additional details and pricing.
  19. 19. AWS Direct Connect
  20. 20. What is AWS Direct Connect? Dedicated, 1 or 10 GE private pipes into AWS Create private (VPC) or public virtual interfaces to AWS Reduced data-out rates (data-in still free) Consistent network performance At least 1 location to each AWS region Option for redundant connections Uses BGP to exchange routing information over a VLAN
  21. 21. Physical connection • Cross-connect at the location • Single-mode optical fiber - 1000Base-LX or 10GBASE-LR • Potential onward delivery through Direct Connect partner • Customer router
  22. 22. At the Direct Connect location CORP AWS Direct Connect Routers Customer Router Colocation DX Location Customer network ` AWS backbone network Cross- connect Customer router Customer’s network Demarcation
  23. 23. Dedicated port through Direct Connect partner CORP AWS Direct Connect Routers Colocation DX Location Partner network AWS backbone network Cross- connect Customer router Partner network Access circuit Demarcation Partner equipment
  24. 24. Direct Connect Locations AWS Region AWS Direct Connect Location Asia Pacific (Singapore) Equinix SG2, GPX, Mumbai Asia Pacific (Seoul) KINX, Seoul Asia Pacific (Sydney) Equinix SY3, Global Switch Asia Pacific (Tokyo) Equinix OS1, Equinix TY2 China (Beijing) Sinnet JiuXianqiao IDC, CIDS Jiachuang IDC EU (Frankfurt) Equinix FR5, Interxion Frankfurt EU (Ireland) TelecityGroup, London Docklands, Eircom Clonshaugh Equinix LD4 - LD6, London South America (Sao Paulo) Terremark NAP do Brasil, Tivit US East (Virginia) CoreSite NY1 & NY2, Equinix DC1 - DC6 & DC10 US West (Northern California) CoreSite One Wilshire & 900 North Alameda, CA, Equinix SV1 & SV5 US West (Oregon) Equinix SE2 & SE3, Switch SUPERNAP, Las Vegas AWS GovCloud (US) Equinix SV1 & SV5
  25. 25. Amazon Kinesis Firehose
  26. 26. Amazon Kinesis Streams • For technical developers • Build your own custom applications that process or analyze streaming data Amazon Kinesis Firehose • For all developers, data scientists • Easily load massive volumes of streaming data into Amazon S3, Amazon Redshift and Amazon Elasticsearch Service Amazon Kinesis Analytics • For all developers, data scientists • Easily analyze data streams using standard SQL queries • Coming soon Amazon Kinesis: Streaming data made easy Services make it easy to capture, deliver, and process streams on AWS
  27. 27. Amazon Kinesis Firehose Load massive volumes of streaming data into Amazon S3, Amazon Redshift and Amazon Elasticsearch Service Zero administration: Capture and deliver streaming data into Amazon S3, Amazon Redshift, and other destinations without writing an application or managing infrastructure. Direct-to-data store integration: Batch, compress, and encrypt streaming data for delivery into data destinations in as little as 60 seconds using simple configurations. Seamless elasticity: Seamlessly scales to match data throughput without intervention. Capture and submit streaming data to Firehose Analyze streaming data using your favorite BI tools Firehose loads streaming data continuously into S3, Amazon Redshift and Amazon Elasticsearch
  28. 28. Vertical/use case Accelerated ingest- load to final destination for analytics Ad tech/ marketing analytics Advertising data aggregation Consumer online/gaming Online customer engagement data aggregation Financial services Market/financial transaction order data collection IoT/sensor data Fitness device, vehicle sensor, telemetry data ingestion Amazon Kinesis Firehose use cases
  29. 29. AWS Storage Gateway
  30. 30. What is AWS Storage Gateway? Works with your existing applications Secure and durable storage in AWS Low latency for frequently used data Scalable and cost-effective on-premises storage - $125 per gateway per month + S3/Amazon Glacier storage fees Service connecting an on-premises software appliance with cloud-based storage
  31. 31. Common uses for AWS Storage Gateway Backup and archive Disaster recovery Data migration
  32. 32. How does AWS Storage Gateway work? Amazon EBS snapshots Amazon S3 Amazon Glacier AWS Storage Gateway appliance Application server AWS Storage Gateway back end Customer premises S3 transfer acceleration AWS Direct Connect Internet
  33. 33. AWS Storage Gateway configurations iSCSI block storage Gateway-stored volumes iSCSI virtual tape storage Low latency for all your data with point-in-time backups to AWS Replacement for on-premises physical tape infrastructure for backup and archive Gateway-cached volumes Gateway virtual tape library (VTL) Low latency for frequently used data with all data stored in AWS
  34. 34. Gateway-virtual tape library (VTL) • Replace or augment your aging tape infrastructure with durable object storage • Virtual tapes stored in AWS, frequently accessed data cached on-premises • Up to 1,500 tapes, up to 2.5 TB each, for up to 150 TB per gateway-VTL • Unlimited number of tapes in virtual tape shelf (VTS) Customer data center VTS storage backed by Amazon Glacier AWS Storage Gateway VM Backup server INITIATOR AWS Storage Gateway service MEDIA CHANGER Upload buffer Cache storage Gateway-VTL storage backed by Amazon S3 VT S TAPE DRIVE
  35. 35. AWS Snowball
  36. 36. What is AWS Snowball? Petabyte-scale data transport E-ink shipping label Ruggedized case 8.5G impact All data encrypted end-to-end Rain- and dust- resistant Tamper-resistant case and electronics 80 TB 10 GE network
  37. 37. How it works
  38. 38. How fast is Snowball? • Less than 1 day to transfer 200 TB through 3 x 10 GB connections with 3 Snowballs, less than 1 week, including shipping • Number of days to transfer 200 TB through the Internet at typical utilizations Internet connection speed Utilization 1 Gbps 500 Mbps 300 Mbps 150 Mbps 25% 71 141 236 471 50% 36 71 118 236 75% 24 47 225 157
  39. 39. Use cases: AWS Snowball Cloud migration Disaster recovery Data center decommissioning Content distribution
  40. 40. Pricing Dimension Price Usage charge per job $200.00 (50 TB) $250.00 (80 TB) Extra day charge (first 10 days* are free) $15.00 Data transfer in $0.00/GB Data transfer out $0.03/GB Shipping** Varies Amazon S3 charges Standard storage and request fees apply * Starts one day after the appliance is delivered to you. The first day the appliance is received at your site and the last day the appliance is shipped are also free and not included in the 10-day free usage time. ** Shipping charges are based on your shipment destination and the shipping option (e.g., overnight, 2-day) you choose.
  41. 41. AWS technology partnerships
  42. 42. Amazon storage partner ecosystem Gateway/NAS Data management Sync and shareBackup/DR Content and acceleration Archive File System
  43. 43. Backup to AWS approaches Amazon S3 Amazon Glacier AWS Direct Connect Internet Amazon S3-IA Application servers Cloud gateway Local disk Media server Cloud gateway Application servers Backup SW cloud connector Local disk Media server with cloud connector
  44. 44. CommVault ties together on-premises and cloud-data strategies Commvault orchestrates the enterprise • Back up in the cloud: Keep backups of cloud workloads internal to the cloud • Back up to the cloud: Allow on-premises workloads the ability to leverage AWS • Disaster recovery to the cloud: Automate disaster recovery to the cloud on a scheduled basis • Workload portability: Rest assured that virtual servers can be moved from on- premises to the cloud and back, keep your data available wherever you need it • Archiving to the cloud: Moving legacy data to tier 2 storage in the cloud for long term archive Together, AWS and Commvault minimize networking, storage, and infrastructure costs while providing the business a sound data protection and disaster-recovery strategy.
  45. 45. Backup to AWS approaches Amazon S3 Amazon Glacier AWS Direct Connect Internet Amazon S3-IA Application servers Cloud gateway Local disk Media server Cloud gateway Application servers Backup SW cloud connector Local disk Media server with cloud connector
  46. 46. NetApp AltaVault backup from on- premises to S3/Amazon Glacier Common backup applications integrated with AltaVaultSolve backup and archive headaches with cloud-integrated storage  90% reduction in time, cost, and data volumes  Shrink recovery times from days to minutes  85% of backup and software providers supported Amazon Glacier On-premises AWS Cloud-integrated storage appliance NetApp AltaVault FAS E-series Non-NetApp storage  NetApp SnapProtect  Arcserve  CommVault Simpana  EMC NetWorker  HP Data Protector  IBM Tivoli Storage Manager  Symantec Backup Exec  Symantec NetBackup  Veeam  Microsoft SQL Server  Oracle RMAN S3 AltaVault also available on Marketplace to protect cloud-native workloads Seamlessly integrates into existing storage and backup software environment Caches recent backups locally, vaults older copies to the cloud Store data in the public or private cloud of choice Deduplicates, compresses, and encrypts
  47. 47. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. unified file services that extend from endpoints, to remote offices, to the cloud snapshots, file versioning, and file sync runs across all access points via the cloud data is secured and optimized at the source all stored in your AWS VPC, data is stored on AWS S3-IA Integrated with trusted enterprise security and management tools ROBO NAS gateways Endpoint apps Cloud server agents Data protection engine File sync engine with centralized automation, management and multi-tenancy identity management data governance cloud orchestration S3 Infrequent Access CTERA global deduplication Ctera: Enterprise file services platform
  48. 48. Summary: When to use each service IF YOU NEED: CONSIDER: An optimized or replacement Internet connection to: connect directly into an AWS regional data center Direct Connect migrate TB or PB of data to the cloud Snowball accelerate data transfer S3 transfer acceleration, CloudFront, AWS partner A friendly interface into S3 to: cache data locally in a hybrid model (for performance reasons) Storage Gateway, AWS partner redirect backups or archives with minimal disruption Storage Gateway, AWS partner aggregate data streams from multiple devices Kinesis Firehose
  49. 49. Thank You!

×