Session 3, Presentation 2 from the AWS Sydney Summit

1. Dean Samuels
Technical lessons on how to do
Backup and Disaster Recovery in the Cloud
Solution Architect, Amazon Web Services

2. Agenda
• Prologue The need for Backup and DR
• Customer Story Ausenco Limited
• Lessons Backup and DR with AWS
• Conclusion Q&A

3. The need for Backup & DR

4. • Archiving is the process of moving data that is no longer actively used to
a separate data storage device for long-term retention. Data archives are
indexed and have search capabilities so that files and parts of files can be
easily located and retrieved.
• A backup or the process of backing up is making copies of data which
may be used to restore the original after a data loss event. The primary
purpose is to recover data after its loss, be it by data deletion or corruption.
The secondary purpose of backups is to recover data from an earlier time.
• Disaster recovery (DR) is the process, policies and procedures related to
preparing for recovery or continuation of technology infrastructure critical to
an organization after a natural or human-induced disaster.
Some Definitions

5. Where is this….Brisbane, QLD
Source: http://www.abc.net.au/news/specials/qld-floods/

6. Where is this….Brisbane, QLD - January 13th 2011
Ausenco
Source: http://www.abc.net.au/news/specials/qld-floods/

7. Milton, QLD (Lang Park)
Source: http://www.abc.net.au/news/specials/qld-floods/

8. Milton, QLD (Lang Park) – January 13th 2011
Source: http://www.abc.net.au/news/specials/qld-floods/

9. Coronation Drive, Milton, QLD
Source: http://www.abc.net.au/news/specials/qld-floods/

10. Coronation Drive, Milton, QLD – January 13th 2011
Source: http://www.abc.net.au/news/specials/qld-floods/

11. Backup, Disaster Recovery and
Data Tiering with AWS
Mike Latsky
Director, IT Operations
Ausenco

12. About Ausenco
We provide:
Engineering,
Procurement,
Construction Management and
Operational Services
to the energy, resource, and process
industries
Presentation title | DD Month Year | 12

13. IT Perspective
• Engineers generate large 2D and 3D
CAD files
• CAD performance demands close
network proximity
• Drawings are Ausenco’s deliverable
and MUST be protected

14. • ASX200 company
• Head office in Brisbane
• 29 Offices in 19 countries
• 3500 Employees
Ausenco Offices

15. Our IT Environment
• Completely virtualised
• Windows
• ERP - Oracle SaaS
• Email, Lync, SharePoint – MS Office 365 SaaS
• Onsite storage
• 2 x primary data centres
• 1 x secondary data centre

16. Our Challenge
• Backup
– Unreliable
• Disaster Recovery
– Sites were exposed
• Local Disk Storage
– At capacity
Possible Solutions
An Enterprise Backup Solution?
Secondary Data Centres?
More Disk Storage?

17. Our initial approach
• Large Integrators
• Leading backup providers
• Leading archiving providers
• Hardware providers
• Data centre hosting providers
• Leading cloud providers

18. Paradigm Shift
• Why use traditional technologies?
• How can we better leverage the AWS cloud?
• How do we do more with less?
• Address the complete data lifecycle
• Flexible, Scalable, Cost Effective
Disaster Recovery ArchivingBackup

19. Solution

20. The Result
• Low effort
• Incremental rollout
• Cloud migration
• Efficient, effective usage of bandwidth
• Application/database servers
• Regionalised deployment

21. Our Roadmap
• Wider deployment
• Decommission costly secondary data centre
• Migration of production systems to AWS

22. Thank-you

23. Backup and DR with AWS
Backup & DR Lessons

24. Business and Technical Drivers….
Reduce costs
Slash DR budgets by up to 50%
Consolidate sites
Eliminate the need to run a
secondary site
Reduce on-premise
Eliminate 30%+ of on-premise
physical equipment
Remove aging
technologies
Eliminate tape for backup and
archive

25. The fundamental economic model…
Utility, on-demand datacenter
Primary Site
Routers
Firewalls
Network
Application Licenses
Operating Systems
Hypervisor
Servers
SAN
Primary Storage
Backup
Archive
AWS
Routers
Firewalls
Network
Application Licenses
Operating Systems
Hypervisor
Servers
SAN
Snapshot Storage
Backup
Archive
Secondary
site costs

26. Backup and DR with AWS
Backup Lessons

27. Backup Lessons – My backup should be accessible
Source: http://www.abc.net.au/news/specials/qld-floods/
a.k.a. the pain
of physical
data transfer

28. AWS Direct Connect
Dedicated bandwidth between
your site and AWS
Amazon Storage Gateway
Shrink-wrapped gateway for volume
synchronization
AWS Import/Export
Physical transfer of media into and
out of AWS
Getting data into the cloud

29. Simple Storage Service
Highly scalable object storage
1 byte to 5TB in size
99.999999999% durability
Elastic Block Store
High performance block storage device
1GB to 1TB in size
Mount as drives to instances with
snapshot/cloning functionalities
Glacier
Long term object archive
Extremely low cost per gigabyte
99.999999999% durability
Storage Options
Very fast
‘instance’ disks
Slow, rare accessFast web object
storage

30. Cost
($/GB/month)
Performance Durability
Amazon S3 0.095  
Amazon Glacier 0.01  
AWS Storage Gateway
0.095
(+ 125/GW)
 
Amazon EBS 0.10  
Amazon EBS (PIOPS) 0.125  
*AWS Oregon pricing
Storage Options

31. Snapshot of
local volumes
Restoration
from snapshots
Backup Lessons – My backup should be accessible

32. VPN
Public / AWS Direct Connect
AWS Import/Export
Backup Lessons – My backup should be accessible

33. GW-Cached volumes
GW-stored volumes
“Cool” storage
“Cold”
Backup Lessons – My backup should be accessible

34. Backup Lessons – My backup should be accessible

35. • “Infinite” scale with Amazon S3 and Amazon Glacier
• Scale to multiple regions
• Seamless
• No need to provision
• Cost tiers (cheaper at scale)
Backup Lessons – My backup should be able to scale

36. • SSL Endpoints (Amazon S3 and Amazon Glacier)
• Signed API calls
• Store encrypted files
• Server-side encryption
• Multiple copies across different data centers
• Local/cloud with AWS Storage Gateway
Backup Lessons – My backup should be safe

37. • Easy to integrate within AWS or Hybrid
• AWS Storage Gateway: Run services on Amazon EC2 (DR)
• Clear costs
• Reduced costs
• I decide redundancy/availability in relation to costs
Backup Lessons – My backup should work with a DR policy

38. Backup Lessons – My backup should work with a DR policy

39. • Clear ownership
• Permissions with IAM: Users, groups  roles
• Logs
• AWS support
Backup Lessons – Someone should care about it

40. 1. My backup should be accessible
1. My backup should be able to scale
1. My backup should be safe
2. My backup should work with a DR policy
3. Someone should care about it
Backup Lessons

41. Backup and DR with AWS
DR Lessons

42. DR is part of a wider set of policies and controls…
DR & business continuity
It’s not an all or nothing thing
Choose what needs to failover and what does not
Some things more important than others
Some things will still be working
High availability Backup Disaster recovery
Keep your applications
running 24x7
Make sure your data is safe Get your applications and
data back after a major
disaster

43. Each set of IT assets will have different requirements…
DR & business continuity
Recovery Time
Objective (RTO)
How quickly you need this asset to be
recovered?
e.g. 1min? 15min? 1hr? 4hrs? 1day?
Recovery Point
Objective (RPO)
How ‘fresh’ the recovery must be for the
asset?
e.g. zero data loss, 15mins out of date?

44. Assets will sit on a spectrum of technical complexity…
DR & business continuity
Rebuild when
required from
offsite backup
Run hot-hot
configuration with
auto-failover

45. DR Lessons – You NEED a DR plan in place
DR with High Availability

46. App DR with Standby
DR Lessons – You NEED a DR plan in place

47. DR Lessons – Testing your DR
• Dev/test in the cloud is super easy
• Spin up capacity only for the test
• Regularly test your DR
• Cost is minimal
• What about data transfer speed?
s3cmd ls --recursive
s3://datasets.elasticmapreduce/ngra
ms/books/ | awk '{print $4;
sub(/s3://datasets.elasticmapredu
ce/, "/array", $4); print $4}' |
parallel -j0 -N2 --progress
/usr/bin/s3cmd --no-progress get
{1} {2}
Copying 2.4 TB
down from 48 hours
to 9 hours (5x
faster)

48. DR Lessons – Reducing Costs
• Dev/test in the cloud is super easy
• Spin up capacity only for the test
• Regularly test your DR
• Cost is minimal
• What about data transfer speed?

49. DR Lessons – You can have different DR solutions
• Easy to integrate existing vendors with DR on AWS
• Approach: One vendor/hybrid/multiple vendors
• One region/multi-regions (if you need geo-diversity)
• Different DR Architectures
Backup & Restore Pilot light
Warm standby in
AWS
Multi-site solution
in AWS & on-
premise

50. Pilot light
Stopped
instances

51. Pilot light
Running
instances

52. 1. You NEED a DR plan in place
1. Testing your DR
1. Reducing costs
2. You can have different DR solutions
DR Lessons

53. aws.amazon.com
get started on the free tier

54. Technical lessons on how to do
Backup and Disaster Recovery in the Cloud