Unblocking The Main Thread Solving ANRs and Frozen Frames
Deploying secure backup on to the Cloud
1. How do we
Deploy a secure backup
to the Cloud
Lahav Savir, lahavs@emind.co
2. Lahav Savir
• 15 years in on-line industry
• Architect and CEO @ Emind Systems (est. 2006)
• AWS solution provider
• Over 30 AWS customers
Hobbies (that’s the . . .)
• MTB cycling
• Mountain hiking
3. Backup scenarios
On premises to off-site On the cloud to other site
• File servers • File servers
• Backup files • Large data volumes
• Data base dumps • Data base dumps
archiving • Large S3 buckets
• Disaster recovery
5. Requirements
Backup
• Keep a replica of the data off-site
• Keep history of the data for X previous months
• Secure transfer
• Encryption of data sets
• Large files
• Delta transfer
Deployment
• Don’t impact existing setup
• Don’t install any SW on servers
• No additional hardware
6. Few more . . .
• Control bandwidth throughput
• Visibility and monitoring
• Simplicity
• Keep the costs down
– License
– Traffic
– Storage
7. Alternatives
• Windows • Storage built-in
– Virtual drive to S3 integration tos3
– Sync application – No monitoring
– Cygwin / delta copy – No visibility to status
– No bandwidth control
• Linux – No feedback
– s3fs (fuse)
– s3cmd
9. Sync Configuration
• rsync (filer to filer)
rsync;/filer/data1/; sync@192.168.61.130:/data1/{A}
rsync;/filer/data2/; sync@porticor_vpd:/data2
• s3 (filer to s3 with / without VPD)
s3;/var/www/wordpress/;s3://bucket1/wordpress-{d}/;-
-no-delete-removed
s3;/mnt/srv1/;s3://bucket2/
10. Bandwidth control
• Tag user traffic
iptables -t mangle -A OUTPUT -m owner --uid-owner $SYNCMGR_UID -j MARK
--set-mark 0x1
• Create root qdisc for eth0
$TC qdisc add dev $IF root handle 1: htb default 30
• Add a class (bucket) with bandwidth restrictions
$TC class add dev $IF parent 1: classid 1:2 htb rate $MAXRATE
• Then add a filter to force packets through the class
$TC filter add dev $IF protocol ip parent 1:0 prio 1 handle 1 fw
classid 1:2
Tip: use iftop to see it in action
13. Hosting on the cloud
• Public cloud
– Instance behind security groups with SSH keys
• VPC
– Instance behind VPN
• AWS VPN Gateway
• IPSec with CheckPoint in the VPC
• IPSec with Swan in the VPC
• SSL VPN with OpenVPN in the VPC
14. Restoring
• rsync back from storage
rsync ; sync@192.168.61.130:/data1/{A} ; /filer/data1/
• 3scmd
s3cmd get s3://bucket2/file /path/to/restore/file
15. Summary
• Simple and open solution
• No impact on customer infrastructure
• No additional HW required
• Control with full visibility
• Fully integrated with NMS
• Reliable
• Secure
16. AWS Tips
• Don’t forget to set AWS console to MFA
• Setup a VPN to your AWS server
• No public SSH
• Monitor traffic coming into your servers
• Multi-region / AZ for high availability
• Use ec2 tools
• Backup backup backup . . .