(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Screens | AWS re:Invent 2014
•
8 likes•7,011 views
Some of the best businesses today are deploying their code dozens of times a day. How? By making heavy use of automation, smart tools, and repeatable patterns to get process out of the way and keep the workflow moving. Come to this session to learn how you can do this too, using services such as AWS OpsWorks, AWS CloudFormation, Amazon Simple Workflow Service, and other tools. We'll discuss a number of different deployment patterns, and what aspects you need to focus on when working toward deployment automation yourself.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (12)
Similar to (ARC402) Deployment Automation: From Developers' Keyboards to End Users' Screens | AWS re:Invent 2014
Similar to (ARC402) Deployment Automation: From Developers' Keyboards to End Users' Screens | AWS re:Invent 2014 (20)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Screens | AWS re:Invent 2014
- 1. November 13, 2014 | Las Vegas NV Chris Munns, AWS Solutions Architect
- 4. Historically, there was no automation for developers:
- 6. Historically, development needed to be tightly controlled:
- 8. ?
- 9. QA Staging Dev Prod
- 10. QA Staging Dev Prod
- 11. QA Staging Dev ProdUnit TestsIntegration TestsUsability TestsPerformance TestsAcceptance TestsSystem TestsRegression TestsMonitoringA/B Tests
- 12. QA Staging Dev Prod
- 13. QA Staging Dev ProdUnit TestsIntegration TestsUsability TestsPerformance TestsAcceptance TestsSystem TestsRegression TestsMonitoringA/B Tests
- 14. ?
- 19. Where do we begin? https://secure.flickr.com/photos/stevendepolo/5749192025/
- 20. If you are part of an Ops / “DevOps” / “DevTools” / “CoreInfra” / whatever team that has developers as an internal customer, it’s your job to help them:
- 22. DEVQAPROD BarkerYour environments should be as similar to each other as possible!
- 23. Make the process so easy a Cavemancould do it* *provided they have the appropriate access to!
- 24. Complexity of the process isn’t necessarily bad, so long as not everyone in the organization HAS to know how the sausage is made https://secure.flickr.com/photos/erix/2657100921
- 25. https://secure.flickr.com/photos/jasoneppink/499531891Make the results of change visible to everyone who causes or deals with change!
- 28. Aim to reduce the “works on my machine” failures inherent with developing on one OS and running production on another:
- 29. Docker is really changing how applications are being built and deployed! NEW!
- 30. “A Better Dev/Test Experience: Docker and AWS” on Medium! https://medium.com/aws-activate-startup-blog/a-better-dev-test-experience- docker-and-aws-291da5ab1238 http://bit.ly/1saojKw
- 31. Dramatically lowers the complexity in running developer environments. Let’s set it up: [munns@maclaptop ~]$ vagrant init chef/centos-6.5 [munns@maclaptop ~]$ vagrant up [munns@maclaptop ~]$ vagrant ssh Last login: Fri Mar 7 16:57:20 2014 from 10.0.2.2 [vagrant@zekaih ~]$ uname -a Linux zekaih.munnsdev.com 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
- 33. •Continuous integration •Continuous deployment
- 35. Continuous deploymentalwayscontinuous integration. Continuous integration doesn’tmean that the code gets deployed to production every commit! https://secure.flickr.com/photos/isherwoodchris/6917253693
- 36. CI tools:
- 37. CI bits
- 39. Green is good!
- 41. Convenience Control Higher-level services Do it yourself AWSElastic Beanstalk AWSOpsWorks AWSCloudFormation AWS CodeDeploy AWS application/infrastructure management tools
- 42. AWS application/infrastructure management tools Convenience Control Higher-level services Do it yourself AWSElastic Beanstalk AWSOpsWorks AWSCloudFormation AWS CodeDeploy NEW!!
- 43. You’re not still configuring your servers by hand, right?
- 44. Options: Deciding factors: How are you getting the bits from your code repository to your destination environments?
- 45. Simplest of all methods. Use a deployment tool to either do a repo sync, or copy the raw files from one environment to another: Pros: –Easy to get started with –No need for midprocess packaging steps Cons: –Rollbacks could become a challenge –Harder to do at large scale AWS services can make it really easy to deploy from a repository: –AWS CodeDeploy –AWS Elastic Beanstalk –AWS OpsWorks
- 46. Example with OpsWorks: [root@saarbrucken infrahelper]# ll /srv/www/infrahelper/ total 8 lrwxrwxrwx 1 deploy apache 44 Oct 21 20:43 current -> /srv/www/infrahelper/releases/20141021204316 drwxr-xr-x 7 deploy apache 4096 Oct 21 20:43 releases drwxrwx---9 deploy apache 4096 Oct 21 20:43 shared <----------------------DEPLOY HAPPENS---------------------> [root@saarbrucken infrahelper]# ll /srv/www/infrahelper/ total 8 lrwxrwxrwx 1 deploy apache 44 Nov 7 21:44 current -> /srv/www/infrahelper/releases/20141107214310
- 47. Bundle up your code, deploy the bundle: Pros: –Very atomic solution to deploying code –Easy to track versions and changes across environments Cons: –Potentially very large deployable assets Package will be largely unique to the OS/language you are using:
- 48. Example using FPM (Effing Package Management) [munns@somehost ~]$ gem install fpm ….. [munns@somehost ~]$ git clone https://github.com/teknogeek0/ReInvent2014-InfraHelper.git …... [munns@somehost ~]$ fpm -s dir -t rpm -n "InfraHelper" -v 1.0 --epoch 1 ReInvent2014-InfraHelper/=/opt/InfraHelper Created package {:path=>"InfraHelper-1.0-1.x86_64.rpm"} [munns@somehost ~]$ rpm -ivh InfraHelper-1.0-1.x86_64.rpm ….. [munns@somehost ~]$ rpm -qa InfraHelper InfraHelper-1.0-1.x86_64 [munns@somehost ~]$ ls -l /opt/InfraHelper/ total 32 drwxr-xr-x 2 root root 4096 Nov 4 23:07 flow -rw-r--r--1 root root 156 Nov 4 23:06 Gemfile -rwxr-xr-x 1 root root 661 Nov 4 23:06 IHQueueWatcher_control.rb -rw-r--r--1 root root 2765 Nov 4 23:06 infrahelper_utils.rb …..
- 49. Generate a new AMI with your updated code on it. Launch it: Pros: –The most atomic way possible •Won’t affect any currently running instances –Can pretty easily run two versions side by side Cons: –Bit more work involved –Really have to think about data persistence –Have to think about how rollbacks would happen Bunch of tools to help you build AMIs quick and easy:
- 50. # ./packer validate webimage.json Template validated successfully. # ./packer build webimage.json amazon-ebs output will be in this color. ==> amazon-ebs: Inspecting the source AMI... ==> amazon-ebs: Creating temporary keypair: packer 5459736e-26a7-5983-db5a-df145dafa7e7 …. Build 'amazon-ebs' finished. ==> Builds finished. The artifacts of successful builds are: --> amazon-ebs: AMIs were created: us-west-2: ami-5f9bd36f # cat webimage.json { "variables": { "aws_access_key": "", "aws_secret_key": "" }, "builders": [{ "type": "amazon-ebs", "access_key": "{{user `aws_access_key`}}", "secret_key": "{{user `aws_secret_key`}}", "region": "us-west-2", "source_ami": "ami-b5a7ea85", "instance_type": ”m3.medium", "ssh_username": "ec2-user", "ami_name": "webserver {{timestamp}}", }] }
- 51. HOST METRICSSERVICEMETRICSLOG ANALYSISBUILD METRICS
- 52. AWS Marketplace and Partners • You can find, research, and buy software • Simple pricing, aligns with Amazon EC2 usage model • Launch in minutes • AWS Marketplace billing integrated into your AWS account • Can also find SaaS offerings from partners! • 1900+ products across 25 categories Learn more at: aws.amazon.com/marketplace
- 54. You’ve picked a deployment method; now how are you going to go about acting upon it? https://secure.flickr.com/photos/wscullin/3770015991
- 55. How do we go about rolling out our code? What gotchas are there?
- 56. Replace code on all of the instances without changing them or taking removing traffic: Elastic Load Balancing (ELB) Web/App instances Amazon DynamoDB MySQL Amazon RDS Instance Amazon ElastiCache Cache Node users v1 v2 Amazon Route 53
- 57. • Go through existing tier updating the application in batches: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users Amazon Route 53 v1
- 58. • Go through existing tier updating the application in batches: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users Amazon Route 53 v2 v1
- 59. • Go through existing tier updating the application in batches: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users Amazon Route 53 v2 v1
- 60. • Go through existing tier updating the application in batches: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users Amazon Route 53 v2
- 61. • Go through existing tier updating the application in batches: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users Amazon Route 53 v2
- 62. ”WebAutoScalingGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "AvailabilityZones" : { "Fn::GetAZs" : { "Ref" : "AWS::Region" } }, "LaunchConfigurationName" : { "Ref" : ”WebAutoScalingLaunchConfig" }, "MaxSize" : ”20", "MinSize" : ”10" }, "UpdatePolicy" : { "AutoScalingRollingUpdate" : { "MinInstancesInService" : “6", "MaxBatchSize" : “2", "PauseTime" : "PT5M" } } } CloudFormation—Auto Scaling with rolling updates:
- 63. ”WebAutoScalingGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "AvailabilityZones" : { "Fn::GetAZs" : { "Ref" : "AWS::Region" } }, "LaunchConfigurationName" : { "Ref" : ”WebAutoScalingLaunchConfig" }, "MaxSize" : ”20", "MinSize" : ”10" }, "UpdatePolicy" : { "AutoScalingRollingUpdate" : { "MinInstancesInService" : “6", "MaxBatchSize" : “2", "PauseTime" : "PT5M" } } } Replace 2 at a time, pause for 5 minutes before doing the next batch CloudFormation—Auto Scaling with rolling updates:
- 64. Elastic Beanstalk—batch deployments and rolling configuration updates:
- 68. We stand up a duplicate part of our infrastructure and slowly cut traffic over to it: As we shift more traffic over, let Auto Scaling grow/shrink our instances of the new or old application: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users v1 Amazon Route 53
- 69. We stand up a duplicate part of our infrastructure and slowly cut traffic over to it: As we shift more traffic over, let Auto Scaling grow/shrink our instances of the new or old application: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users Amazon Route 53 v1 ELB v2
- 70. We stand up a duplicate part of our infrastructure and slowly cut traffic over to it: As we shift more traffic over, let Auto Scaling grow/shrink our instances of the new or old application: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users v1 ELB v2 Amazon Route 53
- 71. We stand up a duplicate part of our infrastructure and slowly cut traffic over to it: As we shift more traffic over, let Auto Scaling grow/shrink our instances of the new or old application: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users v1 ELB v2 Amazon Route 53
- 72. We stand up a duplicate part of our infrastructure and slowly cut traffic over to it: As we shift more traffic over, let Auto Scaling grow/shrink our instances of the new or old application: ELB Web/App instances DynamoDB MySQL RDS Instance ElastiCache Cache Node users v2 Amazon Route 53
- 73. Some words of caution! https://secure.flickr.com/photos/e-coli/3888542890
- 74. Schema changes tied to deployments are a blocker to moving fast: Unlink this from code deploys:
- 75. Be prepared for things to go wrong! https://secure.flickr.com/photos/akyamada/4071735996
- 76. "Resources" : { "myMainSite" : { "Type" : "AWS::Route53::RecordSetGroup", "Properties" : { "HostedZoneName" : {"Ref" : "HostedZoneName"}, "Comment" : "Main site with fail whale.", "RecordSets" : [ { "Name" : {"Ref" : "DNSRecordName"}, "Type" : "A", "SetIdentifier": "Primary", "Failover": "PRIMARY", "AliasTarget" : { "HostedZoneId" : {"Ref" : "ELBHostedZoneID"}, "DNSName" : {"Ref" : "ELBDnsName"}, "EvaluateTargetHealth" : "True" } }, { "Name": {"Ref" : "DNSRecordName"}, "Type": "A", "SetIdentifier" : "Secondary", "Failover": "SECONDARY", "AliasTarget": { "HostedZoneId": { "Fn::FindInMap" : [ "S3RegionWebEndpoints", { "Ref" : "S3BucketRegion" }, "HostedZoneId" ]}, "EvaluateTargetHealth": "False", "DNSName": { "Fn::FindInMap" : [ "S3RegionWebEndpoints", { "Ref" : "S3BucketRegion" }, "DNSEndpoint" ]} } } ] } } } }
- 77. "Resources" : { "myMainSite" : { "Type" : "AWS::Route53::RecordSetGroup", "Properties" : { "HostedZoneName" : {"Ref" : "HostedZoneName"}, "Comment" : "Main site with fail whale.", "RecordSets" : [ { "Name" : {"Ref" : "DNSRecordName"}, "Type" : "A", "SetIdentifier": "Primary", "Failover": "PRIMARY", "AliasTarget" : { "HostedZoneId" : {"Ref" : "ELBHostedZoneID"}, "DNSName" : {"Ref" : "ELBDnsName"}, "EvaluateTargetHealth" : "True" } }, { "Name": {"Ref" : "DNSRecordName"}, "Type": "A", "SetIdentifier" : "Secondary", "Failover": "SECONDARY", "AliasTarget": { "HostedZoneId": { "Fn::FindInMap" : [ "S3RegionWebEndpoints", { "Ref" : "S3BucketRegion" }, "HostedZoneId" ]}, "EvaluateTargetHealth": "False", "DNSName": { "Fn::FindInMap" : [ "S3RegionWebEndpoints", { "Ref" : "S3BucketRegion" }, "DNSEndpoint" ]} } } ] } } } } Example template: http://bit.ly/113cNaa
- 78. HOST METRICSSERVICEMETRICSLOG ANALYSISEXTERNAL SITE METRICS
- 79. When things break during a deploy you’ll need to decide how to react: How do you decide? Deployment pattern and method will be deciding factors:
- 80. Have a real-time communication method for the entire company: Share knowledge:
- 82. QA Staging Dev Prod
- 84. We need robots https://secure.flickr.com/photos/spenceyc/7481166880
- 85. Good news, we’ve got lots of “robots”! https://secure.flickr.com/photos/peyri/10207629
- 86. (NEW!) (NEW!) (SOON!) (NEW!)
- 88. Code Repository
- 89. Code Repository CI Infra CI SaaS
- 90. Code Repository CI Infra CI SaaS
- 91. Code Repository CI Infra CI SaaS Code Bundler
- 92. Code Repository CI Infra CI SaaS Code Bundler Deploy Object Amazon S3 Bucket
- 93. Code Repository CI Infra CI SaaS Code Bundler Deploy Object Amazon S3 Bucket
- 94. Code Repository CI Infra CI SaaS Code Bundler Deploy Object Amazon S3 Bucket AWSOpsWorks
- 95. Code Repository CI Infra CI SaaS Code Bundler Deploy Object Amazon S3 Bucket AWSOpsWorks Dev Web/App Servers
- 96. Code Repository CI Infra CI SaaS Code Bundler Deploy Object Amazon S3 Bucket AWSOpsWorks Dev Web/App Servers Dev/ QA Users
- 97. Deploy Object Amazon S3 Bucket AWSOpsWorks Prod Web/App Servers Users Deployment Interface
- 99. ?
- 100. ✓