Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
James Andrew Vaughn (Andy) @MindTouch
Tear It Down, Build It Back Up:
Empowering Developers with
Amazon CloudFormation
James Andrew Vaughn (Andy)
• Software Architect at MindTouch
• @modethirteen on Twitter & GitHub
• Interests
• Software Bu...
@modethirteen
Agenda
• What is Amazon CloudFormation? Why use it?
• Managing your release testing and production infrastru...
@modethirteen
Why manage infrastructure
as code?
@modethirteen
@modethirteen
All of our customers host their brand on our
common, hosted infrastructure. One mistake
and all customer bra...
@modethirteen
Before CloudFormation
• Infrastructure had grown organically over years
• Hand rolled scripts with boto.py t...
@modethirteen
Weekly releases must be simple,
repeatable, non events
@modethirteen
Developers cannot be isolated from
the infrastructure where their code
will ultimately run
@modethirteen
Code gives context to problems
solved and provides audit trail for
infrastructure design
@modethirteen
Infrastructure code and server
configuration code is versioned with
application code
@modethirteen
CloudFormation: Define creation of AWS
resources (EC2 as well as Security
Groups, SQS, RDS, etc)
Puppet, Che...
@modethirteen
CloudFormation vs Terraform
• Access to nearly every AWS
resource. Better support for
VPC, Security Groups, ...
@modethirteen
CloudFormation Stacks
Main Stack
Sub Stacks
A stack is a collection of AWS resources that can be configured
@modethirteen
App Server Pool
Stack
Database
Stack
ElasticSearch
Stack
App Server Pool
Stack
Main Stack
@modethirteen
CloudFormation Stacks
Resources are things that can be queried, configured in the AWS API (including CloudFo...
@modethirteen
Database
Stack
ElasticSearch
Stack
App Server Pool
Stack
Main Stack
• AutoScaling::AutoScalingGroup
• AutoSc...
@modethirteen
Custom Resources
• CloudFormation::CustomResource
• Sends custom HTTP message (Service Token) to any of your...
@modethirteen
CloudFormation Stacks
Stack parameters come from API input, version controlled JSON templates,
or from the o...
@modethirteen
• MySQL Storage Engine
App Server Pool
Stack
Database
Stack
ElasticSearch
Stack
App Server Pool
Stack
Main S...
@modethirteen
CloudFormation Stacks
Parameters of stack can be outputted to dependent stacks. Example: IP’s,
Security Poli...
@modethirteen
Template: {…}
App Server Pool
Stack
Database
Stack
ElasticSearch
Stack
App Server Pool
Stack
Main Stack
• My...
@modethirteen
Stack Policy: Stack Update
Resource Access Control
@modethirteen
Deploying a Stack
@modethirteen
Troposphere
@modethirteen
Puppet / Chef / SaltStack / Ansible
• Stack includes an EC2 Instance or AutoScaling Group Resource
• Resourc...
@modethirteen
Execute Deployment
@modethirteen
Lessons Learned
• Goal was to put entire existing AWS infrastructure into CloudFormation, no
immediate value...
@modethirteen
Send in the Developers
@modethirteen
Approach #1 : Build your own web
console for launching test
and dev stacks
@modethirteen
Approach #2 : Every developer has
their own AWS account billed to main
AWS account
@modethirteen
Approach #3 : One developer AWS
account billed to main account
@modethirteen
The Teams
• Are developer teams responsible for their own container /
infrastructure templates, are operator...
@modethirteen
TL;DR
• Your product is application code, data, services, and servers
• CloudFormation deploys your product ...
The End. Q?
Tear It Down, Build It Back Up: Empowering Developers with Amazon CloudFormation
Prochain SlideShare
Chargement dans…5
×

Tear It Down, Build It Back Up: Empowering Developers with Amazon CloudFormation

As a product grows, and the infrastructure becomes more complex, the Operations team traditionally shoulders the burden of maintaining this infrastructure while deploying code from Software Engineers. Code is sometimes given to Operations with little to no information regarding how it should run or what the criteria for successful deployment is. This is not due to lack of caring, Software Engineers often lack the context themselves to provide production deployment instructions. To Software Engineers, production can be like a walled off city, filled with pathways and rooms not to be explored, guarded by Operations.

This presentation aims to provide a solution to this problem. We will address how the traditional separation of Operations and Software Engineers slows innovation, and redefine their relationship -- blending responsibilities. We will examine the transition of two real teams, an Operations team and Engineering team, from complete isolation, to closer environments through virtual machines, to one cloud environment shared by all and managed with CloudFormation.

  • Identifiez-vous pour voir les commentaires

Tear It Down, Build It Back Up: Empowering Developers with Amazon CloudFormation

  1. 1. James Andrew Vaughn (Andy) @MindTouch Tear It Down, Build It Back Up: Empowering Developers with Amazon CloudFormation
  2. 2. James Andrew Vaughn (Andy) • Software Architect at MindTouch • @modethirteen on Twitter & GitHub • Interests • Software Build and Testing Automation • Frontend Web Performance • Web Components & Polymer • SSO and Identity Management
  3. 3. @modethirteen Agenda • What is Amazon CloudFormation? Why use it? • Managing your release testing and production infrastructure code • Give developers the power (`cause knowledge is power!)
  4. 4. @modethirteen Why manage infrastructure as code?
  5. 5. @modethirteen
  6. 6. @modethirteen All of our customers host their brand on our common, hosted infrastructure. One mistake and all customer brands look bad #yousuck
  7. 7. @modethirteen Before CloudFormation • Infrastructure had grown organically over years • Hand rolled scripts with boto.py to create different EC2 instance types, and manual Puppet runs to configure them • Non EC2 AWS Resources managed by hand • No infrastructure in different zones or fast, programatic disaster recovery for entire infrastructure • Developers were ignorant of production infrastructure
  8. 8. @modethirteen Weekly releases must be simple, repeatable, non events
  9. 9. @modethirteen Developers cannot be isolated from the infrastructure where their code will ultimately run
  10. 10. @modethirteen Code gives context to problems solved and provides audit trail for infrastructure design
  11. 11. @modethirteen Infrastructure code and server configuration code is versioned with application code
  12. 12. @modethirteen CloudFormation: Define creation of AWS resources (EC2 as well as Security Groups, SQS, RDS, etc) Puppet, Chef, SaltStack, Ansible: Define actions that occur within EC2 instances once they’ve been provisioned
  13. 13. @modethirteen CloudFormation vs Terraform • Access to nearly every AWS resource. Better support for VPC, Security Groups, IAM, Cloudfront, SQS • Stable and mature • JSON infrastructure templates can be generated by Troposphere (with Python logic) • Vendor neutrality: AWS, OpenStack, Heroku, etc • Can execute infrastructure plans as a dry run • DSL for generating infrastructure templates (HCL) • If one resource fails to build, subsequent rebuild will only build tainted resource and those dependent on it • Open source so AWS API coverage can be improved by community Google Docs: Terraform AWS Coverage
  14. 14. @modethirteen CloudFormation Stacks Main Stack Sub Stacks A stack is a collection of AWS resources that can be configured
  15. 15. @modethirteen App Server Pool Stack Database Stack ElasticSearch Stack App Server Pool Stack Main Stack
  16. 16. @modethirteen CloudFormation Stacks Resources are things that can be queried, configured in the AWS API (including CloudFormation sub stacks). Examples: Listing S3 buckets, Adding Route 53 DNS entries, Taking DB snapshots
  17. 17. @modethirteen Database Stack ElasticSearch Stack App Server Pool Stack Main Stack • AutoScaling::AutoScalingGroup • AutoScaling::LaunchConfiguration • IAM::InstanceProfile • IAM::User • AutoScaling::AutoScalingGroup • AutoScaling::LaunchConfiguration • CloudFormation::WaitCondition • IAM::InstanceProfile • IAM::User • RDS::DBInstance • IAM::InstanceProfile • IAM::User
  18. 18. @modethirteen Custom Resources • CloudFormation::CustomResource • Sends custom HTTP message (Service Token) to any of your endpoints, and continues stack execution after response • AWS SNS • AWS Lambda • Node.JS • Your choice!
  19. 19. @modethirteen CloudFormation Stacks Stack parameters come from API input, version controlled JSON templates, or from the output of other stacks
  20. 20. @modethirteen • MySQL Storage Engine App Server Pool Stack Database Stack ElasticSearch Stack App Server Pool Stack Main Stack • ElasticSearch Version • App Server Pool EC2 Group Name • ElasticSearch EC2 Group Name • RDS MySQL IP & Port
  21. 21. @modethirteen CloudFormation Stacks Parameters of stack can be outputted to dependent stacks. Example: IP’s, Security Policies, Custom Values, etc.
  22. 22. @modethirteen Template: {…} App Server Pool Stack Database Stack ElasticSearch Stack App Server Pool Stack Main Stack • MySQL Storage Engine • ElasticSearch EC2 Group Name • RDS MySQL IP & Port • ElasticSearch Version • App Server Pool EC2 Group Name Template: {…}
  23. 23. @modethirteen Stack Policy: Stack Update Resource Access Control
  24. 24. @modethirteen Deploying a Stack
  25. 25. @modethirteen Troposphere
  26. 26. @modethirteen Puppet / Chef / SaltStack / Ansible • Stack includes an EC2 Instance or AutoScaling Group Resource • Resource includes a “UserData” metadata section, for bootstrapping an instance or group of instances • Include data that cloud-init uses to install instance configuration tool of choice • curl http://169.254.169.254/latest/user-data • Example: • cloud-init installs puppet from UserData commands • cloud-init runs puppet (configures instance and installs cfn-signal) • cfn-signal notifies CloudFormation that puppet was success or failure
  27. 27. @modethirteen Execute Deployment
  28. 28. @modethirteen Lessons Learned • Goal was to put entire existing AWS infrastructure into CloudFormation, no immediate value was attained • Difficult getting buy in for incremental improvements to infrastructure management • Existing resources cannot be migrated to CloudFormation • Know the caveats of deleting AWS Resources, they can fail a stack tear down • AWS Resources missing from CloudFormation API can be mitigated with Custom Resources • Must understand what a resource does when it updates
  29. 29. @modethirteen Send in the Developers
  30. 30. @modethirteen Approach #1 : Build your own web console for launching test and dev stacks
  31. 31. @modethirteen Approach #2 : Every developer has their own AWS account billed to main AWS account
  32. 32. @modethirteen Approach #3 : One developer AWS account billed to main account
  33. 33. @modethirteen The Teams • Are developer teams responsible for their own container / infrastructure templates, are operators part of these teams • Are developers just as responsible for troubleshooting when infrastructure goes down • What are operator obligations to developers • What are developer obligations to operations
  34. 34. @modethirteen TL;DR • Your product is application code, data, services, and servers • CloudFormation deploys your product to production • CloudFormation deploys your product for development and testing • Your developers can make better decisions • Your operators can make better decisions • Your customers / users are happy
  35. 35. The End. Q?

×