Managing AWS infrastructure using CloudFormation by Anton Babenko (AWS User Group Norway, 23.11.2015)

1. AWS User Group Norway
http://meetup.com/AWS-User-Group-Norway/
Managing AWS infrastructure using CloudFormation
by Anton Babenko
23.11.2015
Oslo, Norway

2. About the group
Anders BjørnestadAnton Babenko
Arne Solheim Olaf Skaug
“This is a group for people interested in Amazon Web Services. Anyone can participate, ranging from
AWS evangelists to the curious. The main focus of the group is to build up a community around AWS
with socializing and talks on topics like innovations, development and possibilities. Having trouble with
a stack? Maybe you'll meet someone with a solution or an approach that you haven't already tried.”
AWS Certified SysOps,
Sr. Software Eng at Telenor Digital
linkedin.com/in/antonbabenko
anton@antonbabenko.com
AWS Certified Architect and Developer,
Sr. Consultant at Webstep
linkedin.com/in/abjoerne
anders.bjornestad@webstep.no
Organizers:
Cloud Architect at Nordcloud Ltd
linkedin.com/in/olafskaug
olaf.skaug@nordcloud.com
CTO at Nordeca Insight
linkedin.com/in/arnesolheim
arne.solheim@nordeca.com

3. Today’s
meeting
1) News from AWS
2) Anton Babenko - Managing AWS
infrastructure using CloudFormation
3) Arne Solheim - Cloudfront
Pizza and drinks sponsored by:
23NOV2015

5. Agenda:
Basics of CloudFormation
Use cases
Best practices
Challenges and limitations

6. AWS
CloudFormation
“... an easy way to create and manage a
collection of related AWS resources,
provisioning and updating them in an orderly
and predictable fashion.”
https://aws.amazon.com/cloudformation/

8. {
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "JSON string",
"Metadata" : {
template metadata
},
"Parameters" : {
set of parameters
},
"Mappings" : {
set of mappings
},
"Conditions" : {
set of conditions
},
"Resources" : {
set of resources - REQUIRED
},
"Outputs" : {
set of outputs
}
}
Template anatomy

9. ● Collection of AWS resources
● Supports:
○ Create
○ Update
○ Describe
○ Delete
Stack

10. Manage resources
Bootstrapping of instances
Application deployment
Use cases

11. Manage AWS resources
Initial bootstrapping of instances
Use Ansible, Puppet, Chef or AWS OpsWorks for more real things
Application deployment
Consider using AWS alternatives also (CodeDeploy, Elastic Beanstalk, ECS)
Use cases

12. Use generator tool (JSON is not for humans)
Python: https://github.com/cloudtools/troposphere
Ruby: https://github.com/sparkleformation/sparkle_formation
Ruby: https://github.com/tongueroo/lono
Scala: https://github.com/MonsantoCo/cloudformation-template-generator
Manage AWS resources

13. Use generator tools - user-data escaping hell
Ruby example 2:
:UserData =>
base64(interpolate(file(
'userdata.sh')))
Ruby example 1 (erb template):
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
<%=
user_data('userdata.sh.erb
') %>
]
]
}
Cloudformation JSON example 1:
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"", ["#!/bin/bash -
ex","n",
"yum update -y","n",
"# here is my long shell
script. Imagine how long it can
be :)"]
]
}
}
Cloudformation JSON example 2:
"UserData": "here is long &
base64 encoded string. Imagine
how long it can be :)"

14. Use Cloudformer to create templates based on existing AWS resources
CloudFormer: https://aws.amazon.com/developertools/6460180344805680
I have created AWS resources manually

18. CloudFormation Designer view of CloudFormer stack
Template: https://s3.amazonaws.com/cloudformation-templates-eu-west-1/CloudFormer.template

19. 3 ways to organize stacks

20. Fit small deployments
Small size limit (50kb when load from local file; 450kb when load from S3)
1.Stand-alone stacks

21. Easy execution - aws-cli
Allow to put reusable parts into separate stacks
One fails = all fails = all rollback
Pass parameters to the internal stacks through parent
Reference to resources in parent: { "Fn::GetAtt" : [ "myVPCStack", "Outputs.VPCId" ] }
2.Nested stacks

22. Manual execution one after another
Pass outputs as inputs to downstream template
Pass parameters to each stack directly
Independent failure = independent rollback
Can’t use WaitContitions with other stacks
3.Pipelined stacks

23. Continuous Integration: Infrastructure & Application
Infrastructure:
● Parametrize everything
● Validate templates (AWS CLI)
● Version templates
● Probably run on separate AWS account first (CI)
Application deployment:
● EC2 cloud-init + new application version = new deploy

24. CloudFormation challenges
● Can’t import already created resources without deleting them first
● Not all AWS resources/features/services are supported by CloudFormation (eg, EC2 keypairs)
● No officially supported CloudFormation generator available
● No way to see what kind of changes are going to be applied
● Failed state… what to do ?
○ Do not update resources created by CloudFormation manually

25. CloudFormation limitation
● JSON format is not very human-friendly
● No iteration and limited conditional support (and/or/not/equal)
● Limited ability to adjust stack based on dynamic conditions
● Managing dependencies between templates
○ Tieing together inputs/outputs is not directly supported

26. Summary
● Use JSON generators
● Keep templates maintainable and single purposed
● Probably start with stand-alone stack and iterate
● Decide how you can handle failures (rollback just one stack or all)
● Integrate with CI similar way how you do with your application
● Use CloudFormation for very primitive application deployments

27. Available templates: https://aws.amazon.com/cloudformation/aws-cloudformation-
templates/
CloudFormation Masterclass:
http://www.slideshare.net/AmazonWebServices/aws-cloudformation-masterclass
(DVO304) AWS CloudFormation Best Practices:
http://www.slideshare.net/AmazonWebServices/dvo304-aws-cloudformation-best-
practices
Links

28. Questions ?

29. Thank you!
See you at DevOps Norway meetup 14th of December 2015:
Manage AWS infrastructure (as code) using Terraform
http://www.meetup.com/DevOps-Norway/events/226820193/