Ops are overwhelmed with support. Devs are mad because their cannot deploy the changes as fast as they want. Sounds familiar? Infrastructure-as-code can make your life easier by empowering developers and reducing operations' routine toil. It can cut down the lead time for infrastructure provisioning from hours or even days to minutes. This talk reviews several IaC tools and approaches, showing how to integrate them into continuous delivery pipeline. It covers the problems and challenges that engineers may face while working with infrastructure-as-code tools and provides a few hands-on recipes to address them.

1. Infrastructure-as-Code
bridging the gap between Devs and Ops
April 6th, 2019 - DevOps Fest 2019 - Kyiv, Ukraine

2. Who am I?
Mykyta Protsenko
Software developer @ Netflix
(Edge Developer Productivity)
Twitter: @mykyta_p

3. How long does it take
you to provision
infrastructure?

4. How many resources
do you need for one
microservice?

5. How many resources
do you need for one
microservice?

6. ...and how many do you
need for all of them?

7. Do it now and
automate later?

8. Too much stuff for OPS
to handle

9. Bridging the gap
Empowering devs
Ensuring safety

10. What is
infrastructure?

11. Computing
Storage
Networking

12. Everything is software

13. Writing
Testing
Maintaining

14. 12 factor apps
Codebase
Dependencies
Configuration
Backing services
Build, release, run
Processes
Port binding
Concurrency
Disposability
Dev/prod parity
Logs
Admin processes

15. 12 factor apps
Codebase
Dependencies
Configuration
Backing services
Build, release, run
Processes
Port binding
Concurrency
Disposability
Logs
Dev/prod parity
Admin processes

16. 12 factor apps Codebase
Configuration
Logs
Dev/Prod Parity

17. 12 factor apps Codebase
Configuration
Logs
Dev/Prod Parity

18. 12 factor apps Codebase
Configuration
Logs
Dev/Prod Parity

19. 12 factor apps Codebase
Configuration
Logs
Dev/Prod Parity

20. 12 factor apps Codebase
Configuration
Logs
Dev/Prod Parity

21. 12 factor apps Codebase
Configuration
Logs
Dev/Prod Parity

22. Ansible
Chef
Puppet

23. Ansible
Chef
Puppet

24. Immutable
infrastructure

25. What tools do we need?
aws elb create-load-balancer
--load-balancer-name myELB
--listeners
"Protocol=HTTP,
LoadBalancerPort=80,
InstanceProtocol=HTTP,
InstancePort=80"
--subnets subnet-15aaab61
--security-groups sg-a61988c3

26. Imperative tools?
aws elb create-load-balancer
--load-balancer-name myELB
--listeners
"Protocol=HTTP,
LoadBalancerPort=80,
InstanceProtocol=HTTP,
InstancePort=80"
--subnets subnet-15aaab61
--security-groups sg-a61988c3

27. Declarative tools FTW!

28. Cloudformation?
"MyDNSRecord": {
"Type": "AWS::Route53::RecordSet",
"Properties": { "HostedZoneName":
{"Fn::Join":
["", [{"Ref":"HostedZone"},
"."]]},
"Comment" : "DNS for inst.",
"Name" : {"Fn::Join":
["",[{"Ref":"EC2Instance"},
".",{"Ref": "AWS::Region"},
".", {"Ref:"HostedZone"},"."]]},
"Type" : "A",
"TTL" : "300",
"ResourceRecords":
[{"Fn::GetAtt" :
["EC2Instance", PublicIp"]}]}
}

30. resource "aws_route53_record" "www"
{
zone_id = "${...}"
name = "www.example.com"
type = "A"
ttl = "300"
records =
["${aws_eip.lb.public_ip}"]
}
Terraform

31. Terraform
Simple
Human-friendly

32. What else?
@RestController
public class HelloWorld {
@GetMapping("/")
public String hello() {
return "Hello World!n";
}
}
https://github.com/iac-demo

33. Terraform FTW!
resource "aws_ecs_task_definition"
"hello-world" {
...
family = "hello-world"
cpu = "256" // 0.25 vCPU
memory = "512" // 512 MB
container_definitions = <<DEF
[
{
...
"image": "helloworld:latest",
...
}
]
DEF
}
https://github.com/iac-demo

34. Terraform FTW!
resource "aws_ecs_task_definition"
"hello-world" {
...
family = "hello-world"
cpu = "256" // 0.25 vCPU
memory = "512" // 512 MB
container_definitions = <<DEF
[
{
...
"image": "helloworld:latest",
...
}
]
DEF
}
https://github.com/iac-demo

35. Terraform FTW!
resource "aws_ecs_service" "hello-world" {
cluster = "${aws_ecs_cluster.main.id}"
task_definition =
"${aws_ecs_task_definition.hello-world.arn}"
desired_count = "2"
network_configuration {
subnets = ["${aws_subnet.sb_a.id}",...]
security_groups =
["${aws_security_group.web_ecs.id}"]
}
load_balancer {
target_group_arn =
"${aws_alb_target_group.hello-world.id}"
container_name = "helloworld"
container_port = "8080"
}
...
https://github.com/iac-demo

36. Terraform FTW!
resource "aws_ecs_service" "hello-world" {
cluster = "${aws_ecs_cluster.main.id}"
task_definition =
"${aws_ecs_task_definition.hello-world.arn}"
desired_count = "2"
network_configuration {
subnets = ["${aws_subnet.sb_a.id}",...]
security_groups =
["${aws_security_group.web_ecs.id}"]
}
load_balancer {
target_group_arn =
"${aws_alb_target_group.hello-world.id}"
container_name = "helloworld"
container_port = "8080"
}
...
https://github.com/iac-demo

37. Terraform FTW!
resource "aws_ecs_service" "hello-world" {
cluster = "${aws_ecs_cluster.main.id}"
task_definition =
"${aws_ecs_task_definition.hello-world.arn}"
desired_count = "2"
network_configuration {
subnets = ["${aws_subnet.sb_a.id}",...]
security_groups =
["${aws_security_group.web_ecs.id}"]
}
load_balancer {
target_group_arn =
"${aws_alb_target_group.hello-world.id}"
container_name = "helloworld"
container_port = "8080"
}
...
https://github.com/iac-demo

38. Terraform FTW!
resource "aws_alb" "hello-world" {
name = "hello-world"
subnets = [...]
security_groups = [...]
vpc_id = "${aws_vpc.iacdemo_vpc.id}"
}
https://github.com/iac-demo

39. Core Infrastructure
vs
Project Infrastructure

40. Core Infrastructure
resource "aws_vpc" "iacdemo_vpc" {
cidr_block = "10.0.0.0/16"
enable_dns_hostnames = true
}
resource "aws_internet_gateway"
"default" {
vpc_id = "${aws_vpc.iacdemo_vpc.id}"
}
...
https://github.com/iac-demo

41. Core Infrastructure
output "vpc_id" {
value = "${aws_vpc.iacdemo_vpc.id}"
}
terraform {
backend "s3" {
key = "iacdemo.tfstate"
region = "us-west-2"
bucket = "demobucket"
}
}
https://github.com/iac-demo

42. Project Infrastructuredata "terraform_remote_state" "core" {
backend = "s3"
config {
key = "iacdemo.tfstate"
region = "us-west-2"
bucket = "demobucket"
}
}
resource "aws_alb" "hello-world" {
name = "hello-world"
...
vpc_id =
"${data.
terraform_remote_state.core.vpc_id}"
}
https://github.com/iac-demo

43. Project Infrastructuredata "terraform_remote_state" "core" {
backend = "s3"
config {
key = "iacdemo.tfstate"
region = "us-west-2"
bucket = "demobucket"
}
}
resource "aws_alb" "hello-world" {
name = "hello-world"
...
vpc_id =
"${data.
terraform_remote_state.core.vpc_id}"
}
https://github.com/iac-demo

44. https://github.com/iac-demo

46. CELEBRATE!

47. CELEBRATE?

48. State of the world
local terraform.tfstate

49. State of the world
remote S3
local terraform.tfstate

50. State of the world
terraform {
backend "s3" {
key = "iacdemo.tfstate"
region = "us-west-2"
bucket = "demobucket"
dynamodb_table =
"table_for_locking"
}
}

51. State of the world
terraform init

52. Automated Pipeline
git clone git@github.com:my/repo.git

53. Automated Pipeline
git clone git@github.com:my/repo.git
terraform init

54. Automated Pipeline
git clone git@github.com:my/repo.git
terraform init
export TF_VAR_foo="bar"
terraform plan

55. Automated Pipeline
git clone git@github.com:my/repo.git
terraform init
export TF_VAR_foo="bar"
terraform plan
terraform apply

56. COPY-PASTE

57. Encapsulation
Hiding Complexity
Reusing Code

58. Terraform modules
variable "service_name" {}
variable "docker_image" {}

59. Terraform modules
variable "service_name" {}
variable "docker_image" {}
resource "aws_ecs_task_definition"
"service" {
family = "${var.service_name}"
container_definitions = <<DEF
[{
...
"image": "${var.docker_image}",
"name": "${var.service_name}"
...
}]DEF
}

60. Terraform modulesmodule "hello_world" {
source = "./microservice_module"
service_name = "helloworld1"
docker_image= "helloworld:latest"
}

61. Terraform modulesmodule "hello_world" {
source = "./microservice_module"
service_name = "helloworld1"
docker_image= "helloworld:latest"
}
module "another_hello_world" {
source = "./microservice_module"
service_name = "helloworld2"
docker_image= "helloworld:latest"
}

62. Where do I store modules?

63. Terraform modules
Local Storage
S3
Consul
Terraform Registry
Git
… and others
module "hello_world" {
source =
"git::https://example.com/srv.git"
...
}

64. Terraform modules
Local Storage
S3
Consul
Terraform Registry
Git
… and others
module "hello_world" {
source =
"git::https://example.com/srv.git
?ref=1.0.0"
...
}

65. Terraform modulesmodule "hello_world" {
source = ...
service_name = "helloworld"
docker_image= "helloworld:latest"
}

66. Breaking Changes?

67. ...While Running 24/7?

68. Create Before Destroy
resource "aws_alb" "hello-world" {
...
lifecycle {
create_before_destroy = "true"
}
}

69. Create Before Destroy
resource "aws_alb" "hello-world" {
...
lifecycle {
create_before_destroy = "true"
}
}
resource "aws_ecs_service" "hello-world" {
...
lifecycle {
create_before_destroy = "true"
}
}

70. Green/Blue Deployments
V1

71. Green/Blue Deployments
V1
V2

72. Green/Blue Deployments
V1
V2

73. Safety
vs
Complexity/Cost

74. Life after Terraform

75. Kubernetes! And More!

76. Declarative K8S
k8s_template.yaml.sh
#!/bin/bash
cat <<YAML
apiVersion: apps/v1beta1
kind: Deployment
...
spec:
replicas: 1
template:
spec:
containers:
- name: $SERVICE_NAME
image: $DOCKER_IMAGE
imagePullPolicy: Always
ports:
- containerPort: 8090
...
YAML

77. Declarative K8S
k8s_template.yaml.sh
#!/bin/bash
cat <<YAML
apiVersion: apps/v1beta1
kind: Deployment
...
spec:
replicas: 1
template:
spec:
containers:
- name: $SERVICE_NAME
image: $DOCKER_IMAGE
imagePullPolicy: Always
ports:
- containerPort: 8090
...
YAML

78. Declarative K8S
$ export DOCKER_IMAGE=hello:latest
$ export SERVICE_NAME=helloworld
$ k8s_template.yaml.sh |
kubectl apply -f -

79. Still Need Core Infra
provider "google" {
project = "breakme-europe"
region = "europe-west1"
}
resource "google_container_cluster"
"main" {
name = "k8s-cluster"
zone = "europe-west1-d"
initial_node_count = 4
node_config {
machine_type = "n1-standard-2"
...
}
...
}

80. Fear of Changes

81. Show, Don’t Tell

82. 30-40 mins vs 3-4 mins
10x better!

83. Evolution

84. Small Changes

85. Learning Curve?

86. Show, Don’t Tell
Be Patient

87. Tooling Issues?

88. Centralize?

89. Centralize?

90. Centralize?
Vagrant?

91. Centralize?
Vagrant?
Package Repository?

92. Leverage Build System
build.gradle
buildscript {
dependencies {
classpath "com.roku:henka:1.0.0-RELEASE"
}
}
task terraformPlan(type: TerraformTask) {
description "Runs a terraform script"
tfDir = "${projectDir}/terraform"
tfAction = "plan -input=false"
terraformBaseDir = "/opt/terraform"
terraformVersion = "0.11.11"
}
// ...

93. Leverage Build System
$ ./gradlew build
$ ./gradlew terraformPlan

94. Best Practices
Unified Build Logic

95. Best Practices
Unified Build Logic
Unified Monitoring

96. Best Practices
Unified Build Logic
Unified Monitoring
Code Reviews

97. Best Practices
Unified Build Logic
Unified Monitoring
Code Reviews
CI/CD

98. Best Practices
Unified Build Logic
Unified Monitoring
Code Reviews
CI/CD
No Documentation

99. Deploy Faster!

100. Terraform https://www.terraform.io
12 factor app https://12factor.net/
Kubernetes https://kubernetes.io/
Gradle https://gradle.org/
Henka https://github.com/roku-oss/henka

102. Twitter @mykyta_p
Slides http://devopsfest2019.protsenko.com
Sources https://github.com/iac-demo