A talk by Matthew Revell from OpenCredo. Discussing some of the problems with designing infrastructure deployments with Terraform. Sometimes the most obvious and logical approach does not yield the desired outcome. Often the nature and culture of an organisation needs to be considered when structuring infrastructure deployments.

1. Misadventures
With Terraform
Matthew Revell
Senior Consultant at OpenCredo

2. In the beginning...
Bash &
Python

3. A few years ago...
Terraform
0.6.xx
Bash &
Python

4. Into the future...
Terraform
0.6.xx
Terraform
0.12.xx
Bash &
Python

5. when modules get too small
Honey I Shrunk The Mods

6. Monolith Terraform State

7. Reusable Modules

8. Single Resource Modules

9. Complex Deployment

10. Consider whether a
single resource
module adds any
value
Consider whether
the additional
complexity is worth
the perceived value
Consider whether
the module will be
usable by the
intended
consumer(s)
Value Complexity Usability
Lessons Learned (small modules)

11. pray the code works the second time
Run Once And Forget

12. A Service

13. Two Services

14. Two Connected Services

15. FIRE!

16. Testing modules in
isolation can only
validate the
internals
Full deployment
tests are essential
to validate the
entire Terraform
structure
A dedicated
account can allow
continuous testing
without disruption
Testing Testing Testing
Lessons Learned (running code once)

17. or in this case Terraform States
How To Slice The Cake

18. Terraform States and Modules

19. Terraform States and Modules

20. Resources should
be grouped such
that states do not
grow exponentially
States should have
a limited scope to
minimise impact in
the event of
mistakes
Teams should be
able to manage
their own Terraform
independently
Scalability Blast Radius Ownership
Lessons Learned (dividing states)

21. but DRY is divine
To dir Is Human

22. terraform
+ env
+ dev
- terraform.tfvars
- backend.tf
- main.tf
+ prod
- terraform.tfvars
- backend.tf
- main.tf
Terraform Code Repo

23. variable "instance_type" {}
module "vpc" {
source = "git::https://example.com/vpc.git?ref=0.1.0"
}
module "app" {
source = "git::https://example.com/app.git?ref=0.4.20"
instance_type = "${var.instance_type}"
}
Terraform main.tf

24. instance_type = "m4.large"
Terraform terraform.tfvars

25. terraform {
backend "s3" {
bucket = "terraform-states"
key = "prod/terraform.tfstate"
}
}
Terraform backend.tf

26. terraform
+ env
+ dev
- terraform.tfvars
- backend.tf
- main.tf
+ prod
- terraform.tfvars
- backend.tf
- main.tf
Terraform Code Repo

27. ● Thin wrapper for Terraform
● Allows for easier management of backends
● Reduces amount of repeated code
● Developed by Gruntworks
Terragrunt
Image courtesy of Gruntworks Inc.

28. terraform
+ env
- common.tfvars
+ dev
- terraform.tfvars
+ prod
- terraform.tfvars
Terragrunt Code Repo

29. terragrunt = {
remote_state {
backend = "s3"
config {
bucket = "my-terraform-state"
key = "${path_relative_to_include()}/terraform.tfstate"
}
}
}
instance_type = "m4.medium"
Terragrunt common.tfvars

30. terragrunt = {
include {
path = "../common.tfvars"
}
terraform {
source = "git::https://example.com/deployment.git?ref=v0.0.1"
}
}
instance_type = "m4.xlarge"
Terragrunt terraform.tfvars

31. terraform
+ env
- common.tfvars
+ dev
- terraform.tfvars
+ prod
- terraform.tfvars
Terragrunt Code Repo

32. ● Generic templating engine
● Generate output in YAML or JSON
● Uses jsonnet, jinja2 (and more)
● Developed by Deepmind
Kapitan
Image courtesy of Deepmind (Alphabet Inc.)

33. Repeated code and
copy and pasting
will definitely lead
to mistakes
A lack of clarity and
readability will also
lead to confusion
and mistakes
Tooling can help
maintain clean code
in complex
deployments
Keep it DRY Clarity Tooling
Lessons Learned (repo structures)

34. Concluding Remarks

35. comments, or ideas
Questions