Terraform is a tool for building infrastructure in the cloud. When you start using Terraform it can be confusing, you might fall into issues and see yourself trapped into a problem which can result in an infrastructure not fully automated or error prone. This talk will show you best practices and tricks I have learned while building a Kubernetes infrastructure in AWS using Terraform.
3. Agenda
● What is terraform
● Terraform 0.12
Insights on:
● Terraform Modules
● Terraform State
● Naming & Tagging
● Data
● Null Resources
● Local Files & Templating
● Kubernetes with Terraform
● Running Terraform & Pipelines
4. What is Terraform?
Provider: Use AWS
Resource: Create role
Resource:
Attach other
policies
Data: Build policy
https://learn.hashicorp.com/
terraform
https://events.hashicorp.co
m/workshops/amsterdam/
5. Terraform 0.12
● If you are starting to use terraform, use terraform 0.12
● All of the code in this presentation is 0.11
● All best practices in this presentation are terraform 0.12 compatible
Tip #1 - Migrating to 0.12
Use tfswitch to have multiple terraform
versions
6. Agenda
● What is terraform
● Terraform 0.12
Insights on:
● Terraform Modules
● Terraform State
● Naming & Tagging
● Data
● Null Resources
● Local Files & Templating
● Kubernetes with Terraform
● Running Terraform & Pipelines
8. I used to write my own modules all
the time…
Module Writing
9. Modules
Tip #2 - Open Source Modules
Use Open Source modules! Help the community!
https://github.com/terraform-aws-modules
https://github.com/cloudposse
Tip #3 - Build modules & divide to conquer
Use Open Source modules + Your business rules
10. Module Source in Terraform
Repo Module Folder
Version
Local Path Module
11. Modules - Git Repository
● Allow re-use across different
terraform repositories
● Allow versioning
● Versioning allow better
development
Versioning will make your workflow
more complex
12. Modules - One repository for each module
Complexity will increase even more
because you will have thousands of
repositories to manage version & etc
13. Modules - Git Repository
Tip #4 - Use separate repository for modules
Use separate Git Repository for your modules if they are used across
multiple repositories or if you need versioning
Tip #5 - Avoid creating one repository for module
Use one repository with all your modules instead!
14. Modules - Securing Inputs & Outputs
Tip #6 - Secure Variables: Use CLI for passwords
Use var_file or TF_VAR
Tip #7 - Secure Outputs
If Terraform generates password use the sensitive option in your
outputs or a secret manager (Vault, etc…)
15. Document your modules with terraform-docs
Tip #8 - Tool to document your modules
It will generate documentation based in the description attribute
from inputs and outputs
https://github.com/segmentio/terraform-docs
16. Module Order & Dependency
● Terraform executes everything in parallel
● Order execution is done by using one module as input to other module.
This will tell Terraform to first run
run the bucket principal and then
run hosting_bucket
17. Dependency Drama
Troubled Dev: What if there’s no way to link 2
modules? What if only order does the trick?
https://github.com/hashicorp/terraform/issues/10462
18. Dependency Problem
Tip #9 - Use terragrunt to manage dependencies
You will see terragrunt be used to solve issues that terraform does
not during this presentation. More spoilers on next slides!
https://github.com/gruntwork-io/terragrunt
Before running this, run
the kms_role
19. Module Recap
● Avoid hassle: Use Open Source Modules
https://github.com/terraform-aws-modules
https://github.com/cloudposse
● Make your code in modules & divide to conquer
● Use a separate git repository for your modules to have versioning and more control
● Avoid creating one repository to each module to help your sanity
● Document your modules with https://github.com/segmentio/terraform-docs
● Use Terragrunt for module dependency control (more in next slides)
20. Agenda
● What is terraform
● Terraform 0.12
Insights on:
● Terraform Modules
● Terraform State
● Naming & Tagging
● Data
● Null Resources
● Local Files & Templating
● Kubernetes with Terraform
● Running Terraform & Pipelines
22. What is a Terraform State?
Terraform must store state. This state is used by Terraform to map real world resources
to your configuration, keep track of metadata, and to improve performance for large
infrastructures.
Terraform state is a JSON that represents your infrastructure
23. Important State Requirements
● Keep your state secure
● Remote
● Locking Mechanism
● State per environments
● No manual creation of state
25. Terraform State
Tip #10 - Use terragrunt to manage your state for you
https://github.com/gruntwork-io/terragrunt
What is terragrunt?
Terragrunt is a thin wrapper for Terraform that provides extra
tools for keeping your Terraform configurations DRY, working with
multiple Terraform modules, and managing remote state.
36. Tagging your infra - Why it is important?
● Distinguish your infrastructure (footprint)
● Monitor your infrastructure
● Group resources.
● Cost Separation
37. Tag in AWS
myid.label = value
Myid is your identifier and will differentiate your labels from external automation
I also use this format for Kubernetes Labels
38. Tag in AWS / Use labels in Kubernetes
Label can be:
1. Name - Name is not necessary a tag but I always add it for AWS
2. product - Name of your product
3. environment - your environment
4. component - categorization of asset. AWS: This can be your service like bucket. For
Kubernetes it could be deployment, ingress…
5. part-of - Is this component part of a bigger plan? What? Example: Monitoring,
Cluster-XYZ, CI-server
6. terraform.repository
7. terraform.module = name of your module
8. terraform.module.version = version (it can also be branch name)
42. Data Common Mistakes
● Data can make a resource change on every execution if you use as input.
Examples in aws_security_group:
vpc_id = “${data.aws_vpc.cluster_vpc.id}”
Tip #13 - If data gives trouble, use state data
only as last resort
If you use data from state you might suffer
compilation errors during plan/apply terraform
Only use data from state if absolute necessary
43. What is Data from the state?
When you use data you are making a request to your current infra for data
When you use data from state you are actually fetching from the actual terraform state
based on what you built. That’s why you should use output in your modules.
data "terraform_remote_state" "vpc" {}
44. Agenda
● What is terraform
● Terraform 0.12
Insights on:
● Terraform Modules
● Terraform State
● Naming & Tagging
● Data
● Null Resources
● Local Files & Templating
● Kubernetes with Terraform
● Running Terraform & Pipelines
45. null-resource
The Rebel Terraform Resource
Tip #14 - Avoid using null resources
Check if a terraform provider does what needs to be done instead of shell scripting
You will have less work, less coding
46. null-resources
If trigger change the command will run
Tip #15 - Use null resources with triggers & dependencies (explicit or not)
To ensure everything will run when it has to run
Use cases: Run Ansible & restarts pods in Kubernetes after infra change
53. Local File
Tip #17 - Avoid using local_files
Because on every CI/laptop terraform consider local file creation as a
change
54. Coding Recap
● Use naming conventions and use terraform to enforce it
● Use tags/label conventions and use terraform to enforce it
● If data is forcing your resource to change an alternative it is to get data from the
state
● Avoid use of null provider
● Null Provider: always use with triggers & dependency (explicit or implicit)
● Null Provider: add a destroy conditional when is safe
● Avoid using Local File
55. Agenda
● What is terraform
● Terraform 0.12
Insights on:
● Terraform Modules
● Terraform State
● Naming & Tagging
● Data
● Null Resources
● Local Files & Templating
● Kubernetes with Terraform
● Running Terraform & Pipelines
57. Motivation
● One Tool execution
● Seize the templating from terraform instead of HELM or
another templating library
● I need to alternate between terraform & kubernetes
commands.
Example:
○ Run Kubernetes to create load balancer
○ Run terraform to create dns record
61. Using Kubernetes Provider - The issue
Tip #18 - For simple low maintenance kubernetes resources use terraform
Tip #19 - For sensitive/high maintenance kubernetes resources use other tools
Example: Your application
62. Agenda
● What is terraform
● Terraform 0.12
Insights on:
● Terraform Modules
● Terraform State
● Naming & Tagging
● Data
● Null Resources
● Local Files & Templating
● Kubernetes with Terraform
● Running Terraform & Pipelines
64. Terragrunt plan-all/apply-all
Tip #20 - Use Terragrunt plan/apply-all to make sure your infra is always
matching code
● Good for testing if all modules are fine after adding another one
● Module dependency control
● Support for multiple aws-accounts
● And much more...
65. Terraform plan/apply for 1 module
Why? Plan-All output can be really bad and if you are testing/coding or if when the
output is looking horrible.
To inspect your code better I would use plan with terraform-landscape and during initial
testing I would only apply in one module.
Tip #21 - To better see the results of your plan use landscape CLI
67. Why use a Pipeline?
● No more hassle of setting your AWS account every run
● My CLI secret variables are now in the CI and I don’t have to
worry about them in my laptop
● Everyone can run terraform
Tip #22 - Build a pipeline to run terraform for you
70. Tools still to explore
● Terratest - Testing
● Atlantis - CI for terraform
● Helm with Terraform for Kubernetes
● Dependabot - Keep your external modules up to date
● SecretHub - Provision Passwords & Keys for applications
● Will Pulumi beat Terraform? (https://www.pulumi.com/)
○ No more HCLs and Infra Languages: just use pure
Python/JS to build your infra