SlideShare une entreprise Scribd logo
1  sur  38
Télécharger pour lire hors ligne
Terraform Tips and
Tricks
LAOUC Community Tour 2022
August 17th
Nelson Calero
© Pythian Services Inc 2022 | 1
Our Speaker
Nelson Calero
Internal Principal Consultant
Pythian
http://www.linkedin.com/in/ncalero
@ncalerouy
© Pythian Services Inc 2022 | 2
24
Years in Business
_____________________
Premier Partner
140+ Certifications
8 Specializations
360+
Experts Across 5 Continents
500+
Customers Globally
_____________________
Advanced Partner
175+ Certifications
_____________________
Gold Partner
15+ Certifications
_____________________
Platinum Partner
150+ Certifications
_____________________
SAP Certified Partner
40+ Certifications
© Pythian Services Inc 2022 | 3
A G E N D A
● Terraform basics (1 slide)
● Terraform Idiosyncrasy
● Cloud provider Idiosyncrasy
© Pythian Services Inc 2022 | 4
Terraform basics
● Infrastructure as Code, declarative config files (HCL)
● Simple install - standalone binary file
● Few operations - plan/apply/destroy
● Providers - implement how to deal with your resources. 1000+
● State file - mapping of existing resources to TF configuration
● Simple structure
○ Reads code from current directory
○ Split code in files for readability: main.tf, variables.tf, outputs.tf
● Advanced configurations - workspaces, modules, remote state files
● Changes in versions - Features, state file format
© Pythian Services Inc 2021 | 5
Terraform basics
● Infrastructure as Code, declarative config files (HCL)
● Simple install - standalone binary file
● Few operations - plan/apply/destroy
● Providers - implement how to deal with your resources. 1000+
● State file - mapping of existing resources to TF configuration
● Simple structure
○ Reads code from current directory
○ Split code in files for readability: main.tf, variables.tf, outputs.tf
● Advanced configurations - workspaces, modules, remote state files
● Changes in versions - Features, state file format
© Pythian Services Inc 2021 | 6
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.27"
}
}
required_version = ">= 0.14.9"
}
provider "aws" {
profile = "default"
region = "us-west-1"
}
resource "aws_instance" "app_server" {
ami = "ami-830c94e3"
instance_type = "t2.micro"
tags = {
Name = "AppServerInstance"
}
}
● https://www.terraform.io/
● Lucas’ blog: https://technology.amis.nl/continuous-delivery/provisioning/terraform/
Terraform idiosyncrasy
1) State file relevance
2) Concurrent work on same resources
3) Reducing scope of operations
4) Resource parameters validation
5) Re-using code
6) Available features = TF + provider
© Pythian Services Inc 2021 | 7
Terraform idiosyncrasy
● state file drives the changes - make sure is in sync with your resources
● state file - removing resources (state rm), importing (import name_in_state name_aws)
● State file lock from previous canceled/timed out execution: force-unlock to remove it
● Plan and apply only one resource from many: -target name_in_state
● Conditional resources in modules: count
● Re-using code among environments and/or deployments? Welcome terragrunt. Examples:
○ Standardize resource attributes (tags, HA, networking, security, parameters) between deployments - in
same or different environments and regions
○ Support different database engines and versions
○ Validations (possible but limited to same attribute expression)
○ Modules, variables and environments - directory structure
○ Several other features - check https://terragrunt.gruntwork.io/
© Pythian Services Inc 2021 | 8
Terraform idiosyncrasy - 1) state file
Demo 1
Code taken from public Oracle repo (few changes for simplicity):
https://github.com/oracle/terraform-provider-oci/tree/master/examples/database/adb
© Pythian Services Inc 2021 | 9
Terraform idiosyncrasy - state file
© Pythian Services Inc 2021 | 10
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_db_instance.master will be created
+ resource "aws_db_instance" "master" {
+ address = (known after apply)
+ allocated_storage = 500
+ apply_immediately = (known after apply)
...
+ identifier ="nelson-ora19"
...
Plan: 1 to add, 0 to change, 0 to destroy.
Terraform idiosyncrasy - state file
© Pythian Services Inc 2021 | 11
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_db_instance.master will be created
+ resource "aws_db_instance" "master" {
+ address = (known after apply)
+ allocated_storage = 500
+ apply_immediately = (known after apply)
...
+ identifier ="nelson-pg14"
...
Plan: 1 to add, 0 to change, 0 to destroy.
$ aws rds describe-db-instances --region=us-west-1 --db-instance-identifier nelson-ora19
--query
"*[].{DBClusterIdentifier:DBClusterIdentifier,DBInstanceIdentifier:DBInstanceIdentifier,Engin
eVersion:EngineVersion}" --output table
--------------------------------------------------------------------------------------
| DescribeDBInstances |
+---------------------+------------------------+-------------------------------------+
| DBClusterIdentifier | DBInstanceIdentifier | EngineVersion |
+---------------------+------------------------+-------------------------------------+
| None | nelson-ora19 | 19.0.0.0.ru-2021-07.rur-2021-07.r1 |
+---------------------+------------------------+-------------------------------------+
Terraform idiosyncrasy - state file
© Pythian Services Inc 2021 | 12
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_db_instance.master will be created
+ resource "aws_db_instance" "master" {
+ address = (known after apply)
+ allocated_storage = 500
+ apply_immediately = (known after apply)
...
+ identifier ="nelson-pg14"
...
Plan: 1 to add, 0 to change, 0 to destroy.
$ aws rds describe-db-instances --region=us-west-1 --db-instance-identifier nelson-ora19
--query
"*[].{DBClusterIdentifier:DBClusterIdentifier,DBInstanceIdentifier:DBInstanceIdentifier,Engin
eVersion:EngineVersion}" --output table
--------------------------------------------------------------------------------------
| DescribeDBInstances |
+---------------------+------------------------+-------------------------------------+
| DBClusterIdentifier | DBInstanceIdentifier | EngineVersion |
+---------------------+------------------------+-------------------------------------+
| None | nelson-ora19 | 19.0.0.0.ru-2021-07.rur-2021-07.r1 |
+---------------------+------------------------+-------------------------------------+
$ terraform import aws_db_instance.master nelson-ora19
aws_db_instance.master: Importing from ID "nelson-ora19"...
aws_db_instance.master: Import prepared!
Prepared aws_db_instance for import
aws_db_instance.master: Refreshing state... [id=nelson-ora19]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
Terraform idiosyncrasy - state file
© Pythian Services Inc 2021 | 13
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_db_instance.master will be created
+ resource "aws_db_instance" "master" {
+ address = (known after apply)
+ allocated_storage = 500
+ apply_immediately = (known after apply)
...
+ identifier ="nelson-pg14"
...
Plan: 1 to add, 0 to change, 0 to destroy.
$ aws rds describe-db-instances --region=us-west-1 --db-instance-identifier nelson-pg14
--query
"*[].{DBClusterIdentifier:DBClusterIdentifier,DBInstanceIdentifier:DBInstanceIdentifier,Engin
eVersion:EngineVersion}" --output table
--------------------------------------------------------------------------------------
| DescribeDBInstances |
+---------------------+------------------------+-------------------------------------+
| DBClusterIdentifier | DBInstanceIdentifier | EngineVersion |
+---------------------+------------------------+-------------------------------------+
| None | nelson-ora19 | 19.0.0.0.ru-2021-07.rur-2021-07.r1 |
+---------------------+------------------------+-------------------------------------+
$ terraform import aws_db_instance.master nelson-pg14
aws_db_instance.master: Importing from ID "nelson-pg14"...
aws_db_instance.master: Import prepared!
Prepared aws_db_instance for import
aws_db_instance.master: Refreshing state... [id=nelson-pg14]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
aws_db_instance.master: Refreshing state... [id=nelson-ora19]
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# aws_db_instance.master will be updated in-place
~ resource "aws_db_instance" "master" {
...
+ password = (sensitive value)
...
Plan: 0 to add, 1 to change, 0 to destroy.
Terraform idiosyncrasy - state file
© Pythian Services Inc 2021 | 14
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_db_instance.master will be created
+ resource "aws_db_instance" "master" {
+ address = (known after apply)
+ allocated_storage = 500
+ apply_immediately = (known after apply)
...
+ identifier ="nelson-pg14"
...
Plan: 1 to add, 0 to change, 0 to destroy.
$ aws rds describe-db-instances --region=us-west-1 --db-instance-identifier nelson-pg14
--query
"*[].{DBClusterIdentifier:DBClusterIdentifier,DBInstanceIdentifier:DBInstanceIdentifier,Engin
eVersion:EngineVersion}" --output table
--------------------------------------------------------------------------------------
| DescribeDBInstances |
+---------------------+------------------------+-------------------------------------+
| DBClusterIdentifier | DBInstanceIdentifier | EngineVersion |
+---------------------+------------------------+-------------------------------------+
| None | nelson-ora19 | 19.0.0.0.ru-2021-07.rur-2021-07.r1 |
+---------------------+------------------------+-------------------------------------+
$ terraform import aws_db_instance.master nelson-pg14
aws_db_instance.master: Importing from ID "nelson-pg14"...
aws_db_instance.master: Import prepared!
Prepared aws_db_instance for import
aws_db_instance.master: Refreshing state... [id=nelson-pg14]
Import successful!
The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
aws_db_instance.master: Refreshing state... [id=nelson-pg14]
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# aws_db_instance.master will be updated in-place
~ resource "aws_db_instance" "master" {
...
+ password = (sensitive value)
...
Plan: 0 to add, 1 to change, 0 to destroy.
$ terraform apply
…
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
aws_db_instance.master: Refreshing state... [id=nelson-ora19]
------------------------------------------------------------------------
No changes. Infrastructure is up-to-date.
This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.
Terraform idiosyncrasy - state file
● Default state file is terraform.tfstate in current directory
● Critical data, requires good backup policy
● terraform import is not always able to recreate all attributes (ex: passwords, MySQL grants)
● Two key complementary features:
○ remote state file
https://www.terraform.io/language/state/remote
○ OCI discovery
https://registry.terraform.io/providers/oracle/oci/latest/docs/guides/resource_discovery
© Pythian Services Inc 2021 | 15
Terraform idiosyncrasy - state file / remote
● Remote state file - https://www.terraform.io/language/state/remote
© Pythian Services Inc 2021 | 16
remote_state {
backend "s3" {
bucket = "terraform-states"
key = "networking/terraform.tfstate"
region = "us-phoenix-1"
endpoint = "https://acme.compat.objectstorage.us-phoenix-1.oraclecloud.com"
shared_credentials_file = "../terraform-states_bucket_credentials"
skip_region_validation = true
skip_credentials_validation = true
skip_metadata_api_check = true
force_path_style = true
}
}
Ref: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/terraformUsingObjectStore.htm
Terraform idiosyncrasy - state file / OCI discovery
Demo 2
OCI discovery docs: https://registry.terraform.io/providers/oracle/oci/latest/docs/guides/resource_discovery
© Pythian Services Inc 2021 | 17
Terraform idiosyncrasy - state file / OCI discovery
● terraform-provider-oci gets installed by terraform:
● Discover databases and create the state file:
© Pythian Services Inc 2021 | 18
$ ls -lrt .terraform/providers/registry.terraform.io/hashicorp/oci/4.77.0/linux_amd64/
total 129320
-rw-r--r-- 1 calero calero 96384 May 31 17:31 CHANGELOG.md
-rw-r--r-- 1 calero calero 16725 May 31 17:31 LICENSE.md
-rw-r--r-- 1 calero calero 3071 May 31 17:31 README.md
-rwxr-xr-x 1 calero calero 132177920 May 31 17:31 terraform-provider-oci_v4.77.0
$ mkdir discover
$ .terraform/providers/registry.terraform.io/hashicorp/oci/4.77.0/linux_amd64/terraform-provider-oci_v4.77.0
-command=export -compartment_id=$COMPID -services database -generate_state -output_path=discover/
...
$ ls -lrt discover
total 44
-rw-r--r-- 1 calero calero 10109 May 31 18:46 terraform.tfstate.tmp.backup
-rw-r--r-- 1 calero calero 22086 May 31 18:46 terraform.tfstate
-rw-r--r-- 1 calero calero 6085 May 31 18:46 database.tf
-rw-r--r-- 1 calero calero 38 May 31 18:46 provider.tf
-rw-r--r-- 1 calero calero 171 May 31 18:46 vars.tf
$
Terraform idiosyncrasy - state file / OCI discovery
● terraform-provider-oci gets installed by terraform:
● Discover databases and create the state file:
© Pythian Services Inc 2021 | 19
$ ls -lrt .terraform/providers/registry.terraform.io/hashicorp/oci/4.77.0/linux_amd64/
total 129320
-rw-r--r-- 1 calero calero 96384 May 31 17:31 CHANGELOG.md
-rw-r--r-- 1 calero calero 16725 May 31 17:31 LICENSE.md
-rw-r--r-- 1 calero calero 3071 May 31 17:31 README.md
-rwxr-xr-x 1 calero calero 132177920 May 31 17:31 terraform-provider-oci_v4.77.0
$ mkdir discover
$ .terraform/providers/registry.terraform.io/hashicorp/oci/4.77.0/linux_amd64/terraform-provider-oci_v4.77.0
-command=export -compartment_id=$COMPID -services database -generate_state -output_path=../discover/
...
$ ls -lrt ../discover2
total 44
-rw-r--r-- 1 calero calero 10109 May 31 18:46 terraform.tfstate.tmp.backup
-rw-r--r-- 1 calero calero 22086 May 31 18:46 terraform.tfstate
-rw-r--r-- 1 calero calero 6085 May 31 18:46 database.tf
-rw-r--r-- 1 calero calero 38 May 31 18:46 provider.tf
-rw-r--r-- 1 calero calero 171 May 31 18:46 vars.tf
$
$ cd discover
$ terraform plan
oci_database_autonomous_database.export_example_autonomous_data_warehouse: Refreshing state...
[id=ocid1.autonomousdatabase.oc1.iad.anuwcljst77gzoiaaewxote6h4i73oamzoloxdueabnpwprspzngjoce52xq]
oci_database_autonomous_database.export_example_autonomous_database: Refreshing state...
[id=ocid1.autonomousdatabase.oc1.iad.anuwcljtt77gzoiamz3gg5cfsbdqgcoj7lrobis75qp7lwubm7d77bpblqgq]
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences,
so no changes are needed.
$
Terraform idiosyncrasy - 2) concurrent work
© Pythian Services Inc 2021 | 20
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
aws_rds_cluster.master: Refreshing state... [id=nelson-auroramysql56]
aws_rds_cluster_instance.cluster_instances[0]: Refreshing state... [id=nelson-auroramysql56-0]
aws_rds_cluster_instance.cluster_instances[1]: Refreshing state... [id=nelson-auroramysql56-1]
mysql_user.monitor: Refreshing state... [id=monitor@%]
Error: Could not connect to server: dial tcp 127.0.0.1:3306: connect: connection refused
Terraform idiosyncrasy - concurrent work
© Pythian Services Inc 2021 | 21
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
aws_rds_cluster.master: Refreshing state... [id=nelson-auroramysql56]
aws_rds_cluster_instance.cluster_instances[0]: Refreshing state... [id=nelson-auroramysql56-0]
aws_rds_cluster_instance.cluster_instances[1]: Refreshing state... [id=nelson-auroramysql56-1]
mysql_user.checkmk_mon: Refreshing state... [id=checkmk_mon@%]
Error: Could not connect to server: dial tcp 127.0.0.1:3306: connect: connection refused
$ terraform plan
...
Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional
request failed
Lock Info:
ID: f6f04b96-de6d-754a-7bbe-8d0504f75e79
Path: my-tf-state-286052569778/test/us-west-1/services/my-db/aurora-mysql/terraform.tfstate
Operation: OperationTypePlan
Who: ncalero@tst-rds01
Version: 0.12.31
Created: 2021-12-28 13:45:41.113713713 +0000 UTC
Info:
$
Terraform idiosyncrasy - concurrent work
© Pythian Services Inc 2021 | 22
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_security_group.sgpr: Refreshing state...
aws_rds_cluster.master: Refreshing state... [id=nelson-auroramysql56]
aws_rds_cluster_instance.cluster_instances[0]: Refreshing state... [id=nelson-auroramysql56-0]
aws_rds_cluster_instance.cluster_instances[1]: Refreshing state... [id=nelson-auroramysql56-1]
mysql_user.checkmk_mon: Refreshing state... [id=checkmk_mon@%]
Error: Could not connect to server: dial tcp 127.0.0.1:3306: connect: connection refused
$ terraform plan
...
Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional
request failed
Lock Info:
ID: f6f04b96-de6d-754a-7bbe-8d0504f75e79
Path: my-tf-state-286052569778/stable/us-west-1/services/my-db/aurora-mysql/terraform.tfstate
Operation: OperationTypePlan
Who: c_ncalero@gds-stg-rds01
Version: 0.12.31
Created: 2021-12-28 13:45:41.113713713 +0000 UTC
Info:
$
$ terraform force-unlock f6f04b96-de6d-754a-7bbe-8d0504f75e79
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may be still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
$
Terraform idiosyncrasy - 3) reduce scope
© Pythian Services Inc 2021 | 23
$ terraform plan
data.oci_database_autonomous_db_versions.test_adb_versions: Reading...
oci_database_autonomous_database.autonomous_data_warehouse: Refreshing state...
[id=ocid1.autonomousdatabase.oc1.iad.anuwcljst77gzoiaaewxote6h4i73oamzoloxdueabnpwprspzngjoce52xq]
oci_database_autonomous_database.autonomous_database: Refreshing state...
[id=ocid1.autonomousdatabase.oc1.iad.anuwcljtt77gzoiamz3gg5cfsbdqgcoj7lrobis75qp7lwubm7d77bpblqgq]
data.oci_database_autonomous_databases.autonomous_databases: Reading...
...
Plan: 0 to add, 1 to change, 1 to destroy.
$ terraform state list
data.http.ip
data.oci_database_autonomous_db_versions.test_adb_versions
oci_database_autonomous_database.autonomous_data_warehouse
oci_database_autonomous_database.autonomous_database
random_string.adb_admin_password
$ terraform plan -target oci_database_autonomous_database.autonomous_database
data.oci_database_autonomous_db_versions.test_adb_versions: Reading...
oci_database_autonomous_database.autonomous_database: Refreshing state...
[id=ocid1.autonomousdatabase.oc1.iad.anuwcljtt77gzoiamz3gg5cfsbdqgcoj7lrobis75qp7lwubm7d77bpblqgq]
No changes. Your infrastructure matches the configuration.
Terraform idiosyncrasy - 4) parameter validations
© Pythian Services Inc 2021 | 24
$ terraform apply
…
aws_rds_cluster.master[0]: Creating...
aws_rds_cluster.master[0]: Still creating... [10s elapsed]
aws_rds_cluster.master[0]: Still creating... [20s elapsed]
aws_rds_cluster.master[0]: Still creating... [30s elapsed]
aws_rds_cluster.master[0]: Still creating... [40s elapsed]
aws_rds_cluster.master[0]: Creation complete after 42s [id=my-aurora]
aws_rds_cluster_instance.cluster_instances[1]: Creating...
aws_rds_cluster_instance.cluster_instances[0]: Creating...
Error: error creating RDS Cluster (my-aurora) Instance: InvalidParameterCombination: RDS does not support creating a DB instance with the
following combination: DBInstanceClass=db.t3.small, Engine=aurora-mysql, EngineVersion=5.7. For supported combinations of instance class
and database engine version, see the documentation.
status code: 400, request id: 7c15bd63-b7cf-47a3-8a10-faee35c3e9c4
on main.tf line 7, in resource "aws_rds_cluster_instance" "cluster_instances":
7: resource "aws_rds_cluster_instance" "cluster_instances" {
Error: error creating RDS Cluster (my-aurora) Instance: InvalidParameterCombination: RDS does not support creating a DB instance with the
following combination: DBInstanceClass=db.t3.small, Engine=aurora-mysql, EngineVersion=5.7. For supported combinations of instance class
and database engine version, see the documentation.
status code: 400, request id: a8cb9644-fbe2-40ee-929b-18094709cb33
on main.tf line 7, in resource "aws_rds_cluster_instance" "cluster_instances":
7: resource "aws_rds_cluster_instance" "cluster_instances" {
Terraform idiosyncrasy - 4) parameter validations
● Terraform pass parameters to the provider - it runs the operation on the cloud service and it fails
● Some providers include data sources with attributes having a list of valid values for that (only if we use
those when manipulating the resource)
○ Example: OCI Autonomous database
https://registry.terraform.io/providers/oracle/oci/latest/docs/data-sources/database_autonomous_db_versions
● Additionally, simple validations can be incorporated in the variable definition:
© Pythian Services Inc 2021 | 25
$ grep -A13 "autonomous_database_db_workload" variables.tf
variable "autonomous_database_db_workload" {
default = "OLTP"
validation {
condition = anytrue([
var.autonomous_database_db_workload == "OLTP",
var.autonomous_database_db_workload == "DW",
var.autonomous_database_db_workload == "APEX",
var.autonomous_database_db_workload == "AJD"
])
error_message = "Invalid autonomous_database_db_workload value - allowed OLTP, DW, APEX and AJD."
}
}
Terraform idiosyncrasy - parameter validations
© Pythian Services Inc 2021 | 26
Demo 3
Terraform idiosyncrasy - 5) reusing code
● Terraform by default read the current directory only
● Can include other directories using the “module” directive: https://learn.hashicorp.com/collections/terraform/modules
● Third party tools to re-using code (among environments and/or deployments) - terragrunt example:
○ Standardize resource attributes (tags, HA, networking, security, parameters) between deployments - in same or
different environments and regions
○ Support different database engines and versions
○ Validations (possible but limited to same attribute expression)
○ Modules, variables and environments - directory structure
○ Extra features: hooks, functions, etc. - check https://terragrunt.gruntwork.io/
© Pythian Services Inc 2021 | 27
module "ec2_instances" {
source = "terraform-aws-modules/ec2-instance/aws"
version = "3.5.0"
...
Terraform idiosyncrasy - 6) “features”
● Terraform binary
○ Aurora MySQL upgrade didn’t work on TF versions lower than 0.12.31 (current version: 1.1.7)
● Terraform provider
○ MySQL upgrade didn’t work with AWS provider lower than 3.63.0 (several bugs)
○ Resource discovery implemented in OCI (not in AWS as of today)
● Cloud provider limits/restrictions
○ Resource attributes are not validated (service restrictions) - cause runtime errors
■ allowed instance type for a RDS version (both ways)
■ IOPS parameter when using gp2 storage
■ CLI commands to the rescue (to confirm allowed values)
© Pythian Services Inc 2021 | 28
Cloud provider idiosyncrasy
● You can get errors today that does not repeat tomorrow, as services improves
● Don’t assume all services work the same for similar operations - RTFM!
© Pythian Services Inc 2021 | 29
● Examples on AWS:
○ MySQL minor upgrade fails without any error - not reproducible days later when repeating the test
○ RDS “create Aurora replica” is not available in some versions after upgrades
○ Aurora space used should be checked using metrics
○ RDS parameters have apply_mode immediate or pending-reboot, but tricky to update running instances
automatically (immediate mode)
○ Operations on RDS storage does not allow other operations until they finish - and those can take long
■ Automatic space increment - minutes
■ Change type between gp2 and IOPS - hours
Terraform - parameters cheat sheet
© Pythian Services Inc 2021 | 30
Format code in .tf files terraform fmt
Validate configuration terraform validate
List state file resources terraform state list
Inspect current state file content terraform show
Remove a resource from state file terraform state rm <name_in_state>
Import resource into state file terraform import <name_in_state> <name_aws>
Remove lock from state file terraform force-unlock <lock_id>
Operate on a single resource terraform <op> -target <name_in_state>
Console (to test formulas/expressions) terraform console
Debug execution TF_LOG="TRACE" TF_LOG_PATH=file.log ; terraform <op>
Terraform console
© Pythian Services Inc 2021 | 31
[c_ncalero@myhost ~]$ terraform console
> substr("19.2",0,2)
19
> split(".","19.2")
[
"19",
"2",
]
> split(".","19.2")[0]
19
> element(split(".","19.2"),0)
19
> join("",["oracle",substr("19.2",0,2), "small"])
oracle19small
> join("",["oracle",split(".","19.2")[0]])
oracle19
> exit
OCI CLI - useful validations
© Pythian Services Inc 2021 | 32
$ export COMPID=$(oci iam compartment list --query "data [?name=='mycmp'].{id:id}" | jq -r '.[].id')
$ oci db version list -c $COMPID --db-system-shape "VM.Standard1.1" --output table
+-----------------------------+--------------+-----------------+
| is-latest-for-major-version | supports-pdb | version |
+-----------------------------+--------------+-----------------+
| True | False | 11.2.0.4 |
| False | False | 11.2.0.4.210119 |
| False | False | 11.2.0.4.210420 |
| True | True | 12.1.0.2 |
| False | True | 12.1.0.2.211019 |
| False | True | 12.1.0.2.220118 |
| False | True | 12.1.0.2.220419 |
| True | True | 12.2.0.1 |
| False | True | 12.2.0.1.211019 |
| False | True | 12.2.0.1.220118 |
| False | True | 12.2.0.1.220419 |
| True | True | 18.0.0.0 |
| False | True | 18.16.0.0 |
| True | True | 19.0.0.0 |
| False | True | 19.13.0.0 |
| False | True | 19.14.0.0 |
● Available database versions for DB Systems using shape VM.Standard1.1
| False | True | 19.15.0.0 |
| True | True | 21.0.0.0 |
| False | True | 21.4.0.0 |
| False | True | 21.5.0.0 |
| False | True | 21.6.0.0 |
+-------------+--------------+-----------+
OCI CLI - useful validations
© Pythian Services Inc 2021 | 33
$ export COMPID=$(oci iam compartment list --query "data [?name=='mycmp'].{id:id}" | jq -r '.[].id' )
$ oci db autonomous-database list -c $COMPID --query 'data[].{db_name:"db-name", db_version:"db-version",
db_workload:"db-workload", display_name:"display-name", state:"lifecycle-state"}' --output table
+--------------------+------------+-------------+-----------------------------------+-----------+
| db_name | db_version | db_workload | display_name | state |
+--------------------+------------+-------------+-----------------------------------+-----------+
| adbdw | 19c | DW | example_autonomous_data_warehouse | AVAILABLE |
| adboltp | 19c | OLTP | example_autonomous_database | AVAILABLE |
+--------------------+------------+-------------+-----------------------------------+-----------+
● ADBs created in compartment ’mycmp’
AWS CLI - useful validations
© Pythian Services Inc 2021 | 34
$ aws rds describe-orderable-db-instance-options --engine oracle-ee --db-instance-class db.t3.small --query
"OrderableDBInstanceOptions[].{EngineVersion:EngineVersion}" --output text --region eu-west-1
12.1.0.2.v1
12.1.0.2.v10
12.1.0.2.v11
12.1.0.2.v12
12.1.0.2.v13
12.1.0.2.v14
12.1.0.2.v15
12.1.0.2.v16
12.1.0.2.v17
12.1.0.2.v18
12.1.0.2.v19
12.1.0.2.v2
12.1.0.2.v20
12.1.0.2.v21
12.1.0.2.v22
12.1.0.2.v23
12.1.0.2.v24
12.1.0.2.v25
12.1.0.2.v26
12.1.0.2.v27
● Available Oracle EE versions for shape db.t3.small
12.1.0.2.v3
12.1.0.2.v4
12.1.0.2.v5
12.1.0.2.v6
12.1.0.2.v7
12.1.0.2.v8
12.1.0.2.v9
19.0.0.0.ru-2019-07.rur-2019-07.r1
19.0.0.0.ru-2019-10.rur-2019-10.r1
19.0.0.0.ru-2020-01.rur-2020-01.r1
19.0.0.0.ru-2020-04.rur-2020-04.r1
19.0.0.0.ru-2020-07.rur-2020-07.r1
19.0.0.0.ru-2020-10.rur-2020-10.r1
19.0.0.0.ru-2021-01.rur-2021-01.r1
19.0.0.0.ru-2021-01.rur-2021-01.r2
19.0.0.0.ru-2021-04.rur-2021-04.r1
19.0.0.0.ru-2021-07.rur-2021-07.r1
19.0.0.0.ru-2021-10.rur-2021-10.r1
19.0.0.0.ru-2022-01.rur-2022-01.r1
AWS CLI - useful validations
© Pythian Services Inc 2021 | 35
$ aws rds describe-db-engine-versions --engine oracle-ee --engine-version 12.1.0.2.v20 --query
"DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text
12.1.0.2.v21
12.1.0.2.v22
12.1.0.2.v23
12.1.0.2.v24
12.1.0.2.v25
12.1.0.2.v26
12.1.0.2.v27
19.0.0.0.ru-2020-04.rur-2020-04.r1
19.0.0.0.ru-2020-07.rur-2020-07.r1
19.0.0.0.ru-2020-10.rur-2020-10.r1
19.0.0.0.ru-2021-01.rur-2021-01.r1
19.0.0.0.ru-2021-01.rur-2021-01.r2
19.0.0.0.ru-2021-04.rur-2021-04.r1
19.0.0.0.ru-2021-07.rur-2021-07.r1
19.0.0.0.ru-2021-10.rur-2021-10.r1
19.0.0.0.ru-2022-01.rur-2022-01.r1
● Supported upgrade paths for Oracle version 12.1.0.2
AWS CLI - useful validations
© Pythian Services Inc 2021 | 36
$ REGION=us-west-1; for i in $(awscloudwatch list-metrics --region=$REGION --namespace AWS/RDS --metric-name
VolumeBytesUsed --query "Metrics[].Dimensions[].{Name:Name, Value:Value}" --output table | grep
DBClusterIdentifier | cut -d"|" -f3) ; do echo -n $i " " ; aws
cloudwatch get-metric-statistics --region
$REGION --namespace AWS/RDS --metric-name VolumeBytesUsed --dimensions Name=EngineName,Value=aurora-mysql
--dimensions Name=EngineName,Value=aurora Name=DbClusterIdentifier,Value=$i --start-time $(date -u -d "1 days
ago" '+%F') --end-time $(date -u '+%F') --statistics Maximum --period 3600 --query
"Datapoints[].{Timestamp:Timestamp, Maximum:Maximum, Unit:Unit}" | jq '.[] | .Maximum' | sort -u | tail -1 ;
done | sort
bookshop-db 7381361362844
sales-db 179468653458
track-db 20681679872
programs-db 35563371113
analytics-db 8650433415
master-db 17432850551
resourcing-db 6418328690
…
● Space in use by aurora Mysql instances running in region us-west-1
AWS CLI - useful validations
© Pythian Services Inc 2021 | 37
$ aws rds describe-db-parameters --db-parameter-group-name mypg --region us-west-1 --query
"sort_by(Parameters,&ParameterName)[?Source=='user'].{ParameterName:ParameterName, ApplyMethod:ApplyMethod,
IsModifiable:IsModifiable, ParameterValue:ParameterValue}" --output table
+----------------+---------------+-----------------------------------+------------------+
| ApplyMethod | IsModifiable | ParameterName | ParameterValue |
+----------------+---------------+-----------------------------------+------------------+
| immediate | True | event_scheduler | ON |
| immediate | True | group_concat_max_len | 16384 |
| pending-reboot| True | innodb_sort_buffer_size | 8388608 |
| immediate | True | log_bin_trust_function_creators | 1 |
| immediate | True | long_query_time | 0.1 |
| pending-reboot| True | max_allowed_packet | 1073741824 |
| immediate | True | max_connections | 8000 |
| immediate | True | slow_query_log | 1 |
+----------------+---------------+-----------------------------------+------------------+
● Parameters set by user (we) in a RDS parameter group
Questions?
© Pythian Services Inc 2021 | 38
calero@pythian.com
@ncalerouy
http://www.linkedin.com/in/ncalero

Contenu connexe

Tendances

Serverless Application Development with SAM
Serverless Application Development with SAMServerless Application Development with SAM
Serverless Application Development with SAMAmazon Web Services
 
Azure Key Vault - Getting Started
Azure Key Vault - Getting StartedAzure Key Vault - Getting Started
Azure Key Vault - Getting StartedTaswar Bhatti
 
Azure landing zones - Terraform module design considerations - Azure Architec...
Azure landing zones - Terraform module design considerations - Azure Architec...Azure landing zones - Terraform module design considerations - Azure Architec...
Azure landing zones - Terraform module design considerations - Azure Architec...DubemJavapi
 
Terraform modules restructured
Terraform modules restructuredTerraform modules restructured
Terraform modules restructuredAmi Mahloof
 
Azure key vault
Azure key vaultAzure key vault
Azure key vaultRahul Nath
 
[Azure Governance] Lesson 3 : Azure Tags
[Azure Governance] Lesson 3 : Azure Tags[Azure Governance] Lesson 3 : Azure Tags
[Azure Governance] Lesson 3 : Azure Tags☁ Hicham KADIRI ☁
 
Getting Started with Infrastructure as Code
Getting Started with Infrastructure as CodeGetting Started with Infrastructure as Code
Getting Started with Infrastructure as CodeWinWire Technologies Inc
 
Advanced Load Balancer/Traffic Manager and App Gateway for Microsoft Azure
Advanced Load Balancer/Traffic Manager and App Gateway for Microsoft AzureAdvanced Load Balancer/Traffic Manager and App Gateway for Microsoft Azure
Advanced Load Balancer/Traffic Manager and App Gateway for Microsoft AzureKemp
 
Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices Hendri Karisma
 
Introduction to Docker - 2017
Introduction to Docker - 2017Introduction to Docker - 2017
Introduction to Docker - 2017Docker, Inc.
 
AWS solution Architect Associate study material
AWS solution Architect Associate study materialAWS solution Architect Associate study material
AWS solution Architect Associate study materialNagesh Ramamoorthy
 
Using Amazon Inspector to Discover Potential Security Issues - AWS Online Tec...
Using Amazon Inspector to Discover Potential Security Issues - AWS Online Tec...Using Amazon Inspector to Discover Potential Security Issues - AWS Online Tec...
Using Amazon Inspector to Discover Potential Security Issues - AWS Online Tec...Amazon Web Services
 
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Amazon Web Services
 
Microservices Workshop All Topics Deck 2016
Microservices Workshop All Topics Deck 2016Microservices Workshop All Topics Deck 2016
Microservices Workshop All Topics Deck 2016Adrian Cockcroft
 
Comprehensive Terraform Training
Comprehensive Terraform TrainingComprehensive Terraform Training
Comprehensive Terraform TrainingYevgeniy Brikman
 

Tendances (20)

AWS Cloud Security
AWS Cloud SecurityAWS Cloud Security
AWS Cloud Security
 
Serverless Application Development with SAM
Serverless Application Development with SAMServerless Application Development with SAM
Serverless Application Development with SAM
 
Azure Key Vault - Getting Started
Azure Key Vault - Getting StartedAzure Key Vault - Getting Started
Azure Key Vault - Getting Started
 
Effective terraform
Effective terraformEffective terraform
Effective terraform
 
Azure landing zones - Terraform module design considerations - Azure Architec...
Azure landing zones - Terraform module design considerations - Azure Architec...Azure landing zones - Terraform module design considerations - Azure Architec...
Azure landing zones - Terraform module design considerations - Azure Architec...
 
Terraform modules restructured
Terraform modules restructuredTerraform modules restructured
Terraform modules restructured
 
Azure key vault
Azure key vaultAzure key vault
Azure key vault
 
Terraform
TerraformTerraform
Terraform
 
[Azure Governance] Lesson 3 : Azure Tags
[Azure Governance] Lesson 3 : Azure Tags[Azure Governance] Lesson 3 : Azure Tags
[Azure Governance] Lesson 3 : Azure Tags
 
Getting Started with Infrastructure as Code
Getting Started with Infrastructure as CodeGetting Started with Infrastructure as Code
Getting Started with Infrastructure as Code
 
Advanced Load Balancer/Traffic Manager and App Gateway for Microsoft Azure
Advanced Load Balancer/Traffic Manager and App Gateway for Microsoft AzureAdvanced Load Balancer/Traffic Manager and App Gateway for Microsoft Azure
Advanced Load Balancer/Traffic Manager and App Gateway for Microsoft Azure
 
Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices
 
Introduction to Docker - 2017
Introduction to Docker - 2017Introduction to Docker - 2017
Introduction to Docker - 2017
 
AWS solution Architect Associate study material
AWS solution Architect Associate study materialAWS solution Architect Associate study material
AWS solution Architect Associate study material
 
Using Amazon Inspector to Discover Potential Security Issues - AWS Online Tec...
Using Amazon Inspector to Discover Potential Security Issues - AWS Online Tec...Using Amazon Inspector to Discover Potential Security Issues - AWS Online Tec...
Using Amazon Inspector to Discover Potential Security Issues - AWS Online Tec...
 
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
 
Terraform on Azure
Terraform on AzureTerraform on Azure
Terraform on Azure
 
Microservices Workshop All Topics Deck 2016
Microservices Workshop All Topics Deck 2016Microservices Workshop All Topics Deck 2016
Microservices Workshop All Topics Deck 2016
 
Comprehensive Terraform Training
Comprehensive Terraform TrainingComprehensive Terraform Training
Comprehensive Terraform Training
 
Istio on Kubernetes
Istio on KubernetesIstio on Kubernetes
Istio on Kubernetes
 

Similaire à Terraform Tips and Tricks - LAOUC 2022

TerraformとAzureを組み合わせて使うときの勘所
TerraformとAzureを組み合わせて使うときの勘所TerraformとAzureを組み合わせて使うときの勘所
TerraformとAzureを組み合わせて使うときの勘所Kyohei Moriyama
 
glance replicator
glance replicatorglance replicator
glance replicatoririx_jp
 
Getting data into Rudder
Getting data into RudderGetting data into Rudder
Getting data into RudderRUDDER
 
leboncoin DataEngineering / Terraform - beginner to advanced
leboncoin DataEngineering / Terraform - beginner to advancedleboncoin DataEngineering / Terraform - beginner to advanced
leboncoin DataEngineering / Terraform - beginner to advancedleboncoin engineering
 
Debasihish da final.ppt
Debasihish da final.pptDebasihish da final.ppt
Debasihish da final.pptKalkey
 
Terraform 0.9 + good practices
Terraform 0.9 + good practicesTerraform 0.9 + good practices
Terraform 0.9 + good practicesRadek Simko
 
Adding Complex Data to Spark Stack by Tug Grall
Adding Complex Data to Spark Stack by Tug GrallAdding Complex Data to Spark Stack by Tug Grall
Adding Complex Data to Spark Stack by Tug GrallSpark Summit
 
Common schema my sql uc 2012
Common schema   my sql uc 2012Common schema   my sql uc 2012
Common schema my sql uc 2012Roland Bouman
 
Common schema my sql uc 2012
Common schema   my sql uc 2012Common schema   my sql uc 2012
Common schema my sql uc 2012Roland Bouman
 
Understanding Query Optimization with ‘regular’ and ‘Exadata’ Oracle
Understanding Query Optimization with ‘regular’ and ‘Exadata’ OracleUnderstanding Query Optimization with ‘regular’ and ‘Exadata’ Oracle
Understanding Query Optimization with ‘regular’ and ‘Exadata’ OracleGuatemala User Group
 
Evolution of Performance Management: Oracle 12c adaptive optimizations - ukou...
Evolution of Performance Management: Oracle 12c adaptive optimizations - ukou...Evolution of Performance Management: Oracle 12c adaptive optimizations - ukou...
Evolution of Performance Management: Oracle 12c adaptive optimizations - ukou...Nelson Calero
 
Oracle Diagnostics : Explain Plans (Simple)
Oracle Diagnostics : Explain Plans (Simple)Oracle Diagnostics : Explain Plans (Simple)
Oracle Diagnostics : Explain Plans (Simple)Hemant K Chitale
 
Best Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformBest Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformDevOps.com
 
Distributed Queries in IDS: New features.
Distributed Queries in IDS: New features.Distributed Queries in IDS: New features.
Distributed Queries in IDS: New features.Keshav Murthy
 
Terraform at Scale - All Day DevOps 2017
Terraform at Scale - All Day DevOps 2017Terraform at Scale - All Day DevOps 2017
Terraform at Scale - All Day DevOps 2017Jonathon Brouse
 
Oracle dbms_xplan.display_cursor format
Oracle dbms_xplan.display_cursor formatOracle dbms_xplan.display_cursor format
Oracle dbms_xplan.display_cursor formatFranck Pachot
 
Oracle Exadata Cloud Services guide from practical experience - OOW19
Oracle Exadata Cloud Services guide from practical experience - OOW19Oracle Exadata Cloud Services guide from practical experience - OOW19
Oracle Exadata Cloud Services guide from practical experience - OOW19Nelson Calero
 
[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스
[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스
[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스PgDay.Seoul
 
MySQL 5.7 Tutorial Dutch PHP Conference 2015
MySQL 5.7 Tutorial Dutch PHP Conference 2015MySQL 5.7 Tutorial Dutch PHP Conference 2015
MySQL 5.7 Tutorial Dutch PHP Conference 2015Dave Stokes
 

Similaire à Terraform Tips and Tricks - LAOUC 2022 (20)

Terraform Cosmos DB
Terraform Cosmos DBTerraform Cosmos DB
Terraform Cosmos DB
 
TerraformとAzureを組み合わせて使うときの勘所
TerraformとAzureを組み合わせて使うときの勘所TerraformとAzureを組み合わせて使うときの勘所
TerraformとAzureを組み合わせて使うときの勘所
 
glance replicator
glance replicatorglance replicator
glance replicator
 
Getting data into Rudder
Getting data into RudderGetting data into Rudder
Getting data into Rudder
 
leboncoin DataEngineering / Terraform - beginner to advanced
leboncoin DataEngineering / Terraform - beginner to advancedleboncoin DataEngineering / Terraform - beginner to advanced
leboncoin DataEngineering / Terraform - beginner to advanced
 
Debasihish da final.ppt
Debasihish da final.pptDebasihish da final.ppt
Debasihish da final.ppt
 
Terraform 0.9 + good practices
Terraform 0.9 + good practicesTerraform 0.9 + good practices
Terraform 0.9 + good practices
 
Adding Complex Data to Spark Stack by Tug Grall
Adding Complex Data to Spark Stack by Tug GrallAdding Complex Data to Spark Stack by Tug Grall
Adding Complex Data to Spark Stack by Tug Grall
 
Common schema my sql uc 2012
Common schema   my sql uc 2012Common schema   my sql uc 2012
Common schema my sql uc 2012
 
Common schema my sql uc 2012
Common schema   my sql uc 2012Common schema   my sql uc 2012
Common schema my sql uc 2012
 
Understanding Query Optimization with ‘regular’ and ‘Exadata’ Oracle
Understanding Query Optimization with ‘regular’ and ‘Exadata’ OracleUnderstanding Query Optimization with ‘regular’ and ‘Exadata’ Oracle
Understanding Query Optimization with ‘regular’ and ‘Exadata’ Oracle
 
Evolution of Performance Management: Oracle 12c adaptive optimizations - ukou...
Evolution of Performance Management: Oracle 12c adaptive optimizations - ukou...Evolution of Performance Management: Oracle 12c adaptive optimizations - ukou...
Evolution of Performance Management: Oracle 12c adaptive optimizations - ukou...
 
Oracle Diagnostics : Explain Plans (Simple)
Oracle Diagnostics : Explain Plans (Simple)Oracle Diagnostics : Explain Plans (Simple)
Oracle Diagnostics : Explain Plans (Simple)
 
Best Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformBest Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with Terraform
 
Distributed Queries in IDS: New features.
Distributed Queries in IDS: New features.Distributed Queries in IDS: New features.
Distributed Queries in IDS: New features.
 
Terraform at Scale - All Day DevOps 2017
Terraform at Scale - All Day DevOps 2017Terraform at Scale - All Day DevOps 2017
Terraform at Scale - All Day DevOps 2017
 
Oracle dbms_xplan.display_cursor format
Oracle dbms_xplan.display_cursor formatOracle dbms_xplan.display_cursor format
Oracle dbms_xplan.display_cursor format
 
Oracle Exadata Cloud Services guide from practical experience - OOW19
Oracle Exadata Cloud Services guide from practical experience - OOW19Oracle Exadata Cloud Services guide from practical experience - OOW19
Oracle Exadata Cloud Services guide from practical experience - OOW19
 
[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스
[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스
[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스
 
MySQL 5.7 Tutorial Dutch PHP Conference 2015
MySQL 5.7 Tutorial Dutch PHP Conference 2015MySQL 5.7 Tutorial Dutch PHP Conference 2015
MySQL 5.7 Tutorial Dutch PHP Conference 2015
 

Plus de Nelson Calero

Database automation guide - Oracle Community Tour LATAM 2023
Database automation guide - Oracle Community Tour LATAM 2023Database automation guide - Oracle Community Tour LATAM 2023
Database automation guide - Oracle Community Tour LATAM 2023Nelson Calero
 
Oracle on kubernetes 101 - Dec/2021
Oracle on kubernetes 101 - Dec/2021Oracle on kubernetes 101 - Dec/2021
Oracle on kubernetes 101 - Dec/2021Nelson Calero
 
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...Nelson Calero
 
Automate your oracle cloud infrastructure operations v2.0 - OOW19
Automate your oracle cloud infrastructure operations v2.0 - OOW19Automate your oracle cloud infrastructure operations v2.0 - OOW19
Automate your oracle cloud infrastructure operations v2.0 - OOW19Nelson Calero
 
Automate the operation of your Oracle Cloud infrastructure v2.0
Automate the operation of your Oracle Cloud infrastructure v2.0Automate the operation of your Oracle Cloud infrastructure v2.0
Automate the operation of your Oracle Cloud infrastructure v2.0Nelson Calero
 
SSL certificates in the Oracle Database without surprises
SSL certificates in the Oracle Database without surprisesSSL certificates in the Oracle Database without surprises
SSL certificates in the Oracle Database without surprisesNelson Calero
 
Practical guide to Oracle Virtual environments
Practical guide to Oracle Virtual environmentsPractical guide to Oracle Virtual environments
Practical guide to Oracle Virtual environmentsNelson Calero
 
Automate your Oracle Cloud Infrastructure operation
Automate your Oracle Cloud Infrastructure operationAutomate your Oracle Cloud Infrastructure operation
Automate your Oracle Cloud Infrastructure operationNelson Calero
 
Welcome to databases in the Cloud
Welcome to databases in the CloudWelcome to databases in the Cloud
Welcome to databases in the CloudNelson Calero
 
Redefining tables online without surprises
Redefining tables online without surprisesRedefining tables online without surprises
Redefining tables online without surprisesNelson Calero
 
Protect Sensitive Data: Implementing Fine-Grained Access Control in Oracle
Protect Sensitive Data: Implementing Fine-Grained Access Control in OracleProtect Sensitive Data: Implementing Fine-Grained Access Control in Oracle
Protect Sensitive Data: Implementing Fine-Grained Access Control in OracleNelson Calero
 
Oracle Exadata Maintenance tasks 101 - OTN Tour 2015
Oracle Exadata Maintenance tasks 101 - OTN Tour 2015Oracle Exadata Maintenance tasks 101 - OTN Tour 2015
Oracle Exadata Maintenance tasks 101 - OTN Tour 2015Nelson Calero
 
My Experience Using Oracle SQL Plan Baselines 11g/12c
My Experience Using Oracle SQL Plan Baselines 11g/12cMy Experience Using Oracle SQL Plan Baselines 11g/12c
My Experience Using Oracle SQL Plan Baselines 11g/12cNelson Calero
 
Oracle RAC sin sorpresas - v2014
Oracle RAC sin sorpresas - v2014Oracle RAC sin sorpresas - v2014
Oracle RAC sin sorpresas - v2014Nelson Calero
 
Alta disponibilidad con Pacemaker
Alta disponibilidad con PacemakerAlta disponibilidad con Pacemaker
Alta disponibilidad con PacemakerNelson Calero
 
AROUG BIDAY 2013 - Automatizar procesos de ETL con PL/SQL
AROUG BIDAY 2013 - Automatizar procesos de ETL con PL/SQLAROUG BIDAY 2013 - Automatizar procesos de ETL con PL/SQL
AROUG BIDAY 2013 - Automatizar procesos de ETL con PL/SQLNelson Calero
 
MariaDB y FOSS en infraestructura de salud y estándares
MariaDB y FOSS en infraestructura de salud y estándaresMariaDB y FOSS en infraestructura de salud y estándares
MariaDB y FOSS en infraestructura de salud y estándaresNelson Calero
 
UYOUG 2012 - Oracle RAC 11gR2 - New features
UYOUG 2012 - Oracle RAC 11gR2 - New featuresUYOUG 2012 - Oracle RAC 11gR2 - New features
UYOUG 2012 - Oracle RAC 11gR2 - New featuresNelson Calero
 
Collaborate 2012 - Administering MySQL for Oracle DBAs
Collaborate 2012 - Administering MySQL for Oracle DBAsCollaborate 2012 - Administering MySQL for Oracle DBAs
Collaborate 2012 - Administering MySQL for Oracle DBAsNelson Calero
 
Collaborate 2012 - RMAN Eliminate the mystery
Collaborate 2012 - RMAN Eliminate the mysteryCollaborate 2012 - RMAN Eliminate the mystery
Collaborate 2012 - RMAN Eliminate the mysteryNelson Calero
 

Plus de Nelson Calero (20)

Database automation guide - Oracle Community Tour LATAM 2023
Database automation guide - Oracle Community Tour LATAM 2023Database automation guide - Oracle Community Tour LATAM 2023
Database automation guide - Oracle Community Tour LATAM 2023
 
Oracle on kubernetes 101 - Dec/2021
Oracle on kubernetes 101 - Dec/2021Oracle on kubernetes 101 - Dec/2021
Oracle on kubernetes 101 - Dec/2021
 
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
 
Automate your oracle cloud infrastructure operations v2.0 - OOW19
Automate your oracle cloud infrastructure operations v2.0 - OOW19Automate your oracle cloud infrastructure operations v2.0 - OOW19
Automate your oracle cloud infrastructure operations v2.0 - OOW19
 
Automate the operation of your Oracle Cloud infrastructure v2.0
Automate the operation of your Oracle Cloud infrastructure v2.0Automate the operation of your Oracle Cloud infrastructure v2.0
Automate the operation of your Oracle Cloud infrastructure v2.0
 
SSL certificates in the Oracle Database without surprises
SSL certificates in the Oracle Database without surprisesSSL certificates in the Oracle Database without surprises
SSL certificates in the Oracle Database without surprises
 
Practical guide to Oracle Virtual environments
Practical guide to Oracle Virtual environmentsPractical guide to Oracle Virtual environments
Practical guide to Oracle Virtual environments
 
Automate your Oracle Cloud Infrastructure operation
Automate your Oracle Cloud Infrastructure operationAutomate your Oracle Cloud Infrastructure operation
Automate your Oracle Cloud Infrastructure operation
 
Welcome to databases in the Cloud
Welcome to databases in the CloudWelcome to databases in the Cloud
Welcome to databases in the Cloud
 
Redefining tables online without surprises
Redefining tables online without surprisesRedefining tables online without surprises
Redefining tables online without surprises
 
Protect Sensitive Data: Implementing Fine-Grained Access Control in Oracle
Protect Sensitive Data: Implementing Fine-Grained Access Control in OracleProtect Sensitive Data: Implementing Fine-Grained Access Control in Oracle
Protect Sensitive Data: Implementing Fine-Grained Access Control in Oracle
 
Oracle Exadata Maintenance tasks 101 - OTN Tour 2015
Oracle Exadata Maintenance tasks 101 - OTN Tour 2015Oracle Exadata Maintenance tasks 101 - OTN Tour 2015
Oracle Exadata Maintenance tasks 101 - OTN Tour 2015
 
My Experience Using Oracle SQL Plan Baselines 11g/12c
My Experience Using Oracle SQL Plan Baselines 11g/12cMy Experience Using Oracle SQL Plan Baselines 11g/12c
My Experience Using Oracle SQL Plan Baselines 11g/12c
 
Oracle RAC sin sorpresas - v2014
Oracle RAC sin sorpresas - v2014Oracle RAC sin sorpresas - v2014
Oracle RAC sin sorpresas - v2014
 
Alta disponibilidad con Pacemaker
Alta disponibilidad con PacemakerAlta disponibilidad con Pacemaker
Alta disponibilidad con Pacemaker
 
AROUG BIDAY 2013 - Automatizar procesos de ETL con PL/SQL
AROUG BIDAY 2013 - Automatizar procesos de ETL con PL/SQLAROUG BIDAY 2013 - Automatizar procesos de ETL con PL/SQL
AROUG BIDAY 2013 - Automatizar procesos de ETL con PL/SQL
 
MariaDB y FOSS en infraestructura de salud y estándares
MariaDB y FOSS en infraestructura de salud y estándaresMariaDB y FOSS en infraestructura de salud y estándares
MariaDB y FOSS en infraestructura de salud y estándares
 
UYOUG 2012 - Oracle RAC 11gR2 - New features
UYOUG 2012 - Oracle RAC 11gR2 - New featuresUYOUG 2012 - Oracle RAC 11gR2 - New features
UYOUG 2012 - Oracle RAC 11gR2 - New features
 
Collaborate 2012 - Administering MySQL for Oracle DBAs
Collaborate 2012 - Administering MySQL for Oracle DBAsCollaborate 2012 - Administering MySQL for Oracle DBAs
Collaborate 2012 - Administering MySQL for Oracle DBAs
 
Collaborate 2012 - RMAN Eliminate the mystery
Collaborate 2012 - RMAN Eliminate the mysteryCollaborate 2012 - RMAN Eliminate the mystery
Collaborate 2012 - RMAN Eliminate the mystery
 

Dernier

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...apidays
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesrafiqahmad00786416
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxRustici Software
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024The Digital Insurer
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...apidays
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024The Digital Insurer
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsNanddeep Nachan
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Jeffrey Haguewood
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 

Dernier (20)

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 

Terraform Tips and Tricks - LAOUC 2022

  • 1. Terraform Tips and Tricks LAOUC Community Tour 2022 August 17th Nelson Calero © Pythian Services Inc 2022 | 1
  • 2. Our Speaker Nelson Calero Internal Principal Consultant Pythian http://www.linkedin.com/in/ncalero @ncalerouy © Pythian Services Inc 2022 | 2
  • 3. 24 Years in Business _____________________ Premier Partner 140+ Certifications 8 Specializations 360+ Experts Across 5 Continents 500+ Customers Globally _____________________ Advanced Partner 175+ Certifications _____________________ Gold Partner 15+ Certifications _____________________ Platinum Partner 150+ Certifications _____________________ SAP Certified Partner 40+ Certifications © Pythian Services Inc 2022 | 3
  • 4. A G E N D A ● Terraform basics (1 slide) ● Terraform Idiosyncrasy ● Cloud provider Idiosyncrasy © Pythian Services Inc 2022 | 4
  • 5. Terraform basics ● Infrastructure as Code, declarative config files (HCL) ● Simple install - standalone binary file ● Few operations - plan/apply/destroy ● Providers - implement how to deal with your resources. 1000+ ● State file - mapping of existing resources to TF configuration ● Simple structure ○ Reads code from current directory ○ Split code in files for readability: main.tf, variables.tf, outputs.tf ● Advanced configurations - workspaces, modules, remote state files ● Changes in versions - Features, state file format © Pythian Services Inc 2021 | 5
  • 6. Terraform basics ● Infrastructure as Code, declarative config files (HCL) ● Simple install - standalone binary file ● Few operations - plan/apply/destroy ● Providers - implement how to deal with your resources. 1000+ ● State file - mapping of existing resources to TF configuration ● Simple structure ○ Reads code from current directory ○ Split code in files for readability: main.tf, variables.tf, outputs.tf ● Advanced configurations - workspaces, modules, remote state files ● Changes in versions - Features, state file format © Pythian Services Inc 2021 | 6 terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 3.27" } } required_version = ">= 0.14.9" } provider "aws" { profile = "default" region = "us-west-1" } resource "aws_instance" "app_server" { ami = "ami-830c94e3" instance_type = "t2.micro" tags = { Name = "AppServerInstance" } } ● https://www.terraform.io/ ● Lucas’ blog: https://technology.amis.nl/continuous-delivery/provisioning/terraform/
  • 7. Terraform idiosyncrasy 1) State file relevance 2) Concurrent work on same resources 3) Reducing scope of operations 4) Resource parameters validation 5) Re-using code 6) Available features = TF + provider © Pythian Services Inc 2021 | 7
  • 8. Terraform idiosyncrasy ● state file drives the changes - make sure is in sync with your resources ● state file - removing resources (state rm), importing (import name_in_state name_aws) ● State file lock from previous canceled/timed out execution: force-unlock to remove it ● Plan and apply only one resource from many: -target name_in_state ● Conditional resources in modules: count ● Re-using code among environments and/or deployments? Welcome terragrunt. Examples: ○ Standardize resource attributes (tags, HA, networking, security, parameters) between deployments - in same or different environments and regions ○ Support different database engines and versions ○ Validations (possible but limited to same attribute expression) ○ Modules, variables and environments - directory structure ○ Several other features - check https://terragrunt.gruntwork.io/ © Pythian Services Inc 2021 | 8
  • 9. Terraform idiosyncrasy - 1) state file Demo 1 Code taken from public Oracle repo (few changes for simplicity): https://github.com/oracle/terraform-provider-oci/tree/master/examples/database/adb © Pythian Services Inc 2021 | 9
  • 10. Terraform idiosyncrasy - state file © Pythian Services Inc 2021 | 10 $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_db_instance.master will be created + resource "aws_db_instance" "master" { + address = (known after apply) + allocated_storage = 500 + apply_immediately = (known after apply) ... + identifier ="nelson-ora19" ... Plan: 1 to add, 0 to change, 0 to destroy.
  • 11. Terraform idiosyncrasy - state file © Pythian Services Inc 2021 | 11 $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_db_instance.master will be created + resource "aws_db_instance" "master" { + address = (known after apply) + allocated_storage = 500 + apply_immediately = (known after apply) ... + identifier ="nelson-pg14" ... Plan: 1 to add, 0 to change, 0 to destroy. $ aws rds describe-db-instances --region=us-west-1 --db-instance-identifier nelson-ora19 --query "*[].{DBClusterIdentifier:DBClusterIdentifier,DBInstanceIdentifier:DBInstanceIdentifier,Engin eVersion:EngineVersion}" --output table -------------------------------------------------------------------------------------- | DescribeDBInstances | +---------------------+------------------------+-------------------------------------+ | DBClusterIdentifier | DBInstanceIdentifier | EngineVersion | +---------------------+------------------------+-------------------------------------+ | None | nelson-ora19 | 19.0.0.0.ru-2021-07.rur-2021-07.r1 | +---------------------+------------------------+-------------------------------------+
  • 12. Terraform idiosyncrasy - state file © Pythian Services Inc 2021 | 12 $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_db_instance.master will be created + resource "aws_db_instance" "master" { + address = (known after apply) + allocated_storage = 500 + apply_immediately = (known after apply) ... + identifier ="nelson-pg14" ... Plan: 1 to add, 0 to change, 0 to destroy. $ aws rds describe-db-instances --region=us-west-1 --db-instance-identifier nelson-ora19 --query "*[].{DBClusterIdentifier:DBClusterIdentifier,DBInstanceIdentifier:DBInstanceIdentifier,Engin eVersion:EngineVersion}" --output table -------------------------------------------------------------------------------------- | DescribeDBInstances | +---------------------+------------------------+-------------------------------------+ | DBClusterIdentifier | DBInstanceIdentifier | EngineVersion | +---------------------+------------------------+-------------------------------------+ | None | nelson-ora19 | 19.0.0.0.ru-2021-07.rur-2021-07.r1 | +---------------------+------------------------+-------------------------------------+ $ terraform import aws_db_instance.master nelson-ora19 aws_db_instance.master: Importing from ID "nelson-ora19"... aws_db_instance.master: Import prepared! Prepared aws_db_instance for import aws_db_instance.master: Refreshing state... [id=nelson-ora19] Import successful! The resources that were imported are shown above. These resources are now in your Terraform state and will henceforth be managed by Terraform.
  • 13. Terraform idiosyncrasy - state file © Pythian Services Inc 2021 | 13 $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_db_instance.master will be created + resource "aws_db_instance" "master" { + address = (known after apply) + allocated_storage = 500 + apply_immediately = (known after apply) ... + identifier ="nelson-pg14" ... Plan: 1 to add, 0 to change, 0 to destroy. $ aws rds describe-db-instances --region=us-west-1 --db-instance-identifier nelson-pg14 --query "*[].{DBClusterIdentifier:DBClusterIdentifier,DBInstanceIdentifier:DBInstanceIdentifier,Engin eVersion:EngineVersion}" --output table -------------------------------------------------------------------------------------- | DescribeDBInstances | +---------------------+------------------------+-------------------------------------+ | DBClusterIdentifier | DBInstanceIdentifier | EngineVersion | +---------------------+------------------------+-------------------------------------+ | None | nelson-ora19 | 19.0.0.0.ru-2021-07.rur-2021-07.r1 | +---------------------+------------------------+-------------------------------------+ $ terraform import aws_db_instance.master nelson-pg14 aws_db_instance.master: Importing from ID "nelson-pg14"... aws_db_instance.master: Import prepared! Prepared aws_db_instance for import aws_db_instance.master: Refreshing state... [id=nelson-pg14] Import successful! The resources that were imported are shown above. These resources are now in your Terraform state and will henceforth be managed by Terraform. $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... aws_db_instance.master: Refreshing state... [id=nelson-ora19] ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: ~ update in-place Terraform will perform the following actions: # aws_db_instance.master will be updated in-place ~ resource "aws_db_instance" "master" { ... + password = (sensitive value) ... Plan: 0 to add, 1 to change, 0 to destroy.
  • 14. Terraform idiosyncrasy - state file © Pythian Services Inc 2021 | 14 $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_db_instance.master will be created + resource "aws_db_instance" "master" { + address = (known after apply) + allocated_storage = 500 + apply_immediately = (known after apply) ... + identifier ="nelson-pg14" ... Plan: 1 to add, 0 to change, 0 to destroy. $ aws rds describe-db-instances --region=us-west-1 --db-instance-identifier nelson-pg14 --query "*[].{DBClusterIdentifier:DBClusterIdentifier,DBInstanceIdentifier:DBInstanceIdentifier,Engin eVersion:EngineVersion}" --output table -------------------------------------------------------------------------------------- | DescribeDBInstances | +---------------------+------------------------+-------------------------------------+ | DBClusterIdentifier | DBInstanceIdentifier | EngineVersion | +---------------------+------------------------+-------------------------------------+ | None | nelson-ora19 | 19.0.0.0.ru-2021-07.rur-2021-07.r1 | +---------------------+------------------------+-------------------------------------+ $ terraform import aws_db_instance.master nelson-pg14 aws_db_instance.master: Importing from ID "nelson-pg14"... aws_db_instance.master: Import prepared! Prepared aws_db_instance for import aws_db_instance.master: Refreshing state... [id=nelson-pg14] Import successful! The resources that were imported are shown above. These resources are now in your Terraform state and will henceforth be managed by Terraform. $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... aws_db_instance.master: Refreshing state... [id=nelson-pg14] ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: ~ update in-place Terraform will perform the following actions: # aws_db_instance.master will be updated in-place ~ resource "aws_db_instance" "master" { ... + password = (sensitive value) ... Plan: 0 to add, 1 to change, 0 to destroy. $ terraform apply … $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... aws_db_instance.master: Refreshing state... [id=nelson-ora19] ------------------------------------------------------------------------ No changes. Infrastructure is up-to-date. This means that Terraform did not detect any differences between your configuration and real physical resources that exist. As a result, no actions need to be performed.
  • 15. Terraform idiosyncrasy - state file ● Default state file is terraform.tfstate in current directory ● Critical data, requires good backup policy ● terraform import is not always able to recreate all attributes (ex: passwords, MySQL grants) ● Two key complementary features: ○ remote state file https://www.terraform.io/language/state/remote ○ OCI discovery https://registry.terraform.io/providers/oracle/oci/latest/docs/guides/resource_discovery © Pythian Services Inc 2021 | 15
  • 16. Terraform idiosyncrasy - state file / remote ● Remote state file - https://www.terraform.io/language/state/remote © Pythian Services Inc 2021 | 16 remote_state { backend "s3" { bucket = "terraform-states" key = "networking/terraform.tfstate" region = "us-phoenix-1" endpoint = "https://acme.compat.objectstorage.us-phoenix-1.oraclecloud.com" shared_credentials_file = "../terraform-states_bucket_credentials" skip_region_validation = true skip_credentials_validation = true skip_metadata_api_check = true force_path_style = true } } Ref: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/terraformUsingObjectStore.htm
  • 17. Terraform idiosyncrasy - state file / OCI discovery Demo 2 OCI discovery docs: https://registry.terraform.io/providers/oracle/oci/latest/docs/guides/resource_discovery © Pythian Services Inc 2021 | 17
  • 18. Terraform idiosyncrasy - state file / OCI discovery ● terraform-provider-oci gets installed by terraform: ● Discover databases and create the state file: © Pythian Services Inc 2021 | 18 $ ls -lrt .terraform/providers/registry.terraform.io/hashicorp/oci/4.77.0/linux_amd64/ total 129320 -rw-r--r-- 1 calero calero 96384 May 31 17:31 CHANGELOG.md -rw-r--r-- 1 calero calero 16725 May 31 17:31 LICENSE.md -rw-r--r-- 1 calero calero 3071 May 31 17:31 README.md -rwxr-xr-x 1 calero calero 132177920 May 31 17:31 terraform-provider-oci_v4.77.0 $ mkdir discover $ .terraform/providers/registry.terraform.io/hashicorp/oci/4.77.0/linux_amd64/terraform-provider-oci_v4.77.0 -command=export -compartment_id=$COMPID -services database -generate_state -output_path=discover/ ... $ ls -lrt discover total 44 -rw-r--r-- 1 calero calero 10109 May 31 18:46 terraform.tfstate.tmp.backup -rw-r--r-- 1 calero calero 22086 May 31 18:46 terraform.tfstate -rw-r--r-- 1 calero calero 6085 May 31 18:46 database.tf -rw-r--r-- 1 calero calero 38 May 31 18:46 provider.tf -rw-r--r-- 1 calero calero 171 May 31 18:46 vars.tf $
  • 19. Terraform idiosyncrasy - state file / OCI discovery ● terraform-provider-oci gets installed by terraform: ● Discover databases and create the state file: © Pythian Services Inc 2021 | 19 $ ls -lrt .terraform/providers/registry.terraform.io/hashicorp/oci/4.77.0/linux_amd64/ total 129320 -rw-r--r-- 1 calero calero 96384 May 31 17:31 CHANGELOG.md -rw-r--r-- 1 calero calero 16725 May 31 17:31 LICENSE.md -rw-r--r-- 1 calero calero 3071 May 31 17:31 README.md -rwxr-xr-x 1 calero calero 132177920 May 31 17:31 terraform-provider-oci_v4.77.0 $ mkdir discover $ .terraform/providers/registry.terraform.io/hashicorp/oci/4.77.0/linux_amd64/terraform-provider-oci_v4.77.0 -command=export -compartment_id=$COMPID -services database -generate_state -output_path=../discover/ ... $ ls -lrt ../discover2 total 44 -rw-r--r-- 1 calero calero 10109 May 31 18:46 terraform.tfstate.tmp.backup -rw-r--r-- 1 calero calero 22086 May 31 18:46 terraform.tfstate -rw-r--r-- 1 calero calero 6085 May 31 18:46 database.tf -rw-r--r-- 1 calero calero 38 May 31 18:46 provider.tf -rw-r--r-- 1 calero calero 171 May 31 18:46 vars.tf $ $ cd discover $ terraform plan oci_database_autonomous_database.export_example_autonomous_data_warehouse: Refreshing state... [id=ocid1.autonomousdatabase.oc1.iad.anuwcljst77gzoiaaewxote6h4i73oamzoloxdueabnpwprspzngjoce52xq] oci_database_autonomous_database.export_example_autonomous_database: Refreshing state... [id=ocid1.autonomousdatabase.oc1.iad.anuwcljtt77gzoiamz3gg5cfsbdqgcoj7lrobis75qp7lwubm7d77bpblqgq] No changes. Your infrastructure matches the configuration. Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed. $
  • 20. Terraform idiosyncrasy - 2) concurrent work © Pythian Services Inc 2021 | 20 $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... aws_rds_cluster.master: Refreshing state... [id=nelson-auroramysql56] aws_rds_cluster_instance.cluster_instances[0]: Refreshing state... [id=nelson-auroramysql56-0] aws_rds_cluster_instance.cluster_instances[1]: Refreshing state... [id=nelson-auroramysql56-1] mysql_user.monitor: Refreshing state... [id=monitor@%] Error: Could not connect to server: dial tcp 127.0.0.1:3306: connect: connection refused
  • 21. Terraform idiosyncrasy - concurrent work © Pythian Services Inc 2021 | 21 $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... aws_rds_cluster.master: Refreshing state... [id=nelson-auroramysql56] aws_rds_cluster_instance.cluster_instances[0]: Refreshing state... [id=nelson-auroramysql56-0] aws_rds_cluster_instance.cluster_instances[1]: Refreshing state... [id=nelson-auroramysql56-1] mysql_user.checkmk_mon: Refreshing state... [id=checkmk_mon@%] Error: Could not connect to server: dial tcp 127.0.0.1:3306: connect: connection refused $ terraform plan ... Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed Lock Info: ID: f6f04b96-de6d-754a-7bbe-8d0504f75e79 Path: my-tf-state-286052569778/test/us-west-1/services/my-db/aurora-mysql/terraform.tfstate Operation: OperationTypePlan Who: ncalero@tst-rds01 Version: 0.12.31 Created: 2021-12-28 13:45:41.113713713 +0000 UTC Info: $
  • 22. Terraform idiosyncrasy - concurrent work © Pythian Services Inc 2021 | 22 $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_security_group.sgpr: Refreshing state... aws_rds_cluster.master: Refreshing state... [id=nelson-auroramysql56] aws_rds_cluster_instance.cluster_instances[0]: Refreshing state... [id=nelson-auroramysql56-0] aws_rds_cluster_instance.cluster_instances[1]: Refreshing state... [id=nelson-auroramysql56-1] mysql_user.checkmk_mon: Refreshing state... [id=checkmk_mon@%] Error: Could not connect to server: dial tcp 127.0.0.1:3306: connect: connection refused $ terraform plan ... Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed Lock Info: ID: f6f04b96-de6d-754a-7bbe-8d0504f75e79 Path: my-tf-state-286052569778/stable/us-west-1/services/my-db/aurora-mysql/terraform.tfstate Operation: OperationTypePlan Who: c_ncalero@gds-stg-rds01 Version: 0.12.31 Created: 2021-12-28 13:45:41.113713713 +0000 UTC Info: $ $ terraform force-unlock f6f04b96-de6d-754a-7bbe-8d0504f75e79 Do you really want to force-unlock? Terraform will remove the lock on the remote state. This will allow local Terraform commands to modify this state, even though it may be still be in use. Only 'yes' will be accepted to confirm. Enter a value: yes Terraform state has been successfully unlocked! The state has been unlocked, and Terraform commands should now be able to obtain a new lock on the remote state. $
  • 23. Terraform idiosyncrasy - 3) reduce scope © Pythian Services Inc 2021 | 23 $ terraform plan data.oci_database_autonomous_db_versions.test_adb_versions: Reading... oci_database_autonomous_database.autonomous_data_warehouse: Refreshing state... [id=ocid1.autonomousdatabase.oc1.iad.anuwcljst77gzoiaaewxote6h4i73oamzoloxdueabnpwprspzngjoce52xq] oci_database_autonomous_database.autonomous_database: Refreshing state... [id=ocid1.autonomousdatabase.oc1.iad.anuwcljtt77gzoiamz3gg5cfsbdqgcoj7lrobis75qp7lwubm7d77bpblqgq] data.oci_database_autonomous_databases.autonomous_databases: Reading... ... Plan: 0 to add, 1 to change, 1 to destroy. $ terraform state list data.http.ip data.oci_database_autonomous_db_versions.test_adb_versions oci_database_autonomous_database.autonomous_data_warehouse oci_database_autonomous_database.autonomous_database random_string.adb_admin_password $ terraform plan -target oci_database_autonomous_database.autonomous_database data.oci_database_autonomous_db_versions.test_adb_versions: Reading... oci_database_autonomous_database.autonomous_database: Refreshing state... [id=ocid1.autonomousdatabase.oc1.iad.anuwcljtt77gzoiamz3gg5cfsbdqgcoj7lrobis75qp7lwubm7d77bpblqgq] No changes. Your infrastructure matches the configuration.
  • 24. Terraform idiosyncrasy - 4) parameter validations © Pythian Services Inc 2021 | 24 $ terraform apply … aws_rds_cluster.master[0]: Creating... aws_rds_cluster.master[0]: Still creating... [10s elapsed] aws_rds_cluster.master[0]: Still creating... [20s elapsed] aws_rds_cluster.master[0]: Still creating... [30s elapsed] aws_rds_cluster.master[0]: Still creating... [40s elapsed] aws_rds_cluster.master[0]: Creation complete after 42s [id=my-aurora] aws_rds_cluster_instance.cluster_instances[1]: Creating... aws_rds_cluster_instance.cluster_instances[0]: Creating... Error: error creating RDS Cluster (my-aurora) Instance: InvalidParameterCombination: RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t3.small, Engine=aurora-mysql, EngineVersion=5.7. For supported combinations of instance class and database engine version, see the documentation. status code: 400, request id: 7c15bd63-b7cf-47a3-8a10-faee35c3e9c4 on main.tf line 7, in resource "aws_rds_cluster_instance" "cluster_instances": 7: resource "aws_rds_cluster_instance" "cluster_instances" { Error: error creating RDS Cluster (my-aurora) Instance: InvalidParameterCombination: RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t3.small, Engine=aurora-mysql, EngineVersion=5.7. For supported combinations of instance class and database engine version, see the documentation. status code: 400, request id: a8cb9644-fbe2-40ee-929b-18094709cb33 on main.tf line 7, in resource "aws_rds_cluster_instance" "cluster_instances": 7: resource "aws_rds_cluster_instance" "cluster_instances" {
  • 25. Terraform idiosyncrasy - 4) parameter validations ● Terraform pass parameters to the provider - it runs the operation on the cloud service and it fails ● Some providers include data sources with attributes having a list of valid values for that (only if we use those when manipulating the resource) ○ Example: OCI Autonomous database https://registry.terraform.io/providers/oracle/oci/latest/docs/data-sources/database_autonomous_db_versions ● Additionally, simple validations can be incorporated in the variable definition: © Pythian Services Inc 2021 | 25 $ grep -A13 "autonomous_database_db_workload" variables.tf variable "autonomous_database_db_workload" { default = "OLTP" validation { condition = anytrue([ var.autonomous_database_db_workload == "OLTP", var.autonomous_database_db_workload == "DW", var.autonomous_database_db_workload == "APEX", var.autonomous_database_db_workload == "AJD" ]) error_message = "Invalid autonomous_database_db_workload value - allowed OLTP, DW, APEX and AJD." } }
  • 26. Terraform idiosyncrasy - parameter validations © Pythian Services Inc 2021 | 26 Demo 3
  • 27. Terraform idiosyncrasy - 5) reusing code ● Terraform by default read the current directory only ● Can include other directories using the “module” directive: https://learn.hashicorp.com/collections/terraform/modules ● Third party tools to re-using code (among environments and/or deployments) - terragrunt example: ○ Standardize resource attributes (tags, HA, networking, security, parameters) between deployments - in same or different environments and regions ○ Support different database engines and versions ○ Validations (possible but limited to same attribute expression) ○ Modules, variables and environments - directory structure ○ Extra features: hooks, functions, etc. - check https://terragrunt.gruntwork.io/ © Pythian Services Inc 2021 | 27 module "ec2_instances" { source = "terraform-aws-modules/ec2-instance/aws" version = "3.5.0" ...
  • 28. Terraform idiosyncrasy - 6) “features” ● Terraform binary ○ Aurora MySQL upgrade didn’t work on TF versions lower than 0.12.31 (current version: 1.1.7) ● Terraform provider ○ MySQL upgrade didn’t work with AWS provider lower than 3.63.0 (several bugs) ○ Resource discovery implemented in OCI (not in AWS as of today) ● Cloud provider limits/restrictions ○ Resource attributes are not validated (service restrictions) - cause runtime errors ■ allowed instance type for a RDS version (both ways) ■ IOPS parameter when using gp2 storage ■ CLI commands to the rescue (to confirm allowed values) © Pythian Services Inc 2021 | 28
  • 29. Cloud provider idiosyncrasy ● You can get errors today that does not repeat tomorrow, as services improves ● Don’t assume all services work the same for similar operations - RTFM! © Pythian Services Inc 2021 | 29 ● Examples on AWS: ○ MySQL minor upgrade fails without any error - not reproducible days later when repeating the test ○ RDS “create Aurora replica” is not available in some versions after upgrades ○ Aurora space used should be checked using metrics ○ RDS parameters have apply_mode immediate or pending-reboot, but tricky to update running instances automatically (immediate mode) ○ Operations on RDS storage does not allow other operations until they finish - and those can take long ■ Automatic space increment - minutes ■ Change type between gp2 and IOPS - hours
  • 30. Terraform - parameters cheat sheet © Pythian Services Inc 2021 | 30 Format code in .tf files terraform fmt Validate configuration terraform validate List state file resources terraform state list Inspect current state file content terraform show Remove a resource from state file terraform state rm <name_in_state> Import resource into state file terraform import <name_in_state> <name_aws> Remove lock from state file terraform force-unlock <lock_id> Operate on a single resource terraform <op> -target <name_in_state> Console (to test formulas/expressions) terraform console Debug execution TF_LOG="TRACE" TF_LOG_PATH=file.log ; terraform <op>
  • 31. Terraform console © Pythian Services Inc 2021 | 31 [c_ncalero@myhost ~]$ terraform console > substr("19.2",0,2) 19 > split(".","19.2") [ "19", "2", ] > split(".","19.2")[0] 19 > element(split(".","19.2"),0) 19 > join("",["oracle",substr("19.2",0,2), "small"]) oracle19small > join("",["oracle",split(".","19.2")[0]]) oracle19 > exit
  • 32. OCI CLI - useful validations © Pythian Services Inc 2021 | 32 $ export COMPID=$(oci iam compartment list --query "data [?name=='mycmp'].{id:id}" | jq -r '.[].id') $ oci db version list -c $COMPID --db-system-shape "VM.Standard1.1" --output table +-----------------------------+--------------+-----------------+ | is-latest-for-major-version | supports-pdb | version | +-----------------------------+--------------+-----------------+ | True | False | 11.2.0.4 | | False | False | 11.2.0.4.210119 | | False | False | 11.2.0.4.210420 | | True | True | 12.1.0.2 | | False | True | 12.1.0.2.211019 | | False | True | 12.1.0.2.220118 | | False | True | 12.1.0.2.220419 | | True | True | 12.2.0.1 | | False | True | 12.2.0.1.211019 | | False | True | 12.2.0.1.220118 | | False | True | 12.2.0.1.220419 | | True | True | 18.0.0.0 | | False | True | 18.16.0.0 | | True | True | 19.0.0.0 | | False | True | 19.13.0.0 | | False | True | 19.14.0.0 | ● Available database versions for DB Systems using shape VM.Standard1.1 | False | True | 19.15.0.0 | | True | True | 21.0.0.0 | | False | True | 21.4.0.0 | | False | True | 21.5.0.0 | | False | True | 21.6.0.0 | +-------------+--------------+-----------+
  • 33. OCI CLI - useful validations © Pythian Services Inc 2021 | 33 $ export COMPID=$(oci iam compartment list --query "data [?name=='mycmp'].{id:id}" | jq -r '.[].id' ) $ oci db autonomous-database list -c $COMPID --query 'data[].{db_name:"db-name", db_version:"db-version", db_workload:"db-workload", display_name:"display-name", state:"lifecycle-state"}' --output table +--------------------+------------+-------------+-----------------------------------+-----------+ | db_name | db_version | db_workload | display_name | state | +--------------------+------------+-------------+-----------------------------------+-----------+ | adbdw | 19c | DW | example_autonomous_data_warehouse | AVAILABLE | | adboltp | 19c | OLTP | example_autonomous_database | AVAILABLE | +--------------------+------------+-------------+-----------------------------------+-----------+ ● ADBs created in compartment ’mycmp’
  • 34. AWS CLI - useful validations © Pythian Services Inc 2021 | 34 $ aws rds describe-orderable-db-instance-options --engine oracle-ee --db-instance-class db.t3.small --query "OrderableDBInstanceOptions[].{EngineVersion:EngineVersion}" --output text --region eu-west-1 12.1.0.2.v1 12.1.0.2.v10 12.1.0.2.v11 12.1.0.2.v12 12.1.0.2.v13 12.1.0.2.v14 12.1.0.2.v15 12.1.0.2.v16 12.1.0.2.v17 12.1.0.2.v18 12.1.0.2.v19 12.1.0.2.v2 12.1.0.2.v20 12.1.0.2.v21 12.1.0.2.v22 12.1.0.2.v23 12.1.0.2.v24 12.1.0.2.v25 12.1.0.2.v26 12.1.0.2.v27 ● Available Oracle EE versions for shape db.t3.small 12.1.0.2.v3 12.1.0.2.v4 12.1.0.2.v5 12.1.0.2.v6 12.1.0.2.v7 12.1.0.2.v8 12.1.0.2.v9 19.0.0.0.ru-2019-07.rur-2019-07.r1 19.0.0.0.ru-2019-10.rur-2019-10.r1 19.0.0.0.ru-2020-01.rur-2020-01.r1 19.0.0.0.ru-2020-04.rur-2020-04.r1 19.0.0.0.ru-2020-07.rur-2020-07.r1 19.0.0.0.ru-2020-10.rur-2020-10.r1 19.0.0.0.ru-2021-01.rur-2021-01.r1 19.0.0.0.ru-2021-01.rur-2021-01.r2 19.0.0.0.ru-2021-04.rur-2021-04.r1 19.0.0.0.ru-2021-07.rur-2021-07.r1 19.0.0.0.ru-2021-10.rur-2021-10.r1 19.0.0.0.ru-2022-01.rur-2022-01.r1
  • 35. AWS CLI - useful validations © Pythian Services Inc 2021 | 35 $ aws rds describe-db-engine-versions --engine oracle-ee --engine-version 12.1.0.2.v20 --query "DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text 12.1.0.2.v21 12.1.0.2.v22 12.1.0.2.v23 12.1.0.2.v24 12.1.0.2.v25 12.1.0.2.v26 12.1.0.2.v27 19.0.0.0.ru-2020-04.rur-2020-04.r1 19.0.0.0.ru-2020-07.rur-2020-07.r1 19.0.0.0.ru-2020-10.rur-2020-10.r1 19.0.0.0.ru-2021-01.rur-2021-01.r1 19.0.0.0.ru-2021-01.rur-2021-01.r2 19.0.0.0.ru-2021-04.rur-2021-04.r1 19.0.0.0.ru-2021-07.rur-2021-07.r1 19.0.0.0.ru-2021-10.rur-2021-10.r1 19.0.0.0.ru-2022-01.rur-2022-01.r1 ● Supported upgrade paths for Oracle version 12.1.0.2
  • 36. AWS CLI - useful validations © Pythian Services Inc 2021 | 36 $ REGION=us-west-1; for i in $(awscloudwatch list-metrics --region=$REGION --namespace AWS/RDS --metric-name VolumeBytesUsed --query "Metrics[].Dimensions[].{Name:Name, Value:Value}" --output table | grep DBClusterIdentifier | cut -d"|" -f3) ; do echo -n $i " " ; aws cloudwatch get-metric-statistics --region $REGION --namespace AWS/RDS --metric-name VolumeBytesUsed --dimensions Name=EngineName,Value=aurora-mysql --dimensions Name=EngineName,Value=aurora Name=DbClusterIdentifier,Value=$i --start-time $(date -u -d "1 days ago" '+%F') --end-time $(date -u '+%F') --statistics Maximum --period 3600 --query "Datapoints[].{Timestamp:Timestamp, Maximum:Maximum, Unit:Unit}" | jq '.[] | .Maximum' | sort -u | tail -1 ; done | sort bookshop-db 7381361362844 sales-db 179468653458 track-db 20681679872 programs-db 35563371113 analytics-db 8650433415 master-db 17432850551 resourcing-db 6418328690 … ● Space in use by aurora Mysql instances running in region us-west-1
  • 37. AWS CLI - useful validations © Pythian Services Inc 2021 | 37 $ aws rds describe-db-parameters --db-parameter-group-name mypg --region us-west-1 --query "sort_by(Parameters,&ParameterName)[?Source=='user'].{ParameterName:ParameterName, ApplyMethod:ApplyMethod, IsModifiable:IsModifiable, ParameterValue:ParameterValue}" --output table +----------------+---------------+-----------------------------------+------------------+ | ApplyMethod | IsModifiable | ParameterName | ParameterValue | +----------------+---------------+-----------------------------------+------------------+ | immediate | True | event_scheduler | ON | | immediate | True | group_concat_max_len | 16384 | | pending-reboot| True | innodb_sort_buffer_size | 8388608 | | immediate | True | log_bin_trust_function_creators | 1 | | immediate | True | long_query_time | 0.1 | | pending-reboot| True | max_allowed_packet | 1073741824 | | immediate | True | max_connections | 8000 | | immediate | True | slow_query_log | 1 | +----------------+---------------+-----------------------------------+------------------+ ● Parameters set by user (we) in a RDS parameter group
  • 38. Questions? © Pythian Services Inc 2021 | 38 calero@pythian.com @ncalerouy http://www.linkedin.com/in/ncalero