This document summarizes a talk on automating database infrastructures using MariaDB, MySQL and Ansible. It discusses Ansible concepts like inventories, modules, playbooks, roles, plays, variables and facts. It provides code examples of using Ansible to automate the deployment and configuration of MariaDB and MySQL database servers through plays, roles, variables and templates. It also discusses best practices for making tasks idempotent, using conditional tasks, tags and validating variables.

1. MariaDB, MySQL and Ansible:
automating database infrastructures
Federico Razzoli

2. $ whoami
Hi, I’m Federico Razzoli from Vettabase Ltd
Database consultant, open source supporter,
long time MariaDB and MySQL user
● vettabase.com
● Federico-Razzoli.com

3. This talk
● General Ansible concepts
● Opinionated code examples, automating MariaDB/MySQL
● Good and bad practices

4. Ansible Overview

5. What is Ansible
● A tool for deployment and configuration automation
● Infrastructure as code
● Idempotent
● Simple, non-centralised

6. Ansible Concepts
● Inventory
● Module
● Playbook
● Role
● Play
● Variable
● Facts

7. Inventories

8. Inventory example
[pmm]
pmm-1
[mariadb_main]
mariadb-main-1
mariadb-main-2
mariadb-main-3
[mariadb_main:vars]
pmm_server_host=pmm-1
pmm_server_port=80

9. Ansible hosts
[pmm]
pmm-1 ansible_host=123.123.123.123
[mariadb_main]
mariadb-main-1 ansible_host=123.0.0.1
mariadb-main-2 ansible_host=123.0.0.2
mariadb-main-3 ansible_host=123.0.0.3
[mariadb_main:vars]
pmm_server_host=123.123.123.123
pmm_server_port=80
● Sometimes we cannot use meaningful hostnames

10. Hierarchical groups
[mariadb_master]
mariadb-master-1
mariadb-master-2
[mariadb_replica]
mariadb-replica-1
mariadb-replica-2
[mariadb:children]
mariadb_master
mariadb_replica
● We have different groups for masters and replicas, and a supergroup that
includes them both

11. Groups goals
● You assign each group some roles
● You can deploy one or more group

12. Multiple inventories
production_db:
[mariadb_master]
mariadb-master-1 ansible_host=111.0.0.1
mariadb-master-2 ansible_host=111.0.0.2
staging_db:
[mariadb_master]
mariadb-master-1 ansible_host=222.0.0.1
mariadb-master-2 ansible_host=222.0.0.2

13. Plays

14. Play example
---
- hosts: mariadb_main
roles:
- linux-for-mysql
- mariadb
- hosts: mysql_main
roles:
- linux-for-mysql
- mysql
● You assign roles to groups (or individual hosts)

15. Applying all roles to a certain host
Call Ansible specifying an inventory and a play:
% ansible-playbook -i staging -l mariadb_main main.yml

16. Play example
Here is where it can be convenient to have multiple inventories:
% ansible-playbook -i staging -l mariadb_main main.yml
% ansible-playbook -i production -l mariadb_main main.yml

17. Roles, variables and templates

18. Example
- name: Create script dir
file:
name: "{{ script_dir }}"
owner: root
group: root
mode: 755
state: directory
● This is a task, but the term is misleading
● A role is a list of tasks
● The name appears on screen
● "file" is the module
● script_dir is a variable

19. Defining variables
script_dir: /opt/scripts
base_dir: /usr/local/mysql
...
● /host_vars/<host_name>.yml
● /group_vars/<group_name>.yml
● /roles/<role_name>/defaults/main.yml

20. Notes on variables
● The same variable can be used by multiple roles
● Roles from Ansible Galaxy can’t use this opportunity
● This is a good reason to make your own roles
● Document what each variable does in README.md

21. Copying a configuration file
- name: Copy my.cnf
copy:
src: ./files/my.cnf
dest: "/etc/mysql/{{ inventory_hostname }}.cnf"
● my.cnf is copied to /etc/mysql

22. Using a template
- name: Copy my.cnf
template:
src: ./templates/my.cnf .j2
dest: /etc/mysql/my.cnf
● This time my.cnf is a Jinja 2 template

23. Template example
[mysqld]
user = mysql
datadir = {{ mysql_data_dir }}
innodb_buffer_pool_size = {{ innodb_buffer_pool_size }}
● This time my.cnf is a Jinja 2 template

24. Problems
● If we want to set a variable, we should be able to add it in one place
○ host_vars or group_vars or role's defaults
● We can make a template more dynamic

25. Dynamic template example
In group_vars we create a list of dictionaries:
(arrays of objects, if you prefer)
group_mysql_variables:
- { name: 'innodb_buffer_pool_size', value: '50G' }
- { name: 'innodb_log_file_size', value: '50G' }
In host_vars we optionally create another list:
host_mysql_variables:
- { name: 'innodb_buffer_pool_size', value: '80G' }

26. Dynamic template example
We loop over both the lists in the template:
{% for var in group_mysql_variables %}
{{ var.name }} = {{ var.value }}
{% endfor %}
{% if host_mysql_variables is defined %}
{% for var in host_mysql_variables %}
{{ var.name }} = {{ var.value }}
{% endfor %}
{% endif %}

27. Notes
● We cannot use the same list in group_vars and host_vars, because the
one in host_vars would overwrite the whole list
● Some variables may appear twice in the configuration file
● This is fine: the last occurrence of a variable will override the previous one

28. Another use for lists
Hosts of a cluster can be a list too:
# print the IPs separated by commas
wsrep_cluster_address = gcomm://{{ private_ips|join(',') }}
# first node is the donor
wsrep_sst_donor = {{ cluster_hosts[0].node_name }}
# Only the first node imports a backup when the cluster is created
- name: Import backup
...
when: cluster_hosts[0] == hostvars[inventory_hostname]['ansible_default_ipv4']['address']

29. Validate variables
Validate variables for complex roles:
- name: Validate cluster size
assert:
that: cluster_hosts|length > 0
fail_msg: cluster_hosts must contain at least 1 element
success_msg: "cluster_hosts size: {{ cluster_hosts|length }}"
- name: Validate cluster IPs
assert:
that: "'{{ item.public_ip }}'|ipaddr"
fail_msg: "{{ item.public_ip }} is not a valid IP"
success_msg: "{{ item.public_ip }}: OK"
with_items: "{{ cluster_hosts }}"

30. Tags and similar features

31. Conditional tasks
● Certain tasks should only be run for certain servers:
- name: Make server read only
mysql_variables:
variable: read_only
value: 1
when:
is_replica is sameas true
● Sometimes this avoids the need to create multiple roles which are very similar
● Instead of creating mariadb and mariadb_replica roles, we may create
mariadb only, with some conditional tasks
○ Sometimes it makes sense, sometimes it doesn't

32. Conditional tasks
● We can group optional tasks into separate files:
- name: Make server read only
include: replica.yml
when:
is_replica is sameas true

33. Idempotency of Tasks
● Most modules are idempotent
● This means that you can run a task like this multiple times safely:
- name: Start MariaDB
service:
state: started
● If the service is already running, nothing happens

34. Idempotency of Tasks
● Sometimes you make non-idempotent tasks on purpose:
- Copy my.cnf
...
- name: Restart MariaDB
service:
state: restarted
● If the service is already running, nothing happens

35. Idempotency of Tasks
● Sometimes you'd like a task to be idempotent, but it can't be:
- name: Run a system command
shell: >
rm "{{ mysql_data_dir }}"/*
- name: Run a script
shell: >
"{{ mysql_script_dir }}"/my_script.py
- name: Run some SQL
mysql_query:
query: CREATE OR REPLACE TABLE db.table ( ... );
● Ansible doesn't understand system commands, scripts or queries, so it has not
way to check if they were already run. It runs them every time.

36. Tags
● Ansible supports tags
● Each task can be assigned a list of tags:
- name: (Re)create users
...
- name: Update timezone info
tags: [ tzinfo-update ]
...
- name: Do something else
...
ansible-playbook -i production_mysql --tag tzinfo-update production.yml

37. Tags
● Typical tasks I want to have:
- name: Do something
- name: Copy my.cnf
tags: [ mysql-config- static-update ]
- name: Restart MySQL
tags: [ mysql-config- static-update ]
- name: Set variables at runtime
tags: [ mysql-config- dynamic-update ]
- name: Do something else

38. Tags
● So there is a way to selectively run certain tasks
● But what if we want to selectively exclude certain tasks?
● There is not built-in way, but we can do something like this:
- name: Do something
- name: Restart MySQL
service:
state: restarted
when: {{ mysql_restart }} is defined and {{ mysql_restart }} is sameas true
- name: Do something else mething else
ansible-playbook -i production_mysql
--tag mysql-config-dynamic-update -e 'mysql_no_restart=1' production.yml

39. Thanks for attending!
Question time :-)

40. xxxxx
● Xx
● yy

41. Example
CREATE OR REPLACE TABLE ticket (
id INT PRIMARY KEY NOT NULL AUTO_INCREMENT,
state ENUM('OPEN', 'VERIFIED', 'FIXED', 'INVALID') NOT NULL
DEFAULT 'OPEN',
summary VARCHAR(200) NOT NULL,
description TEXT NOT NULL
)
ENGINE InnoDB
;
● We want to start to track changes to bugs over time