Or: how to build a complete system from scratch. It begins by the requirements to have an installation process easy to repeat, documented and auditable.

1. Deployment Automation
Riccardo Lemmi aka AXA

2. Who am I?
● Python, Zope and Plone Developer (from 2001)
● Sysadmin (Debian, Ubuntu)
● ...but I like to say “Über Developer”
● Metalhead
● Multi-instrumentalist

3. Deployment Automation
Or: how to build a complete system from scratch
It begins by the requirements to have an installation process easy to
repeat, documented and auditable.
We are going to discuss about
● "Vagrant” to create virtual machine,
● "fabric" to automate operations
● and the tools to deploy on Amazon Web Services (AWS)

4. Motivation
To find an easy way to reproduce the installation process,
so our co-workers can replicate the deployment without pain.

5. Some definitions
SW deployment
Software deployment is the set of all the activities that make a software
system available for use
"Deployment" should be interpreted as a general process that has to be
customized according to specific requirements or characteristics.

6. Some definitions
Deployment activities
Release
It follows from the completed development process
Installation
It can vary from simple to a complex set of activities
and it can involve continuous delivery

7. Some definitions
Deployment activities
Update
It replaces an earlier version of sw with a newer release
Continuous delivery
It produces software in short cycles,
ensuring that the software can be reliably released at any time

8. Employed Tools
● Vagrant
● Fabric
● AWS
● Boto3 / AWS CLI

9. Vagrant
What is it?
It is a tool to manage Virtual Machines and Containers
and a repository of pre-installed VM

10. Vagrant: some definitions
"Provisioners"
Tools like Puppet, Chef and Ansible but I use Fabric
They run commands on a machine
"Providers"
Providers are the services that Vagrant uses to set up and create
virtual environments (VirtualBox, Hyper-V, and Docker…)

11. Vagrant: How I use it
Choose a pre-installed VM: a Box
https://vagrantcloud.com/boxes/search
Initialize the machine:
$ vagrant init ubuntu/bionic64

12. Vagrant: How I use it
The ‘init’ command create a Vagrantfile
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/bionic64"
end
where one can configure some parameters as
# of cpus and memory size

13. Vagrant: How I use it
Start the VM with:
$ vagrant up
then login
$ vagrant ssh
and stop it
$ vagrant halt

14. Fabric: What is it?
It is a python library
It is designed to execute shell commands remotely over SSH connection
It is similar to Ansible more than Chef or Puppet
It uses “Invoke” library to manage shell command execution
It uses “Paramiko” library to manage low/mid level SSH functionality

15. Fabric: How I use it
To install packages
Configure servers (as apache, monit, postfix, slapd)
Compile sources
Run buildouts

16. Fabric: How I use it
Ansible terminology
Control machine
Machine from where we can manage other machines.
Remote machine
Machines which are handled/controlled by control machine.

17. Fabric: Basics
The most basic use of Fabric is:
to execute a shell command on a remote system via SSH,
then check the result.
Other typical operations are:
● interact with some cli commands (with Responder)
● transfer files
● run the same commands on multiple machines

18. Fabric: Methods
Run
It is the simplest method: run a single command on a server:
from fabric import Connection
server = Connection('web1')
result = server.run('hostname')
if result.ok:
...

19. Fabric: Methods
Sudo
To run commands as the remote system's superuser
server.sudo('useradd axa')
server.sudo('mkdir /opt/aws')
server.sudo('apachectl restart')

20. Fabric: Methods
Put
Send a file to the server
server.put('apache2.conf', remote='/etc/apache2/')

21. Fabric: More Libraries
● Fabtools
● Cuisine

22. Fabric: Fabtools
https://fabtools.readthedocs.io/
It provides methods to manage system users, packages, databases, etc.
The 'require' methods are a declarative style similar to Chef or Puppet.

23. Fabric: Fabtools
Some modules:
● fabtools.require.apache
● fabtools.require.git
● fabtools.require.groups
● fabtools.require.nodejs
● fabtools.require.postfix
● fabtools.require.postgres
● fabtools.require.python
● fabtools.require.system
● fabtools.require.users

24. Fabric: Fabtools Examples
fabtools.require.apache.server()
Require the Apache HTTP server to be installed and running.
fabtools.require.users.user('axa')
Require a user and its home directory.
fabtools.require.postgres.database('userdb', 'axa')
Create a Database in postgres

25. Fabric: Cuisine
https://github.com/sebastien/cuisine
It has some methods similar to fabtools (Chef-like)
It provides common administration operations such as:
● file/dir operations
● user/group creation
● package install/upgrade

26. Fabric: Cuisine Methods
text_*
Text-processing functions
file_*
File operations
dir_*
Directory operations

27. Fabric: Cuisine Methods
package_*
Package management operations
command_*
Shell commands availability
user_*
User creation commands

28. Fabric: Cuisine Methods
group*
Group creation commands
mode_*
Configures cuisine's behaviour within the current session.
select_*
Selects a specific option, such as package back-end
(apt, yum, zypper, or pacman)

29. Fabric: Cuisine Examples
file_update(
"/etc/monit/monitrc",
lambda _:text_replace_line(
_,
"# with start delay 240 # optional: delay the first check by 4-minutes (by ",
" with start delay 60",
)[0]
)

30. Fabric: Cuisine Examples
user_ensure('axa')
Ensures that the given users exists,
optionally updating their passwd/home/uid/gid/shell.
package_ensure('imagemagick')
Tests if the given package is installed, and installs it in
case it's not already there.

31. Fabric example
from fabric.api import task, run
@task
def check_system():
""" """
run('uname -a')
run('lsb_release -a')
run('hostnamectl status')
$ fab -H sanctuary check_system

32. Amazon Web Services (AWS)
Motivations
To deploy production and stage machines in a simple and replicable
way
With the chance to choose and change the size of the machine
whenever required (cpu, ram, disk)

33. AWS: What is it
"Amazon Web Services offers a broad set of global cloud-based products
including compute, storage, databases, analytics, networking, mobile,
developer tools, management tools, IoT, security and enterprise
applications."

34. AWS: How I use it
EC2, EBS and EIP for the most
Snapshots as simple backup tool
Security group rules as a firewall to control traffic
… and more

35. AWS: How do I create a machine?
● Boto3
● AWS CLI

36. AWS: Why scripting installation?
To have “Infrastructure as code” so:
"...the process of managing and provisioning computer data centers
through machine-readable definition files, rather than physical
hardware configuration or interactive configuration tools."

37. AWS: Boto3
"Boto is the Amazon Web Services (AWS) SDK for Python. It enables
Python developers to create, configure, and manage AWS services, such
as EC2 and S3. Boto provides an easy to use, object-oriented API, as well
as low-level access to AWS services."

38. AWS: Boto3 - How I use it
Boto3 is a library providing methods to create and configure resources
● EC2 Elastic Compute Cloud
● EBS Elastic Block storage
● EIP Elastic IP
● CF Cloud Front (CDN: Content Delivery Network)
● RDS Relational Database

39. AWS: Boto3 - Examples
Show status and parameters of all instances
ec2 = boto3.client('ec2')
response = ec2.describe_instances()
print(response)

40. AWS: Boto3 - Examples
Create an Instance
ec2 = boto3.client('ec2')
ec2.run_instances(
ImageId='ami-0286372f78291e588',
InstanceType='t2.nano',
...)

41. AWS: Boto3 - Examples
Create a volume
ec2 = boto3.client('ec2')
volume = ec2.create_volume(
AvailabilityZone='eu-west-1a',
Size=50,
VolumeType='gp2',
...)

42. AWS: Boto3 - Examples
Start EC2 Instance
ec2 = boto3.client('ec2')
ec2.start_instances(InstanceIds=[instance_id])

43. AWS: AWS CLI
The AWS Command Line Interface (CLI) is a unified tool to manage your
AWS services.
The commands are the same available in boto3 but they can be used
from bash scripts.
How to install it:
$ pip install awscli
Yes, it is in Python and it is based on boto3!

44. Thank you

45. ● https://www.fabfile.org/
● https://12factor.net/