[HKOSCON][20200613][ Ansible: From VM to Kubernetes]

1. Hong Kong Open Source
Conference 2020
Ansible: From VM to Kubernetes
Edison Wong
2020-06-13

2. Wong Hoi Sing, Edison
●
2005 - Drupal Developer & Contributor
– https://drupal.org/user/33940
●
2008 - HKDUG Co-founder
– https://groups.drupal.org/drupalhk
●
2010 - CEO, PantaRei Design
– hswong3i@pantarei-design.com

7. PantaRei Design
●
Everything Changes and Nothing Remains Still
●
Reinvent Enterprise with Open Source Software and Cloud Computing
●
Hong Kong based FOSS service provider
– Content Management System (CMS) with Drupal
– Cloud Hosting Solution with Amazon Web Services (AWS)
– Team collaborate solution with Atlassian
●
Business Partner with industry leaders
– 2012, AWS Consulting Partner
– 2013, Acquia Partner
– 2013, Atlassian Experts
– 2014, Rackspace Hosting Partner
●
http://pantarei-design.com

9. Outline
●
HKOSCON 2019
●
Why DevOps with Ansible?
●
Ansible with VM
●
Ansible with Docker
●
Ansible with Kubernetes
●
Tips & Tricks
●
Roadmap
●
Q&A

10. HKOSCON 2019
●
Ansible Role with Molecule + LXD
●
Docker Build with Ansible
●
Kubernetes with Molecule +
Vagrant + VirtualBox

11. Ansible Role with Molecule +
LXD
●
Molecule LXD driver + Travis CI
●
Could mock up 80% use cases
●
Lack of cgroup/network/device support
●
(2020) Improved with Vagrant + Libvirt
+ Travis CI

12. Docker Build with Ansible
●
Ansible playbook drive by
Dockerfile, inside target container
●
Reduce custom bash shell scripting
●
(2020) Improved with Molecule
Docker driver + `docker commit`

13. Kubernetes with Molecule +
Vagrant + VirtualBox
●
Molecule Vagrant driver +
VirtualBox for local test
●
Slow, limited OS, no Travis CI
●
(2020) Improved with Vagrant +
Libvirt + Travis CI

14. Why DevOps with Ansible?
●
SysAdmin Daily Difficulties
●
Why DevOps?
●
Why Ansible?

15. SysAdmin Daily Difficulties
●
Different deployment target
●
Test logic before deploy
●
Complex cluster management
●
Documentation
●
No time for learning

16. SysAdmin Daily Difficulties
(cont.)
●
Write-once for all cases
– Native/Bare Metal/VM
– Docker/LXD/Vagrant
– OpenStack/AWS/GCE/Azure
– Kubernetes/OpenShift/AKS/GKE/EKS

18. Why DevOps?
●
Manual install
– Non-repeatable
●
Manual install with document
– Difficult to manage (Docs to Action)
– Always async with production
●
Manual scripting
– Difficult for everything: learn, write, error detection,
debug, etc…

19. Why DevOps? (cont.)
●
DevOps
– Deployment logic as code (i.e. revision
with GIT)
– With error detection and debug tools
– Manage multiple deployment target at
once (e.g. data center, clustering)

20. Why Ansible?
●
Writing “tasks” in YAML
– Human readable == minimize
documentation
– Easy to learn, when compare with
Ruby for Chef or Puppet

21. Why Ansible? (cont.)
A lot of reusable modules
– Simplify complicated logic with error
detection
– Or running “shell” command directly

22. Why Ansible? (cont.)
●
Simple requirement
– Python and Password-less SSH
– Agent-less for managed node

25. Ansible with VM
●
Ansible CLI
●
Ansible Playbook
●
Ansible Role
●
Molecule + Delegate
●
Demo: ansible-role-sshd

26. Ansible CLI
●
Running command on remote guest is
simple
– ansible -i guest1,guest2, -m ping
– ansible -i guest1,guest2, -m apt -a ‘name=vim
state=present’
– ansible -i guest1,guest2, -m shell -a ‘uname -a’

27. Ansible Playbook
●
Running multiple “task” once together
●
Finer control than running with CLI
●
Define your inventory then play with it
– ansible-playbook -i inventory/all/hosts
playbooks/setup-everything.yml

30. Ansible Role
●
Not just “Tasks”, but also:
– Default over-writable variables
– Internal static variables
– Static files for copy
– Template files
– Event handlers
●
A basic build block for complex architecture
– Use Playbook to include different Roles

31. Ansible Role (cont.)
●
Create a new role with ansible-galaxy
– mkdir ~/.ansible/roles
– cd ~/.ansible/roles
– ansible-galaxy init dummy
●
You could now test it (run via your localhost)
– cd ~/.ansible/roles/dummy
– ansible-playbook -i tests/inventory tests/test.yml
●
Limited functionality

33. Ansible Role (cont.)
●
Molecule
– Testing framework for Ansible
– Written in Ansible and Python style
– Write your test case in standard Ansible style
– Manage test environment life-cycle for you
– Code lint
– Idempotence (i.e. run twice to confirm no extra
changes)

35. Ansible Role (cont.)
●
Create a new Role with molecule
– cd ~/.ansible/roles
– molecule init role -r dummy2 -d docker
– molecule init role -r dummy3 -d lxd
– molecule init role -r dummy4 -d vagrant
●
Now you could run test inside Docker
– cd ~/.ansible/roles/dummy2
– molecule test

37. Molecule + Delegate (cont.)
●
Molecule + Delegate
= Ansible Role Installer
– Roles dependency management
– No custom wrapper playbook
– Install into localhost

38. Demo: ansible-role-sshd
●
https://github.com/alvistack/ansible-role
-sshd
– mkdir ~/.ansible/roles && cd ~/.ansible/roles
– git clone
https://github.com/alvistack/ansible-role-
sshd.git sshd && cd sshd
– molecule converge

43. Ansible with Docker
●
Why NOT Dockerfile?
●
Why NOT ansible-bender?
●
Molecule + Docker
●
Demo: docker-jira

44. Why NOT Dockerfile?
●
Back to the origin: why still
custom shell scripting?
– Difficult for everything: learn, write,
error detection, debug, etc…

45. Why NOT ansible-bender?
●
https://github.com/ansible-commu
nity/ansible-bender
– Build Docker Image with standard
Ansible Playbook
– Podman + Buildah based
– Just need basic Python support inside
target container

46. Why NOT ansible-bender?
(cont.)
●
PROS
– Push image to DockerHub once build
successful
●
CONS
– Could NOT integrate with Travis CI
– Only support Podman + Buildah
– Not compatible with Molecule

48. Molecule + Docker
●
Molecule + Docker
= Docker image creator
●
Support both Docker and Podman
●
Run as standard Molecule test case
●
`docker commit` during destroy phase
●
Push result Docker image to remote registry

49. Molecule + Docker (cont.)
●
molecule/*/Dockerfile.j2
– Just define meta data (e.g. FROM,
EXPOSE, ENTRYPOINT, CMD, etc)
– Minimal RUN (e.g. groupadd,
useradd, etc)

51. Molecule + Docker (cont.)
●
molecule/*/create.yml
– Create initial base image with meta
data as Dockerfile.j2
– Override CMD with `base -c “sleep
infinity”` on-the-fly for running test

53. Molecule + Docker (cont.)
●
molecule/*/destroy.yml
– Fetch base image meta data
– Commit running Docker instance
with base image’s CMD/ENTRYPOINT

55. Demo: docker-jira
●
https://github.com/alvistack/docker-jira
– Docker Image packaging for Atlassian JIRA
– Molecule + Docker
– All used Roles are Vagrant + Libvirt tested
– Push to DockerHub once Travis CI passed

59. Ansible with Kubernetes
●
Molecule + Vagrant + Libvirt
●
Demo: ansible-collection-
kubernetes

60. Molecule + Vagrant + Libvirt
●
In case of Ceph OSD, truth block device is
required
– Not support file-based loop device
●
In case of Weave, each Kubernetes node
must have unique machine ID
– With LXD all instance get the same host
machine ID

61. Molecule + Vagrant + Libvirt
(cont.)
●
Molecule + Vagrant + Libvirt
= 100% mock up
●
Support Travis CI
●
Support multiple instances for cluster test
●
Support multiple OS
●
Fully support cgroup/network/block/etc

64. Demo: ansible-collection-
kubernetes
●
Ansible + Ceph + Kubernetes + Addon
– All Roles tested with Vagrant + Libvirt individually
– Simply clone-and-play
●
Multiple OS Support
– Ubuntu 18.04/19.10/20.04
– RHEL/CentOS 7/8
– openSUSE Leap 15.1
– Debian 10
– Fedora 32
●
https://github.com/alvistack/ansible-collection-kubernetes

66. Demo: ansible-collection-
kubernetes (cont.)
●
Support different deployment style
– Single All-in-One
– (Kubernetes + Ceph) x3
– Kubernetes x3 + Ceph x3
– Kubernetes xN + Ceph xN

68. Demo: ansible-collection-
kubernetes (cont.)
●
Kubernetes 1.18.3
– CRI-O
– CNI: Weave
– CSI: CephFS
– Ingress Nginx
●
Ceph 15.2.3

70. Demo: ansible-collection-
kubernetes (cont.)
●
Support individual application
deployment per namespace, e.g.
– Drupal + Apache + PHP-FPM + MariaDB
– Jira + Apache + PostgreSQL
●
Support HTTPS with Let’s Encrypt

72. Demo: ansible-collection-
kubernetes (cont.)
●
Fetch source
– git clone https://github.com/alvistack/
ansible-collection-kubernetes.git && cd
ansible-collection-kubernetes
– git submodule update --init –recursive

73. alvistack/ansible-collection-
kubernetes (cont.)
●
Setup inventory
– cp -rfp inventory/default inventory/all
– vi inventory/all/hosts

74. alvistack/ansible-collection-
kubernetes (cont.)
●
Run the playbook
– ansible-playbook -i inventory/all/hosts
playbooks/coverge.yml

78. Tips & Tricks
●
Always Start with Test Cases
●
Simple Deployment Goes Molecule + Delegate
●
Test Cases Always Goes Molecule + Vagrant +
Libvirt + Travis CI
●
Create Docker Image After Molecule Test Case
by Commit

79. Roadmap
●
Migrate everything from Docker
to Podman/Buildah/Skopeo
●
Handle Kubernetes Addons with
Ansible Operator

80. Q&A

81. Contact Us
●
Address: Unit 326, 3/F, Building 16W, No.16
Science Park West Avenue, Hong Kong Science
Park, Shatin, N.T.
●
Phone: +852 3576 3812
●
Fax: +852 3753 3663
●
Email: sales@pantarei-design.com
●
Web: http://pantarei-design.com