Christophe Romain (ProcessOne) introduces Docker and ProcessOne work in progress to support Docker-based ejabberd deployments.

1. Managing ejabberd
platform with
Docker containers

2. What is Docker
• Linux tool, written in Go language
• Started in 2010 as a concept at dotCloud SARL
• Published 2y ago by creator Solomon Hykes with Open
Source license (Apache 2.0)
• Handled by Docker Inc
• Partnership with Red Hat, IBM, Google, Amazon...
• Automates the deployment of applications as standalone
software containers

3. High level view
• Own process space
• Own network interface
• Include all binaries/libs required by your software
• Works the same on every Linux host
• Just need Linux Kernel on host machine

4. Low level view
• Like chroot++ or improved BSD jails, without need
of custom configuration and deployment scripts
• Shares Kernel with Host and other containers, but
in isolated environment
• No need for device emulation (neither HVM nor PV)
• Adds API to existing Linux Kernel features: LXC,
cgroups, namespaces

5. Container vs VM
• Virtual Machine: emulates hardware, runs the
Operating System and applications
• Docker: system process with isolated view of the
namespace and resources, runs libraries and
applications. No emulation layer. Accumulates
AUFS mounts for container filesystem.

6. Container vs VM

7. Why it matters
• Containers start fast, are smaller than VM
• Build once run everywhere, matches cloud needs
• Containers are easy to replicate (docker hub)
• No runtime resources overhead, no emulation
• Allow low level packaging optimisation/setup

8. Drawbacks
• Works only on Linux (for now?)
• Requires Linux 3.8+
• Kernels older than 3.10 lack some of the features
required to run some containers

9. ejabberd requirements
• Ejabberd beam files
• Erlang dependencies beam and NIF
• Configuration and runtime files
• Erlang VM and system library dependencies:
libpthread, libm, libssl, libc...

10. Installer vs Container
• Erlang is already a VM after all ! Installed may be sufficient ?
• Installer must include everything (code+database) and can not
handle upgrade without external and custom scripts
• Installer is standalone but uses hosts's libc, which may differ from
one host to another
• No clean way to use custom configuration and database with
Installer/rpm/deb
• Containers gives deeper application integration, makes upgrade
process simpler (more control on global environment), and ensure
same environment deployed everywhere. (no hazardous
dependencies issues)

11. Installations issues
• Heterogeneous setup: installed from sources, from official binary
installer, from deb/rpm/pkgsrc/brew/etc... => impossible to
reproduce clean, secure and optimised setup everywhere, with
same version of erlang for a given ejabberd release
• Way too much combination, from system libs to erlang version.
Many possible configuration, many patches on packaged Erlang
and/or system libraries, etc...
• No hands on host's network settings and tunning
• Binary installer gives more control on software dependencies,
but still depends on host's libc compatibility and does not ensure
user uses bundled libraries.

12. Simple container
• From Debian Jessie
• Download ProcessOne binary installer
• Install using unattended mode
• Apply own configuration
=> Build new container for every release
=> Depends on ProcessOne binary packaging

13. ejabberd container
• By Rafael Römhild (rroemhild/ejabberd)
• Complete and customisable container
• Available for all tags since 14.12 up to 15.10
• Based on Debian Jessie
• Uses debian's Erlang/OTP, build ejabberd from
sources.

14. Environment
EJABBERD_HTTPS true
EJABBERD_STARTTLS true
EJABBERD_S2S_SSL true
EJABBERD_PROTOCOL_OPTIONS_TLSV1 false
EJABBERD_LOGLEVEL 4
EJABBERD_MUC_CREATE_ADMIN_ONLY
EJABBERD_REGISTER_ADMIN_ONLY
EJABBERD_MOD_ADMIN_EXTRA true
EJABBERD_MOD_MUC_ADMIN false
EJABBERD_REGISTER_TRUSTED_NETWORK_ONLY
EJABBERD_SKIP_MODULES_UPDATE false
EJABBERD_AUTH_METHOD internal
EJABBERD_ADMINS
EJABBERD_USERS
ERLANG_NODE ejabberd@localhost
ERLANG_COOKIE
ERLANG_OPTIONS -noshell
XMPP_DOMAIN
TZ Europe/Paris

15. Configuration
• Few environment variables allows to change simple
configuration settings, but default configuration
remains very generic, too generic...
• Better create your own container with your own
configuration and certificates:
FROM rroemhild/ejabberd:15.10
ADD ./ejabberd.yml /etc/ejabberd/ejabberd.yml
ADD ./ejabberdctl.cfg /etc/ejabberd/ejabberdctl.cfg
ADD ./example.com.pem /opt/ejabberd/ssl/example.com.pem
$ docker build -t p1/ejabberd:15.10 .

16. Persistent storage
• Using a data container, hum... really ?
docker create --name ejabberd-data rroemhild/ejabberd-data
docker run -d
--name "ejabberd"
--volumes-from ejabberd-data
....
rroemhild/ejabberd:15.10
• Using mounts from host filesystem
docker run -d
--name "ejabberd"
-v $PWD/database:/opt/ejabberd/database
-v $PWD/logs:/var/logs/ejabberd
....
rroemhild/ejabberd:15.10

17. Using custom container
• Simplify both configuration and customisations
• Allows better integration with your environment
• Allows implicit linking with your database container
• Very simple to write and maintain
• Build "ready to use" containers for your use

18. Deploy/share containers
• Need a docker hub account (public/private)
• docker push me/ejabberd:15.10
• docker pull me/ejabberd:15.10
• docker run me/ejabberd:15.10

19. Docker hub
• Public repository to share your app
• Private repository to share container with dev team
• Private repository to easy deploy app in-house
• https://docs.docker.com/docker-hub/

20. Start ejabberd
docker run -i -t -P
--name "15.09"
-p 5222:5222
-p 5269:5269
-p 5280:5280
-h 'example.net'
-e "XMPP_DOMAIN=example.net"
-e "ERLANG_NODE=node1"
-e "ERLANG_COOKIE=demo"
-e "EJABBERD_ADMINS=admin@example.net"
-e "EJABBERD_USERS=admin@example.net:password1234"
-e "TZ=Europe/Paris"
-v $PWD/database:/opt/ejabberd/database
-v $PWD/logs/node1:/var/log/ejabberd
p1/ejabberd:15.09 live
docker start 15.09
docker exec 15.09 ejabberdctl status
docker stop 15.09

21. Simple upgrade
• Simple switch from 15.09 to 15.10 when using
same data container or mounting point, and also
using same configuration.
• Allow easy upgrade/rollback but stops service if
serving with a single node.
docker start 15.09
docker exec 15.09 ejabberdctl status
docker stop 15.09
docker start 15.10
docker exec 15.10 ejabberdctl status
DEMO

22. So what ?
• Using our custom container, upgrading from 15.09 to
15.10
• Using host mounts for data and preconfigured
ejabberd
• a simple start/stop of the right container is enough
=> NO EXTRA CHANGE REQUIRED !
• improvement: N+1 / N-1 automatic upgrade/migration
code needed ejabberd side for full coverage.

23. Smooth upgrade
• Using haproxy as front-end, two ejabberd containers running on
same host, mapping them to their own ports and own erlang node
name to play with cluster
current container A: -p 15222:5222
next container B: -p 25222:5222
• Haproxy redirects 5222 to 15222, then start container B, then add B
in the cluster, then haproxy redirects 5222 to 25222, after a while
remove A from cluster and stop A.
• - : Requires lots of automated stuff, complex, not very clean
• + : Allows using only one host without disconnecting all your users

24. Cluster upgrade
• Run ejabberd container on several hosts
• Get them as one ejabberd cluster
• Apply simple upgrade node after node
• Disconnected users automatically reconnect to
another running node
• Load can be redirected at balancer level to reduce
impact of disconnections (takes more time, depends
on service use)

25. Cluster issue
• Ejabberd nodes needs direct network link at each
other, being part of same network
• Containers runs on different hosts
• Docker links are only bound to local host
• Containers are isolated
• Need to resolve remote hosts: multi-host networking
using overlay network driver is docker solution

26. Multi host networking
• Needs Linux 3.16+
• Access to key-value store (Consul, Etcd or ZooKeeper)
• A configured Engine daemon on each host in the cluster
• Not straight forward solution
$ docker network create -d overlay
--subnet=192.168.0.0/16 --subnet=192.170.0.0/16
--gateway=192.168.0.100 --gateway=192.170.0.100
--ip-range=192.168.1.0/24
--aux-address a=192.168.1.5 --aux-address b=192.168.1.6
--aux-address a=192.170.1.5 --aux-address b=192.170.1.6
my-multihost-network

27. Simpler multi host
• Docker container can access host interface being
bridged by its own interface
• Need private internal network (ip alias can work)
• Containers can ping by internal network IP
• Custom container needs /etc/hosts and inetrc with static
IP for all your nodes
• But needs update /etc/hosts when adding nodes. So this
is OK until our cluster changes over time (no easy scale)

28. Other DNS solutions
• dnsdocker using docker compose
• etcd, consul or zookeeper for use with overlay
• other container management systems like Rancher
• possibles improvement with new Networking in
Docker 1.9 (relay on an official image to provide
clean DNS resolution across containers ?)
=> need improvements to support dynamic cluster

29. ejabberd container
improvements (WIP)
• Allow easy custom configuration
• Native support of multi-host clustering and transient
joincluster/leavecluster
• Consistency with standard installation
• Custom minimalist Erlang/OTP+Elixir with hipe support
• Smaller container (based on Alpine instead of Debian,
from 245Mb to 35Mb !)

30. Questions
Christophe Romain <cromain@process-one.net>